Application Programming Interface Design and Scalability Questions
Designing application programming interfaces that remain reliable, performant, and maintainable at high scale. Candidates should understand how interface decisions affect scalability, availability, latency, and operational complexity and be able to reason about trade offs between client complexity and server responsibility. Core areas include stateless interface design, pagination and cursor strategies for large result sets, filtering and search optimization, payload minimization, batching and streaming, and techniques to reduce server load while preserving client experience. Resilience and operational controls include rate limiting and quota management, throttling, backpressure and flow control, retry semantics and idempotency patterns, error format design and explicit identification of retryable errors, and strategies for graceful degradation under overload. Evolution and compatibility topics include backward compatible versioning strategies, deprecation policies, contract design and testing approaches to avoid breaking consumers. Infrastructure and deployment considerations include API gateway and edge patterns, interaction with load balancers and traffic distribution, caching and content delivery, routing fault tolerance, health checks and canary rollout strategies, and observability through metrics, distributed tracing, and logging to support capacity planning and incident response. Security considerations such as scalable authentication and authorization, credential and key management, and permission models are also important. Candidates should be prepared to discuss concrete patterns, trade offs, algorithms, and operational playbooks for designing and running high traffic application programming interfaces.
MediumTechnical
58 practiced
Evaluate GraphQL and REST for a large, multi-product API platform focusing on operational concerns: edge caching, rate limiting complexity, query complexity attacks, batching, schema federation, telemetry, and developer onboarding. Provide concrete mitigations for GraphQL-specific issues such as depth limiting, persisted queries, and query cost analysis.
Sample Answer
High-level verdict: REST is operationally simpler and predictable; GraphQL offers flexibility and fewer round-trips but introduces operational complexity that must be addressed with gateway controls and observability. As a solutions architect I’d pick REST for simpler multi-product APIs where caching & rate controls are primary concerns; choose GraphQL when client-driven fetching, schema evolution, and reduced client code justify additional operational investments.Edge caching- REST: straightforward—HTTP semantics (Cache-Control, ETag) at CDN/edge. Cache keys map to URL+headers.- GraphQL: single POST endpoint makes CDN caching harder. Mitigations: use persisted queries (stable IDs/GET endpoints), operation-specific URLs, or cache per-response fingerprinting; add CDN rules keyed by persisted query ID + variables hash.Rate limiting complexity- REST: per-route / per-client limits are simple.- GraphQL: one query can be cheap or expensive. Implement combined controls: per-client global quotas + per-operation (persisted-query) quotas; enforce per-field or per-resolver throttles in the gateway.Query complexity attacks & batching- Threat: deeply nested or expensive aggregations.- Mitigations: - Depth limiting: reject queries beyond N nesting levels at gateway. - Query cost analysis: assign cost weights per field/resolver and reject requests exceeding budget. - Persisted queries: require pre-registered operations to enable static vetting and allow caching and per-operation rate limits. - Disable introspection in prod or gate it behind auth to reduce reconnaissance. - Batching: use automatic persisted queries + gateway-level query batching carefully—limit batch size and total cost per batch.Schema federation & multi-product ownership- GraphQL: Apollo Federation or schema stitching allows product teams to own subgraphs; requires clear contract/versioning, ownership metadata, and CI checks. Use federated gateway for runtime composition and a registry for persisted queries and schema history.- REST: straightforward API versioning and service ownership; composition via API gateway and orchestration.Telemetry and observability- REST: standard access logs, distributed tracing per route, metrics per endpoint.- GraphQL: require richer telemetry: trace at field/resolver granularity, per-operation latency/cost, aggregated usage per persisted query, error rates, and field-level hotspots. Integrate trace IDs through gateway to backend resolvers; export costs and depths to monitoring and alerting.Developer onboarding- REST: clear contracts (OpenAPI), easy mocking, stable caching semantics.- GraphQL: faster client iteration, auto-generated types, but needs schema registry, examples, tooling (GraphiQL/Playground), and enforced persisted-query workflow to keep prod-safe. Provide templates, CI checks, and a local mocking layer.Concrete GraphQL mitigations summary- Depth limiting: gateway enforces max nesting; tune per client type and monitor rejection rates.- Persisted queries: require registration pipeline (CI + registry), use query IDs in CDN/edge caching, and enable per-OP ACLs & quotas.- Query cost analysis: define cost model (weights for scalars, lists, db-backed resolvers), compute static cost at parse-time where possible, and add runtime instrumentation to account for dynamic costs (e.g., list sizes). Reject or throttle queries above thresholds and log for review.Trade-offs and recommendation- If operational simplicity, predictable caching, and easy rate limiting are highest priorities, use REST with fine-grained endpoints and an API gateway. If client agility and reduced chattiness drive product value, adopt GraphQL but invest in a hardened gateway that enforces persisted queries, depth/cost limits, per-operation quotas, federated schema lifecycle, and field-level telemetry. Provide a developer experience: schema registry, CI validation, examples, and SDKs to keep risk low while enabling flexibility.
MediumTechnical
61 practiced
As a solutions architect, list the key metrics, traces, and logs you would instrument for a public API to support capacity planning and incident response. Include dimensionality examples (by endpoint, tenant, region), latency percentiles, error budgets, sampled distributed traces, and log correlation keys. Explain a tagging strategy and trade-offs with metric cardinality.
Sample Answer
Key metrics, traces, and logs to instrument for a public API (to support capacity planning and incident response):Metrics (aggregated + dimensionality examples)- Request rate (rps) by endpoint, tenant, region, method, API key- Concurrent connections / active sessions by region, tenant- Latency percentiles per endpoint: p50, p90, p95, p99, p999 (also bucketed histograms for percentiles)- Error rate (5xx, 4xx) and error counts by endpoint, error code, tenant, region- Backend/DB call latency and saturation (CPU, mem, queue length) by service- Throughput (bytes in/out) per endpoint/tenant- Capacity signals: thread pool usage, queue depth, GC pause, file descriptorsSLOs / Error budget- Define SLOs per critical endpoint (e.g., 99.9% requests p99 < 300ms) and track error budget burn rate metric (errors / allowed errors) per tenant and global- Alert on sustained high burn (>X% in rolling window)Traces- Sampled distributed traces (trace sampling ~1–5% of requests, adaptive sampling on errors/high latency)- Capture span names for gateway, auth, business service, DB, cache; include service durations and annotations (retry, circuit-breaker events)- High-signal sampling: always sample errors and requests exceeding p95/p99Logs & Correlation- Structured logs (JSON) with timestamp, level, message, service, endpoint, tenant_id, region, request_id, trace_id, user_id (where allowed), status_code, latency_ms- Log correlation keys: request_id (generated at edge), trace_id (propagated), span_id; ensure request_id is included in all logs and metrics where possibleTagging strategy- Use small stable set of tags: service, environment (prod/stage), region, endpoint (path template), tenant_id (or tenant_bucket for high-cardinality), status_code, deployment_version- For tenants, avoid including raw tenant IDs in high-cardinality metrics. Use coarse buckets (top-N tenants by volume + "others") and send tenant-specific metrics only for VIP tenants or sampled telemetry.Trade-offs and cardinality- High cardinality (full tenant_id × endpoint × region × version) gives precise root-cause and capacity insight but increases storage and query costs and can harm metric aggregation performance.- Mitigation: roll up metrics (global + per-region + per-endpoint), use tagging limits, cardinality caps, sampling, pre-aggregate at edge, and maintain separate detailed telemetry pipeline for forensic investigations.Operational notes- Expose dashboards: traffic heatmap, p95/p99 latency trends, SLO burn-rate, error-rate by component, capacity headroom- Alerts: latency SLO breach, error budget burn, sudden drop in throughput, queue-depth high, trace-based spike in downstream latency- Regular capacity planning: use peak RPS by region × p99 CPU/memory per request to model headroom with safety factor (e.g., 2x peak), validate with load tests.This mix supports both real-time incident response (alerts, correlated traces + logs) and longer-term capacity planning (traffic trends, percentiles, saturation metrics) while controlling cardinality costs.
MediumTechnical
114 practiced
You operate a recommendation API that occasionally overloads under traffic spikes, causing high latency for some requests. As solutions architect, outline a plan to gracefully degrade the API so core functionality remains available. Include strategies like circuit breakers, degraded models, cached fallbacks, request hedging, and client feature flags, and explain how to prioritize which features degrade first.
Sample Answer
Situation: Our recommendation API suffers latency spikes under traffic surges, impacting customer experience and SLAs. Goal: implement graceful degradation so core functionality remains available and user impact is minimized.Plan (high-level architecture):- Traffic gate + circuit breaker layer at API edge (envoy/NGINX/service mesh) to detect high error/latency and short-circuit expensive downstream calls.- Tiered fallback stack: 1. In-memory local cache (hot recommendations per user/segment). 2. CDN/Redis cached responses for recent queries. 3. Lightweight “degraded model” (rule-based or small similarity lookup) that's orders-of-magnitude faster. 4. Static popular/top-N recommendations as last resort.- Request hedging: send fast degraded-model request in parallel with full model when latency risk is detected; accept first successful response.- Client feature flags & capability negotiation: clients expose tolerance levels (latency vs quality); server-side flags enable degradation per client or cohort.- Telemetry & control plane: real-time metrics (p95/p99 latency, error rate, cache hit ratio), automated policies to trigger degradation and manual override in dashboard.Circuit breaker behavior:- Open when latency/error thresholds exceeded for a sliding window; route traffic to fallbacks, incrementally probe downstream when closed-half.Prioritization for feature degradation:1. Protect core actions that generate revenue/critical flows (checkout, search, login).2. Degrade low-value, high-cost features first (personalized promotions, optional widgets).3. Use user segmentation: premium/SLAs get higher fidelity; anonymous/free users get degraded paths earlier.4. Evaluate by metrics: conversion impact, revenue per request, and user retention risk.Operational practices:- Chaos tests and load drills to validate policies.- SLOs around correctness vs availability; document degradation contract for clients.- Gradual rollout using canary rules; rollback via feature flags.Result: This preserves core API availability, gives predictable behavior under load, and lets product teams control acceptable quality–latency trade-offs.
HardTechnical
62 practiced
Draft an incident response playbook for an API outage caused by primary database degradation. Outline immediate mitigation steps such as diverting traffic, enabling read-only mode, increasing caching, throttling, failover to replicas, communication templates for customers and stakeholders, controlled rollbacks, and post-incident RCA actions.
Sample Answer
Situation: Primary DB is degrading causing API errors/latency affecting customers. Objective: restore service quickly with minimal data-loss risk, keep stakeholders informed, and perform a thorough RCA.Immediate Triage (first 0–10 mins)- Triage lead (on-call SRE) acknowledges incident and sets priority/severity.- Collect telemetry: error rates, tail latency, DB CPU/I/O, replication lag, connection pool exhaustion, slow queries.- Open incident channel (Slack/Zoom) and incident doc with timeline.Immediate Mitigations (10–30 mins)1. Divert traffic - Shift non-critical traffic to standby regions/CDN origin if multi-region available. - Use load-balancer weights to reduce traffic to affected region.2. Enable read-only mode - Put API tier into read-only for write paths (feature-flag + gateway rule) to stop write amplification and reduce DB load.3. Increase caching - Extend TTLs for cacheable endpoints (edge + app cache). - Warm cache for high-value keys; prepopulate common responses.4. Throttling & graceful degradation - Apply rate-limits per client and global backpressure (token-bucket). - Serve degraded responses (cached/stale) with 503 Retry-After when necessary.5. Failover to replicas - Evaluate replica freshness and promotion risk. If replicas lag < acceptable threshold, promote a replica following documented failover playbook; otherwise avoid split-brain. - If read replicas are healthy, route read traffic to replicas.6. Controlled rollbacks - If recent deploy suspected, perform controlled rollback of the change to a known-good release with canary gating.Communication Templates- Customer-facing (email/status page): Subject: Service degradation impacting API — [INCIDENT ID] Body: We are investigating increased errors/latency impacting API calls. Mitigations: read-only mode, increased caching, rate limits. No data loss expected. Next update: in 30 minutes. Contact: support@company.com- Internal (exec): Subject: Incident [ID] — Impact & Actions Body: Impact summary, user-visible symptoms, actions taken, ETA for next update, business impact estimate, escalation needs.Post-Incident (within 24–72 hrs)- Controlled recovery: revert temporary throttles/TTL changes gradually; monitor error/latency metrics for regressions.- Postmortem / RCA: - Timeline with decisions, evidence, root cause analysis (5 whys), contributing factors. - Action Items: e.g., add automated failover runbooks, increase replica capacity, limit long-running queries, add circuit-breakers, test read-only workflows, improve observability (DB queue depth, replica lag), and update runbooks. - Assign owners and deadlines; track to closure.- Customer follow-up: summary, impact, remediation steps taken, SLA credits if applicable.Key trade-offs & decision principles- Prioritize data integrity over availability (preferring read-only vs unsafe promotion).- Prefer automated, tested failover methods; manual promotion only with clear checks.- Communicate early and often; avoid speculative statements.This playbook balances rapid mitigation, risk containment, clear communications, and learning to reduce recurrence.
EasySystem Design
60 practiced
Design health check endpoints for a microservice-backed API. Specify the semantics and expected responses for at least three endpoints: liveness, readiness, and dependency checks. Explain how each should be used by load balancers, orchestration systems, and external monitoring, and what costly checks (like long DB probes) should be avoided in certain endpoints.
Sample Answer
Requirements & goals:- Fast, unambiguous probes that guide orchestration/load‑balancers and provide actionable monitoring signals.- Separate transient startup vs permanent failure signals.- Avoid long-running or resource-heavy checks on probes invoked frequently.Endpoint semantics and expected responses:1) /health/live (liveness)- Purpose: Is the process alive (not stuck, crashed, or deadlocked)?- Response: 200 OK + short JSON {"status":"alive","uptime_s":123} OR 5xx on fatal internal state.- Implementation: Very cheap — check event loop/thread responsiveness, leader heartbeat, or simple in-memory flag.- Usage: Kubernetes livenessProbe; if failing, orchestration restarts the container. Load balancers should NOT remove traffic on transient liveness failures (restart is preferred).- Avoid: DB/network calls, long I/O.2) /health/ready (readiness)- Purpose: Is the service ready to accept traffic?- Response: 200 OK + {"status":"ready","version":"1.2.3"} or 503 Service Unavailable + {"status":"initializing","eta_s":30}- Implementation: Lightweight checks that confirm required subsystems are initialized (config loaded, caches warmed, connection pools established).- Usage: LB and orchestration use this to add/remove instance from rotation. Should fail during startup, graceful shutdown, or when overloaded.- Avoid: Deep dependency probes that can be slow.3) /health/deps or /health/checks (dependency checks / detailed)- Purpose: Health of external dependencies (DB, message broker, external APIs) for monitoring and alerting.- Response: 200 OK + detailed JSON per dependency {"db":"ok","cache":"degraded","es":"down"}; include latency/timestamps.- Implementation: Run asynchronously or cached (e.g., background poll every N seconds) to avoid probe slowness; include TTL for freshness.- Usage: External monitoring (Prometheus, synthetic checks) should poll this less frequently; orchestration should NOT use this for liveness/readiness decisions unless strict SLA demands it.- Avoid: Performing heavy migrations or long queries synchronously; return last-known-state if real-time check exceeds threshold.Best practices:- Keep liveness/readiness sub-second responses; set short timeouts in orchestrator.- Use separate HTTP status codes: 200 OK for healthy, 503 for not ready, 5xx for unhealthy.- Expose metrics and logs for root-cause: probe failure reasons, timestamps, and recent errors.- Support graceful shutdown: readiness -> 503, drain connections, then stop.- Rate-limit/secure dependency endpoint (auth or internal network) to prevent abuse.- For Kubernetes: livenessProbe -> /health/live (exec/http), readinessProbe -> /health/ready; dependency endpoint scraped by monitoring.
Unlock Full Question Bank
Get access to hundreds of Application Programming Interface Design and Scalability interview questions and detailed answers.
