API and Full Stack Coding Patterns Questions

Beyond pure algorithms, be prepared for problems that combine algorithmic thinking with API design, rate limiting, caching strategies, or distributed system concepts. Understanding how a coding solution fits into a larger fullstack architecture.

MediumSystem Design

75 practiced

Design an observability plan for a public API platform: list the metrics, traces, and logs you would collect; describe how to correlate a request end-to-end across services (trace-id propagation); and propose dashboards and alerts to detect p95/p99 latency regressions or SLO burn.

Sample Answer

Requirements:- End-to-end visibility for public API requests across services, detect p95/p99 latency regressions, SLO burn, and rapid root-cause identification.- Low overhead, compatible with distributed tracing (OpenTelemetry), metrics (Prometheus), logs (structured JSON), and alerting (PagerDuty).

Metrics to collect:- Request-level: request_count (by service, route, method, status_code), latency_histogram (buckets for 10ms..10s), p50/p95/p99 gauges, success_rate.- System: CPU, memory, threadpool queue length, connection pool usage, GC pause, disk IO.- Dependency: downstream call latency/error rates, circuit-breaker state.- Business: API key usage, per-tenant rate, quota consumption.

Traces:- Instrument entry gateway + all services with OpenTelemetry; record spans for gateway auth, routing, service handlers, DB calls, external HTTP calls.- Capture attributes: trace_id, span_id, parent_id, service, route, method, status_code, tenant_id, sampling_priority, error flag.- Sample: head-based adaptive sampling + retain all error traces and high-latency traces (>= p95 threshold).

Logs:- Structured JSON logs including trace_id, span_id, request_id, timestamp, severity, message, service, route, user_id, tenant_id, error details, stack traces.- Export to centralized store (ELK/Opensearch/Loki) with fast indexed fields: trace_id, status_code, tenant_id.

Trace-id propagation:- Use W3C Trace Context headers (traceparent, tracestate) at gateway; libraries propagate context automatically; ensure every outgoing HTTP/gRPC/DB client injects traceparent; fall back to generated request_id for non-instrumented components.- Correlate logs and metrics by exposing trace_id in logs and metric labels (for sampled slow/error traces).

Dashboards:- Global API Overview: QPS, error rate, p50/p95/p99 latency heatmap, SLO burn chart (error budget remaining) with per-tenant filters.- Service-level: latency histogram, top N slow endpoints, dependency latencies, resource saturation panels.- Tracing view: recent p99 traces, longest spans, trace waterfall for selected trace_id.- SLO page: SLOs, burn-rate, rolling error budget window, incident history.

Alerts:- Latency regressions: Alert when p95 > baseline + X% for 10m (warning) and 30m (critical); p99 spike immediate critical.- SLO burn: use burn-rate (e.g., 14-day burn): Fire warning when burn-rate > 2x and critical when > 4x or when error budget <= 20% remaining.- Error rate: sustained error_rate > SLO error_budget_rate for 5m.- Resource alerts: queue length > threshold, CPU/Memory saturation > 90% for 5m.- Alert routing: page critical on-call, notify Slack for warnings; include runbook link, key dashboards, top slow traces, and sample trace_id.

Operational notes:- Keep metric cardinality bounded (avoid per-request labels); use aggregation and tagging (service, route, tenant-tier).- Regularly review sampling and histogram buckets; automate on-call playbooks with runbooks attached to alerts.- Post-incident: attach traces/logs/metrics to postmortem and tune alerts to reduce noise.

MediumTechnical

81 practiced

Explain the difference between throttling (smoothing) and strict rate-limiting. Give concrete examples where throttling is preferable for user experience, and outline how you would implement smoothing at the API gateway level (e.g., token bucket / leaky bucket).

Sample Answer

Throttling (smoothing) vs strict rate-limiting — difference- Strict rate-limiting: enforces a hard cap (e.g., 100 req/min). Once exceeded, requests are rejected (HTTP 429) immediately. Good for protecting backend capacity and enforcing quotas.- Throttling/smoothing: evens out bursts by delaying or pacing requests rather than outright rejecting them. It allows short bursts but enforces a long-term average rate, improving perceived reliability and UX.

When smoothing is preferable (concrete UX examples)- Interactive UIs: typing-autocomplete that emits bursts — smoothing avoids frequent 429s and keeps suggestions flowing with slight delay.- Bulk uploads or imports from clients that send clustered requests — smoothing avoids failing many items and lets the queue drain.- Real-time feeds: allow short spikes so users don’t see drops in data during normal bursty behavior.In these cases smoothing reduces error exposure, improves perceived latency, and reduces client retry storms.

Implementing smoothing at API gateway (token bucket / leaky bucket)1. Choose algorithm: - Token bucket: tokens accrue at rate r; a request consumes token(s). Allows bursts up to bucket capacity (burst_size). - Leaky bucket: incoming requests queue and are processed at fixed rate; excess over queue capacity are dropped.2. Parameters to set: - rate r (tokens/sec), burst size B, per-key scope (client IP, API key, user id), queue size for leaky bucket.3. Architecture: - Enforce at gateway or edge (NGINX, Envoy filter, API GW plugin). For multi-instance gateways, use consistent hashing for sticky keys or a centralized store (Redis with atomic INCR/EXPIRE or Lua scripts) for global quotas.4. Implementation sketch (token bucket semantics): - For key K, store {tokens, last_ts} in Redis. - On request: - now = current_ts - tokens += (now - last_ts) * r; tokens = min(tokens, B) - if tokens >= 1: tokens -= 1; allow and update last_ts/tokens atomically - else: either enqueue/delay (smoothing) or return 429 - Use Redis Lua script to make it atomic and low-latency.5. Smoothing behavior: - Instead of immediately 429 when tokens=0, return HTTP 202 with Retry-After or place request in short FIFO queue with timeout, or use HTTP/2 server push with backpressure signals.6. Observability and safety: - Emit metrics: allowed, delayed, dropped, current_tokens, queue_length. - Alerts when drop rate > threshold. - Implement client-facing signals (Retry-After, X-RateLimit-Remaining, estimated delay) so clients can adapt.Trade-offs and considerations- Token bucket + Redis gives burst tolerance and global coordination but adds latency and central dependency—use local bucket with occasional sync for best performance.- Leaky bucket provides steady throughput but can increase end-to-end latency during sustained bursts.- Always test with realistic traffic and tune r and B to meet SLOs and backend capacity.

MediumSystem Design

72 practiced

Design a caching tier for an API serving user profile objects: scale 1M users, 100k RPS reads, and writes are 1% of reads. Requirements: read-heavy, strong read consistency within 1 second after an update, fail-open if cache is down. Describe architecture (cache layer placement), invalidation strategy, TTLs, and monitoring/alerts.

Sample Answer

Requirements clarification:- 1M user profiles, 100k RPS reads, writes = 1% of reads (~1k RPS writes).- Read-heavy, must reflect updates within 1 second, and degrade to DB reads (fail-open) if cache is unavailable.

High-level architecture:- Client -> API layer (stateless) -> Cache tier (Redis Cluster, sharded) -> Primary DB (authoritative)- Redis cluster: multiple master shards with replicas for HA. Use cluster mode to scale to throughput (~100k RPS). Place cache logically between API and DB (API talks to cache first).- Optional CDN/edge cache for public profile fields (reduces RPS) if applicable.

Cache pattern & strong-consistency:- Use cache-aside for reads, but for writes use synchronous cache-update pattern to meet 1s consistency: - Write flow: API writes to DB (primary), then atomically update the cache entry (SET with new value and updated TTL) before returning success. If cache update fails, still return success but publish an invalidate job to ensure eventual consistency (fail-open). - Use Redis transactions (MULTI/EXEC) or Lua script to ensure atomic update if multi-key operations needed.- For systems that cannot synchronously update cache on every write (latency concerns), use write-behind combined with a strong invalidation: write DB, publish invalidation message on Redis PUB/SUB or Kafka topic, consumers invalidate local caches; fall back so reads after update will miss cache and read DB then repopulate—this must guarantee propagation <1s (use low-latency messaging and TTLs to bound window).

TTL strategy:- Default TTL: 5–15 minutes for profile objects to limit staleness and memory. However, for entries updated recently, refresh TTL on write (or set a longer TTL after explicit write).- For hot accounts, consider no TTL (or very long TTL) and rely on explicit invalidation on writes.- For fields with stricter freshness (last-active), use shorter TTLs (≤1s) or store separately and always read from DB.

Fail-open behavior:- API cache client treats Redis errors/timeouts as transparent: on cache miss or cache error, read from DB and serve. Track these events in metrics/alerts.- Circuit-breaker: if cache error rate > threshold, temporarily bypass cache to protect latency and DB overload.

Capacity & perf notes:- Estimate avg profile size (e.g., 2–4KB). 1M profiles = 2–4GB raw; with overhead and replicas plan ~3–8 nodes per shard set. Provision Redis nodes for peak RPS and replication.- Use client-side connection pooling and pipelining/batching for bursts.

Monitoring & Alerts:Metrics to collect:- Cache hit rate, miss rate, request QPS, average latency (p99/p50)- Write/update success rate to cache, cache error rate, timeouts- Evictions/sec, memory usage per node, fragmentation ratio, CPU- Replication lag and replica health, cluster slot migrations- DB QPS when cache bypass active

Suggested alerts (examples):- Hit rate < 85% sustained -> investigate TTLs/evictions/CDN usefulness- Cache error rate > 1% or >100 errors/min -> page on-call- Evictions/sec > threshold -> warn and scale or increase memory- Memory usage > 80% per node -> autoscale/add shard- Redis replication lag > 100ms or replica down -> page- Cache bypass rate > 5% of reads -> page (indicates large outage)- DB QPS spike > baseline*1.5 when cache bypass occurs -> page

Operational practices:- Run redis in multiple AZs, enable persistence (RDB/AOF) per RPO needs.- Chaos exercises (simulate cache failure) to validate fail-open and DB capacity.- Use canary deploys for cache client changes; versioned keys if schema changes.- Instrument end-to-end latency and SLOs (e.g., 95% reads < 50ms). Maintain runbooks for cache incidents.

Trade-offs:- Synchronous cache-update gives strong freshness but adds write latency and complexity.- Pub/sub invalidation reduces write latency but requires strict propagation guarantees to meet 1s window.

This design balances read throughput with strong near-real-time consistency, safe fail-open behavior, and actionable monitoring for SRE operations.

MediumTechnical

78 practiced

Using Redis and Lua, implement a sliding-window rate limiter for multiple API servers that enforces 200 requests per minute per user. Describe the Redis keys, the Lua script operations (add timestamp, remove old, get count), and how this remains atomic across concurrent checkers.

Sample Answer

Approach: use a per-user sorted set (ZSET) where members are request timestamps (ms). Each check is a single Redis EVAL of a Lua script that: adds current timestamp, removes timestamps older than window (60s), counts remaining members, decides allow/deny, sets a TTL on the key to auto-clean. Using a Lua script ensures the add/remove/count/ttl operations are atomic across concurrent API servers.

Redis keys:- Key: rate:{user_id} — ZSET of timestamps (score = timestamp, member = unique id like timestamp:uuid)- TTL on key: 61 seconds to auto-expire when idle

Lua script (arguments explained in comments):

lua

-- ARGV[1] = now_ms
-- ARGV[2] = window_ms (60000)
-- ARGV[3] = limit (200)
-- KEYS[1] = rate:{user_id}

local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
local zkey = KEYS[1]

-- remove old entries (score <= now - window)
local min_score = 0
local max_score = now - window
redis.call('ZREMRANGEBYSCORE', zkey, min_score, max_score)

-- add this request with a unique member
local member = tostring(now) .. "-" .. redis.sha1hex(tostring(now) .. math.random())
redis.call('ZADD', zkey, now, member)

-- get current count
local count = redis.call('ZCARD', zkey)

-- ensure key expires shortly after window to free memory
redis.call('PEXPIRE', zkey, window + 1000)

if count <= limit then
  return {1, count} -- allowed, current count
else
  return {0, count} -- denied, current count
end

Key points / reasoning:- ZSET supports range deletions by score and cardinality checks efficiently.- All steps run inside Redis atomically via EVAL => no race between concurrent servers.- Use millisecond timestamps and unique members to avoid collisions.- TTL prevents stale keys from accumulating.

Complexity:- ZREMRANGEBYSCORE: O(log N + M) where M removed; ZADD/ZCARD: O(log N) / O(1) amortized. For limit 200, N bounded ~200 so operations are effectively constant.

Edge cases:- Clock skew: prefer monotonic server clocks or let API servers use Redis TIME for now_ms (call Redis TIME first).- Bursts: sliding window enforces exact rolling rate; if bursts across windows are desired, consider token-bucket.- Member explosion if callers flood; TTL and removals mitigate.

Alternatives:- Token bucket via Redis INCR with refill scheduled or Lua; approximate leaky-bucket implementations; Redis built-in RedisBloom or third-party modules for high-scale.

HardTechnical

70 practiced

A library upgrade introduces a subtle p99 latency regression in your API. As an SRE, outline a detailed investigation plan: how to use distributed traces, flamegraphs, CPU/GC/heap profiling, dependency latencies, and deployment artifacts to isolate the cause. Describe immediate mitigations and long-term fixes.

Sample Answer

Situation & goal: p99 latency regressed after a library upgrade. Objective: isolate root cause quickly (is it CPU, GC, blocking I/O, serialization, or client-side retry behavior), mitigate to meet SLOs, then implement long-term fixes.

Investigation plan (stepwise):

1. Confirm and scope- Verify alerting graphs (p99, p95, p50, error rate, QPS) and timeframe of regression relative to deploy.- Identify impacted services, endpoints, and versions via deployment tags.

2. Distributed traces- Filter traces for slow p99 traces during window. Compare trace histograms before/after upgrade.- Inspect span durations and hang points: look for increased service processing time vs downstream/wait time, and increased span counts (retries/loops).- Use sampling to capture full traces for impacted endpoints.

3. Flamegraphs & CPU profiling- Capture CPU profiles (e.g., perf/async-profiler, pprof) on service instances serving slow requests.- Generate flamegraphs and diff with pre-upgrade baseline; look for new hot methods (library symbols) or increased lock contention.

4. GC and heap profiling- Collect GC metrics (pause times, GC frequency, allocation rate) and heap/allocator profiles.- Heap dumps and allocation flamegraphs to find increased allocations or finalizer pressure introduced by the library.

5. Dependency latencies- Measure downstream call latencies (DB, caches, third-party APIs). Correlate trace spans to see if regression is upstream or due to slower dependencies.- Validate network metrics (socket counts, retries, TCP retransmits).

6. Deployment artifacts and config- Compare build artifacts, dependency versions, compile flags, JVM/GC flags, container images, and environment variables between versions.- Check for shading/packaging changes, additional instrumentation, or changed serializers.

7. Reproduce in controlled environment- Roll forward test with canary or synthetic load (loadtest to replicate p99).- Use binary search: deploy previous version vs upgraded lib in A/B to isolate.

Immediate mitigations- Rollback the library upgrade for affected service(s) or route traffic away (kill switch, feature flag, canary rollback).- If rollback impossible, apply rate-limiting, degrade noncritical features, increase concurrency limits cautiously, or scale out instances to absorb p99 temporarily.- Tune GC or thread-pool sizes if profiling shows clear GC or thread starvation issues and it's low-risk.

Long-term fixes- Fix root cause: patch or replace offending library, contribute upstream fix, or apply workaround (e.g., reuse buffers, avoid sync APIs).- Add regression tests: microbenchmarks and end-to-end p99 performance tests as part of CI.- Add better observability: histogrammed latencies, allocation/GC dashboards, and distributed tracing with automatic p99 trace capture.- Improve deployment safety: stricter canarying, automatic rollback policies, dependency compatibility checks, and pre-merge performance gates.

Example commands/tools- Traces: Jaeger/Tempo + traceQL filters by latency- CPU: async-profiler → flamegraph.svg- JVM GC: jstat/jcmd/GC logs; heap: jmap + Eclipse MAT- Native: perf/top/pprof- Diffing: compare flamegraphs and trace histograms

Outcome: short-term SLO recovery via rollback/mitigation; long-term prevention through fixes, tests, and improved observability.

Unlock Full Question Bank

Get access to hundreds of API and Full Stack Coding Patterns interview questions and detailed answers.

Join thousands of developers preparing for their dream job.