Backend Architecture and Service Design Questions

Focuses on designing backend systems and services with clear responsibilities, interfaces, and manageable dependencies. Topics include service oriented architecture and microservices trade offs, service boundaries and domain driven design, data ownership and consistency models, idempotency and error handling, application programming interface design and versioning, separation of concerns and modular code organization, testing strategies, deployment and release patterns, scalability and fault tolerance patterns, observability and monitoring, operational concerns including capacity planning and dependency management, and maintainability practices to enable teams to extend systems without constant refactoring. Senior level discussion includes designing for evolution, team ownership models, and balancing complexity versus simplicity across the service landscape.

HardTechnical

62 practiced

Design a consistent cache invalidation strategy for user profile data served from caches across multiple regions. Discuss use of TTLs, versioned cache keys, explicit invalidation events, cross-region event propagation, and how to avoid cache stampedes and stale reads without heavy locking.

Sample Answer

Requirements & goals:- Keep user profile reads fast globally, minimize stale reads, ensure eventual consistency across regions, avoid heavy locking or global coordination, and resist cache stampedes.

High-level strategy:1) Cache model: read-through caches in each region (local CDN/edge + regional Redis/Memory). Authoritative source = central user-profile service + durable store (DB).

2) Versioned keys + TTLs:- Store cache entries as key: user:{userId}:v{version}. Index latest version in a small pointer key user:{userId}:latest -> v{N}.- Default TTL short-to-medium (e.g., 5–60 mins) depending on SLA; use per-key TTLs. Versioning lets explicit invalidation create a new version without immediately evicting stale keys.

3) Explicit invalidation events & cross-region propagation:- When profile changes, profile service increments version (or writes new version) and emits an invalidation event to a global, reliable message bus (e.g., Kafka, SNS+SQS, or a geo-replicated CDC stream).- Each region subscribes and updates its local pointer user:{id}:latest -> v{N} and optionally evicts old version key asynchronously.- Use at-least-once semantics; idempotent updates and dedupe by version number.

4) Avoiding stampedes & stale reads:- Stagger TTLs and add jitter to expiry to avoid synchronized expirations.- Implement stale-while-revalidate: on cache miss or if entry expired-but-allowed-stale, return stale data immediately and trigger background refresh (singleflight/coalescing per key to prevent N concurrent refreshes). Use a lightweight in-process or distributed singleflight (e.g., Redis SETNX lock with short TTL) to let one worker refresh.- Probabilistic early refresh: on read, if remaining TTL < threshold compute probability p to trigger async refresh to smooth load.- Conditional revalidation: cache stores ETag/last-modified; refresh uses conditional GET to authoritative service to reduce payloads.

5) Handling partition and eventual consistency:- Readers using latest pointer may point to new version before regional caches have populated it; if the v{N} key is missing, do read-through to authoritative service and populate local cache. This avoids stale reads without global locking.- For strong-currency operations (sensitive updates), bypass cache or perform read-after-write by writing both DB and local cache + emitting invalidation; client can be routed to primary region.

Trade-offs and rationale:- Versioned keys + latest pointer allow low-latency invalidation without global synchronous eviction.- Event propagation gives near-real-time invalidations; eventual consistency window = propagation latency + refresh time.- Singleflight + stale-while-revalidate balances freshness and availability, avoiding blocking requests.- Extra storage for versioned entries and pointer adds complexity but simplifies correctness and rollback.

Operational considerations:- Monitor propagation lag, cache hit ratio, refresh rate, and lock contention.- Backpressure: throttle refreshes under overload.- Security: sign invalidation events; validate version monotonicity.- Test failure modes (message bus outage, region isolation) and have fallback: TTL ensures eventual expiry.

This approach provides low-latency reads, allows explicit fast invalidation, limits stale reads via version pointers and read-through, and prevents stampedes via coalescing, jitter, and probabilistic refresh—without heavy global locking.

HardSystem Design

78 practiced

Design an end-to-end backpressure strategy for an architecture where async microservices communicate using durable queues and some synchronous HTTP calls. Explain how you would signal producers, manage consumer queues, prevent message loss, and maintain latency SLAs under load.

Sample Answer

Requirements & constraints:- Durable async message bus + some synchronous HTTP paths.- Preserve ordered delivery per key where required.- Latency SLA (e.g., p95 < 200ms for sync paths), availability and no message loss.- Variable load spikes, multi-tenant.

High-level approach:- Combine multi-layer backpressure: proactive signaling to producers, queue-rate shaping, consumer-side admission control, dynamic throttling and shed/priority policies; persistent storage for durability and replay.

Components and responsibilities:1. Ingress Gateways (producers): enforce client quotas, token-bucket local rate limiters, and accept “backpressure hint” from broker responses.2. Durable Broker/Queue (Kafka/RabbitMQ/SQS+DLQ): per-topic/tenant partitions; expose metrics (queue depth, consumer lag, oldest message age).3. Consumer Pool: adaptive concurrency controller that measures processing latency and inflight work; performs worker scaling and per-partition fetch rate control.4. Control Plane: central rate orchestrator that collects broker + consumer metrics and computes target publish rates, priority windows and circuit states. Exposes signals via HTTP 429-like responses, Retry-After headers, or push via producer SDK.

Signaling producers:- Immediate: broker responds with 429/503 + Retry-After or throttling header when acceptance would violate SLAs.- Soft: publish accepted but tagged with lower priority / delayed delivery (time-to-process).- Push: control-plane can notify long-lived producers via Webhook/stream to reduce rate or pause.- Token-bucket across client/tenant + global lease from control-plane ensures fair-share.

Managing consumer queues:- Adaptive fetching: consumers adjust fetch sizes and concurrency based on p95 processing time and queue oldest age.- Priority queues: honor high-priority messages; low priority can be rerouted to slower lanes or delayed.- Partitioned consumption for ordering; enable pause/resume per partition.- Autoscale consumers based on lag and CPU, bounded to avoid overloading downstream sync calls.

Preventing message loss:- Durable persistence with replication, write-ahead logs.- At-least-once processing with idempotent consumer handlers; where necessary implement exactly-once semantics via transactional writes (Kafka transactions) or deduplication keys.- Dead-letter queue (DLQ) with alerting and replay tools.- Graceful shutdown & relocation protocols: consumers commit offsets only after durable sink ack.

Maintaining latency SLAs:- Reserve a fast-path: synchronous HTTP flows bypass heavy queues when capacity available; control-plane preserves headroom by throttling low-priority async producers preemptively.- Admission control: when queue oldest age or consumer p95 approaches threshold, reduce inbound rates to protect SLA.- Circuit-breakers on consumer to stop pulling when downstream sync latency degrades.- Backpressure windows: maintain target queue depth thresholds (soft/hard). Soft triggers autoscale; hard triggers reject/persist-to-bulk-store and notify producers.

Operational & metrics:- Key metrics: queue depth, consumer lag, oldest message age, p50/p95/p99 processing latency, CPU/memory, error rates, publish 429 rate.- Automation: control-plane runbook for scaling, throttling, tenant isolation.- Testing: chaos injection (broker failover, consumer slowdowns), load tests validating SLA boundaries.

Trade-offs:- Strong throttling protects SLA but increases producer latency and potential rejections.- Large autoscaling reduces rejections but increases cost.- Exactly-once adds complexity; prefer idempotency + at-least-once unless business requires otherwise.

This design preserves durability, gives multiple ways to signal producers, adapts consumption under load, and enforces latency SLAs via preemptive admission control, priority lanes, and operational automation.

HardTechnical

69 practiced

Design a distributed rate-limiting system that enforces per-user and per-organization quotas across multiple regions with low latency. Compare token-bucket vs leaky-bucket, centralized counters vs local caches with periodic sync, and propose a hybrid that balances accuracy, latency, and fault tolerance.

Sample Answer

Requirements & constraints:- Per-user and per-organization quotas, enforced globally across regions- Low tail latency for authorizing requests (<~10ms preferred)- Multi-region availability, partition tolerance, eventual consistency acceptable for some uses, but strong-enough accuracy to avoid large quota breaches- Fault-tolerance and graceful degradation when control plane is unreachable

High-level architecture:- Regional API gateways (edge) enforce local fast-path checks- Global control plane (clustered service + durable store) holds authoritative quotas and policy- Distributed in-memory caches (per-region) with local token buckets and background sync to control plane- Optional coordination via lightweight consensus (e.g., Redis Streams or gossip) for burst credits

Token-bucket vs Leaky-bucket (comparison):- Token-bucket: allows bursts up to bucket capacity; replenishes tokens at rate R. Good for smoothing and permitting bursts, simple to implement locally.- Leaky-bucket: enforces constant outflow rate; better for uniform traffic shaping but more restrictive for client bursts.- For quotas that must allow occasional bursts (user-driven), token-bucket fits best; leaky-bucket suits steady throughput guarantees.

Centralized counters vs Local caches with periodic sync:- Centralized counters (strong consistency): accurate, simple to reason about, but increases latency and is a single operational hotspot; requires partition-tolerant design (quorum) and will reject/slow requests if network to COU is unstable.- Local caches with periodic sync: low-latency decisions, high availability, but risk of overshoot between syncs.

Proposed hybrid design (accuracy + latency + fault tolerance):1. Fast-path: regional token-bucket in local cache. Each region provisions a delegated token allowance for each (org, user) — e.g., allocate N tokens per minute from control plane. Gateways decrement locally; decision is sub-ms.2. Control plane: authoritative counters track global usage. Periodically (or on threshold), regions debit their usage and request top-up via an atomic operation (CAS or a small transactional write).3. Borrow & reconciliation: when local tokens exhausted, gateway can: - Try synchronous refill from control plane (higher latency) - Or allow limited borrowing (small emergency buffer) up to configurable overshoot with auditing4. Sync strategies: - Push-based: region reports usage every T seconds or when X tokens consumed - Event-driven: report when remaining local tokens < threshold - Use idempotent, append-only logs to reconcile in face of retries5. Safety: control plane enforces strict global ceilings by refusing top-ups once quota exhausted; reconciler subtracts overshoot and enforces penalties or throttles.6. Fault tolerance & degradation: - If control plane unreachable, continue using local allowance and reduce refill frequency; once exhausted, apply conservative fallback: deny requests or degrade to leaky-bucket local behavior (rate-limited, no bursts). - Use exponential backoff and circuit-breaker for control plane calls.

Trade-offs & tuning:- Larger delegated allowances => fewer remote calls, lower latency, more potential overshoot.- Smaller allowances => higher accuracy, more calls and latency spikes.- Choose delegation window based on SLA (e.g., 1–10s for strict quotas, 60s+ for lenient quotas).- Use metrics: overshoot rate, refill latency, rejected refill count to tune thresholds.

Implementation notes:- Persist authoritative state in strongly-consistent datastore (CockroachDB, Spanner) or leader-backed Redis with persistence.- For cross-region coordination use CRDT counters or bounded counters to simplify reconciliation.- Audit logs and monitoring to detect abuse and tune borrow policies.

This hybrid balances low-latency enforcement at the edge, global accuracy via a central control plane, and robustness through bounded local autonomy and reconciliation.

HardTechnical

64 practiced

Design a schema/versioning strategy for evolving message and API schemas across a large microservice ecosystem to minimize coordination friction. Discuss backward and forward compatibility, use of schemas (JSON Schema, Protobuf, Avro), compatibility testing, governance, and migration techniques like consumer-driven evolution and feature flags.

Sample Answer

High-level approach: use schema-first contracts (Protobuf/Avro for binary/streaming; JSON Schema or OpenAPI for HTTP) with semantic compatibility rules, automated compatibility tests, lightweight governance, and migration patterns driven by consumers and feature flags to avoid team-to-team coordination friction.

Key principles- Prefer additive changes: add optional fields, new messages/types; avoid removing/renaming fields.- Define backward-compatible (new consumers can read old data) vs forward-compatible (old consumers can read new data) rules per transport and schema format.- Use schema registries (Confluent Schema Registry / Apicurio) for Avro/Protobuf with compatibility modes (BACKWARD/ FORWARD/ FULL).

Schema choices & rules- Protobuf/Avro for high-throughput, strongly-typed messaging. Enforce numeric tag stability (Protobuf) and field defaults (Avro).- JSON Schema/OpenAPI for REST: use versionless endpoints where possible and make schema extensions optional; avoid breaking path/response contract changes.- Document compatibility matrix per format.

Compatibility testing & CI- Gate all PRs with contract tests: consumer-driven contract tests (PACT) and registry compatibility checks.- Run schema evolution tests: validate new schema against registered previous versions under chosen compatibility mode.- Include end-to-end staging with representative consumers.

Governance & discovery- Lightweight governance board (one architect per domain) for exceptions and major schema changes.- Catalog + registry: searchable schemas, owners, change logs, migration guides, deprecation timelines (e.g., 90/180 days).- Enforce ownership and contact metadata in registry.

Migration techniques- Consumer-driven evolution: let consumers pull new optional fields; producers emit both old and new fields when necessary.- Use feature flags and dual-write or translation services to progressively shift traffic.- Facade/adapter services to translate between versions when immediate consumer changes aren’t possible.- Deprecation process: announce, monitor usage, retire after SLA and automated verification.

Trade-offs- Strict schema enforcement reduces friction long-term but increases short-term velocity; balance via per-domain risk levels.- Binary schemas add speed but require tooling support; JSON is flexible but riskier for silent breakages.

Outcome- Reduced cross-team coordination by automating compatibility checks, providing clear rules, and enabling gradual migrations via consumer-driven patterns and feature flags.

EasyTechnical

61 practiced

Design an idempotent create-order HTTP API for a high-throughput ordering system where clients may retry requests. Explain how you would use an Idempotency-Key header,where and how to store keys, expiration strategy, and how to handle partial failures and concurrent retries.

Sample Answer

Situation: Clients will retry create-order calls; we must ensure a retry doesn’t create duplicate orders while supporting very high throughput.

Design summary:- Client includes an Idempotency-Key header (UUID or client-generated unique token) on every create-order request. Keys should be unique per logical request and scoped by client (e.g., Idempotency-Key + Client-ID).- API behavior: on request arrival, check store for (Client-ID, Idempotency-Key).

Storage and schema:- Use a fast key-value store (Redis with AOF/RDB or Cassandra for multi-region durability) to map (Client-ID, Idempotency-Key) → {status, request-hash, order-id, response-body, created-at, last-updated}.- Status values: IN_PROGRESS, COMPLETED, FAILED.- Persist final successful result to primary DB (orders table) and copy summary to idempotency store for quick responses.

Concurrency and race prevention:- Implement atomic insert-if-not-exists (SET NX in Redis or lightweight conditional write in Cassandra) when saving a new key. If insert succeeds, mark IN_PROGRESS and proceed to create order. If it exists: - If COMPLETED: return stored response (200/201) with stored order-id. - If IN_PROGRESS: return 202 Accepted with Retry-After or wait with short backoff/polling to return final result. - If FAILED: allow client to retry (optionally increment retry count).

Partial failures and retries:- Use two-step commit internally: create order in DB with idempotency metadata or use unique constraint on orders (order_idempotency_key) to prevent duplicates. If external downstream calls fail after order creation, record compensation actions and mark FINAL state accordingly.- If process crashes while IN_PROGRESS, background worker scans keys older than threshold and either retry processing or mark FAILED for client to retry.

Expiration & retention:- Keep idempotency entries for a configurable TTL (e.g., 24–72 hours) to cover client retry windows; retain COMLETED entries longer if business requires (e.g., 30 days) for audit/dispute resolution.- Use background TTL eviction and periodic compaction to remove stale IN_PROGRESS entries older than a safety cutoff.

Additional considerations:- Limit idempotency-key size and enforce collision-resistant tokens.- Log metrics: collision rate, stale IN_PROGRESS count, retries.- For multi-region, replicate idempotency store or use causal consistency to avoid split-brain duplicates.This approach provides atomic deduplication, fast lookups for retries, clear handling of partial failures, and operational controls for high throughput.

Unlock Full Question Bank

Get access to hundreds of Backend Architecture and Service Design interview questions and detailed answers.

Join thousands of developers preparing for their dream job.