Scalable System Architecture and Design Principles Questions

Architectural patterns principles and decision making for building systems that are maintainable resilient and able to scale. Coverage includes service decomposition and trade offs between microservice architectures and monoliths; layered and n tier architecture patterns; event driven design and command query responsibility segregation pattern; choosing synchronous versus asynchronous communication and its impact on correctness and latency; design principles such as loose coupling high cohesion separation of concerns and single responsibility; state management and session handling and when to favor stateless designs; application programming interface design versioning and contract management; front end and user experience considerations such as resource loading and progressive rendering; migration strategies for evolving systems and incremental refactoring; and how to weigh latency throughput reliability cost and development velocity when selecting architectures. Candidates should illustrate pattern selection with concrete examples and justify operational and developer experience implications.

HardSystem Design

65 practiced

Design an inventory service for an e-commerce platform with multiple warehouses that prevents oversell while maximizing throughput. Discuss reservation strategies, pessimistic locking vs optimistic reservation, distributed reservation caches, heartbeat/lease mechanisms for long checkouts, and how to handle failures that must release reserved stock.

Sample Answer

Requirements & constraints:- Prevent oversell across multiple warehouses, support high throughput (thousands TPS), low latency for checkout, eventual consistency for inventory sync, support long-lived checkouts (minutes), tolerate node failures and network partitions.

High-level architecture:- Inventory Service (central authoritative store per SKU-shard) + Reservation Service + Distributed Reservation Cache (hot items) + Warehouse adapters.- Use consistent hashing to shard SKUs; each shard owns authoritative stock counts and persistent ledger of reservations (append-only events).

Reservation strategies:- Optimistic reservation (fast path): decrement available_count in a distributed cache (e.g., Redis with Lua atomic script) and create a tentative reservation token; asynchronously persist to authoritative shard. Good for high throughput and low-latency checkout.- Pessimistic locking (slow path): for low-stock/high-value SKUs, acquire a short-lived lock on the shard (or use DB row lock/compare-and-swap) before confirming—prevents contention/oversell when stock is scarce.

Pessimistic vs Optimistic trade-offs:- Optimistic: higher throughput, lower latency, requires robust reconciliation to fix races; risk of temporary over-allocation if async persist fails.- Pessimistic: strong correctness, higher latency and lower throughput due to blocking/locks; use selectively for hot contention SKUs.

Distributed reservation cache:- Cache stores available_count and reserved_count per SKU with atomic operations via Lua scripts (decr_if_ge). Cache is a performance layer, not source of truth. On cache miss or reconciliation need, query authoritative shard.- Use write-through or async-backfill with idempotent reservation commands to shard.

Heartbeat / lease for long checkouts:- Reservations issued with TTL/lease (e.g., 15 min). Client must heartbeat (renew) to extend lease; renewals update reservation expiry in shard and cache.- If checkout completes, convert reservation -> committed sale and decrement physical stock ledger.- Implement max total lease to prevent indefinite holds.

Failure handling & releasing reserved stock:- On heartbeat expiry, background reaper process on shard marks reservation as released and increments available_count atomically; publish release events for cache invalidation and potential rerouting to waiting requests.- On service crash/network partition: remaining reservations are still persisted in authoritative shard. If cache lost, rebuild from ledger events.- For async persist failures (cache accepted but shard persist failed): retry with exponential backoff; use idempotent reservation IDs so retries are safe. If persist ultimately fails, background reconciliation detects "orphaned" cache reservations by comparing cache vs shard and releases cache-held stock.- For partial commit (payment success not received): use two-phase-ish pattern: reserve -> payment -> commit. If payment gateway times out, allow manual inspection and auto-release after TTL.

Consistency & reconciliation:- Periodic ledger replay and reconciliation job to ensure cache and shards align; cross-warehouse aggregate uses CRDT-style merge of committed sales per warehouse.- Provide monitoring/alerts on reservation churn, TTL expiries, cache-shard divergences.

Scalability & optimizations:- Prioritize optimistic path for majority of traffic; route scarce SKUs to pessimistic path.- Use batching for ledger writes, idempotent writes, and backpressure.- Local per-warehouse caches for warehouse-local fulfillment to reduce cross-datacenter latency; authoritative decisions still at SKU shard.

Key takeaways:- Combine optimistic reservations + authoritative shard ledger + TTL leases + heartbeat renewals to maximize throughput while preventing oversell.- Use pessimistic locking selectively for high-contention items.- Rely on idempotent operations, background reapers, and periodic reconciliation to handle failures safely.

EasyTechnical

63 practiced

Design session management for a web application expected to serve 10M monthly active users across multiple regions. Compare stateless approaches (JWTs/signed tokens) versus stateful server-side sessions. Discuss tradeoffs for security, revocation, session size, logout and immediate invalidation, per-request latency, and how you would implement each approach at scale.

Sample Answer

Requirements (clarify): 10M MAU, multi-region, low latency (<100ms), strong security (ability to revoke sessions immediately), and operational simplicity.

High-level comparison:- Stateless (JWT / signed tokens) - Security: Token integrity is verifiable without DB reads (signing + short TTL). Vulnerable if private key leaked; use rotating keys and short expiry. - Revocation / logout: Hard — JWTs valid until expiry. Implement revocation list (blacklist) or use short-lived access tokens + refresh tokens; revoke refresh tokens server-side. - Session size: Safe to embed small claims (user id, roles, exp). Keep token minimal to avoid large cookies and network overhead. - Immediate invalidation: Achieved only with server-side revocation store (adds state + lookup). - Per-request latency: Very low — local signature verification; no network hop unless checking revocation. - At scale: Use asymmetric signing (RS256) for regional verification; distribute public keys via JWKS; rotate keys and propagate quickly via CDN/keystore.

- Stateful (server-side sessions) - Security: Sensitive data stays server-side; session id in cookie references server store. Easier to invalidate and audit. - Revocation / logout: Immediate — delete session from store. - Session size: Can store arbitrary data; keep small to reduce storage and serialization overhead. - Immediate invalidation: Native. - Per-request latency: Requires session lookup. To keep latency low, use in-memory caches (regional Redis clusters) with read replicas and local caching. - At scale: Use a consistent session store per region (Redis cluster with replication). Route users to nearest region (geo-DNS or global LB) or use sticky sessions at the edge + session replication. For global failover, persist sessions to a central DB or use cross-region replication with CRDTs or active-active Redis (with trade-offs).

Implementation recommendations:- Prefer hybrid: short-lived JWT access tokens for low-latency auth + refresh tokens stored server-side (stateful) to allow revocation. This gives stateless speed for normal requests and stateful control for long-lived sessions.- Security best practices: HTTPS-only cookies, SameSite, HttpOnly, rotate keys, monitor for token replay, rate-limit refresh endpoints, and log session events.- Multi-region ops: Deploy verification keys and token validation logic in each region; replicate revocation/refresh-store with eventual consistency but design to fail-safe (treat unknown refresh as invalid) for security.- Metrics & tooling: Track token issuance, revocations, refresh frequency, latency of session lookups; run drills for key rotation and cross-region failover.

Trade-offs summary: Pure JWT = simple, low-latency, harder revocation. Pure stateful = immediate control, higher per-request cost and operational complexity. Hybrid yields best practical balance for 10M MAU.

HardTechnical

66 practiced

For an online payments system that requires strong correctness and low latency, compare architectural options: synchronous replication vs consensus protocols (Raft/Paxos), leader-based writes, and asynchronous replication. Evaluate their impact on write latency, throughput, availability, and operational complexity and explain which you'd choose under different SLA/availability constraints.

Sample Answer

Situation: We need an online payments system where correctness (no lost/duplicated money), low write latency, and high availability are primary concerns. Below I compare three architectural options and recommend choices depending on SLA/availability constraints.

Definitions / quick trade-offs- Synchronous replication: Primary writes synchronously copied to replicas before ack. Guarantees durability/consistency at commit time; latency includes network round-trips to replicas.- Consensus protocols (Raft/Paxos): A coordinated protocol across nodes that elects a leader and replicates logs with strong consistency and safety; handles leader changes and guarantees linearizability when configured correctly.- Asynchronous replication: Primary acknowledges writes immediately; replicas catch up later. Low write latency but risks data loss on primary failure.

Impact summary (write latency / throughput / availability / operational complexity)- Synchronous replication: - Write latency: Higher — must wait for n acknowledgements (network RTT × required replicas). - Throughput: Moderate; limited by slowest replica and sync bottlenecks. - Availability: Dependent on replica quorum policy; can become unavailable if required replicas are unreachable. - Ops complexity: Moderate — simpler than full consensus but needs health checks, failover scripts, quorum config.- Consensus (Raft/Paxos, leader-based writes are typical in Raft): - Write latency: Comparable to synchronous replication when configured to require quorum ACKs (leader → majority). Optimizable via leader co-location and batching. - Throughput: High with append-only log batching; concurrency limited by single leader for writes. - Availability: Strong — handles leader failover, maintains safety, survives minority failures; availability depends on majority being up. - Ops complexity: Higher — must manage election tuning, membership changes, and complex debugging; but mature libraries/tooling exist.- Asynchronous replication: - Write latency: Lowest — primary acks immediately. - Throughput: Highest for writes on primary; replicas catch up asynchronously. - Availability: Read-after-write consistency on primary only; risk of data loss if primary fails before replication. - Ops complexity: Low-to-moderate; but needs careful recovery procedures and conflict resolution.

Recommendations by SLA / constraints- If correctness (no data loss) is mandatory and you can tolerate modest latency (e.g., ~50–200ms depending on RTTs): Use a consensus protocol like Raft with leader-based writes configured to require majority commit before ack. It gives linearizability, automatic failover, and clear safety properties. Tune for low latency by placing replicas in low-latency regions and using batching.- If sub-10ms write latency is critical (e.g., POS terminals) and occasional small-window data loss is acceptable or mitigated by business compensations: Use synchronous replication to a small set of local replicas (within same region) or hybrid: local async for speed plus cross-region consensus for durability.- If availability and write throughput outweigh strict durability (rare for payments): Asynchronous replication only if paired with additional safeguards — e.g., durable WAL on primary, synchronous cross-checks for high-value transactions, or transaction-level two-phase commit with ledger service.

Operational notes and mitigations- Use leader-based writes (consensus) for global correctness; colocate leader or use partitioning/sharding so each shard’s leader is low-latency to its clients.- For payments, prefer consensus+leader-based writes for core ledger; use async replication for analytics or reporting replicas.- Measure and tune: quorum size, election timeouts, batching, flow-control. Implement monitoring, automated failover tests, and disaster recovery runbooks.- Hybrid pattern: synchronous/consensus within primary region for strict correctness; async cross-region replication for DR and read scaling.

Conclusion: For payments where correctness is non-negotiable, choose consensus (Raft/Paxos) with leader-based writes and majority commit. If extreme latency constraints exist for low-value operations, use local synchronous replicas or hybrid approaches, but keep the canonical ledger on consensus to preserve safety.

MediumSystem Design

56 practiced

Design a multi-region API front-end for a globally distributed service with 10M daily active users. Discuss traffic routing options (DNS, anycast, geo-load balancing), health checks and failover, regional caching, geo-affinity for writes, and how to set consistency expectations for cross-region writes and reads.

Sample Answer

Requirements & constraints:- 10M DAU global, low read latency (<100ms target), high availability (99.95%), cross-region consistency relaxed but predictable.- Must optimize for read-heavy traffic, tolerate region failures, and provide sensible write behavior.

High-level architecture:- Regionally deployed API frontends behind local load balancers + regional app servers + regional data stores (read replicas + optional regional primary).- Global traffic entry via a Global Server Load Balancer (GSLB) offering DNS-based geo-routing or Anycast IPs; combine with cloud provider global LB where available.

Traffic routing options (trade-offs):- DNS (GeoDNS/GSLB): Simple, widespread. Pros: easy geo-steering, health-aware TTLs. Cons: DNS caching delays for failover; TTL tuning required.- Anycast: Single IP advertised from multiple regions; very fast failover and simple client config. Cons: difficult to do fine-grained capacity steering and per-request health control; balancing relies on BGP.- Geo-load balancing (global LB + latency-based routing): Best for per-request steering and health awareness. Use for active capacity-based routing.

Recommended approach:- Primary: use provider Global LB (latency + capacity aware) for most traffic.- Fall back: Anycast for DDoS mitigation + consistent IPs for clients if needed.- DNS TTLs: low-ish (30–60s) for quicker reroute but balance with DNS resolver caching.

Health checks & failover:- Multi-layered health checks: coarse (BGP/anycast reachability), LB-level active HTTP health probes (/healthz) that test app + downstream DB, and passive checks (error rates).- Failover strategy: mark region unhealthy at global LB when probes fail; route traffic away. Use circuit-breakers at client/API layer to avoid cascading.- Graceful draining: LB signals new deployments, connection draining to avoid dropped requests.

Regional caching:- Use CDN for static assets and cacheable API responses at edge (cache-control, surrogate-keys).- Regional read caches (in-memory, e.g., Redis) for hot reads; populate via cache-aside. Use short TTLs and cache invalidation via pub/sub across regions for near-real-time invalidation.- For user-specific state, prefer regional caches pinned to user’s primary region.

Geo-affinity for writes:- Prefer write affinity: route writes to a “home” region for the user (geo-affinity based on user profile or nearest region). This reduces cross-region write conflicts and improves latency.- Implement sticky sessions / session tokens that indicate home-region to ensure subsequent write requests go to the same region.- For global operations (e.g., admin writes), use a designated authoritative region or coordinate via a global sequencer.

Consistency model and expectations:- Default: “Read-your-writes” and strong consistency within-region; eventual consistency cross-region.- Implement synchronous replication within-region; asynchronous cross-region replication with timestamps/vector clocks for conflict detection.- Offer stronger guarantees optionally: for critical writes, provide synchronous cross-region commit (higher latency) or a quorum-based API (client can request WAIT until N replicas ack).- Conflict resolution: application-level CRDTs or last-writer-wins with causal metadata; for complex objects, prefer application merge logic.- Communicate SLA: e.g., 99% reads reflect writes in <500ms cross-region; critical consistency available via explicit endpoints that incur latency.

Operational notes:- Metrics & SLOs: monitor tail latency, error rates, replication lag, DNS health.- Chaos testing: simulate region failures, network partitions.- Security & rate limits: global WAF, per-region throttles.

Summary:Use a global LB for per-request intelligent routing, complemented by Anycast/DNS as needed. Keep reads cached at edge/region, use write geo-affinity for latency and conflict reduction, and expose clear consistency options (fast eventual by default, stronger on demand) with observable replication lag and failover behavior.

HardTechnical

67 practiced

Design a cross-team API contract management strategy that includes consumer-driven contract tests, a schema registry or specification store, version gate checks in CI, automated contract verification, and a policy for introducing incompatible changes without blocking independent team deployments.

Sample Answer

Requirements:- Allow multiple teams to evolve APIs independently while preventing consumer breakage- Provide automated, CI-gated verification (consumer-driven where possible)- Centralized schema/spec store for discovery, validation, and governance- Process for introducing incompatible changes without blocking deploys- Low friction for teams; fast feedback loops

High-level architecture:Consumer Test Suites → Contract Broker/Registry ← Provider CI ↑ Governance & Policy Service ↓ Automated Verification Pipeline

Core components and responsibilities:1. Schema/Specification Store (e.g., Schema Registry / OpenAPI store)- Single source of truth for published schemas and historical versions- Metadata: owners, stability level (experimental/stable/deprecated), compatibility rules- Enforce upload validation (format, linting)

2. Consumer-Driven Contract Tests (CDC)- Consumers produce tests (Pact, Spring Cloud Contract) and publish “consumer contracts” to the broker on CI success.- Contracts are lightweight assertions describing expected provider behavior.

3. Provider-Side Contract Verification- Provider CI fetches consumer contracts from broker and runs automated verification.- Fail CI if verification fails for contracts against provider’s current code (subject to policy exceptions—see below).

4. Version Gate Checks in CI- Gate checks when publishing schemas or merging provider changes: - Automatic compatibility check (semantic: backward/forward) based on declared policy. - Linting and security scanning. - If incompatible, require explicit approval and an accompanying mitigation plan.

5. Automated Contract Verification & Canarying- After CI, auto-deploy canaries that run integration tests against a subset of traffic.- Monitor contract violations, error rates, and provide automatic rollbacks.

Policy for incompatible changes without blocking deployments:- Introduce an "Incompatible Change Flag" workflow: - Developer marks PR/Schema change as incompatible and must supply a Migration Plan: consumer impact matrix, migration window, feature toggles, or dual-write/dual-read strategy. - The registry accepts the incompatible version but marks it with a higher risk label and records required approvals. - Provider CI is allowed to merge/deploy to isolated environments and production can be gated by: time-based gradual rollout (feature flag + canary), consumer opt-in endpoints, or a compatibility shim. - Consumers are notified via automated channels (Slack/email/webhooks) listing impacted contracts and deadlines. Consumers can publish updated contracts demonstrating compatibility; providers must run verification. - Governance board/owners must approve if rollback or extended mitigation needed.

Data flow & lifecycle:- Consumer CI publishes contract → Broker validates & stores → Provider CI pulls contracts → Runs verification → Result posted to Broker & Slack → If incompatible and approved, provider deploys behind flag + canary → Observability monitors for regressions → After migration window, deprecated versions are retired.

Scalability & practices:- Automate publishing and verification; keep contracts small and focused.- Use semantic versioning and add "stability" tags to reduce noise.- Provide SDKs and templates for producer/consumer testing to reduce friction.- Track metrics: contract verification success rate, time-to-fix, number of incompatible changes, consumer adoption lag.

Trade-offs:- Strict blocking enforcement reduces runtime incidents but slows teams; the incompatible-change policy balances safety and autonomy.- Central registry adds operational overhead; mitigate by using managed or open-source broker with automation.

This strategy preserves independent deployments while using consumer-driven contracts, registry-backed governance, CI gate checks, and an explicit, auditable path for incompatible changes.

Unlock Full Question Bank

Get access to hundreds of Scalable System Architecture and Design Principles interview questions and detailed answers.

Join thousands of developers preparing for their dream job.