Scaling Systems and Platforms Through Growth Questions

Describe experiences scaling systems, platforms, or services through significant growth phases. Examples: scaling from 1 million to 100 million users, migrating from monolith to microservices as organization grew, or building infrastructure to support 10x team growth. For each example: What was working before that stopped working at scale? What bottlenecks did you encounter? How did you identify and address them? What architectural changes were necessary? How did you sequence the work to minimize disruption? What did you learn? Discuss both technical and organizational scaling—they're intertwined.

MediumTechnical

72 practiced

You are migrating a synchronous monolith where a key transaction flows synchronously through multiple modules and shows high end-to-end latency spikes. Describe a refactor approach to reduce latency, considering async processing, event-driven decoupling, compensating actions, and request fibers such as idempotency and retries.

Sample Answer

Clarify constraints first: which steps must complete before returning a response (user-facing latency) vs which can be deferred; SLA for end-to-end consistency; failure tolerance; throughput and scale.

Refactor approach (stepwise):

1) Identify the hot path: profile the monolith, trace the transaction, flag modules causing spikes. Add correlation IDs and distributed tracing.

2) Separate sync vs async responsibilities: keep only the minimal, user-critical steps synchronous (authentication, basic validation, immediate debit/ack). Push non-critical work (notifications, analytics, downstream enrichment) to async pipelines.

3) Introduce event-driven decoupling: emit domain events after each logical step to a durable broker (Kafka/Rabbit/SQS). Consumers process downstream tasks independently. This reduces blocking and enables retries/backpressure.

4) Implement sagas/compensating actions: for multi-step business transactions that previously relied on a single DB transaction, use a choreography or orchestrator pattern. Each step publishes success/failure events; failures trigger compensating actions to rollback previous effects. Model state transitions explicitly.

5) Ensure idempotency and safe retries: design idempotent handlers (idempotency keys per transaction) and store processed-event IDs to avoid duplication. Use exponential backoff + dead-letter queues for poison messages.

6) Protect user experience: return a definitive immediate response (accepted / pending) with status endpoint or push updates. For strictly-required sync results, consider request fibers: optimistic local commits + eventual reconciliation, or two-phase where a fast provisional response is given and finalized asynchronously.

7) Reliability and performance controls: add timeouts, circuit breakers for slow downstreams, rate limiting, batching where appropriate, and backpressure propagation from queues.

8) Observability and roll-out: monitor latency, queue lengths, error rates. Roll out incrementally by strangler pattern—route a % of traffic to refactored flow and validate.

Example: On a payment flow, keep card authorization sync, immediately return "payment accepted (processing)", emit PaymentInitiated event. Async processors perform ledger posting, fraud checks, settlement. If ledger posting fails, publish PaymentFailed and run compensating refund via stored idempotent refund action.

Why this works: decoupling reduces end-to-end blocking, events provide durability and retryability, sagas maintain business correctness without distributed transactions, and idempotency ensures safety under retries.

HardSystem Design

75 practiced

Design a cache invalidation strategy for a global inventory system where reads are high and writes are frequent. The system requires most reads to reflect updates within 1 second for correctness. Discuss mechanisms such as event-driven invalidation, versioning, TTLs, and selective sync, and detail trade-offs between consistency and performance.

Sample Answer

Requirements & constraints- Global inventory, very high reads, frequent writes.- Reads must see updates within 1s (soft real-time correctness).- Low read latency, high availability, regional replication.

High-level design- Origin DB (strongly-consistent master per SKU partition) + global read replicas.- Distributed caching tier (regional edge caches + a global consistent cache for critical metadata).- Event bus (Kafka or cloud pub/sub) carrying inventory change events (CDC from DB or write path).- Invalidation service subscribed to events, plus per-region cache brokers to push invalidations.

Mechanisms1) Event-driven invalidation (primary)- On write commit, publish event with SKU, version, delta, timestamp.- Regional brokers consume and apply: either Evict(key) or Update(cache[key]=newValue).- Push model gives <1s propagation if brokers colocated and consumer lag small (<100ms).

2) Versioning & read validation- Include a monotonically-increasing version (or Lamport/timestamp) per SKU.- Cache stores value + version. Readers can accept cached value only if version >= required; otherwise fetch origin.- Optionally, serve stale but attach a freshness flag and background validate.

3) TTLs for safety net- Short TTL (e.g., 1–2s) on cached entries to bound staleness if events lost.- TTL compensates for transient failures but increases origin load slightly.

4) Selective sync / selective bypass- For hot SKUs or transactional flows, use synchronous write-through cache or read-through with immediate update to cache in write path.- For less critical SKUs, rely on event-invalidation + TTL.

Reliability, ordering, idempotence- Use partitioning by SKU in event bus to preserve order; consumers process in-order.- Events must be idempotent (include version) so replay is safe.- Emit tombstones for deletes; handle duplicate/out-of-order via version compare.

Trade-offs: consistency vs performance- Strong push + synchronous cache write (write-through) gives best consistency but increases write latency and complexity globally.- Event-driven async invalidation + short TTL balances read latency and write throughput; slight window for stale reads but bounded (<1s) if pipeline healthy.- Versioned reads + immediate invalidation minimizes wrong answers without blocking writers; requires readers to check version or accept fallback to origin (adds read latency on misses).- Short TTL reduces risk but increases origin QPS and cache churn.

Recommendation (practical)- Use event-driven invalidation as primary with per-SKU versioning and regional brokers to push updates. Set TTL ~1–2s as backup. For top-traffic SKUs, apply synchronous cache updates during write path (write-through) or pin to a strongly-consistent global cache. Instrument end-to-end latency and consumer lag, and implement dead-letter and replay to ensure correctness. This yields low read latency, scalable writes, and bounded staleness meeting the 1s requirement while allowing trade-offs per SKU class.

MediumSystem Design

73 practiced

A client requires an active-active multi-region deployment across NA and EU for low latency. Explain choices for traffic routing, state synchronization, conflict resolution for user data, and how you'd ensure eventual consistency while minimizing user-visible anomalies.

Sample Answer

Requirements & constraints:- Active-active across NA and EU with low read/write latency for regional users, tolerating region failure, and ensuring eventual consistency with minimal user-visible anomalies.

High-level approach:- Route users to nearest region for low latency; keep all regions writable and replicate state asynchronously with conflict resolution strategies.

Traffic routing:- Use GeoDNS/Global Server Load Balancer + Anycast for initial region affinity.- Put regional edge/load balancers (ALB/NLB) behind a global traffic manager; optionally use CDN for static content.- Implement sticky affinity for session-bound state (short TTLs) and allow failover to another region if needed.

State synchronization & data plane:- Partition data by user/tenant where possible so a user's writes are routed to their “home” region (logical affinity).- Use a multi-master, leaderless replication model (e.g., Dynamo-style) or use CRDT-backed stores for specific data types.- Store metadata (vector clocks or causal timestamps) with writes to detect concurrent updates.

Conflict resolution:- Prefer convergent approaches: - For simple counters/sets use CRDTs (G-Counter, LWW-Element-Set). - For richer domain objects use application-level merge rules (merge function that understands business semantics) with vector clocks to detect concurrency. - Fallback LWW with hybrid logical clocks when simple resolution acceptable, but log conflicts for human review if needed.- Expose conflict metadata to clients where UX can present “merged” view or prompt reconciliation.

Ensuring eventual consistency while minimizing anomalies:- Provide read-your-writes and monotonic reads via: - Session tokens including causal metadata; direct subsequent reads to region with the write or use read-after-write via quorum reads. - Optionally perform fast-path local reads and background read-repair/quorum reconciliation for stronger guarantees.- Use configurable quorum strategies: prefer R+W > N for stronger consistency on critical ops; allow lower quorums for high-throughput, low-latency reads.- Implement background anti-entropy (Merkle trees) to ensure replicas converge.- Surface conflict resolution outcomes to monitoring/alerts and maintain an audit log for reconciliation.

Operational considerations & trade-offs:- Latency vs consistency: synchronous cross-region writes increase latency—avoid for common paths.- Complexity vs correctness: CRDTs reduce conflicts but require careful design; application merges are more flexible but costlier.- Test with chaos engineering, simulate region outages, measure tail latencies, and educate product/UX on how to display eventual-consistent states (e.g., “Saving…”, optimistic UI with rollback).

Monitoring & rollback:- Instrument replication lag, conflict rates, and client-visible anomalies. Provide runbooks for manual conflict resolution and automated reconciliation jobs.

This design balances low-latency regional access with robust, convergent replication and application-aware conflict resolution to minimize user-visible inconsistencies.

MediumSystem Design

84 practiced

Outline your approach to decomposing a large monolith into microservices: state the criteria to define service boundaries, how you'd sequence the extractions (first 3 services), and techniques to limit cross-service coupling and repeated refactors.

Sample Answer

Requirements & constraints:- Preserve business continuity (zero/low downtime), support incremental delivery, keep team autonomy, and minimize customer impact. Non-functional: scalability, deployability, observability, and security.

High-level approach:1. Discover bounded contexts: map business domains (billing, orders, catalog, auth, notifications) using domain-driven design (events, aggregates, owners).2. Define service boundaries by criteria: - Single business capability / bounded context - Clear data ownership (single source of truth) - Independent release & scaling needs - Low coupling (fewest synchronous calls) - Team boundaries and SLAs - Compliance or security isolation requirements3. Sequence extractions (first 3 services): - Service A — Authentication & Authorization: small surface, central to security, enables token-based contracts and reduces blast radius. - Service B — Catalog / Product: read-heavy, clear data model, allows independent scaling and caching; replaces monolith reads with API and cache. - Service C — Orders (or Checkout): high-business-value flow, but dependent on auth/catalog; after auth/catalog exist, extract orders to own transaction boundaries. Sequence justification: start with low-risk, high-independence pieces (auth, catalog) to establish deployment patterns, CI/CD, observability before extracting transaction-heavy orders.4. Techniques to limit coupling & repeated refactors: - Define explicit service contracts (REST/gRPC + protobuf/OpenAPI) and version them; push backward-compatible changes. - Prefer asynchronous communication (events, message bus) for eventual consistency to decouple write/read workflows. - Introduce an anti-corruption layer / adapter when integrating legacy monolith APIs. - Use consumer-driven contract testing (PACT) and contract verification in CI. - Encapsulate data access: each service owns its datastore; use change data capture (CDC) or event sourcing to propagate state rather than shared DB. - Start with API facades / strangler pattern to route traffic gradually; feature toggles to switch to new service. - Invest in observability (tracing, metrics, logs) and runbooks to detect coupling hotspots. - Establish API governance, schema registry, and a shared library for client-side resilience (retries, circuit breakers).5. Trade-offs & operational considerations: - Accept increased operational overhead (Kubernetes, service mesh) for autonomy; mitigate with platform automation. - Plan for eventual consistency, compensating transactions, and idempotency. - Measure success: reduced lead time for changes, lower incident blast radius, independent deploys.

This approach balances business risk, technical decoupling, and operational readiness to minimize rework while delivering measurable value early.

HardTechnical

85 practiced

An e-commerce checkout service must handle 10x traffic during peak sales while keeping checkout latency almost unchanged, but you cannot change the underlying database engine. Propose an architecture and sequence of changes using patterns like CQRS, caching, async fulfillment, and traffic shaping to meet the target without breaking transactional guarantees customers expect.

Sample Answer

Requirements & constraints:- Handle 10x peak traffic with near-constant checkout latency.- Cannot change DB engine; transactional guarantees for payments/orders must hold.- Use CQRS, caching, async fulfillment, traffic shaping.

High-level architecture:- Frontend/API Gateway → Traffic Shaper & Rate Limiter → Checkout Service (Stateless) → Command Bus → Write Model (primary DB) + Event Store → Read Model (materialized views in cache / fast DB) → Fulfillment/Payment Workers (async) → External payment/gateway services.- Add a cache layer (Redis) for read-heavy data (cart, pricing, inventory snapshots).- Use idempotent command handlers and SAGA coordinator for distributed transaction choreography.

Sequence of changes (phased, safe):1. Observability & baseline: add detailed metrics, tracing, and feature flags.2. Introduce CQRS read model: project order/cart views into Redis for sub-100ms reads; keep writes routed to primary DB.3. Implement optimistic command queue + idempotency keys so repeated commands safe under retries.4. Deploy async fulfillment: accept order, reserve inventory (soft-reserve in DB), respond to customer immediately; push fulfillment to worker queue. Use compensating actions if fulfillment fails.5. Add traffic shaping: token-bucket per-customer & global admission to smooth spikes; priority lanes for payments/checkout vs noncritical traffic.6. Inventory & pricing caches with short TTL + version checks to avoid stale commits; on write, validate version against DB (compare-and-swap).7. Scale stateless checkout services and workers horizontally; autoscale on queue depth and latency.8. Conduct flood tests behind feature flags, tune TTLs, queue sizes, and rate limits; rollback capability.

Preserving transactional guarantees:- Critical operations (final payment capture, final inventory commit) use synchronous DB writes with optimistic concurrency (version/row locks minimal scope).- Use SAGA (orchestrator) to provide atomicity across services: local DB transaction + event emission; compensating transactions handle failures.- Idempotency keys and event sourcing ensure exactly-once semantics for external effects.

Trade-offs:- Slightly higher complexity and eventual consistency for read views, but user-facing latency improves.- Soft-reserves risk temporary inventory contention; mitigated by short hold windows and fast reconciliation.

This phased approach meets 10x peak by offloading reads to cached projections, smoothing spikes via traffic shaping, and converting slow downstream steps to async while preserving financial/transactional correctness through strong write-side validations and SAGA/coordinator patterns.

Unlock Full Question Bank

Get access to hundreds of Scaling Systems and Platforms Through Growth interview questions and detailed answers.

Join thousands of developers preparing for their dream job.