Trade-Off Analysis in System Design Questions

Ability to identify key nonfunctional requirements and constraints and to compare alternative designs with clear, quantitative reasoning. Expect discussion of consistency versus availability, latency versus throughput, cost versus performance, operational complexity, and implementation risk. Candidates should demonstrate how to quantify trade offs using metrics such as latency percentiles, throughput, cost per request, and availability targets, how to choose appropriate consistency models and failure modes, and how to document and justify the selected architecture given product and business priorities.

MediumSystem Design

71 practiced

You need to split a monolith handling orders into two services: Orders and Inventory. Design a migration approach that preserves correctness (no oversells), minimizes user-visible latency, and reduces implementation risk. Discuss trade-offs among synchronous calls, asynchronous events, distributed transactions, and SAGA patterns, and propose metrics to evaluate success.

Sample Answer

**Clarify goals & constraints**- Preserve correctness: zero oversells.- Minimize user-visible latency for order placement.- Reduce implementation risk; incremental rollout.

**Proposed migration (strangler + hybrid flow)**1. Strangler: introduce Orders service first; Inventory stays in monolith.2. Read side: replicate Inventory read models to Orders via CDC (change-data-capture) to keep fast availability checks locally.3. Write path (two-phase rollout): - Phase A (conservative): Orders calls Inventory via synchronous guarded API for final stock reservation — fallback to monolith for unresolved cases. - Phase B (optimized): switch to local reservation using the replicated read model + async confirm-reserve event to Inventory. Inventory performs authoritative commit and publishes confirmation or compensation events (SAGA).

**Why this reduces risk**- CDC replication keeps latency low for optimistic availability checks.- Synchronous guarded calls for edge cases preserve correctness during transition.- Async confirm + SAGA prevents long distributed locks; compensations handle conflicts.

**Trade-offs**- Synchronous calls: simple and consistent but increase latency and coupling.- Asynchronous events + SAGA: low latency, eventual consistency, requires compensating logic and careful idempotency.- Distributed transactions (2PC): strong consistency but high complexity, poor scalability, and fragile across services.- CDC/local read model: speeds checks but is eventually consistent; needs reconciliation.

**Operational concerns**- Idempotency, dedup keys, and ordered consumer processing.- Reconciliation job to detect and resolve drift.- Feature flags and canary rollout.

**Success metrics**- Oversell count (target 0).- 95th/99th percentile order-placement latency.- Rate of compensating transactions per 1k orders.- Time-to-detect and time-to-reconcile state drift.- Error/retry rates on Inventory APIs.- Deployment rollbacks and incident frequency during migration.

This hybrid approach balances correctness, latency, and risk by using synchronous fallbacks during migration and moving to async SAGA with authoritative Inventory commits once confidence is high.

EasyTechnical

56 practiced

Define latency and throughput and explain the relationship between them. As a Cloud Architect, describe how you would present latency percentiles (p50, p95, p99) and throughput curves to product and SRE stakeholders to make a decision about scaling and cost trade-offs.

EasyTechnical

72 practiced

A service currently receives 100,000 reads/sec and 200 writes/sec. The database can support 5,000 reads/sec per read-replica, and each read-replica costs $1,000/month. A Redis caching cluster costs $4,000/month and is estimated to achieve a 95% read hit rate. Estimate DB load, p99 latency impact, and cost-per-request for (a) adding read replicas vs (b) adding the Redis cache. Show calculations and recommend an approach.

Sample Answer

**Summary of assumptions**- Traffic: 100,000 reads/sec, 200 writes/sec (steady).- Read-replica capacity: 5,000 reads/sec each; cost $1,000/month each.- Redis cluster: $4,000/month, 95% read hit rate.- Month = 30 days = 2,592,000 seconds.

**Raw monthly request counts**- Total reads/month = 100,000 * 2,592,000 = 259,200,000,000- Total writes/month = 200 * 2,592,000 = 518,400,000- Total requests/month = 259,718,400,000

(a) Adding read replicas (to serve all reads)- Needed replicas = ceil(100,000 / 5,000) = 20 replicas- Monthly cost = 20 * $1,000 = $20,000- DB load after: assume reads fully served by replicas → master handles writes only = 200/sec (518.4M writes/month)- P99 latency impact: - Reads: local read latency served by replicas — p99 for reads depends on replica sizing/placement but likely similar to current read latency per DB node; reduces master read contention. - Master p99 for writes improves because master no longer contends for reads.- Cost per request (replica approach): - Cost / total requests = $20,000 / 259,718,400,000 ≈ $7.70e-8 ≈ $0.000000077 per request - If charging only reads: $20,000 / 259,200,000,000 ≈ $7.72e-8 per read

(b) Adding Redis cache (95% hit rate)- Cache serves 95% of reads -> cache-served reads = 95,000/sec; DB reads remaining = 5,000/sec- DB load after: reads to DB = 5,000/sec; writes = 200/sec → total DB ops = 5,200/sec- Replicas required to support remaining reads (if you still use replicas): ceil(5,000 / 5,000) = 1 replica ($1,000/month). But question compares pure cache vs pure replicas—so analyze cache alone: master still handles writes + those reads; if single master can handle 5,200/sec (assume yes).- Monthly cost = $4,000- P99 latency impact: - Most reads served from Redis (in-memory) → large reduction in p99 read latency (from DB-ms to sub-ms). - DB p99 improves because read load drops massively; write p99 improved modestly but depends on write path.- Cost per request (cache approach): - Cost / total requests = $4,000 / 259,718,400,000 ≈ $1.54e-8 ≈ $0.0000000154 per request - If counting only reads: $4,000 / 259,200,000,000 ≈ $1.54e-8 per read

**Comparison & recommendation**- Cost: Redis is significantly cheaper monthly ($4k vs $20k) and gives lower cost-per-request.- Latency: Redis provides much better p99 for reads (in-memory sub-ms) and reduces DB p99; read replicas reduce DB read load but each read still pays DB latency (higher than Redis).- Operational complexity: 20 replicas increases operational surface (replica lag, sync issues, failover). Redis adds caching complexity (cache invalidation, cold-warm behavior) but is standard for read-heavy workloads.- Hybrid note: Best practice often is cache-first (Redis) to cut ~95% of reads, then add 1–2 replicas for remaining read scaling and HA.

Recommendation (Cloud Architect): Deploy the Redis caching cluster first to achieve large latency gains and cost savings. Monitor DB residual load and add 1–2 read replicas only if DB CPU/IO or p99 for writes/remaining reads requires further isolation or HA.

HardTechnical

57 practiced

You must advise leadership whether to standardize on a single cloud provider or adopt a multi-cloud approach for an analytics platform. Analyze trade-offs in vendor lock-in, cost, time-to-market, feature parity, operational complexity, migration cost, and negotiating leverage. Propose a quantitative decision model (including estimated migration cost and operational overhead) and recommend an approach for (a) a fast-growing startup and (b) an established enterprise.

Sample Answer

**Situation & goal**Advise whether to standardize on one cloud or adopt multi-cloud for an analytics platform; weigh lock-in, cost, time-to-market, feature parity, operational complexity, migration cost, negotiating leverage.

**High-level trade-offs**- Vendor lock-in: single-cloud increases technical and contractual lock-in; multi-cloud reduces provider dependence but demands abstraction layers.- Cost: single-cloud often yields volume discounts and simpler ops (lower Opex); multi-cloud can exploit best-price services but has duplication.- Time-to-market: single-cloud typically faster (1 provider CI/CD, single skillset); multi-cloud slows delivery.- Feature parity: single-cloud access to native services faster; multi-cloud may lag or need custom equivalents.- Operational complexity & migration cost: multi-cloud raises SRE, networking, security complexity and inter-cloud data egress.- Negotiating leverage: multi-cloud increases strategic leverage but requires real ability to move workloads.

**Quantitative decision model (simplified)**

Define annual Total Cost of Ownership (TCO):

text

TCO = InfraCost + OpsCost + MigrationAmort + BusinessPenalty

- InfraCost = baseline provider bills (incl. egress)- OpsCost = staffing, tool, complexity premium- MigrationAmort = one-time migration cost amortized: MigrationOneTime / Years- BusinessPenalty = expected opportunity cost from slower time-to-market (revenue loss)

Estimate inputs (example numbers, USD/year):- Single-cloud: InfraCost=1,200k; OpsCost=400k; MigrationAmort=0 (existing); BusinessPenalty= - (faster launch) 0- Multi-cloud: InfraCost=1,350k (duplicate services + egress); OpsCost=650k (+50% complexity); MigrationAmort=300k/3yrs =100k; BusinessPenalty= - (slower) 150k

Compute and compare.

Intuition: multi-cloud TCO often 20–40% higher initially. Negotiation leverage should be valued only if migration capability exists (<12 months, <15% of annual spend).

**Recommendation**- Fast-growing startup: single-cloud. Rationale: prioritize time-to-market, lower OpsCost, developer velocity. Use IaC, modular design and 12–18 month review to add portability (avoid deep proprietary bindings for core logic).- Established enterprise: hybrid approach: standardize core analytics on primary cloud for efficiency, implement multi-cloud for critical data resilience and strategic leverage where justified. Invest in abstraction (Kubernetes, data mesh, Terraform modules), formal migration runway and measure switch cost against negotiated savings.

**Operational guardrails (both)**- Define portability score per workload (0–10). Move only workloads with score >=7 to multi-cloud.- Maintain CI/CD, IaC and data exportability tests.- Target break-glass migration time < 6 months for critical workloads to preserve leverage.

MediumSystem Design

72 practiced

A social feed service requires replies to appear after their parent posts (causal ordering) but otherwise can be eventually consistent for propagation. Propose a consistency model and architecture to guarantee causal ordering for replies in a globally distributed deployment. Specify which metrics you would use to quantify staleness and ordering correctness and how you would tune the system to balance cost and user experience.

Sample Answer

**Clarifying requirement**- Strong ordering only for reply → parent causality; otherwise eventual global propagation is acceptable. Goal: users never see a reply delivered before its parent post (no causal violations), while keeping low cost and good UX.

**Consistency model**- Causal+Eventual: record causal dependencies for each write (reply) and deliver only when dependencies are visible locally. Implement with lightweight causal metadata (parent ID + causal timestamp using hybrid logical clocks (HLC) or dotted version vectors) rather than full vectors to limit metadata size.

**High-level architecture**- Frontend/API layer (region-local) → write to local shard leader.- Local write path: - Assign HLC timestamp; persist post and reply with dependency = parent ID + parent HLC. - Append to local durable log (Kafka/Cloud PubSub).- Global propagation: - Geo-replicated log (mirror/replication) carries item + dependency metadata. - Each region has a delivery service that consumes the replicated log and applies items to local read store (Cassandra/Scylla or DynamoDB) only after dependencies are satisfied.- Read path: - Query local read store; if a reply is encountered whose parent is not yet present locally, either: - Hold replies in a tiny local buffer until parent arrives (bounded wait), or - Show a placeholder (e.g., “Parent loading”) and fetch parent on-demand from remote region (read-repair).- Optional optimization: for hot parents, synchronously replicate parent to other regions (fast-path) to avoid holdbacks.

**Key mechanisms**- Dependency check: delivery service keeps a small dependency index; when parent arrives, drain buffered replies.- Metadata compactness: only store parent ID + HLC, not full causal history.- Garbage-collect dependency state after durable local application.

**Metrics to quantify correctness and staleness**- Ordering correctness: - Causal-violation rate: fraction of reads where reply appears before its parent (target ~0). - Dependency-buffer events: count of replies buffered due to missing parents.- Staleness / visibility: - Write-to-visibility latency: time from write commit in origin to first visibility in region R (P50/P95/P99). - Parent-visibility gap for replies: time between parent visible in region and reply visible.- UX metrics: - Placeholder rate: fraction of reads that show “parent missing” placeholders. - Tail latency of read requests (P99).- Operational: - Replication lag distribution; buffer sizes and drop rates.

**Tuning knobs to balance cost vs. UX**- Buffer timeout: short timeout (e.g., 100–500ms) minimizes UX hiccups but may increase cross-region fetchs; longer timeout reduces remote calls but raises perceived staleness.- Synchronous vs asynchronous replication for hot parents: sync replicate hot parent to N regions (costly) to eliminate buffering for large conversations.- Replica fanout / replication frequency: increase replication throughput for active shards to lower visibility latency.- Read-repair aggressiveness: eager remote fetch on user demand vs lazy background repair.- Metadata granularity: HLC (compact) vs full vector (expensive). Use HLC for scale; switch to per-conversation sequence numbers for tight threads.- Placement: pin hot conversation shards to fewer regions to reduce cross-region dependency.

**Trade-offs and rationale**- Using HLC + dependency index gives causal guarantees with small metadata overhead and scalable geo-replication; holding replies locally prevents user-visible violations without requiring synchronous global commits.- Cost saved by eventual propagation for cold content; UX improved for active threads via targeted sync/replication.- Conservative defaults: aim for zero causal violations (monitoring), tune buffer timeout and hot-parent replication to hit target read latencies and acceptable cross-region bandwidth.

This design provides causal ordering for replies with operational levers (timeouts, targeted sync) to balance cost and user experience while giving measurable metrics to iterate.

Unlock Full Question Bank

Get access to hundreds of Trade-Off Analysis in System Design interview questions and detailed answers.

Join thousands of developers preparing for their dream job.