System Capacity and Cost Estimation Questions

Translating business requirements into technical capacity needs and cost estimates for infrastructure and operations. Topics include back of envelope and Fermi estimation for storage compute and bandwidth, understanding throughput metrics such as queries per second and latency targets, provisioning strategies including autoscaling versus reserved capacity, how architectural choices drive operational cost over time, and approaches to estimate cloud and infrastructure expenses. Candidates should be able to justify assumptions, demonstrate simple calculations for resource sizing, discuss trade offs between performance reliability and cost, and explain how to align provisioning strategy with budget constraints.

MediumSystem Design

38 practiced

Design a capacity and cost estimate for an e-commerce checkout API that expects a flash-sale peak of 5,000 concurrent users for 10 minutes. Users make on average 4 requests per checkout session within 2 minutes. Size the stateless application tier, caching layer, and suggest a DB approach. State all assumptions and show calculations.

Sample Answer

Requirements & assumptions- Flash sale peak: 5,000 concurrent users for 10 min.- Each checkout session lasts 2 minutes and issues 4 requests on average (cart validation, address, payment, confirm).- Traffic distribution uniform over peak 10 min; provision for 25% headroom.- Average request size: 50 KB payload; processing CPU ~50ms, mem per request negligible; target p99 latency <200ms.- Use AWS-like pricing for ballpark.

Calculate request rate- 5,000 concurrent users × (4 requests / 120s) = 166.7 req/s- Add 25% headroom → ≈ 208 req/s (≈12,480 req/min)

Stateless application tier- Assume a t3.medium-class app (or equivalent) can handle ~100 req/s steady with p99 <200ms (conservative).- Required instances = ceil(208 / 100) = 3 → with headroom use 4 instances.- Add autoscaling + health checks; use ALB + container ECS/Fargate or k8s.- Cost (approx): 4 × $0.04/hr (if container vCPU pricing) ≈ $0.16/hr → ~$115/month for peak-period 24/7; if only for short flash windows, use on-demand or spot/burst.

Caching layer- Use Redis/Memcached for session & product/cart hotspots.- Read cache hit target 90% for product metadata; checkout-specific ephemeral session in cache.- Estimate items cached: 100k product objects avg 2 KB → 200 MB; sessions: 10k concurrent sessions × 1 KB → 10 MB.- Include overhead → provision 1GB–2GB Redis instance. Use managed Redis (e.g., AWS ElastiCache cache.t3.medium ~ $0.05–0.10/hr) or clustered if cross-AZ.

Database approach- Use primary transactional DB for writes (orders/payments) + read replica(s) for read-heavy lookups.- Expected write rate: assume 10% conversion of 5,000 concurrent = 500 orders over 2 minutes → 250 orders/min → ~4.2 writes/s. Add writes for audit/logs → <10 w/s.- Single strong-primary RDS (db.m5.large) can handle this; add 1–2 read replicas for catalog-heavy reads.- Storage: start 100GB SSD; provision IOPS if needed (≥3000 IOPS during peak).- Consider using separate DBs: OLTP for checkout, NoSQL (DynamoDB/Cassandra) for scaleable session/cart alternative, and queue (SQS/Kafka) to buffer downstream work (payment processing, fulfillment).

Trade-offs & resilience- Keep app stateless; store sessions in Redis or signed client tokens.- Use circuit breakers, retries with exponential backoff.- For extreme spikes, use throttling or queuing to protect DB.- Use multi-AZ for Redis and RDS for failover.

Summary numbers (ballpark)- App: 4 application instances (autoscale to 6 for safety)- Cache: 1 x 2GB managed Redis- DB: 1 primary db.m5.large + 1 read replica; 100GB GP2/GP3 storage with 3k IOPS- Add CDN for static assets; monitoring & alarms.

This plan is conservative and tuned for <<1k req/s spikes; re-bench with real app load tests to refine instance types, request capacity per node, and cost.

EasyTechnical

43 practiced

Estimate how many VMs are required to process 1,000,000 images per day if a single VM can process 200 images per minute. Include assumptions about parallelism, utilization, and a 25% capacity buffer for failures and spikes.

EasyTechnical

52 practiced

Describe your approach to documenting and tracking assumptions used in capacity estimates (for example assumed CPU per request or storage growth). How do you version those assumptions, validate them post-deployment, and communicate changes to stakeholders?

HardSystem Design

40 practiced

Estimate incremental capacity and cost to move a batch analytics pipeline that runs nightly (10-minute SLA) to a near-real-time architecture with 1-second latency. Propose an architecture (streaming, in-memory aggregation) and quantify compute/memory costs and trade-offs versus the current batch cost.

Sample Answer

Requirements & clarification- Move nightly batch (10-minute SLA) to near‑real‑time 1s end‑to‑end latency for same workload and accuracy.- Assume current nightly volume = 1 TB of input logs → ~300M events; steady average ingest rate if spread over 8 hours = ~10k events/sec; for 1s realtime we must sustain peak bursts (assume 100k events/sec peak).- Target: per-event aggregation/windowing (minute/second windows), joins enriched by small dimension lookups, and serving aggregated metrics to dashboards/alerts.

High‑level architecture- Ingest: Kafka / MSK (durable partitioned stream).- Stream processing: Flink (or ksqlDB/Beam) for event-time windows, stateful aggregations, joins.- State backend: RocksDB (local keyed state) + periodic checkpointing to S3.- Low‑latency serving: Redis/Elasticache for hot aggregated metrics; materialized views in ClickHouse/ClickStreamDB for analytics.- Observability: Prometheus/Grafana, tracing.- Delivery: Kafka topics → Flink → write to Redis for 1s reads; longer retention to ClickHouse.

Sizing & incremental capacity estimate (rough)- Kafka: assume 100k eps peak, avg message 1 KB → 100 MB/s peak ingress. Plan 6 Kafka partitions per broker for throughput; use 3 brokers x i3.large (sufficient I/O) or managed MSK with throughput plan of ~1 Gbps headroom.- Flink cluster: stateful processing for 300M events/day with RocksDB: - Parallelism: target ~100 task slots (to handle 100k eps with headroom). - Nodes: 10 x m5.4xlarge (16 vCPU, 64 GiB) — offers CPU for CPU-bound ops and memory for RocksDB working set. Each node has 10 task slots → 100 slots. - State size: assume 200 GB keyed state → RocksDB distributed across nodes; require ~2x memory/disk headroom.- Redis serving: cache of latest aggregates ~50 GB → Elasticache cluster: 3 x cache.r5.large (~13 GB each) or 1 x cache.r5.xlarge + replicas depending on HA.- Storage & checkpointing: S3 costs negligible relative to compute.

Cost estimate (onsite cloud on‑demand, ballpark)- Current batch (example): EMR cluster 8 x m5.2xlarge for 4 hours nightly = 8 * $0.384/hr * 4 = ~$12.3/day ≈ $370/month (compute dominated) plus S3 and small services → assume $500/month total.- Realtime incremental (continuous): - Kafka/MSK (3 brokers): ~$0.40/hr * 24 * 30 * 3 ≈ $864/month (managed overhead varies). - Flink (10 x m5.4xlarge): ~$0.77/hr * 10 * 24 * 30 ≈ $5,600/month. - Redis (Elasticache cluster): ~$0.40/hr * 3 * 24 * 30 ≈ $864/month. - Storage/monitoring/other (S3, Prometheus, ClickHouse): ~$500/month. - Total ≈ $7k–8k/month incremental vs batch $0.5k/month = ~14× cost increase.Notes: Use reserved/spot instances and right-sizing (smaller nodes, autoscaling) to reduce costs significantly; managed services pricing varies by provider and region.

Trade‑offs & mitigations- Latency vs cost: 1s SLA requires always‑on compute; cost rises from batch’s short-lived footprint to continuous clusters. Use autoscaling to reduce baseline during low traffic; use burstable instances/spot to lower cost (~40–60%).- State durability vs latency: RocksDB + checkpoints gives durability with sub‑second processing; but checkpoint frequency impacts I/O — tune to balance recovery vs overhead.- Complexity: Real‑time stack increases operational complexity (stream processing, state management). Consider alternative hybrid: micro‑batches (e.g., 1s micro-batches with Spark Structured Streaming) to reduce engineering overhead.- Accuracy: Exactly‑once semantics require careful configuration (Flink checkpoints, Kafka transactions) — small cost/latency overhead.

Recommendation- Proof of concept: deploy MSK + Flink with a smaller parallelism (20 slots) and scale to production after validating latency and state growth. Use cost controls: reserved instances, autoscaling, and use managed services only where it reduces TCO. Provide precise sizing after measuring actual event sizes, burst patterns, and state cardinality.

MediumSystem Design

47 practiced

Design capacity for a streaming ingestion pipeline that must accept 50k events/sec, each 1KB, with 7-day retention on the broker (replication factor 3). Estimate needed broker throughput, storage, number of partitions, and monthly storage cost. State compression and retention assumptions.

Sample Answer

Requirements & assumptions- Ingest: 50,000 events/sec, 1 KB/event → raw ingest = 50,000 KB/s ≈ 50 MB/s.- Retention window: 7 days.- Kafka replication factor (RF) = 3.- Assume producer + broker-level compression (Snappy or LZ4) with conservative 2× compression ratio (real ratio depends on payload).- Storage cost assumption: $0.10 per GB-month (adjust to your cloud vendor).

Throughput calculations- Raw incoming traffic to cluster (before compression): 50 MB/s.- With 2× compression: stored data rate = 50 / 2 = 25 MB/s.- Aggregate disk write across replicas (RF=3): 25 MB/s × 3 = 75 MB/s sustained disk writes.- Network replication traffic roughly equals the replica writes (plus producer traffic) — expect ~75 MB/s internal replication plus 50 MB/s ingress from producers (can be compressed).

Storage calculations (7-day window)- Per-replica stored bytes for 7 days: 25 MB/s × 86,400 s/day × 7 days = 15,120,000 MB ≈ 15,120 GB ≈ 15.1 TB.- Total cluster storage (3 replicas): 15.1 TB × 3 = 45.36 TB.

Monthly storage cost (using $0.10/GB-month)- Total GB stored = 45,360 GB → cost ≈ $4,536 / month.(If using cheaper object storage or higher compression, cost reduces; if 1.5× compression use lower figures, if no compression double storage/cost.)

Partitions & broker sizing guidance- Partitions: choose partitions for both throughput and consumer parallelism. Practical guidance: - Throughput side: compressed write rate 25 MB/s. A single partition can typically sustain 50–100 MB/s depending on broker disk/network. But partitions are primarily for parallel consumers and rebalancing. - Recommendation: 100–200 partitions to allow consumer parallelism, fast recovery, and even leader distribution. For low-latency recovery you may choose more partitions (e.g., 200).- Broker count & per-broker capacity: - Aim to keep per-broker storage <= ~8–12 TB (depends on disk type). With 45.36 TB total, 6 brokers → ~7.56 TB per broker (good). So start with 6 brokers. - Per-broker sustained disk write: cluster replica writes 75 MB/s distributed across brokers; with balanced leaders/followers each broker sees a fraction of leader writes + follower replication. With 6 brokers the per-broker write is modest (~12–15 MB/s average), well within typical I/O/network for modern instances. - Choose brokers with network >=1–10 Gbps (10 Gbps preferred for headroom) and NVMe or high-throughput disks.

Other considerations & trade-offs- Compression: using producer-side LZ4/Snappy reduces network+storage; CPU cost is low for LZ4/Snappy. Gzip gives better compression but higher CPU and latency.- Retention: if retention is strict 7 days, storage = 7-day window. If you keep a longer window, multiply accordingly.- Replication factor: RF=3 provides durability/availability; RF=2 halves storage cost but reduces fault tolerance.- Ops: account for additional headroom (60–80% utilization comfortable), monitoring, compaction/segment overhead, and uncompressed indexes — add ~10–20% overhead.- Cost sensitivity: adjust $/GB-month to your vendor (e.g., $0.05–0.12), and adjust compression ratio based on payload entropy.

Short summary- Broker write throughput (cluster, compressed): 25 MB/s logical; aggregate disk writes with RF=3 = 75 MB/s.- Stored data (7 days): ~15.1 TB per replica, ~45.4 TB total.- Partitions: 100–200 recommended.- Broker count: ~6 brokers (adjust for disk size, IO, and high availability).- Estimated monthly storage cost (at $0.10/GB-month): ~$4,536.

Unlock Full Question Bank

Get access to hundreds of System Capacity and Cost Estimation interview questions and detailed answers.

Join thousands of developers preparing for their dream job.