Scalability & Capacity Planning Questions

Analyzing how a system's performance changes as load grows and planning the resources to keep it healthy. Covers horizontal vs vertical scaling, throughput vs latency under load, headroom and saturation, load modeling, and forecasting capacity for expected traffic. Includes identifying the scaling bottleneck that will bind first as demand increases.

HardTechnical

69 practiced

Explain how you'd build a capacity plan for a data-serving fleet that must handle seasonal spikes up to 10x baseline. Include the metrics to track (traffic, CPU, latency, queue lengths), autoscaling strategies, pre-warming or reserved capacity approaches, testing for scale, and trade-offs between cost and availability.

Sample Answer

Requirements & goals: support up to 10x seasonal peak with SLOs (p99 latency < X ms, availability 99.95%). Budget constraint target and acceptable degradation window during ramp.

Metrics to track (real-time + historical):- Traffic: requests/sec, input bytes/sec, distinct keys- Utilization: CPU, memory, disk I/O, network- Service-level: p50/p95/p99 latency, error rate, timeouts- Queueing: queue lengths, backlog growth rate, processing rate- Platform: instance boot time, container startup time, cold-cache miss rate- Business: revenue-impacting requests, SLA breaches

Autoscaling strategies:- Reactive horizontal autoscaling (target CPU, request latency, or custom metric like in-flight requests per host) with conservative cooldowns to prevent thrash.- Predictive scaling using historical seasonality (time-series forecasting: ARIMA/Prophet/ML) to start scale-out before expected spike.- Hybrid: predictive scale-up + reactive fine-tuning. Use rate-based and queue-length based policies to capture bursts.- Use graceful draining and connection-aware load balancers; ensure state is externalized.

Pre-warming / reserved capacity:- Maintain a warm pool of standby instances or containers (fraction of expected peak) for fast handoff.- Use reserved/committed instances for base baseline and autoscaling for incremental capacity. Consider savings plans for steady baseline.- For spot/interruptible instances, mix with reserved for cost: keep critical portion on reserved, opportunistic capacity for bulk.

Testing for scale:- Rehearsals: regular load tests that simulate 1x–10x traffic including realistic request patterns and cache-warm vs cold scenarios.- Chaos and failure injection (instance termination, AZ outage) during scaled conditions.- Canary and gradual rollout during real spikes.- Measure boot times, cache warm-up curves, and end-to-end SLOs.

Trade-offs (cost vs availability):- 100% reserved capacity avoids cold starts but is costly. Heavily predictive scaling reduces cold time but risks overprovisioning if forecasts miss.- Spot instances reduce cost but increase risk; use for noncritical batch or as pooled capacity with fast replacement.- SLA targets dictate min reserved capacity: higher availability -> higher reserved %.- Optimize by caching, batching, request shaping, and graceful degradation for low-value requests.

Operational pieces:- Clear runbooks for scale events, automated alerts on forecast deviations.- Dashboards for metrics and cost.- Post-mortems after spikes to refine forecasts and policies.

Example: baseline 1000 rps, peak 10k rps. Reserve capacity for 2x baseline (handles growth + safety), predictive autoscale to 8x starting 2 hours before expected peak, warm pool to cover remaining 0–20% sudden bursts, reactive autoscale with queue-length target to reach full 10x. Adjust based on rehearsals and cost SLO.

Unlock Full Question Bank

Get access to hundreds of Scalability & Capacity Planning interview questions and detailed answers.

Join thousands of developers preparing for their dream job.