Infrastructure Scaling and Capacity Planning Questions

Operational and infrastructure level planning to ensure systems meet current demand and projected growth. Topics include forecasting demand headroom planning and three to five year capacity roadmaps; autoscaling policies and metrics driven scaling using central processing unit memory and custom application metrics; load testing benchmarking and performance validation methodologies; cost modeling and right sizing in cloud environments and trade offs between managed services and self hosted solutions; designing non disruptive upgrade and migration strategies; multi region and availability zone deployment strategies and implications for data placement and latency; instrumentation and observability for capacity metrics; and mapping business growth projections into infrastructure acquisition and scaling decisions. Candidates should demonstrate how to translate requirements into capacity plans and how to validate assumptions with experiments and measurements.

EasyTechnical

73 practiced

What is headroom planning? Describe how you would decide an appropriate headroom percentage for a new service with uncertain demand, including which business inputs (SLA, procurement lead time, marketing events) and engineering constraints (cold-start time, autoscaling speed) you would consider.

HardTechnical

69 practiced

Describe how to forecast 3–5 year capacity using probabilistic statistical models. Specify required input data (historical metrics, marketing plans, seasonality), feature engineering steps, model choices (ARIMA, Prophet, Bayesian hierarchical models), how to generate confidence intervals for capacity needs, and how to validate model accuracy.

Sample Answer

Approach: treat capacity forecasting as a probabilistic time-series problem with exogenous drivers (marketing, product launches). Build a pipeline that produces point forecasts plus full predictive distributions for 3–5 year horizons and operational confidence intervals for capacity planning.

Required input data:- Historical metrics: weekly/daily demand, users, transactions, latency, error rates (3–5+ years if available).- Exogenous signals: marketing spend/tactics, feature launches, pricing changes, macro indicators.- Calendar/seasonality: day-of-week, holidays, promotional windows.- Operational constraints: provisioning lead times, max scaling rates.- Metadata: geography, customer segments, service tiers for hierarchical modeling.

Feature engineering:- Time features: trend, day/week/month, holiday flags, cyclical encodings (sin/cos).- Lag features and rolling aggregates (7/30/90-day means, std).- Interaction terms: marketing_spend × seasonality, segment × trend.- Event indicators and decay functions for promotions.- Align and impute missing exogenous data; normalize or log-transform skewed metrics.- Aggregate at multiple granularities (global, region, customer tier) for hierarchical models.

Model choices (pros/cons):- ARIMA / SARIMA / State-space (Kalman): good for linear autocorrelation and formal CI; struggles with many exogenous regressors and nonlinearity.- Prophet: fast, handles multiple seasonalities, changepoints, holiday effects; offers uncertainty via trend+season components — easy baseline.- Exponential smoothing (ETS): robust for level/seasonal patterns.- Bayesian hierarchical time-series (e.g., dynamic hierarchical models, Bayesian structural time series): best for combining segment-level data, sharing information across groups, and producing coherent predictive posteriors; accommodates uncertainty in parameters and exogenous effects.- Machine-learning hybrids: gradient-boosted trees or RNNs for complex nonlinearities; wrap with quantile regression or conformal prediction for intervals.- Ensemble: combine statistical + ML models to improve robustness.

Generating confidence intervals:- Analytical intervals: ARIMA/ETS provide forecast variance from model equations.- Bayesian posterior: sample from posterior predictive distribution (MCMC/variational) to get credible intervals; naturally handles hierarchical uncertainty and parameter uncertainty.- Bootstrapped residuals / block bootstrap: resample residuals to create predictive distributions when analytic forms are unreliable.- Monte Carlo scenario simulation: sample exogenous future paths (e.g., marketing scenarios: baseline, ramp-up) and forward-simulate to produce capacity percentiles.- For operational planning, compute percentiles (e.g., 50th, 95th) and translate to provisioning decisions given SLAs and lead times.

Validation and accuracy:- Rolling-origin backtesting (time-series cross-validation): evaluate forecasts at multiple cutoffs across historical windows.- Metrics: MAE, RMSE for point forecasts; MAPE or SMAPE for scale-free; proper scoring rules for distributions (CRPS, log-likelihood); calibration metrics: empirical coverage (e.g., fraction of true values within 95% PI).- Diagnostic checks: residual autocorrelation (ACF/PACF), heteroskedasticity; PIT histograms for Bayesian models.- Stress tests: simulate extreme marketing or demand shocks, validate model behavior and CI width.- Segment-level checks: ensure coherent aggregation (sum of segment forecasts ≈ global forecast) or use hierarchical models that enforce coherence.

Practical considerations (as a software engineer):- Automate ETL, feature computation, model training, and evaluation with reproducible pipelines (Airflow, Kedro).- Version data/models; store model artifacts and metrics.- Deploy models as services that can ingest scenario inputs (e.g., marketing plan) and return predictive distributions and recommended capacity-percentiles.- Monitor drift and recalibrate: schedule retraining cadence, alert on coverage degradation or residual anomalies.- Communicate outputs to stakeholders: provide scenario-based capacity recommendations tied to percentiles and provisioning lead times.

Example quick workflow:1. Ingest 5 years daily demand + marketing.2. Build features (lags, rolling means, holiday flags).3. Fit Bayesian hierarchical model per region with marketing as covariate; sample posterior predictive for 5-year horizon under multiple marketing scenarios.4. Validate with rolling-origin: report MAE and 95% credible interval coverage.5. Export 50/95th percentile capacity curves into provisioning system and schedule monthly retrain.

EasyTechnical

63 practiced

Explain right-sizing in cloud environments. How would you use historical utilization histograms and percentile-based sizing (for example p95 CPU) to choose instance sizes and counts? Discuss pitfalls (outliers, bursty workloads, cold-starts) and how to mitigate them.

Sample Answer

Right-sizing is choosing the smallest combination of VM/container instance types and counts that meet performance targets while minimizing cost. The practical approach uses historical utilization histograms and percentile-based sizing to balance typical load vs headroom.

Approach:- Collect per-instance metrics (CPU, memory, I/O) as time-series and build histograms (e.g., percent of time CPU at x%).- Choose a target percentile (p95 or p99) that represents acceptable tail risk. For p95 CPU = 60% means 95% of samples are ≤60%.- Sizing rule: pick an instance type whose capacity keeps the chosen percentile below a safety threshold (e.g., p95 ≤ 70%). For scale-out, divide peak or percentile-based aggregate demand by per-instance capacity to get count, round up.- Example: aggregate p95 CPU demand = 320 vCPU-seconds → with 4-vCPU instances, p95 per-instance ≈ 320/4 = 80% → choose larger instance or increase count to keep per-instance p95 ≤70%.

Pitfalls & mitigations:- Outliers: single anomalous spikes can skew naïve decisions. Mitigate by using robust percentiles (p95) not max, and applying outlier filters or trimming top 0.1%.- Bursty workloads: short bursts may exceed p95 capacity. Use autoscaling with short cooldowns, burst-friendly instance types (CPU credits), or add small headroom cushion.- Cold-starts (serverless/container startup): percentile of utilization ignores latency during cold starts. Mitigate with warm pools, provisioned concurrency, or pre-warming based on traffic patterns.- Multi-resource constraints: CPU-focused sizing can starve memory or I/O—use multi-dimensional profiling and choose instance types that satisfy all hotspots.- Measurement bias: sampling interval matters—use fine-grained samples for bursty systems; ensure baseline periods represent real traffic (weekday/weekend, deployments).

Operational best practices:- Automate collection and simulate “what-if” with different percentiles.- Continuously re-evaluate (monthly) and tag changes.- Combine rightsizing with autoscaling, SSE (spot) mixes, and CI/CD deployment windows to avoid flapping.

MediumTechnical

58 practiced

For a latency-sensitive, read-heavy service with global users, how would you decide data placement and replication strategy across regions? Discuss consistency models (strong vs eventual), read/write locality, failover, cost, and legal/compliance constraints that influence the decision.

Sample Answer

Start by clarifying requirements: target p95 read latency, acceptable staleness, write rate and distribution, RTO/RPO, number of regions, and legal/data‑residency constraints. With that, evaluate options along latency, consistency, cost, and complexity.

Pattern choices- Read-local, single-writer (primary in one region + read replicas elsewhere): low read latency, simple conflict model (strong for writes in primary), eventual consistency for replicas unless you use read-after-write routing. Good when writes are less frequent or can accept short staleness.- Geo-partitioning (shard by user/tenant region): strong locality for both reads and writes, minimizes cross-region write traffic, avoids global consensus. Best if user data can be partitioned cleanly.- Multi-master (active-active) with conflict resolution or CRDTs: low latency for both reads/writes globally but higher complexity and potential for divergent state; requires application-level merge rules or convergent data types.- Global consensus (Raft/Paxos across regions): provides strong consistency globally but incurs higher write latency (round-trips across regions) and cost.

How to choose1. Consistency vs latency: If the product needs strict correctness (financial transactions), prefer strong consistency (single-master or global consensus) despite higher write latency. If reads dominate and some staleness is acceptable (social feeds), eventual consistency + read-local replicas is preferable.2. Read/write locality: Route reads to nearest replica (DNS + global LB + local caches). Route writes to the owning region: either the primary for that shard or the user’s home region.3. Failover: Use health-checked global load balancer and automated leader election. Maintain warm replicas with up-to-date WAL shipping or CDC for quick promotion. Test failover runbooks and ensure client retry/backoff semantics.4. Cost: More replicas and cross-region sync increase bandwidth and storage costs; consensus protocols increase request latency and cloud egress. Balance SLAs vs budget — e.g., fewer read replicas in low-traffic regions.5. Legal/compliance: Enforce data residency by placing shards in required jurisdictions and prevent cross-border replication for regulated fields. Use encryption-at-rest, audit logging, and data removal workflows to meet GDPR/CCPA.

Operational knobs- Tune read quorum vs write quorum (for quorum-based stores) to favor reads or writes.- Use TTLs, cache invalidation and read-after-write routing for user sessions.- Monitor staleness metrics (replica lag), p95/p99 latencies, error rates and failover time.

Example recommendation (read-heavy, global, latency-sensitive, tolerates slight staleness): Geo-partition users by region; run a primary for each partition in its home region with read replicas in nearby regions; route reads to nearest replica, route writes to home primary; enable async replication with monitored lag and optional read-after-write affinity for recent writers. For regulated users, restrict replicas to compliant regions only. For parts requiring strict consistency (billing), centralize writes to a strongly consistent global service (or use a strongly consistent datastore) and keep other services eventually consistent.

MediumTechnical

53 practiced

Create a high-level cost model for scaling a web application on AWS to support 1,000,000 monthly active users. Which assumptions do you make about RPS per MAU, average session length, caching hit rate, instance sizing, storage throughput, network egress, and monitoring? Show how sensitivity to caching hit rate or instance choice changes monthly cost estimates.

Sample Answer

Situation: We need a high-level monthly cost model on AWS to support 1,000,000 MAU. I’ll state assumptions, compute baseline costs, then show sensitivity to caching hit rate and instance sizing.

Key assumptions (reasonable engineering defaults):- Traffic: 1,000,000 MAU → 33,333 daily active users (DAU) assuming 30-day month.- RPS per MAU: Average sessions/day = 1.5, average session length = 5 minutes (300s). Concurrent users ≈ (DAU * session_length) / 86400 = (33,333 * 300) / 86,400 ≈ 116 concurrent users. Peak factor 5 → peak concurrent ~580. Translate to RPS: assume each concurrent user generates 0.2 req/s → baseline RPS ~23 (avg) and peak RPS ~116.- Caching: cache hit rate baseline 70% (edge CDN + app cache). Misses go to app servers/DB.- Instance sizing: app servers: t3.medium (2 vCPU, 4GB) can handle ~150 RPS app-layer (conservative). Use auto-scaling with min 2, avg 3, peak 6.- DB: Amazon RDS (db.m5.large) primary + read-replica for reads. Storage: 200GB gp3, baseline IOPS 3000.- Storage throughput: 200 GB S3 for assets, 1 TB/month egress initial.- Network egress: 1 TB/month to internet (CDN reduces origin egress).- Monitoring: CloudWatch + X-Ray ~ $200/month.- Other: ALB, NAT, Route53, backups, IAM, small budgets.

Baseline monthly cost estimate (ballpark):- CDN (CloudFront): $0.09/GB for first TB => ~1 TB => $90- EC2 app (3 avg t3.medium): 3 * $0.0416/hr * 24 * 30 ≈ $90- RDS primary (db.m5.large ~ $0.192/hr) + replica: 2 * $0.192 * 24 *30 ≈ $277- Storage (RDS gp3 200GB): ~$20- S3 storage 200GB: ~$5- ALB: ~$25- EBS + snapshots: ~$30- Network egress (beyond CDN): additional $50- Monitoring/logging: $200- Misc (Route53, NAT, data transfer between AZs): $80

Total baseline ≈ $970/month (rounded ~ $1k).

Sensitivity examples:1) Caching hit rate effect (assuming 70% baseline; each cache miss adds app + DB cost proportional to requests):- If hit rate increases to 90%: 20% of requests previously hitting app/DB are eliminated → reduce app CPU load by ~29% and DB reads by ~29%. Fewer app instances needed: avg app servers from 3 → 2 (cost drop ~$30/month). DB replica maybe downscaled or lower IO costs (save ~$50). Net monthly ≈ $890 (~8% savings).- If hit rate drops to 50%: more load → app avg instances 3→5 (add ~$60), DB IOPS increases raising RDS / storage IO ~$80. Net monthly ≈ $1,110 (~14% increase).

2) Instance choice effect:- Use c5.large (2 vCPU, 4GB, higher network/CPU) instead of t3.medium: c5.large ~$0.086/hr ≈ double cost per instance but can handle ~3x CPU-bound throughput. If we can reduce instances from 3→1 (with better packing), cost = 1 * $0.086 * 24 *30 ≈ $62 vs prior $90 → slight savings but risk of single point and less burstable credit. If instead we need 2 c5.large for redundancy ≈ $124 → +$34 compared to baseline.- Using larger RDS (db.m5.xlarge) doubles DB cost (+$277) if throughput or IOPS require it.

Key takeaways and recommendations:- CDN + high cache hit rate yields largest wins; invest in CloudFront + Redis/ElastiCache and cache warming.- Right-size instances: prefer instance families with better network/CPU for web workloads; use auto-scaling policies based on RPS and latency.- Monitor and model IOPS and egress closely—DB IO and network are common cost drivers.- Build knobs: cache TTLs, read-replicas, compression for egress, and infra automation to scale down non-peak.

This is a high-level model; for production refine with real traffic traces (RPS by minute), percent of dynamic vs static requests, DB query profile, and more granular cost items (reserved instances, savings plans, peering).

Unlock Full Question Bank

Get access to hundreds of Infrastructure Scaling and Capacity Planning interview questions and detailed answers.

Join thousands of developers preparing for their dream job.