Infrastructure Scaling and Capacity Planning Questions

Operational and infrastructure level planning to ensure systems meet current demand and projected growth. Topics include forecasting demand headroom planning and three to five year capacity roadmaps; autoscaling policies and metrics driven scaling using central processing unit memory and custom application metrics; load testing benchmarking and performance validation methodologies; cost modeling and right sizing in cloud environments and trade offs between managed services and self hosted solutions; designing non disruptive upgrade and migration strategies; multi region and availability zone deployment strategies and implications for data placement and latency; instrumentation and observability for capacity metrics; and mapping business growth projections into infrastructure acquisition and scaling decisions. Candidates should demonstrate how to translate requirements into capacity plans and how to validate assumptions with experiments and measurements.

HardTechnical

106 practiced

You observe periodic latency spikes caused by JVM garbage collection on server nodes serving user requests. Design capacity and deployment changes to mitigate user impact while optimizing cost: include GC tuning, pod sizing, instance sizing, rolling restarts, and architectural changes (e.g., service partitioning or more pods with smaller heap).

HardTechnical

75 practiced

Design an A/B experiment in production to compare autoscaling based on CPU utilization (A) versus request-per-second (RPS) target tracking (B). Specify cohort allocation, duration, success metrics (latency p95, error rate, cost per request), statistical significance approach, rollback criteria, and safety mechanisms if one model performs poorly.

Sample Answer

**Overview & goal**Compare CPU-based autoscaling (A) vs RPS target-tracking (B) in production to evaluate latency p95, error rate, and cost/request, while keeping service safe.

**Cohort allocation**- 50/50 traffic split by user-session or by availability-zone (sticky by session-id) to avoid cross-contamination.- Start with small canary (5% B) for 12–24h sanity, then scale to 50/50.

**Duration & sample size**- Run for at least 2 weeks covering weekdays/weekend; minimum of N requests per cohort estimated via power calc (e.g., detect 5% p95 difference, 80% power, α=0.05) — use historical variance to compute N.

**Success metrics**- Primary: p95 latency (ms)- Secondary: error rate (% of 5xx), cost per request (cloud billing / request count)- Also capture scaling behavior: average instances, scale-up frequency, cold starts.

**Statistical significance**- Aggregate per-minute buckets per cohort; use bootstrapped confidence intervals for p95.- For error rate, use proportion z-test (or Fisher exact if small counts).- Adjust for multiple comparisons (Bonferroni) since 3 metrics.- Require p < 0.05 and effect size threshold (e.g., ≥5% improvement).

**Rollback criteria & safety**- Immediate automated rollback if: - Error rate in B > A by ≥2% absolute and p < 0.01 over 5-minute rolling window. - p95 worsens by ≥30% and sustained 10 minutes. - Instances exceed budget threshold (cost spike >30% in 1 hour).- Manual alert to on-call for degradations not hitting auto-rollback.

**Safety mechanisms**- Pre-deployment: canary, feature flag to switch cohorts.- Rate-limited rollout and circuit breaker to redirect traffic to A.- Autoscaling guardrails: max instance cap, scale-up cooldowns, warm pools to reduce cold starts.- Continuous monitoring dashboards and automated alerts for SLO breaches.

**Post-experiment**- Analyze aggregated metrics, operational behavior, cost trade-offs, produce runbook changes (threshold tuning, guardrail adjustments), then promote winner gradually.

HardTechnical

73 practiced

Build a comprehensive cost model comparing a managed Kafka offering vs self-hosting Kafka on EC2. Include upfront infrastructure costs, operational engineering hours, scaling behavior, fault tolerance differences, data transfer costs, and scenarios where each option becomes more cost-effective. Describe assumptions you would capture and sensitivity factors.

Sample Answer

**Clarify scope & assumptions**- Throughput: 1 TB/day ingress, 3 TB/day egress; retention 7 days; 3 topics, avg message size 1 KB.- SLAs: 99.95% for managed; self-hosted target 99.9%. Zookeeper vs KRaft considered.- EC2 types: m5.2xlarge for brokers, gp3 volumes; networking 10 Gbps. Ops hourly rate $60/hr. Managed Kafka price example: $0.25/GB ingress, $0.05/GB egress, $0.20/vCPU-hr (example vendor).- Time horizon: 3 years, USD.

**High-level model components**- Upfront infra (self-hosted): EC2 instances (broker+controller+nimbus), storage (replicated), load balancers, networking, backups — CAPEX-like first month cost.- Recurring infra: EC2 hours, EBS IOPS, data transfer, monitoring agents.- Managed: no upfront servers; monthly per-throughput + storage + network fees.- Operational labor: provisioning, patching, upgrades, capacity planning, incident response vs managed where vendor handles major upgrades but ops still integrate/auth, monitor, and handle incidents.

**Quantitative example (simplified)**- Self-hosted upfront month 1: 9 x m5.2xlarge brokers (~$2k/mo each) + EBS = ~$25k first month. Annual ops: 1.5 FTEs (~$180k/yr) for Kafka + infra.- Managed: $0 upfront; monthly bill ~ $30k (throughput+storage+replication+network). Ops: 0.3 FTE (~$36k/yr).- 3-year TCO: run numbers with NPV; often managed cheaper for low-to-medium sustained throughput due to lower ops headcount; self-hosted wins at very high sustained throughput where vendor per-GB costs exceed marginal EC2+ops.

**Scaling & fault tolerance**- Self-hosted: vertical/horizontal scaling slower (procurement, rebalancing), requires careful partition reassignment; control over replication, ISR tuning, cross-AZ replicas, faster recovery if well-engineered.- Managed: elastic scaling, automated rebalancing, multi-AZ replicas, better default durability; less control over fine tuning and custom JVM/config.

**When each is cost-effective**- Choose managed when: variable workloads, teams small (≤1-2 Kafka admins), need fast time-to-market, moderate throughput.- Choose self-hosted when: extremely high sustained throughput, strict compliance/custom configs, existing Ops capacity to amortize headcount, predictable growth.

**Sensitivity factors to model**- Throughput (GB/day), retention days, replication factor, number of partitions, broker instance pricing, ops FTE cost, vendor per-GB pricing, multi-AZ cross-region replication, data egress charges, incident frequency.

**Decision checklist for interview**- Show 3-year TCO table, run break-even throughput, stress-test sensitivity on ops FTE and egress, and include SLA/organizational risk (compliance, control).

EasyTechnical

66 practiced

Define capacity planning concepts a systems administrator must know. Explain the terms 'headroom', 'safety margin', 'peak vs average utilization', and 'capacity runway'. For two example services (a low-latency web frontend and a nightly batch-processing job), recommend a headroom percentage and justify your choice with operational trade-offs.

EasyTechnical

68 practiced

Explain how to configure autoscaling cooldowns and stabilization windows to prevent flapping while still providing timely responses to load changes. Provide example numeric values for scale-up vs scale-down windows, explain why scale-down windows are typically longer, and how to test tuning in a safe manner.

Unlock Full Question Bank

Get access to hundreds of Infrastructure Scaling and Capacity Planning interview questions and detailed answers.

Join thousands of developers preparing for their dream job.