Infrastructure Scaling, Capacity Planning, and High Availability Questions

How to make running infrastructure scale and stay available: the operational and architectural mechanics of doing it, not the growth-modeling math behind it. Covers autoscaling policy design and Kubernetes cluster scaling (target-tracking, step, scheduled, and predictive triggers, cooldowns, warm pools, HPA, VPA, Cluster Autoscaler, GPU/node scheduling, bin-packing), load balancing algorithms and architecture (round-robin, least-connections, consistent hashing, L4 vs L7, health checks, connection draining, session affinity), horizontal versus vertical scaling choices, and high-availability and redundancy design (multi-AZ and multi-region failover, active-active vs active-passive, split-brain and leader election). Also covers turning a given demand or growth figure into concrete provisioning: headroom and safety-margin sizing, back-of-envelope instance, IOPS, and replica-count math, right-sizing, and multi-year procurement planning; validating a sizing or scaling change through load, stress, soak, and chaos testing; scaling stateful tiers such as databases, caches, and message queues; cost-aware trade-offs including reserved versus spot capacity and managed versus self-hosted infrastructure; and the observability needed to catch capacity saturation before it breaches an SLO.

MediumTechnical
58 practiced

Using queuing theory, model a front-end service as an M/M/c queue. Given arrival rate λ=1000 req/s and average service rate μ=50 req/s per server, compute the minimum number of servers c required to keep average response time <= 100 ms. Show key equations (Erlang C), calculation steps and discuss the limitations of M/M/c assumptions for real production systems.

HardTechnical
113 practiced

Case study: you're responsible for capacity planning and cost optimization for a public cloud service. Traffic exhibits daily peaks and occasional flash crowds around marketing events. Propose an autoscaling architecture and purchasing mix that balances cost against a strict SLA, and describe how you would test and pre-warm capacity ahead of a marketing event.

MediumSystem Design
67 practiced

You run daily Spark ETL jobs with variable input sizes (100 GB to 10 TB). Jobs must complete within a 3-hour window. Design an autoscaling strategy for Spark-on-Kubernetes (or EMR) including cluster sizing, scaling triggers (scheduled, data-size-based, queue-length-based), pre-warming, and trade-offs between cost and job completion SLAs.

HardTechnical
74 practiced

New autoscaled instances are repeatedly failing health checks because the application takes more than two minutes to start up, and it's blowing your SLA. How would you redesign the autoscaling and load balancer interaction to fix this while keeping cost under control?

MediumTechnical
70 practiced

Describe, step-by-step, how a typical cloud load balancer and autoscaling group interact during a sudden traffic spike that increases requests/sec by 5x in one minute. Explain registration/deregistration, health checks, scale-out latency, connection queueing, and short-term mitigation options to avoid degraded user experience.

Unlock Full Question Bank

Get access to all Infrastructure Scaling, Capacity Planning, and High Availability interview questions and detailed answers.

Sign in to Continue

Join thousands of developers preparing for their dream job.