Infrastructure Scaling, Capacity Planning, and High Availability Questions

How to make running infrastructure scale and stay available: the operational and architectural mechanics of doing it, not the growth-modeling math behind it. Covers autoscaling policy design and Kubernetes cluster scaling (target-tracking, step, scheduled, and predictive triggers, cooldowns, warm pools, HPA, VPA, Cluster Autoscaler, GPU/node scheduling, bin-packing), load balancing algorithms and architecture (round-robin, least-connections, consistent hashing, L4 vs L7, health checks, connection draining, session affinity), horizontal versus vertical scaling choices, and high-availability and redundancy design (multi-AZ and multi-region failover, active-active vs active-passive, split-brain and leader election). Also covers turning a given demand or growth figure into concrete provisioning: headroom and safety-margin sizing, back-of-envelope instance, IOPS, and replica-count math, right-sizing, and multi-year procurement planning; validating a sizing or scaling change through load, stress, soak, and chaos testing; scaling stateful tiers such as databases, caches, and message queues; cost-aware trade-offs including reserved versus spot capacity and managed versus self-hosted infrastructure; and the observability needed to catch capacity saturation before it breaches an SLO.

HardTechnical
64 practiced

Explain how a service mesh (e.g., Istio, Linkerd) interacts with and augments traditional load balancing. Describe sidecar responsibilities versus ingress controller, L7 routing and retries, circuit-breaking primitives, observability improvements, and how mesh telemetry can feed autoscaling decisions. Discuss the latency and operational complexity trade-offs.

MediumTechnical
66 practiced

Describe the types of health checks a load balancer should support (e.g., TCP, HTTP(s)/gRPC probe, application-level heartbeat). Explain how you choose probe interval, timeout, and unhealthy/healthy thresholds for a service with average 200ms request latency and occasional 2s background jobs. Include trade-offs (false positives vs detection speed).

MediumTechnical
72 practiced

Describe how you would implement connection draining for a backend being taken out of service: stopping new connections, notifying active clients gracefully, and closing remaining connections after a timeout. What concurrency and race conditions would you need to handle, and how would this integrate with the load balancer's health checks?

HardSystem Design
75 practiced

Architect a system that needs to handle promotional traffic spikes 100 times its baseline for short bursts, without keeping 100x capacity provisioned around the clock. Walk me through your approach, and how you'd validate it actually holds up before the real spike hits.

HardTechnical
76 practiced

A system stores 5 TB of transactional data, experiences 1,000 writes/sec and 10,000 reads/sec. Propose instance sizing for primary and read replicas, estimate IOPS and storage performance requirements, and provide a monthly cost estimate. State assumptions about instance throughput and storage performance.

Unlock Full Question Bank

Get access to all Infrastructure Scaling, Capacity Planning, and High Availability interview questions and detailed answers.

Sign in to Continue

Join thousands of developers preparing for their dream job.