Infrastructure Scaling, Capacity Planning, and High Availability Questions
How to make running infrastructure scale and stay available: the operational and architectural mechanics of doing it, not the growth-modeling math behind it. Covers autoscaling policy design and Kubernetes cluster scaling (target-tracking, step, scheduled, and predictive triggers, cooldowns, warm pools, HPA, VPA, Cluster Autoscaler, GPU/node scheduling, bin-packing), load balancing algorithms and architecture (round-robin, least-connections, consistent hashing, L4 vs L7, health checks, connection draining, session affinity), horizontal versus vertical scaling choices, and high-availability and redundancy design (multi-AZ and multi-region failover, active-active vs active-passive, split-brain and leader election). Also covers turning a given demand or growth figure into concrete provisioning: headroom and safety-margin sizing, back-of-envelope instance, IOPS, and replica-count math, right-sizing, and multi-year procurement planning; validating a sizing or scaling change through load, stress, soak, and chaos testing; scaling stateful tiers such as databases, caches, and message queues; cost-aware trade-offs including reserved versus spot capacity and managed versus self-hosted infrastructure; and the observability needed to catch capacity saturation before it breaches an SLO.
Explain the role of health checks, liveness probes, and readiness probes in autoscaling and load balancing. Provide specific examples of probe implementations for services that have long startup times or that perform expensive cache warming, and explain how incorrect probes can cause unhealthy autoscaling behavior.
Describe the design and security considerations for implementing a Kubernetes custom metrics adapter that makes application metrics available to the HPA (Prometheus Adapter or similar). Discuss authentication, rate limits, aggregation, cardinality control, latency, and how to handle cases where metrics become unavailable.
Write a Python script or clear pseudocode that reads a CSV of historic batch jobs (columns: job_id, avg_runtime_seconds, avg_cpu_seconds, avg_concurrent_runs) and outputs a recommended minimum cluster vCPU count to meet a target average job latency. Explain the math and assumptions (parallelism, headroom), and provide sample output for three example jobs.
You must size a Kubernetes cluster that will run 200 pods, each requesting 0.5 CPU and 1 GiB memory. Design node types and counts, include system overhead, and show calculations including a 20% buffer for bin-packing inefficiency. Discuss use of spot instances and mixed node pools.
Walk me through how an autoscaling group or managed instance group works: what desired, minimum, and maximum capacity control, how a scaling policy decides when to add or remove instances, and why health checks, instance warm-up, and termination policy choices matter for stability. Then sketch a scaling policy you'd use for a CPU-bound web tier, and a different one for a queue-depth-driven background worker fleet.
Unlock Full Question Bank
Get access to all Infrastructure Scaling, Capacity Planning, and High Availability interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.