Capacity Planning and Resource Optimization Questions

Covers forecasting, provisioning, and operating compute, memory, storage, and network resources efficiently to meet demand and service level objectives. Key skills include monitoring resource utilization metrics such as central processing unit usage, memory consumption, storage input and output and network throughput; analyzing historical trends and workload patterns to predict future demand; and planning capacity additions, safety margins, and buffer sizing. Candidates should understand vertical versus horizontal scaling, autoscaling policy design and cooldowns, right sizing instances or containers, workload placement and isolation, load balancing algorithms, and use of spot or preemptible capacity for interruptible workloads. Practical topics include storage planning and archival strategies, database memory tuning and buffer sizing, batching and off peak processing, model compression and inference optimization for machine learning workloads, alerts and dashboards, stress and validation testing of planned changes, and methods to measure that capacity decisions meet both performance and cost objectives.

EasyTechnical

24 practiced

Describe best practices for setting resource requests and limits for CPU and memory in Kubernetes. Explain how requests and limits affect scheduler decisions, QoS classes, and OOM kill behavior. Provide a practical step-by-step approach to determine initial request/limit values for a new microservice in production.

Sample Answer

Best practices — short version:- Requests = what scheduler uses to place a pod; Limits = runtime cap enforced by cgroups.- CPU: exceeding limit is throttled (not killed). Memory: exceeding limit causes OOM kill.- Scheduler only considers requests. To get Guaranteed QoS, set request == limit for both CPU and memory. Request < limit → Burstable. No requests → BestEffort.- Prefer realistic requests per-pod so the scheduler packs nodes efficiently and avoids eviction cascades.

How requests/limits affect behavior:- Scheduling: Kubernetes sums requests per node; pod is scheduled only if node has enough free requested resources. Limits are ignored by scheduler.- QoS classes: Guaranteed (request==limit for all containers) → highest eviction priority. Burstable (has requests) → middle. BestEffort (no requests) → lowest.- OOM: If a container uses > memory limit, kernel OOM kills that container. If node runs out of memory, system considers QoS and requests to pick victims (BestEffort then Burstable; Guaranteed least likely).

Practical step-by-step to determine initial values for a new microservice:1. Local dev profiling: run representative workloads locally or in a dev cluster; capture CPU/memory usage over time with metrics (p50/p95/p99).2. Baseline request: set requests to the p95 steady-state usage observed (e.g., 200m CPU, 150Mi memory) so normal scheduling is safe.3. Limit margin: set limits to 1.5–2× requests (or to p99 peak) for CPU; for memory be conservative—prefer smaller multiples because memory > limit ⇒ OOM. Example: request=200m / limit=400m; request=150Mi / limit=300Mi.4. QoS strategy: if you need the pod protected from eviction, set request==limit (Guaranteed); otherwise use Burstable.5. Load-test: run stress tests / soak tests; validate behavior under peak. Observe throttling and OOMs.6. Autoscaling & safety: enable HPA/VPA where appropriate; leave node/system reserved capacity (kube-reserved/system-reserved) in cluster sizing.7. Monitor & iterate: collect real metrics (Prometheus), alert on high throttling, high OOM kills, or frequent evictions; refine requests/limits periodically.

Tips:- Prefer conservative requests to avoid node pressure, but not so high that bin-packing becomes inefficient.- Use VPA for memory tuning in production (mode: recommend or auto) and HPA for CPU/requests scaling.- Document assumptions and add alerts for OOMKilled and CPU throttling (container_cpu_cfs_throttled_seconds_total).

EasySystem Design

21 practiced

Design a minimal set of dashboards and alerts for capacity planning of a stateless web service that must meet latency and availability SLOs. List the key metrics (including percentiles and trend indicators), suggested threshold rules tied to SLOs or capacity buffers, and how you would surface these dashboards to on-call teams to support operational decisions.

Sample Answer

Requirements (assume stateless service with latency SLO 95% < 200ms, availability SLO 99.9%): monitor latency, errors, traffic, and capacity/saturation.

Minimal dashboards (3):1) User-facing SLO dashboard- Key metrics: request rate (RPS) with 1m/5m trend, success rate (HTTP 2xx %) & error rate (4xx/5xx) with rolling 1h/24h, latency percentiles: p50/p95/p99 + histogram heatmap, error budget burn rate (30m/1h).- Threshold rules: page if success rate breaches SLO (availability) or p95 latency > 200ms for 5m; page on sustained error budget burn > 1x for 30m.

2) Capacity / autoscaling dashboard- Key metrics: RPS, concurrency, average CPU, memory, request queue length, pod/instance count, autoscaler target & actual, GC/pause times.- Thresholds: warn when CPU > 60% or concurrency per instance > 70% for 10m; page if CPU > 85% or queue length rising and autoscaler not scaling for 5m. Tie thresholds to capacity buffer (e.g., maintain 25–30% spare capacity).

3) Infrastructure health & degradation dashboard- Key metrics: host/container restart rate, OOMs, thread pool exhaustion, downstream latency/errors, network errors, deployment/rollout status.- Thresholds: page on host restarts > X/hour, OOM > 1 per 15m, or downstream SLA breaches impacting >5% of requests.

Alerting design & rules- Alert types: P1 (page) for SLO breaches, autoscaler failure, infra failure; P2 (notify) for approaching buffers.- Use multi-condition alerts combining symptoms (e.g., p95 latency > SLO AND CPU high) to reduce noise.- Use burn-rate alerts: page when burn rate > 2x and error budget remaining < 25%.

Surface to on-call teams- Place SLO dashboard as primary “on-call home” in the incident tool and link in pager alerts.- Attach concise runbooks to each alert with probable causes, quick checks, and remediation steps (scale up, rollback, clear queues).- Configure alert routing: immediate paging to on-call SRE, Slack channel for ops context, and include dashboard links and last 30m snapshots in alert payload.- Weekly capacity digest: automated report of trends (growth %, capacity headroom) to inform scaling and purchase decisions.

Rationale: three focused views cover user impact, capacity mechanics, and infrastructure causes; percentiles capture tail latency; trend windows and capacity buffers prevent reactive firefighting.

MediumTechnical

30 practiced

Explain how to translate SLOs into concrete capacity targets. Given a requirement of 99.95% availability and p95 latency under 100ms, explain how you'd set CPU/memory headroom, instance counts, redundancy levels (N+1/N+2), and load balancing choices to meet SLOs including failure scenarios such as instance or AZ loss.

Sample Answer

Start from SLO → allowed error budget, then convert to capacity targets.

1) Quantify SLOs:- 99.95% availability → 0.05% downtime ≈ 21.6 minutes/month. That’s total budget for all failures.- p95 < 100ms → dimension for tail capacity under peak load (not just mean).

2) Baseline capacity math (example):- Measure: peak QPS = 10,000. Measured per-instance p95 capacity = 2,500 QPS while keeping p95 <100ms.- Needed nominal instances = ceil(10,000 / 2,500) = 4.- Add runtime headroom (to absorb spikes, GC, noisy neighbors): 25–50%. Use 30% typical → scale target = 4 * 1.3 = 5.2 → round up to 6 instances.

3) Redundancy for failures:- Use N+1/N+2 depending on risk. For 99.95% with multi-AZ, N+1 often sufficient; for higher blast-radius or stateful services use N+2.- In example: choose N+1 → provision 1 extra: 6 → +1 = 7 instances total. Practically run 6 active, 1 warm spare or let autoscaler maintain min=6 and target=7.

4) AZ distribution and failure scenarios:- Prefer at least 3 AZs. Spread instances evenly (7 → 3,2,2). Losing one AZ removes ~33% capacity; with headroom you still have > required capacity (example: if AZ with 3 lost, remaining 4–5 instances handle 10k QPS if each can be slightly overloaded or autoscale).- Plan autoscaling policies (CPU/RPS-based) with aggressive cool-downs and predictive scaling for planned peaks.

5) CPU/memory headroom:- Reserve 25–50% headroom per instance; set requests/limits (k8s) so that eviction/GC doesn’t push p95 over target.- Monitor real p95 vs CPU %; tune headroom empirically.

6) Load balancing & operational choices:- Use cross-AZ LB with health checks and fast failover.- Client-side retries + exponential backoff and circuit breakers to avoid overload during failover.- Graceful draining and connection draining windows tuned to SLOs.- Chaos tests (AZ/instance kill) to validate capacity during real failures and measure time-to-recover vs error budget.

7) Validation & iteration:- Run load tests and game days to verify p95 under AZ loss and instance loss.- Convert observations back to headroom/instance counts and adjust autoscaling thresholds.

Key principle: translate SLO → required capacity under peak, add empirical headroom, distribute across AZs, choose N+1/N+2 by risk, and validate with tests so total downtime stays within error budget.

HardSystem Design

23 practiced

Design capacity and scaling for services constrained by strict data-locality regulations: some user data must remain in region X while traffic originates globally. Explain how to place compute and storage, what replication patterns are allowed, how to handle global traffic routing, capacity duplication costs, and how to provision buffer capacity in region X to meet SLOs while complying with regulations.

Sample Answer

Requirements & constraints:- Functional: global traffic can access services, but any data classified as “region X” (PII, regulated) must be stored and processed only within region X.- Non-functional: SLOs (p99 latency, availability), cost limits, disaster recovery, and regulatory auditability.

High-level architecture:- Global ingress (CDN + global LB) → Edge region(s) for stateless handling → Region-aware routing to compute/storage.- Two-tier compute: stateless frontends globally; stateful/regulated processing only in region X.- Storage: authoritative data stores for region-X data live exclusively in region X. Use read-only replicas elsewhere only if regulation permits; otherwise only metadata or tokenized references leave region X.

Allowed replication patterns:- Authoritative master and synchronous or async replicas that remain inside region X.- Cross-region replication only of anonymized/tokenized or aggregated data with explicit regulatory approval.- For disaster recovery, maintain cold backups (encrypted) in approved zones per policy.

Global traffic routing:- Use global LB (GeoDNS + Anycast) to route to nearest frontend. Frontend determines whether request touches region-X data; if so, route the request (via internal network/vaulted API) to compute endpoints in region X. Use sticky routing or session tokens to minimize cross-region hops.- Use service mesh with region tags and circuit breakers to route to region X endpoints and failover gracefully.

Capacity & duplication cost strategy:- Stateless layers: fully global, scale independently.- Stateful/regulatory compute+storage: must be duplicated in region X capacity; this is unavoidable. Optimize by: - Right-sizing with autoscaling + warm pools. - Using burstable instances and spot capacity for non-SLO-critical workloads. - Sharing non-sensitive services across regions where allowed.

Provisioning buffer capacity in region X:- Define SLO-driven capacity model: Capacity = peak QPS * (p99 processing time) * safety factor.- Use demand forecasting (traffic trends, seasonality) + tail-latency margin; set buffer (e.g., 20–40% headroom) to cover spikes and instance warm-up.- Maintain warm standby instances (pre-warmed containers) and an autoscaler with predictive scaling (using scheduled scale-ups for known events and ML-based forecasts).- Implement burst queues and backpressure: accept requests at edge, enqueue or degrade non-critical features if region-X compute is saturated; surface graceful degradation to clients.- Test with chaos and load testing to validate headroom and autoscaling thresholds.

Operational controls & compliance:- Strict IAM, network isolation (private subnets), audited key management within region X.- Monitoring: per-region SLO dashboards, synthetic tests from global vantage points, alerting for region-X saturation, cross-region traffic leakage.- Runbooks: automated failover limited by regulation; legal-approved DR playbooks.

Trade-offs:- Higher cost in region X vs lower latency and compliance. Use caching (non-sensitive) at edges, tokenization, and careful partitioning of workload to minimize region-X compute needs while preserving SLOs.

This approach balances compliance, cost, and reliability through region-isolated state, global stateless frontends, predictive capacity planning, and operational guardrails.

MediumSystem Design

21 practiced

Outline a plan to stress-test a service before executing a planned capacity reduction of 30% fewer instances. Which test scenarios (spike, soak, step), what metrics to collect (latency percentiles, error rates, queue length, CPU steal), success criteria, and automated rollback steps would you include? Discuss trade-offs between testing in staging vs production.

Sample Answer

Requirements: verify service behavior when instance count is reduced by 30% without violating SLOs, identify regressions (latency, errors, saturation), and enable safe automated rollback.

Test plan (scenarios):- Step (gradual): reduce traffic or instances in 5% increments until -30%. At each step hold 5–10 mins to observe metrics. Purpose: find threshold where degradation appears.- Spike: short high-concurrency bursts (2–5x normal RPS for 1–3 minutes) on the reduced-capacity configuration to surface queuing and throttling hot paths.- Soak: run steady production-like load at -30% capacity for 1–4 hours to catch memory leaks, connection exhaustion, and CPU drift.

Metrics to collect (per instance & aggregate):- Latency percentiles: p50/p90/p95/p99, and tail (p999 if possible)- Error rates: 4xx/5xx per minute, cbreaker/trip counts- Throughput: requests/sec and success/sec- Queue/backlog lengths: request queues, message queues, connection queue- Resource: CPU, mem, disk I/O, network, file descriptors- CPU steal/wait (iowait/steal) and GC pause times- Infrastructure: pod restarts, OOMs, thread pools exhausted, connection pool saturation- Service-level: SLO error budget burn rate, downstream latency amplification- Business metrics: conversion rate, UX timeouts

Success criteria:- SLOs maintained (e.g., p95 < target, error rate < threshold) and no sustained resource saturation- No increase in incidents: no pod crashes, <X% increase in retry/error rate (define X)- Queue length returns to baseline after spikes- No cascading downstream failures

Automated rollback / safety:- Pre-flight: snapshot current autoscaling and deployment config, enable circuit breakers and rate-limits, ensure runbook exists.- Canary: deploy capacity reduction to a small subset (e.g., 10% of clusters) first, run tests.- Automated checks: run health probes and compare chosen metrics vs thresholds for a monitoring window (e.g., 2 mins). If any threshold breached (p95 latency increase >20% AND error rate >2% absolute), trigger rollback.- Rollback steps (automated): revert instance count/config via IaC or orchestration API; scale to previous replica count; mark canary unhealthy; notify on-call; run post-rollback validation to confirm metrics recover.- Safety nets: disable reduction if cluster-level autoscaler intervenes; set short cooldowns and allow horizontal autoscaling.

Trade-offs: staging vs production- Staging pros: safe, repeatable, controlled; can push extreme scenarios without customer impact. Cons: traffic/profile differences, missing noisy neighbors, smaller scale, different infra.- Production pros: realistic, exposes true downstream impacts and shared infra behavior (noisy neighbors, real traffic patterns). Cons: risk to users; must use canaries, throttled tests, strict automated rollback and business-hour scheduling.Recommended approach: iterate in staging to validate the plan, then perform controlled canary tests in production during low-traffic windows with automated rollback and observability guardrails.

Unlock Full Question Bank

Get access to hundreds of Capacity Planning and Resource Optimization interview questions and detailed answers.

Join thousands of developers preparing for their dream job.