Infrastructure Scaling, Capacity Planning, and High Availability Questions

Steps and data sources

1. Gather historical usage trends. Pull 12-24 months of infrastructure utilization and traffic data (compute, storage, request volume) as the trend baseline; a short history understates real seasonality.
2. Fold in business signals. Pull sales/growth projections, planned feature launches, and expected customer-count growth from product and sales leadership, since infrastructure demand tracks business growth, not just its own historical trend.
3. Account for seasonality. Identify recurring patterns (end-of-quarter, holiday season, back-to-school) from the historical data and layer them onto the trend rather than smoothing them away, a flat trend line understates true peak demand.
4. Account for event-driven spikes. Separately track known one-off drivers (a planned marketing campaign, a major customer onboarding) that won't show up in historical trend data at all.
5. Combine into a forecast with a confidence range, not a single number, since business projections and infrastructure trends both carry real uncertainty that compounds over a 3-year horizon.

Documenting assumptions and confidence ranges

For each input (growth rate, seasonality multiplier, event-driven spike size), document the assumption explicitly, its source (measured historical data versus a business projection versus a guess), and a rough confidence level. State the forecast itself as a range (for example, "P50 estimate: X; P90 estimate: 1.4X", where P50 is the median or expected-case estimate, and P90 is a more conservative estimate that actual demand exceeds only 10% of the time, the same percentile idea used for latency elsewhere, just applied to a demand forecast instead of a response time) rather than a point figure, and note which specific assumption the range is most sensitive to, so a reviewer knows what to double check first. Revisit and update the forecast on a fixed cadence (quarterly is common) as real data replaces projections.

Turning the forecast into a phased, defensible procurement plan

• Break the 3-year forecast into phases (for example, 6-month increments) rather than committing to a single 3-year capacity purchase upfront, since committing far in advance against an uncertain forecast concentrates risk.
• Tie each phase's spend to a re-validation checkpoint: before committing phase 2's budget, check phase 1's actual usage against the forecast and adjust.
• Present the budget to finance with the range and its drivers attached, not just a bottom-line number, "we're requesting budget for the P50 case, with a described trigger and pre-approved path to the P90 case if growth outpaces plan" is a far more defensible ask than a single committed figure that either wastes money (if growth is slower) or requires an emergency request later (if growth is faster).
• Include risk buckets in the plan explicitly: what's committed regardless of outcome (baseline growth), what's contingent on a specific trigger (a named event or growth threshold), and what's speculative headroom that can be deferred if unneeded.

Proactive versus reactive triggers for this same exercise

This forecasting and procurement-planning method applies whether it's run proactively, as part of normal annual/quarterly planning, or reactively, triggered by a capacity-exhaustion incident that made clear the existing forecast was wrong. The reactive case adds one extra step at the front: before re-forecasting, diagnose why the prior forecast missed (an unmodeled business driver, an underestimated seasonality effect, or simply stale data), so the corrected forecast doesn't repeat the same blind spot, and be prepared to present the same cost-versus-risk trade-off to finance and product under more time pressure than the proactive version would carry.

What each controller actually decides

• HPA (Horizontal Pod Autoscaler) decides how many replicas of a pod to run, reacting to a metric like CPU utilization.
• VPA (Vertical Pod Autoscaler) decides how big each individual pod should be, adjusting its CPU and memory requests based on observed usage.
• Cluster autoscaler decides how many nodes exist, adding nodes when pods can't be scheduled and removing underutilized ones.

Where each one addresses, or misses, this failure mode

HPA handles the variable CPU load well on its own: standard target-tracking scaling adds replicas as CPU rises. But HPA does nothing about memory pressure inside a single pod. If a pod is genuinely undersized (or leaking), adding more replicas doesn't stop any individual pod from hitting its memory limit and getting OOMKilled (Out Of Memory Killed, the kernel/kubelet terminating a container for exceeding its memory limit), more copies of an undersized pod are still undersized.

VPA can fix that by right-sizing memory requests from observed usage, but it has a real gap here too: VPA's default Recreate update mode restarts a pod to apply a new size, so tuning memory this way causes churn. More importantly, Kubernetes explicitly warns against running HPA and VPA on the SAME metric at the same time (for example both reacting to CPU), because they fight each other: VPA resizes the pod, which changes utilization percentage, which changes what HPA sees, causing oscillation.

The cluster autoscaler doesn't address either problem directly, it only reacts once pods are already unschedulable due to insufficient node capacity, it can't fix an undersized pod or decide replica count.

Combined approach

Split responsibility so nothing fights over the same signal:

• Run VPA in recommendation mode (updateMode: "Off" or "Initial") targeting memory only, and periodically apply its recommended memory requests to the deployment in a controlled rollout, rather than letting VPA auto-restart pods live.
• Run HPA on CPU utilization (and a custom latency metric if available) for replica count, since that's the axis HPA is actually good at.
• Let the cluster autoscaler respond to whatever scheduling pressure that combination produces, adding nodes when the right-sized, correctly-counted pods don't fit, removing them when they're not needed.

Set memory limits comfortably above VPA's recommended request (not equal to it) to avoid throttling/OOM on legitimate short spikes, and put a PodDisruptionBudget on the deployment so a VPA-driven rollout, or an HPA scale-down, never drops availability below what the service needs.

Framing

"Concurrent users" here means people with an active session hitting the service at the same moment, not total registered users. For a launch expecting 1,000 concurrent users, a reasonable starting assumption is that this maps to somewhere around 50 to 150 requests per second at the API tier, depending on how chatty the app is (a dashboard that polls every few seconds looks very different from a mostly-read content site). State that assumption explicitly to stakeholders rather than treating 1,000 as a request-rate number.

12-week timeline

Weeks 1-4, baseline and sizing. Define the SLOs (Service Level Objectives, the target thresholds you commit to, e.g. p99 latency, the value below which 99 percent of requests complete, under 300ms) for the launch. Build a back-of-envelope capacity model from the 1,000-concurrent-user target (requests/sec per tier, DB connections, cache size). Identify the riskiest component (usually the database or a third-party dependency) and size it first.

Weeks 5-8, build and load test. Stand up the environment at the sized capacity. Run load tests ramping from 0 to 2 to 3 times the 1,000-user target, not just to the target itself, since launch day traffic is a forecast with real error bars. Fix bottlenecks found (connection pool exhaustion, N+1 queries, an unrelated database term for an ORM issuing one extra query per row instead of one batched query, not the redundancy N+1 used above, and missing indexes). Run a failover drill: kill an instance under load and confirm the system self-heals within your SLO.

Weeks 9-12, validate and go-live readiness. Run a full dress rehearsal (a "game day") that simulates launch conditions end to end, including the marketing push if there is one. Freeze non-essential changes in the final week. Confirm on-call rotation, runbooks, and rollback procedure are in place. Go/no-go review with stakeholders 3-5 days before launch.

Buffers (safety margins)

• Sizing buffer: provision for roughly 2x the 1,000-user estimate at minimum, and closer to 3x if the launch has an unpredictable marketing driver (a press mention, a paid campaign with a specific start time). New-product launch estimates carry much more uncertainty than steady-state forecasts, so the buffer should be larger here than you would carry for an established service.
• Redundancy buffer (separate from sizing buffer): run N+1 instances per tier at minimum, so a single instance failure does not eat into your sizing headroom.
• Time buffer: keep 3-5 days at the end of the 12 weeks unscheduled, so a bad load-test result in week 10 does not force cutting the rehearsal.

Pre-launch validation activities

• Load testing to the buffered target (2-3x), not just the raw estimate.
• Chaos/failover drills (kill an instance, kill a dependency, verify graceful degradation).
• Monitoring and alert validation: trigger each alert deliberately and confirm someone gets paged.
• A launch-day rehearsal covering the actual runbook, not just the infrastructure.
• Rollback rehearsal: time how long a rollback actually takes, since "we can roll back" is only true if it's been practiced.

Stakeholders to align with

• Product/marketing: they know the real traffic driver (press, ads, an email blast) and its timing, which capacity planning depends on.
• Finance/budget owner: headroom capacity costs real money; get sign-off on the buffer level rather than surprising them with the bill.
• On-call/SRE: need the runbook and escalation path finalized before launch, not during it.
• Customer support: needs a status-page and communication plan for the (plausible) case of a launch-day incident.
• Executive sponsor: owns the final go/no-go decision, ideally 3-5 days out with time to react.

CPU

SLI (service level indicator, the actual measured value): average and p95 (the value below which 95 percent of samples fall) CPU utilization per instance over a rolling 5-minute window.
Why it helps: it's the most direct signal of how close a compute-bound service is to needing more capacity, and it's what most autoscaling policies key off.
Alert or dashboard: alert when p95 CPU exceeds 75% sustained for 15 minutes, leaving headroom before saturation actually degrades latency; dashboard it as a fleet-wide heatmap over time so uneven load across a few hot hosts is visible separately from a genuine capacity shortage.

Memory

SLI: resident memory usage as a percentage of instance memory, plus, for garbage-collected runtimes like the JVM, time spent in garbage collection per minute.
Why it helps: memory exhaustion causes a hard failure (an out-of-memory kill) rather than gradual degradation, so it needs a different response than CPU. Memory problems are usually a leak or a workload-size regression, not something you autoscale away.
Alert or dashboard: alert at 85% memory utilization, before the OS starts swapping or the kernel's out-of-memory killer engages, and separately alert if garbage-collection time exceeds 10% of wall-clock time; dashboard it as a trend line per instance to catch slow leaks a point-in-time snapshot would miss.

Storage I/O

SLI: disk read and write latency (p95, in milliseconds) and IOPS (input/output operations per second) as a percentage of the volume's provisioned IOPS limit.
Why it helps: storage capacity planning is really about staying under a provisioned ceiling, since most cloud block storage has a hard IOPS or throughput limit, so "I/O latency spiking" usually means you're hitting that ceiling, not that the disk failed.
Alert or dashboard: alert when IOPS utilization exceeds 80% of the provisioned limit for 10 minutes; dashboard p95 I/O latency next to IOPS utilization so it's visible exactly when latency starts climbing as utilization approaches the ceiling.

Network throughput

SLI: bytes per second in and out as a percentage of the instance's or interface's provisioned bandwidth cap.
Why it helps: bandwidth-bound services, like large file transfer or video, can be fully healthy on CPU, memory, and disk and still degrade because they're capped on the network, so this metric catches a capacity limit the other three would miss entirely.
Alert or dashboard: alert at 70% of the provisioned bandwidth cap, earlier than the 75 to 85% range used across CPU, memory, and storage I/O above (75% for CPU, 85% for memory, 80% for storage I/O), since network saturation degrades sharply near the limit; dashboard it as a stacked area chart of inbound vs outbound traffic per instance type, since asymmetric patterns often reveal a specific workload driving the need.

Practical note on collection

A concrete example: emit all four via a host agent at 60-second resolution, retained at full resolution for 15 days and downsampled to 5-minute or hourly aggregates beyond that for long-term capacity trend analysis.

Framing

The ask here is explicitly upstream of load testing: before spending money provisioning for a 3x forecast, first find out whether the forecast itself is trustworthy, and whether the answer is even "add capacity" at all, the launch might shift traffic MIX rather than volume, or existing headroom might already cover it.

Validating the forecast itself

• Compare the forecast's methodology against a comparable past launch if one exists. If a similar past feature launch predicted 2x and actually delivered 1.3x, that's real evidence to widen or discount the current 3x forecast's confidence, not just trust the point estimate at face value.
• Break the 3x forecast into its components: is it 3x total requests, 3x users making the same per-user request volume, or the same users making 3x more requests each? Each has different infrastructure implications, for example 3x users usually scales roughly linearly across the system, but 3x per-user activity might concentrate load on one specific endpoint or database table rather than scaling everything uniformly.
• Run a smaller, controlled real-world test where possible: a percentage rollout or a beta cohort exposed to the new feature ahead of full launch, to get an actual measured multiplier from real traffic instead of relying purely on a forecast model.

Measurements to build confidence

Instrument the beta or percentage-rollout cohort's actual traffic multiplier and compare it against the 3x forecast in real time as rollout percentage increases. Separately, confirm CURRENT headroom, if the system already runs at 30% average utilization with room to 3x before hitting any known ceiling, the forecast may not require new provisioning at all, though that conclusion should be checked against PEAK utilization, not average, since peak is the number that actually matters for headroom.

Deciding whether additional capacity is actually needed

Set the decision threshold explicitly, ahead of time, for example: "if the beta cohort's measured multiplier plus a safety margin exceeds current peak headroom, provision ahead of full launch; if it's within headroom, monitor and scale reactively instead." This avoids two common failure modes: provisioning for every forecast regardless of whether current headroom already covers it (wasted cost), and always waiting for reactive autoscaling on a launch-day traffic profile whose ramp speed reactive autoscaling's own reaction time genuinely can't keep up with.