Scalability & Capacity Planning Questions

Framing

Monthly active users (MAU) is not the number that drives capacity, peak concurrent load is. The first job is translating MAU into an expected peak concurrency, and that ratio usually isn't constant as the user base grows: early adopters tend to be more engaged than the mainstream users who join later, so the concurrency-to-MAU ratio often drifts down as the base grows tenfold.

Staged ramp plan

I'd break the 100K to 1M MAU journey into checkpoints rather than one big jump, say roughly monthly: 100K (baseline), 400K, 700K, 1M. For each checkpoint I'd estimate peak concurrency using an engagement ratio that declines slightly as the base grows, for example 2.0% at 100K, 1.8% at 400K, 1.6% at 700K, and 1.5% at 1M:

• 100K MAU x 2.0% = 2,000 peak concurrent
• 400K MAU x 1.8% = 7,200 peak concurrent
• 700K MAU x 1.6% = 11,200 peak concurrent
• 1,000K MAU x 1.5% = 15,000 peak concurrent

That's the useful insight: MAU grows 10x but concurrent load only grows about 7.5x, because engagement dilutes as the base scales. (The opposite can also happen: a social feed where growth increases cross-user reads can make load grow faster than MAU, so I'd validate the ratio against real engagement data rather than assume it always falls.)

What I'd test before each stage

Before opening the gate to the next checkpoint, I'd load test at roughly 1.3x that stage's projected peak, using the real feature mix, and I'd specifically test the dependency that looked weakest at the prior stage rather than re-testing everything uniformly. That way each stage gate is validated ahead of where traffic will actually be, not behind it.

Managing cost so it doesn't spike ahead of growth

The failure mode to avoid is provisioning for the full 1M-MAU ceiling on day one and paying for months of unused capacity. Instead, I'd buy capacity in increments tied to each stage gate, short reservations or committed-use terms (a discount from a cloud provider in exchange for committing to a set spend or usage level over a period, similar to a reserved instance) scoped to the checkpoint rather than the full 3-month horizon, and only commit the next increment once that stage's load test confirms the forecast is holding. That keeps spend roughly tracking actual usage instead of a worst-case guess made three months early.

Why this happened

An outage from running out of capacity usually means there was no ongoing visibility into how much headroom existed and how fast it was shrinking, capacity was being assessed reactively instead of tracked as a trend. Building a practice means turning a one-time sizing exercise into a recurring process with an early-warning signal.

Estimating headroom

For whatever resource caused the outage, say it was disk space on a stateful data store, I'd start from the current usage against the provisioned ceiling and the recent growth rate. For example: 3.2TB used out of 4TB provisioned (80%), growing at roughly 60GB per week. Time to saturation = (4,000GB - 3,200GB) / 60GB per week = about 13.3 weeks of runway at the current growth rate. That runway number, not just the current percentage, is what tells you when action is actually needed.

Metrics to catch the trend early

• The utilization trend over weeks, not just the current snapshot, since a static "80% full" reading tells you nothing about urgency.
• Leading indicators specific to the resource: queue depth or connection pool usage tend to move before latency does, and latency tends to move before hard failures do.
• A runway metric (time-to-saturation) computed from the trend, so it shows up as a number of weeks remaining rather than a percentage that has to be mentally translated.

Setting safety margins

Systems rarely degrade smoothly up to 100%; most show a latency or error "knee" (the point on the utilization curve where latency or errors start climbing sharply, well before the resource is literally 100% full) well before full saturation, often somewhere around 80-90% utilization for a database's disk or connection pool. So the safety margin should target staying comfortably below that knee, not below 100%, and the trigger threshold should account for how long it takes to actually add capacity. If procurement or migration takes 3 weeks, I would not wait until 3 weeks of runway remain, I'd set the alert at something like 8 weeks of runway so there's time to review, decide, and execute before it becomes urgent.

Communicating capacity risk before it's an incident

I'd set up a recurring capacity review (say monthly) with a simple dashboard showing runway per key resource, shared with the product team, plus an automatic alert when projected runway crosses the trigger threshold. The goal is that product teams see "13 weeks of runway, trending down" as a routine planning input well before it becomes "we're out of disk and the service is down."

Metrics to track to forecast the spikes

• Request rate over time, to see the actual shape and onset speed of each spike, not just its peak magnitude.
• CPU utilization trend as a spike approaches, to confirm compute is tracking the traffic curve as expected.
• Latency percentiles (p95/p99, meaning the response time that the slowest 5% and 1% of requests exceed), an early sign of saturation before outright failures appear.
• Queue length or backlog depth, which for a data-serving fleet is often the earliest leading indicator: a queue growing faster than it's draining signals the fleet can't keep up before latency or error rates show it.

Pre-warming versus reserving on standby

Because these are recurring, predictable seasonal spikes with historical data behind them (unlike a one-off event with no track record), I'd favor pre-warming ahead of the known window over holding idle standby capacity year-round: scale up and warm caches shortly before the historically observed onset time, rather than paying for capacity that sits unused most of the year. Reserving idle standby makes more sense when the timing itself is unpredictable and there's no reliable lead-time signal to pre-warm against.

Cost versus availability risk, worked example

Say the baseline fleet costs $15K/month. Holding the full 10x capacity in standby year-round would add roughly $80K/month, about $960K/year. Pre-warming that same capacity only for the known spike windows, say 6 weeks total across the year, costs closer to $80K x (6/4.33), where 4.33 is the average number of weeks in a month (52 weeks / 12 months), which is about $110K for the year. Pre-warming into known windows is dramatically cheaper, but it only works if the timing is predictable enough to trust and there's enough lead time to actually warm the fleet (populate caches, ramp connection pools) before the spike arrives.

The availability risk to watch for

The main risk with pre-warming is being late: if warming starts even a little after the spike's real onset, the fleet hits the spike cold, and for a data-serving fleet specifically, a backlog that starts forming under a 10x load compounds quickly rather than draining. I'd build in a safety lead-time buffer, starting the warm-up a fixed amount of time before the earliest historically observed onset, not the average onset, so a slightly-early spike doesn't catch the fleet unprepared.

From SLOs to a headroom target

A service level objective (SLO) of 99.99% availability implies an error budget, the amount of allowed failure before the SLO itself is breached:

Allowed downtime = (1 - 0.9999) x 30 days x 24 hours x 60 minutes

That works out to about 4.3 minutes per month. Intuition: four nines of availability doesn't mean zero room for error, it means a very small, explicit, spendable budget of failure per month.

For the p99 latency target (the 99th percentile response time must stay under 200ms), the key idea is that latency doesn't rise smoothly with load, it stays flat and then climbs sharply past some utilization point, the "knee." The capacity plan has to keep peak load safely below that knee, not riding right at it.

Determining headroom

Say load testing shows the knee, where p99 crosses 200ms, sits at about 75% of the fleet's tested ceiling. Operating right at 75% is fragile: any small load variance breaches the SLO immediately. So I'd target sustained peak utilization well below that, say 55-60% of ceiling, treating the 75% point as a true emergency reserve rather than the normal operating target.

Using the error budget to know when it's too close

I'd track how much of the monthly error budget gets consumed specifically by capacity-related incidents (isolated from other causes like deploy bugs), and watch the burn rate, not just the total consumed so far. Worked example: 10 days into a 30-day month, capacity-linked errors have already burned 2.5 of the 4.3-minute budget. At that rate, the full budget would be exhausted by around day 17, well before month end. That trend, not the raw "2.5 of 4.3 minutes used" snapshot, is the actual trigger to add headroom immediately.

Communicating to stakeholders who only care that the SLO is met

I'd skip the RPS and CPU numbers and lead with SLO risk directly: "at current headroom, we estimate an elevated chance of breaching the 99.99% target under this quarter's expected peak growth; closing that gap costs about $X per month." I'd frame under-provisioning explicitly as a decision to accept a stated increase in SLO-breach risk, since stakeholders who only care that the SLO holds need to see that not funding headroom is itself a choice with a cost, just a hidden one.

Headroom sizing

Say baseline traffic is 2,000 requests per second (RPS); 10x gives a forecast peak of 20,000 RPS. Because this is a one-off event with limited prior data for this specific sale, I'd add a larger buffer than I would for a recurring, well-understood pattern, say 30% for forecast uncertainty, putting the target ceiling at about 26,000 RPS.

Hold-ready versus scale-into-it

For a 72-hour hard deadline with no room to react mid-event, I'd hold the full 26,000 RPS capacity ready in advance rather than plan to scale into it reactively. The reasons: there's essentially zero time to safely validate new capacity once the sale is live, and retail checkout paths usually have state that needs warming, caches, database connection pools, CDN edge caches, which can't be spun up instantly under load. This is different from a slow, sustained growth scenario, where reactive scaling is fine because there's time to react.

Cost versus risk

Holding the extra 26,000 RPS-capable capacity ready for about a week around the event, covering setup, the sale itself, and teardown, might cost on the order of a few thousand dollars in extra infrastructure spend. Against that, a marquee 72-hour sale falling over mid-event risks a large multiple of that in lost revenue and reputational damage, on a fixed, unrepeatable window (unlike ongoing growth, where over-provisioning forever really would be wasteful). The asymmetry between a small, bounded extra cost and a large, irrecoverable failure cost is what justifies over-provisioning for this specific case.

Three measurable commitments I'd propose to the client

1. A latency commitment: for example, checkout p99 latency (the slowest 1% of checkout requests) stays under 500ms up to 22,000 RPS, comfortably inside the tested 26,000 RPS ceiling.
2. An error-rate commitment: error rate stays under 0.1% for the full 72-hour window.
3. A headroom commitment as a business metric: at least 15% spare capacity above observed peak traffic is maintained throughout the event, with an internal alert if it drops below 10%, so there's a documented safety margin the client can see, not just a promise.