Cloud Cost Optimization and FinOps Questions
Controlling and optimizing cloud spend: cost modeling and forecasting, rightsizing, reserved capacity and savings plans, autoscaling for cost, tagging and chargeback, and the FinOps operating model. Covers building the business justification for infrastructure spend and continuously driving efficiency at scale without sacrificing reliability. Cost as a first-class architectural concern.
What does 'rightsizing' mean for cloud compute, and what monitoring signals would tell you an instance is overprovisioned? Name one signal that looks convincing but can actually be misleading, and explain why.
Sample Answer
Direct answer
Rightsizing means matching an instance's provisioned CPU, memory, and I/O capacity to what the workload actually uses, so you're neither overpaying for idle headroom nor risking a service-level breach from underprovisioning. The signal that looks convincing but is actually misleading is a low average CPU utilization number by itself: averaged over a wide window, it can hide a workload with a sharp, business-critical peak that genuinely needs the capacity the average makes look wasted.
Structured elaboration
Signals that indicate genuine overprovisioning
- Sustained low CPU utilization (for example, consistently under 15%) across weeks, not just a quiet day, with request latency and error rates staying flat, meaning the extra CPU headroom isn't being called on even under normal load variation.
- Low memory usage with no swap activity and no out-of-memory events, indicating the instance could run on a smaller memory tier without risking a crash under load.
- Low disk I/O and network throughput relative to the instance's provisioned limits, with no I/O wait or throttling observed, indicating storage and network capacity are also oversized.
The misleading signal, and why
A low average CPU utilization computed over a long window (a week or a month) can average away a real, recurring peak: a batch job that runs at 90% CPU for two hours every night looks like "8% average utilization, clearly oversized" if you only look at the mean. Downsizing based on that average would cause the instance to fail or badly degrade during the exact two hours it matters most. The fix is to look at the P95 or P99 (95th or 99th percentile) utilization over the same window, not just the mean, and to check for scheduled or bursty jobs explicitly before resizing anything with a suspiciously low average.
How to act on the signals safely
Downsize in one step at a time, not straight to the theoretical minimum, and monitor for a full peak cycle (including any weekly or monthly batch jobs) before taking the next step down. Keep enough headroom above the P95, not the mean, to absorb normal variance and the occasional unplanned spike, and treat anything customer-facing or on the request path more conservatively than an internal batch worker, since the cost of underprovisioning a batch job (it runs slower) is much cheaper than the cost of underprovisioning a user-facing service (it errors out).
Worked example
An instance shows a 4-week average CPU utilization of 9%, memory averaging 22%, and no I/O throttling, which reads as a clear rightsizing candidate on the average alone. Pulling the P95 utilization for the same window shows CPU at 78%, driven by a nightly reconciliation job that runs for roughly 90 minutes each night at high CPU. The 9% average was real, but it was averaging 22.5 hours a day of near-idle time against 1.5 hours a day of near-saturation, and a downsize based on the average alone would have made that nightly job fail or run far past its window. The correct rightsizing move here is not "downsize," it's "keep capacity sized to the P95 peak, and consider whether the nightly job should run on separate, right-sized capacity instead of sharing the always-on instance," which is a different fix than the average suggested.
Trade-offs and pitfalls
The main pitfall is exactly the one this question is testing: resizing off a mean utilization number without checking for a P95/P99 tail or a scheduled workload hiding inside the average. A second is resizing too aggressively in one step and finding out only under the next real traffic spike that there's no headroom left, which turns a cost optimization into an incident. A third is treating rightsizing as a one-time cleanup instead of an ongoing signal to monitor, since workload shape changes over time (a service that grows its user base or adds a new scheduled job needs its rightsizing baseline revisited, not assumed permanent).
Compare reserved instances, savings plans, and committed-use discounts across the major cloud providers. What is the mechanical difference between them in commitment scope, term, and flexibility across instance types, and how would you decide what percentage of a steady-state workload's capacity to commit?
Sample Answer
Direct answer
All three mechanisms trade a usage commitment for a lower price, but they commit to different things: AWS Reserved Instances (RIs) commit to a specific instance configuration, AWS Savings Plans commit to a dollar-per-hour spend level that flexes across instance types, Google Cloud committed-use discounts (CUDs) commit to either a resource quantity or a dollar-per-hour spend depending on which CUD type you buy, and Azure Reservations commit to a specific VM configuration similar to AWS RIs. The general pattern across every provider is the same trade-off: the more precisely you commit to a specific instance shape, the bigger the discount; the more flexibility you keep, the smaller the discount but the lower your risk if the workload changes shape.
Structured elaboration
Mechanism comparison
| Mechanism | Provider | Commits to | Term | Flexibility |
|---|---|---|---|---|
| Standard Reserved Instance | AWS | Specific instance family, size, region | 1 or 3 yr | Least flexible: can change availability zone and, within limits, instance size in the same family, but not family or OS |
| Convertible Reserved Instance | AWS | Instance family (exchangeable) | 1 or 3 yr | Can exchange for a different family, size, or OS during the term, at a lower discount than Standard |
| Compute Savings Plan | AWS | Dollar-per-hour compute spend | 1 or 3 yr | Most flexible: applies across instance family, size, OS, tenancy, and region, and across EC2, Fargate, and Lambda |
| EC2 Instance Savings Plan | AWS | Dollar-per-hour spend, locked to one instance family and region | 1 or 3 yr | Flexible on size and OS within that family and region only; typically a larger discount than Compute Savings Plans for the same term because it's narrower |
| Resource-based CUD | Google Cloud | A quantity of vCPUs, memory, GPUs, or similar, on Compute Engine | Typically 1 or 3 yr | Locked to the committed resource type and quantity; scope can be a single project or shared across a billing account |
| Flexible (spend-based) CUD | Google Cloud | Dollar-per-hour spend | 1 or 3 yr | Pools eligible spend across only three services, Compute Engine, Google Kubernetes Engine (GKE), and Cloud Run, similar in spirit to an AWS Compute Savings Plan in that the discount follows a dollar-per-hour spend level rather than a specific SKU. BigQuery and Cloud SQL are NOT part of this pool: each has its own separate, service-specific spend-based commitment, purchased and applied independently |
| Reserved VM Instance | Azure | Specific VM series, size, and region | 1 or 3 yr | Instance-size flexibility within the same VM size-flexibility group; can be rescoped after purchase to a subscription, resource group, shared billing scope, or management group without a new commercial transaction |
All four providers offer some form of upfront, partial-upfront, or no-upfront (pay monthly) payment on these commitments at the same total cost, so the payment option is a cash-flow decision, not a discount-size decision on most of these products.
Why the scope difference matters in practice
A resource-level commitment (Standard RI, resource-based CUD, Azure Reservation) only pays off if the workload keeps needing that exact shape for the whole term; if the team migrates to a different instance family six months in, the commitment sits partially wasted (though AWS and Azure both allow some exchange or resale mechanisms to recover part of that). A spend-based commitment (Compute Savings Plan, flexible CUD) survives an instance-family change automatically, because the discount is applied to dollars spent on eligible usage, not to a specific SKU, at the cost of a somewhat smaller discount than the narrowest resource-level option.
Deciding what percentage of steady-state capacity to commit
Start from the floor, not the average: pull 3 to 6 months of utilization history for the workload, and find the usage level that held true on the worst week, not the typical week. That floor, not the mean, is the safe commitment baseline, because a commitment above the actual steady floor pays for idle capacity on every low-usage day. From there, the commitment size is a risk trade-off, not a fixed rule: a stable, mature workload with a long recent history of holding above that floor supports committing close to the full floor, while a workload still changing shape (recent re-architecture, aggressive growth, planned migration) justifies leaving more of the floor on-demand or covering it with a flexible, spend-based commitment instead of a rigid resource-level one, specifically because the risk being managed is "commitment outlives the workload's actual shape," not "commitment size in the abstract."
Worked example
A team's steady-state EC2 fleet held at a minimum of 40 instances of a given family over the last 4 months, with normal weekday peaks around 55 and occasional bursts to 70. The 40-instance floor is the commitment candidate, not the 55-instance average and not the 70-instance peak: committing at 55 would mean paying the commitment rate for capacity that isn't reliably used on quieter days, and any spike above 40 (up to and including the 70-instance bursts) is served by on-demand or spot capacity regardless of the commitment size. If this workload is expected to stay on the same instance family for the full term, an EC2 Instance Savings Plan or Standard RI sized to 40 instances captures the largest discount available on that stable floor; if a re-platforming project is likely to change instance family within the year, a Compute Savings Plan sized to the equivalent dollar-per-hour spend protects the same floor's discount while surviving the family change.
Trade-offs and pitfalls
The most common mistake is committing to the peak or the average instead of the floor, which either overpays for capacity that isn't reliably used or, worse, sizes a "safe" commitment so conservatively it captures almost none of the available discount. The second is choosing the narrowest, highest-discount resource-level commitment for a workload that's still changing shape, and then discovering the commitment doesn't match the new instance family, wasting real money for the rest of the term. The third, specific to the flexible/spend-based products, is assuming "flexible" means "no attention needed": a spend-based commitment still needs the underlying usage to stay above the committed dollar level, or the unused portion is still paid for and simply not applied to any usage.
That is every published Cloud Cost Optimization and FinOps question for Systems Engineer so far. Browse the other topics in this category, or practice this one interactively.