Picking SLO targets
Start from what actually affects the business and users, not an arbitrary round number: for a billing API, a slow or failed checkout call directly risks lost revenue and support load, so the SLO (Service Level Objective: the internal target you set for a metric like latency or availability) should track what customers actually feel. Anchor the target against your own baseline distribution first (don't set a target far tighter than what the system currently, realistically achieves without major investment), and set it off percentiles, not the average, since the average can look healthy while a real slice of checkout attempts are painfully slow.
Metrics to use
- Latency: p95 and p99 response time for the billing endpoints, being explicit about whether failed or timed-out requests are included in the latency population or tracked separately (excluding them silently is a common way an SLI (Service Level Indicator: the actual measured metric, e.g. p99 latency, that gets compared against the SLO) quietly stops meaning what people think it means).
- Availability: the percentage of requests that complete successfully (not 5xx, not timed out), often tracked as its own SLO alongside latency rather than folded into it.
Translating business impact into thresholds
Look at your own funnel or conversion data for where checkout abandonment starts climbing as latency increases, and set the latency ceiling with real headroom below that point, not right at the edge of where users start leaving. The same logic applies to availability: estimate what a percentage point of failed billing requests costs in lost transactions and support tickets, and use that to justify how tight (and how expensive) the target should be, since tighter targets generally cost more engineering effort and infrastructure to sustain.
Error budgets
The error budget is simply what's left over: 1 minus the SLO. A 99.9% availability SLO over a 30-day month leaves a 0.1% error budget:
text
30 days = 43,200 minutes
0.1% of 43,200 minutes = 43.2 minutes
That's roughly 43 minutes of full-equivalent downtime allowed per month, whether spent as one incident or many smaller partial-degradation periods that add up.
How the error budget should drive release cadence and incident response
- Release cadence: while budget remains healthy, ship at normal (or even accelerated) velocity. As the budget gets consumed, slow down: require canary rollouts (releasing a change to a small slice of traffic or users first, before rolling out to everyone), extra review, or freeze risky changes entirely until the budget recovers. This turns "reliability" from a vague value into a concrete, negotiable resource that product and engineering both watch.
- Incident response: alert on burn RATE (how fast the budget is being consumed) rather than only on absolute SLO breach, so a fast, severe burn pages immediately while a slow, minor burn becomes a lower-urgency ticket. Postmortems and remediation priority should also weigh directly against how much budget an incident consumed, giving the team a shared, quantitative way to decide whether to prioritize a reliability fix over the next feature.
