SLIs, SLOs, SLAs, and Error Budgets Questions

A fraud-detection model's two error types (false positives and false negatives) have genuinely different dollar costs, so the SLO needs to be built on a COMBINED financial-impact metric, not a single blended accuracy number that treats the two error types as equally bad.

Structured elaboration

Converting error types to dollar impact: a false negative (a fraudulent transaction incorrectly approved) typically costs roughly the transaction's dollar value plus any associated chargeback/fraud-liability cost; a false positive (a legitimate transaction incorrectly blocked) costs a smaller, but real, amount per incident (lost revenue from the declined transaction, customer friction/churn risk, support cost), and the two costs are rarely equal, meaning a naive "minimize total error rate" target can actually optimize for the WRONG trade-off if false negatives are far more expensive per incident than false positives (or vice versa, depending on the specific business). SLO thresholds should therefore be set on the expected DOLLAR COST of both error types combined over a period, not on a raw accuracy or error-rate percentage alone.

Worked example

If a false negative costs an average of $400 (fraud loss) and a false positive costs an average of $15 (lost transaction + support friction), a model with a lower FALSE-NEGATIVE rate but a somewhat higher false-positive rate could have a LOWER total dollar cost than a model that's more "accurate" by a naive combined error-rate metric; the SLO should be framed as "expected fraud-related dollar cost per 10,000 transactions must stay under $X," letting the model's actual false-positive/false-negative balance be tuned to minimize that cost rather than to hit an arbitrary symmetric accuracy target. Rollback/escalation: if the trailing dollar-cost metric crosses its threshold, the response depends on WHICH error type is driving it (a false-negative spike triggers an urgent fraud-team review and possibly a temporary tightening of the decision threshold, accepting more false positives as a deliberate short-term trade-off; a false-positive spike triggers a review of whether the model has become too conservative, hurting legitimate revenue).

Trade-offs and pitfalls

The dollar-cost estimates themselves (average cost per false negative, per false positive) are assumptions that need periodic re-validation against actual realized costs, since they can drift as fraud patterns, transaction sizes, or customer sensitivity to false declines change over time; an SLO built on stale cost assumptions optimizes for a trade-off that no longer reflects reality. It's also worth being explicit that these two error types trade off against each other via the model's decision threshold, so "improving" one in isolation (tightening to reduce false negatives) mechanically worsens the other, and the SLO needs to be framed around the COMBINED cost specifically so that trade-off is managed deliberately rather than each error type being independently, and contradictorily, targeted.

An ML inference service needs SLIs that capture both classic service health (is it responding, how fast) and model-specific health (is it still making good predictions), because the two can diverge: a service can be perfectly "up" while silently producing degraded predictions.

Structured elaboration

Three concrete SLIs: (1) p95 latency - standard service-health signal, measured continuously; (2) inference error rate - proportion of requests that fail outright (timeout, exception, malformed input), distinct from model quality; (3) model-accuracy-on-labeled-sample - a proxy quality signal computed against a held-out or periodically-labeled sample, since ground truth for most predictions may not be available in real time. Realistic targets might be p95 < 150ms, error rate < 0.5%, and accuracy-on-sample staying within an agreed band (e.g. no more than 2 percentage points below the validated offline baseline) over a rolling window.

Worked example

If the model's offline-validated accuracy was 92% and the live labeled-sample accuracy drops to 88% (a 4-point drop, exceeding the 2-point tolerance), that alone should trigger an alert distinct from the latency/error-rate alerts, since nothing about latency or error rate would catch this: the service is "up" by every conventional measure while quietly serving worse predictions. The alerting action for this SLI might be a model-rollback to the last known-good version rather than any infrastructure-level mitigation.

Trade-offs and pitfalls

The labeled-sample accuracy SLI has an inherent lag: ground truth often isn't available instantly (sometimes it takes hours or days), so this SLI is necessarily a trailing indicator and needs a DIFFERENT, faster proxy (like prediction-distribution drift) if you need to catch model degradation before ground truth confirms it. It's also worth being explicit that latency and accuracy can trade off against each other (a more thorough model is often slower), so the SLO targets for each need to be set together, not independently, or optimizing one can silently violate the other.

An internal platform serving 30 teams needs SLA-like rigor even without external money changing hands, because the accountability mechanism that makes an SLA work (a real consequence for missing the target) has to exist internally too, or the SLO becomes advisory rather than binding.

Structured elaboration

SLO targets, tiered by criticality (a shared auth service probably needs 99.95%+, a lower-traffic internal reporting tool might be fine at 99%); monitoring and alerting owned by the platform team, with dashboards visible to all 30 consuming teams so status isn't opaque; reporting cadence (e.g. a monthly reliability report circulated to all consuming teams' leads); support response expectations (e.g. P1 acknowledged within 15 minutes, P3 within one business day) that function like an internal support SLA. Since there's no direct revenue on the line, roadmap-level accountability is the real enforcement mechanism: sustained SLO misses should trigger a documented reallocation of the platform team's OWN roadmap toward reliability work, visible to and reviewed by the consuming teams' leadership, rather than being a purely internal, invisible metric with no real consequence.

Worked example

An internal feature-flag platform serving 30 teams sets a tiered target: 99.95% availability with p95 latency < 50ms for its critical, synchronous-lookup tier, versus 99% availability with p95 latency < 500ms for a lower-traffic, non-blocking reporting tier, so the latency commitment scales with criticality the same way availability does, not a single flat number applied regardless of tier. Monthly reports go to all consuming teams; if the SLO is missed two months running, the platform team's next-quarter roadmap must allocate at least 30% of capacity to reliability work, reviewed and confirmed by an internal steering group representing the consuming teams, not decided unilaterally by the platform team itself. For a platform specifically supporting ML workflows, this same mechanism additionally includes a documented ONBOARDING process (new ML teams joining the platform are shown the current SLO status and support tiers up front) and ties platform-improvement PRIORITIZATION directly to which internal SLA commitments are most at risk, so onboarding and roadmap decisions are grounded in the same accountability data rather than treated as a separate, informal process.

Trade-offs and pitfalls

Without a genuine consequence for missing the target, an internal SLO quietly becomes theater: teams cite it in retros but nothing structurally changes when it's repeatedly missed, which is why the roadmap-reallocation mechanism needs external visibility (to the consuming teams, not just the platform team's own management) to have teeth. Incentive design matters here too: if the platform team's own leadership is evaluated purely on feature delivery velocity with no weight given to SLO adherence, the roadmap-reallocation mechanism will face constant internal pressure to be deprioritized, so the incentive structure for the platform team's OWN leadership needs to genuinely reward reliability investment, not just tolerate it.

A real-time predictive API serving multiple internal product teams is effectively an internal platform with paying (in engineering-time or budget-allocation terms) tenants, so its SLA design should mirror a genuine multi-tenant platform contract, including cost allocation.

Structured elaboration

SLIs/SLOs: latency (p95/p99, since prediction consumers are often themselves latency-sensitive downstream services), availability (standard success rate), and prediction freshness (how current the underlying model or feature data is, since a "successful" but stale prediction is a real quality problem distinct from an outright failure). Error budgets and escalation follow the standard pattern, with the added wrinkle that MULTIPLE consuming teams share the same underlying service, so a budget-exhaustion event needs a policy for how competing teams' needs get prioritized (e.g. a designated highest-criticality consumer gets priority during a capacity crunch, decided in advance, not negotiated live during an incident). On-call/support tiers: a standard support-ticket SLA (response time by severity) alongside the technical SLO, since consuming teams need to know not just "is the service reliable" but "how fast will my question or issue get a human response." Cost allocation: usage-based chargeback (e.g. cost proportional to each consuming team's request volume or compute consumption) is the most common and fairest model, though it requires accurate per-team usage attribution to be credible.

Worked example

SLO: 99.9% availability, p95 latency < 80ms, freshness < 5 minutes; support tiers: P1 (service down) acknowledged in 15 minutes, P3 (a specific team's edge-case question) acknowledged within 1 business day; cost allocation: each consuming team is billed (internally) proportional to its share of total requests over the month, with a documented, auditable usage report so no team can dispute the allocation without visibility into the underlying data.

Trade-offs and pitfalls

Without a pre-agreed priority policy for a capacity crunch, an incident where multiple teams are simultaneously affected turns into a real-time argument about whose traffic gets protected first, which wastes precious incident-response time exactly when it's most costly; deciding this in advance, even if imperfectly, beats deciding it live every time. Usage-based cost allocation also needs to guard against a specific perverse incentive: if cost is purely proportional to raw request volume, a team might be discouraged from legitimate request patterns (retries for genuine reliability) purely to save on their internal bill, so the allocation model should distinguish between necessary retries (the platform's own responsibility) and a consuming team's own inefficient usage pattern (their responsibility) rather than billing both identically.

A feature built on a third-party managed AI service like Azure OpenAI needs SLOs that explicitly separate what YOU control (your own integration code, your own error handling) from what the VENDOR controls (their service's own availability, latency, and rate limits), since an incident caused by vendor throttling needs a genuinely different response than one caused by your own bug.

Structured elaboration

Measurable SLOs with illustrative targets: latency p95 end-to-end (including your own integration overhead, not just the vendor's raw response time) targeted at, e.g., under 3 seconds; availability percentage targeted at 99.5% monthly for the feature overall (deliberately set below Azure OpenAI's own published SLA, since your feature's ceiling is bounded by the vendor's real, observed reliability plus your own integration's added risk, not just the vendor's number alone); error rate targeted at under 2% of requests, tracked separately from vendor-attributed 429/rate-limit responses (which are logged and trended but not counted against your own code's error-rate SLO, since they are not a bug in your integration); and cost per 1,000 requests targeted at a defined budget ceiling (e.g. $X/1000 requests), a genuinely important SLI for an AI-service-backed feature specifically, since cost can vary meaningfully with usage patterns and retry behavior in a way that's less prominent for typical infrastructure SLOs.

Worked example

Monitoring and alert thresholds: instrument both your own integration layer's health AND, separately, the vendor's observed latency/error/rate-limit behavior as experienced by your actual traffic, so you can distinguish "our code broke" from "the vendor throttled or degraded us" when an incident occurs. Page on-call if p95 latency exceeds 5 seconds sustained for 10 minutes, OR if your own-code error rate (explicitly excluding vendor 429 responses) exceeds 5% for 5 minutes. Track vendor-attributed 429/rate-limit rate as its own dashboard metric with a lower-severity, non-paging alert (a ticket, not a page) if it exceeds 10% of requests for 15 minutes, since that condition requires a different response path (backoff, vendor engagement) than your own bug would. Alert if cost per 1,000 requests drifts more than 20% above its trailing 7-day baseline, since a spike often indicates aggressive retries against a struggling vendor endpoint rather than genuine traffic growth.

Handling an incident caused by external rate-limit throttling from the vendor: first, confirm via your instrumentation that the failure signature (a specific rate-limit error code/response, not a generic timeout or 500) matches vendor throttling rather than your own bug, since misdiagnosing a self-inflicted problem as "the vendor's fault" wastes response time and delays the real fix. Once confirmed as genuine vendor throttling, the response is largely OUTSIDE your control to fix directly (you can't force the vendor to raise your rate limit instantly), so the mitigation is architectural: implement request queuing/backoff with jitter to smooth bursts against the vendor's actual limit, consider whether a fallback (a simpler, non-AI-backed response, or a cached/previous result) can degrade gracefully rather than failing outright during the throttled period, and separately, engage the vendor relationship (requesting a rate-limit increase, or reviewing your usage pattern against their published limits) as a parallel, slower-moving track.

Trade-offs and pitfalls

Setting your own SLO tighter than what the vendor's OWN published SLA can support is a common and avoidable mistake specifically with managed AI services, since your feature's ceiling is fundamentally bounded by the vendor's own commitments (and often, in practice, well below their nominal SLA during periods of high demand across all their customers, not just yours); the SLO-setting process here needs to explicitly account for the vendor's real, observed (not just advertised) reliability as an input, not just aspirational internal targets. It's also worth tracking cost per 1,000 requests as a genuine SLI, not just a finance metric, since certain failure-handling strategies (aggressive retries against a struggling vendor endpoint) can quietly and significantly increase cost while doing little to actually improve the user-facing outcome, a trade-off worth making visible rather than discovering only on the next bill.