SLIs, SLOs, SLAs, and Error Budgets Questions
Defining and operating reliability targets. Covers choosing service level indicators, setting service level objectives and agreements, computing and spending error budgets, and using them to drive engineering decisions. Includes negotiating reliability targets with stakeholders.
Design SLOs for an e-commerce checkout service that depends on payment gateway, inventory, and search systems. Explain which SLOs map to business KPIs (e.g., conversion rate), which are service-level, how to handle downstream failures, and how to surface these SLOs in dashboards for product and engineering stakeholders.
Sample Answer
A checkout service's SLOs need to separate what's genuinely SYSTEM-level (is the service itself healthy) from what's USER-IMPACTING (did the customer actually get to complete their purchase), because the two can diverge when a downstream dependency fails gracefully versus catastrophically.
Structured elaboration
Business-KPI-mapped SLO: checkout completion rate, directly tied to conversion rate, the number product and leadership actually care about; this should reflect the FULL user journey, including downstream calls to payment gateway, inventory, and search. Service-level SLOs: each downstream dependency (payment gateway latency/error rate, inventory-check latency, search-availability) gets its own SLO so a specific failing component can be identified quickly, distinct from the aggregate customer-facing number. Handling downstream failures: define explicit fallback behavior (e.g. if inventory-check times out, default to "assume available, reconcile after" rather than blocking checkout entirely) and make sure the aggregate checkout SLO reflects the ACTUAL user outcome after fallback, not a naive AND of every dependency's individual health.
Worked example
Three concrete SLIs balanced together: checkout success rate (99.9% target, the primary business-facing SLI), checkout latency (p95 < 2s end-to-end), and payment-gateway error rate specifically (< 0.1%, tracked separately since it's the dependency most likely to cause a user-impacting failure even when the checkout service's own code is healthy). Dashboards for engineering show all three plus each downstream dependency's own health; the dashboard shown to product and executives shows primarily the business-facing checkout success rate and conversion-rate correlation, with the ability to drill down into the technical SLIs only if something looks wrong, since flooding an executive dashboard with every service-level SLI obscures the one number that actually matters to them.
Trade-offs and pitfalls
A common mistake is defining the checkout SLO as a strict AND across every downstream dependency's own SLO (checkout is "up" only if payment gateway, inventory, and search are ALL simultaneously healthy), which massively UNDERSTATES real checkout availability if the service has reasonable fallback behavior for a degraded dependency; the checkout SLO should reflect actual USER outcomes after fallback logic runs, not a naive composition of dependency health. It's equally a mistake to surface every technical service-level SLI to executive stakeholders undifferentiated from the business-facing number; the audience-appropriate layering (business KPI first, technical detail on demand) is itself part of designing this SLO set well, not an afterthought.
Design SLAs and SLOs for an internal platform consumed by 30 teams and operated by a platform team. Define appropriate SLO targets (availability, latency tiers), monitoring and alerting strategy, reporting cadence, support response expectations, remediation steps for missed SLOs, and cultural or contractual enforcement mechanisms to ensure accountability.
Sample Answer
An internal platform serving 30 teams needs SLA-like rigor even without external money changing hands, because the accountability mechanism that makes an SLA work (a real consequence for missing the target) has to exist internally too, or the SLO becomes advisory rather than binding.
Structured elaboration
SLO targets, tiered by criticality (a shared auth service probably needs 99.95%+, a lower-traffic internal reporting tool might be fine at 99%); monitoring and alerting owned by the platform team, with dashboards visible to all 30 consuming teams so status isn't opaque; reporting cadence (e.g. a monthly reliability report circulated to all consuming teams' leads); support response expectations (e.g. P1 acknowledged within 15 minutes, P3 within one business day) that function like an internal support SLA. Since there's no direct revenue on the line, roadmap-level accountability is the real enforcement mechanism: sustained SLO misses should trigger a documented reallocation of the platform team's OWN roadmap toward reliability work, visible to and reviewed by the consuming teams' leadership, rather than being a purely internal, invisible metric with no real consequence.
Worked example
An internal feature-flag platform serving 30 teams sets a tiered target: 99.95% availability with p95 latency < 50ms for its critical, synchronous-lookup tier, versus 99% availability with p95 latency < 500ms for a lower-traffic, non-blocking reporting tier, so the latency commitment scales with criticality the same way availability does, not a single flat number applied regardless of tier. Monthly reports go to all consuming teams; if the SLO is missed two months running, the platform team's next-quarter roadmap must allocate at least 30% of capacity to reliability work, reviewed and confirmed by an internal steering group representing the consuming teams, not decided unilaterally by the platform team itself. For a platform specifically supporting ML workflows, this same mechanism additionally includes a documented ONBOARDING process (new ML teams joining the platform are shown the current SLO status and support tiers up front) and ties platform-improvement PRIORITIZATION directly to which internal SLA commitments are most at risk, so onboarding and roadmap decisions are grounded in the same accountability data rather than treated as a separate, informal process.
Trade-offs and pitfalls
Without a genuine consequence for missing the target, an internal SLO quietly becomes theater: teams cite it in retros but nothing structurally changes when it's repeatedly missed, which is why the roadmap-reallocation mechanism needs external visibility (to the consuming teams, not just the platform team's own management) to have teeth. Incentive design matters here too: if the platform team's own leadership is evaluated purely on feature delivery velocity with no weight given to SLO adherence, the roadmap-reallocation mechanism will face constant internal pressure to be deprioritized, so the incentive structure for the platform team's OWN leadership needs to genuinely reward reliability investment, not just tolerate it.
You operate an ML inference service where model accuracy drifts over time and higher latency often correlates with better accuracy. Propose SLIs and SLOs that capture both user experience and model quality, and design an error budget policy that handles accuracy drift and latency trade-offs.
Sample Answer
An ML inference service where accuracy and latency trade off against each other needs SLIs on BOTH dimensions tracked together, plus a signal that catches accuracy degradation directly, since a silent accuracy drift can happen with the service returning a perfectly normal-looking, fast, 200-status response the whole time.
Structured elaboration
SLIs: latency (p95/p99, standard), inference error rate (outright failures, standard), AND a model-quality signal that does NOT depend on an obvious error status: accuracy-on-labeled-sample (measured against a periodically-refreshed held-out or delayed-ground-truth set) and input-distribution drift (comparing the live input feature distribution against the training/validation distribution, which can serve as a FASTER proxy for likely accuracy degradation than waiting for labeled ground truth to confirm it). This last point matters specifically because a model returning subtly wrong predictions looks completely healthy on every conventional service-health metric (200 status, normal latency, no exceptions); only a dedicated drift or accuracy-tracking SLI would ever catch it.
Worked example
An error-budget policy that explicitly handles the latency-accuracy trade-off: if serving a MORE accurate (but slower) model variant would push p95 latency past its SLO, that's a genuine trade-off decision, not a bug, and the policy should specify who's authorized to make that call (e.g. product/ML leadership jointly, not an automated system defaulting to either extreme). Separately, if input-distribution drift crosses a threshold (suggesting the model is now seeing meaningfully different data than it was validated on), that should trigger a model-quality investigation regardless of whether latency or error-rate SLIs show anything wrong at all.
Trade-offs and pitfalls
Treating error rate as the only "is the model broken" signal is the central mistake this scenario is designed to catch: a model that's wrong 30% of the time but never throws an exception or returns a non-200 status will show a perfect error-rate SLI while genuinely failing its purpose, which is exactly the silent-failure risk that drift and accuracy-on-sample SLIs are meant to close. It's also worth being explicit that the labeled-accuracy signal is necessarily a TRAILING indicator (ground truth often isn't available immediately), so the drift-based proxy signal is what actually gives you a chance to react before the slower, more authoritative accuracy signal eventually confirms the problem.
Implement a Python function compute_error_budget_consumption(slo_percent: float, window_minutes: int, good_minute_flags: List[int]) that returns the percentage of the error budget consumed in the window. good_minute_flags is a list with 1 for a 'good' minute and 0 for a 'bad' minute. Assume the list length equals window_minutes. Show examples and complexity.
Sample Answer
The function needs to translate a minute-by-minute good/bad record into a single "percent of the allowed error budget already spent" number, which is the quantity teams actually watch (not the raw bad-minute count).
Structured elaboration
Given an SLO percentage, the allowed BAD fraction of the window is 1−slo/100, and multiplying that by the window length gives the allowed bad minutes. Consumption is then the ratio of actual bad minutes to allowed bad minutes, expressed as a percentage: a result over 100% means the budget for that window has already been overspent, which is a normal and important state to represent (not an error condition). Complexity: the function makes a single pass over good_minute_flags to sum it (sum() is O(window_minutes) time), plus O(1) arithmetic afterward, so overall time complexity is O(window_minutes) and space complexity is O(1) beyond the input list itself (no auxiliary data structures are allocated).
Worked example (executed; python3, verified against 5 cases including an edge case)
from typing import List
def compute_error_budget_consumption(slo_percent: float, window_minutes: int,
good_minute_flags: List[int]) -> float:
if len(good_minute_flags) != window_minutes:
raise ValueError("good_minute_flags length must equal window_minutes")
allowed_bad_fraction = 1 - slo_percent / 100.0
allowed_bad_minutes = allowed_bad_fraction * window_minutes
actual_bad_minutes = window_minutes - sum(good_minute_flags)
if allowed_bad_minutes == 0:
return 100.0 if actual_bad_minutes > 0 else 0.0
return (actual_bad_minutes / allowed_bad_minutes) * 100.0
Verified results: a 99.9% SLO over 1440 minutes (1 day) allows 0.001×1440=1.44 bad minutes; with exactly 1 bad minute observed, consumption is 1/1.44×100≈69.44% (confirmed by running the code). With 720 of 1440 minutes bad, consumption comes out to 50,000% (720/1.44*100), correctly signalling a catastrophic overspend rather than clamping silently to 100%. A perfectly clean window returns exactly 0.0%, and a length mismatch between the flag list and the stated window raises a ValueError rather than silently truncating.
Trade-offs and pitfalls
Clamping the output at 100% is tempting for a dashboard, but it destroys the information a responder needs to judge HOW badly the budget was blown, which matters for deciding release-freeze severity; better to let it exceed 100% and clamp only at the presentation layer if needed. A real production version also needs to handle partial-credit minutes (a minute that was only degraded, not fully down) rather than the binary 1/0 flag used here, and needs to decide how to treat missing minutes (this implementation implicitly requires every minute to be explicitly flagged, which forces the caller to make a conscious choice for gaps rather than silently defaulting them to good).
Propose a measurable Service-Level Objective (SLO) related to code quality for a public API (e.g., regression-caused errors per 1K requests). Explain how you'd instrument the SLO, what data sources you'd use, how to alert on breaches, and what operational playbooks you'd trigger.
Sample Answer
Code quality is usually tracked informally (code review comments, static-analysis scores), but turning "regressions caused by recent code changes" into an actual SLO forces the same operational discipline onto it that latency and availability already get.
Structured elaboration
A concrete SLO: "no more than 2 regression-caused errors per 1,000 requests, attributable to a change deployed within the last 7 days, measured over a rolling 30-day window." Instrumentation: every error needs to be tagged, at the point it's logged, with whether it correlates to a recent deploy (via a deploy-timestamp lookup against the error's occurrence time and a code-ownership/blame mapping), distinguishing a "regression" (newly introduced) from a long-standing, pre-existing bug (which is a different, separate reliability problem, not a code-quality-regression one). Data sources: deploy-event logs (timestamped, tied to a specific commit/PR), error-tracking system output (with stack traces mapped back to the responsible code area), and a code-ownership mapping (so a regression can be attributed to a specific team or even a specific recent PR).
Worked example
Alerting: if the rolling regression-error rate crosses the 2-per-1000 threshold, the alert should include the SPECIFIC recent deploys that correlate with the spike (not just "regressions are up"), since actionable remediation depends on knowing which change to investigate first. Operational playbook triggered: first, confirm the correlated deploy via the error-tracking system's stack traces; second, if confirmed, roll back that specific deploy (not the whole service) if the fix isn't trivial and fast; third, require the offending PR's original author (or team) to add a regression test covering the specific failure before the fix is re-attempted, closing the loop so the same class of regression doesn't recur silently.
Trade-offs and pitfalls
Attribution accuracy is the hardest part of this SLO to get right: a naive "any error within N days of any deploy" heuristic will over-attribute unrelated errors to recent deploys purely by coincidental timing, so the attribution logic needs real evidence (a stack trace pointing into recently-changed code, not just temporal proximity) before counting an error as a genuine regression. It's also worth deciding whether this SLO should apply uniformly to every code change or be weighted by the CHANGE's actual risk profile (a large refactor plausibly deserves closer regression scrutiny than a one-line config tweak), similar to the release-risk-scoring discussion elsewhere, rather than treating every deploy as an equally likely source of regression.
Unlock Full Question Bank
Get access to all SLIs, SLOs, SLAs, and Error Budgets interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.