Airbnb Site Reliability Engineer Interview Preparation Guide (Mid-Level)

1. Rapid root-cause identification

• Establish incident commander and central Slack/war-room. Triage by impact: which services first show errors, which clients fail, timestamps.
• Use distributed tracing (Jaeger/X-Ray), request-id logs, and metrics (error-rate, p95 latency, CPU/memory, queue depths) to build a causal chain. Query traces to find the earliest increased latency/error span.
• Generate a quick dependency graph (service A → B → C) from service metadata or orchestration labels to map propagation paths.

2. Prioritize mitigations to stop the cascade

• Apply the simplest, lowest-risk controls first:
  • Throttle or rate-limit traffic at the edge or API gateway to reduce load.
  • Put nonessential downstream calls into degraded mode (feature flags).
  • Increase timeouts and backoff on retrying clients to avoid retry storms.
  • If a single service is overloaded, scale it up (horizontal/vertical) or roll back recent deploys.
• Communicate changes and expected customer impact to stakeholders.

3. Implement circuit-breaker patterns to limit blast radius

• Define circuit-breaker policy per dependency: sliding window, failure threshold (e.g., 5xx > 10% over 30s), minimum request volume, and cool-down.
• Use libraries/frameworks (Hystrix-like, resilience4j, Envoy route-level circuit breaking) or sidecars.
• Complement with bulkheads (isolate thread pools/connection pools), timeouts, and exponential backoff with jitter to prevent synchronized retries.
• Example rules: open if failures > 20% and requests >100 in 30s; half-open trial of 5 requests.

4. Validate stabilization and coordinate cross-team

• Track key signals: decreasing error-rate, stable p95/p99 latency, queue drain, reduced retries, and SLO/error budget burn rate returning to baseline.
• Run targeted synthetic tests and smoke testing from multiple regions to ensure end-to-end paths.
• Keep teams in sync: incident timeline, mitigations applied, rollback notes. After stable, run a blameless postmortem to capture root cause, permanent fixes (circuit-breaker configs, autoscaling, runbook updates), and follow-up action owners with deadlines.

This structured approach isolates cause quickly, stops damage fast, prevents recurrence with defensive patterns, and ensures cross-team validation and learning.

Case where faster costs more

If the ask is to reduce response time by provisioning a bigger or faster instance type, or by holding extra capacity ready ahead of demand, that genuinely costs more every month. You're paying for headroom and speed together, and the bill goes up.

Case where faster saves money

If the slowness is actually caused by waste, an inefficient query scanning far more data than it needs, or a retry storm hammering a dependency, fixing that root cause makes the system both faster and cheaper at the same time, because you stop paying for work that was never useful in the first place.

How I'd explain the difference

I'd walk through their specific bill with them and point to the actual line item driving cost, rather than talking about performance in the abstract. Then I'd be explicit about which of the two situations applies to their particular request: "in your case, the slowdown is [provisioning headroom / wasted work], so speeding it up will [cost more / save money], and here's the data behind that."

Building trust

I'd avoid asserting a conclusion and instead show the reasoning: the before-and-after cost breakdown, or an offer to prototype the fix on a small slice first so they can see the actual before/after numbers rather than taking my word for it. Being upfront that "faster" and "cheaper" aren't the same thing, and don't always move together, tends to build more trust than promising both every time, since customers notice when a vendor is honest about a trade-off that could have gone the other way.

Direct answer

Union-Find (disjoint set union) maintains a partition of elements into disjoint sets and supports two operations: find(x), which returns a representative (root) identifying which set x currently belongs to, and union(x, y), which merges the sets containing x and y into one. On its own, a naive version can degrade to $O(n)$ per operation on an adversarial sequence of unions, but adding path compression (flattening the tree toward the root during find) and union by rank or size (always attaching the smaller tree under the larger one) brings the amortized cost per operation down to $O(\alpha(n))$, the inverse Ackermann function, a value effectively bounded by 4 for every input size that could exist in practice. Two concrete infrastructure use cases: batching offline connectivity queries over a static snapshot of a network graph, and grouping servers or hosts into failure domains from a stream of "these two are on the same rack/switch/power circuit" observations.

Structured elaboration

The two operations in detail:

• find(x): walk x's parent pointer chain up to the root (the node that is its own parent). With path compression, every node visited during this walk gets repointed to point more directly at the root (either straight to the root, or to its grandparent under "path halving"), so subsequent find calls on those same nodes are faster.
• union(x, y): call find on both, and if the roots differ, attach one root under the other. Union by rank tracks an upper bound on each tree's height and always attaches the shorter tree under the taller one (breaking ties arbitrarily and bumping the surviving root's rank); union by size does the analogous thing tracking subtree size instead of height. Both give the same asymptotic guarantee.

Why amortized cost is near-constant. Union by rank alone bounds any tree's height at $O(\log n)$, so even without path compression, each find is $O(\log n)$ worst case. Path compression alone also gives an amortized $O(\log n)$ bound. The combination of both is what produces the much stronger $O(\alpha(n))$ amortized bound (a result due to Tarjan, 1975): every find call that walks through a node also flattens that node's future lookups, and union by rank ensures that no tree is allowed to grow needlessly tall in the first place, so the "expensive" flattening work a find call does gets amortized (spread out) over all the future cheap lookups it enables.

Two infrastructure use cases:

1. Offline connectivity queries over a static graph. Given a fixed snapshot of a network topology (which hosts can reach which others through some set of links) and a batch of "are A and B in the same reachable group" queries, build the Union-Find structure once by unioning every known edge, then answer every query in near-constant time via two find calls and an equality check, instead of re-running a graph traversal per query.
2. Grouping resources by failure domain. Given a stream of pairwise "these two hosts share a failure domain" facts (same rack, same top-of-rack switch, same power distribution unit, same availability zone), union each pair as the facts arrive. The resulting sets are exactly the maximal failure domain groups, useful for placement decisions (spread replicas across different find roots to avoid correlated failure) and for blast-radius estimation during an incident (given one affected host, find its root and treat every other member of that set as at elevated risk).

Worked example

For a tiny illustration of the failure-domain use case: suppose 6 hosts and the observed co-location facts arrive in this order: (host1, host2) share a rack, (host3, host4) share a rack, (host2, host3) share a power distribution unit, (host5, host6) share nothing observed yet. After processing the three known facts, the disjoint sets are {host1, host2, host3, host4} and {host5} and {host6}, three failure domains, even though no single fact directly named host1 and host4 together. This transitive-merging behavior (the union of a union) is exactly what a manual "same rack" lookup table would miss without repeatedly re-deriving connectivity, and is what Union-Find gives you for free as facts accumulate.

Trade-offs and pitfalls

• Union-Find has no efficient "un-union." Once you have merged the failure domains for host2 and host3 based on a shared power distribution unit, there is no cheap way to later split them back apart if that fact turns out to be stale (the unit was decommissioned, hosts were physically moved). A structure that must support both merges and un-merges over time needs periodic rebuilds or a genuinely dynamic-connectivity data structure, a materially harder problem.
• Common mistake: treating Union-Find as if it directly answers "list every host in this failure domain." It answers "are these two in the same domain" and "how many domains are there" in near-constant time, but producing the actual member list of a given domain requires a separate $O(n)$ grouping pass over all elements by their final root, not a property of the structure itself.
• Common mistake: implementing find/union without EITHER optimization and assuming it is "fine because it is a small graph." On a pathological build order (facts arriving in a long chain, host1-host2, host2-host3, host3-host4, ...), an unoptimized version degrades to a linked list and each subsequent find costs $O(n)$, which matters even at moderate scale if the facts stream is large or adversarially ordered.
• Failure-domain grouping is a good fit for Union-Find specifically because facts only ever ADD information (a shared rack or shared power unit is discovered, not un-discovered, in the normal operating case); if the underlying use case instead needed to react to topology changes that REMOVE a shared-domain relationship, this technique alone would not be enough.

Direct answer
Default to Terraform for the foundational, provider-broad layer, since it has the deepest multi-cloud provider coverage and the most mature ecosystem for infra teams. Reach for Pulumi specifically when the team is already strong in a general-purpose language and the win is expressiveness, real loops and functions and existing test frameworks, rather than provider coverage. Reach for Crossplane (a Kubernetes-native IaC tool that manages cloud resources as Kubernetes objects) specifically when continuous reconciliation matters more than plan/apply control, typically because the platform is already Kubernetes-centric. Treat cloud-native templates as a single-cloud tool, not a multi-cloud contender, useful only for the narrow slice of a platform that's genuinely cloud-specific anyway.

Structured elaboration

The axis that actually decides this
"Multi-cloud parity" collapses into three separable questions most comparisons blur together: does the tool have mature provider support for everything needed across all the clouds in play; is drift corrected continuously or only when someone runs a plan; and does the team's existing skill set make the tool's config language an asset or a liability. Answering those three first makes the tool choice close to mechanical; debating "Terraform vs Pulumi" in the abstract without them just becomes a preference argument.

Comparison

What tips the recommendation in practice

• If the team has no strong Kubernetes investment yet, Crossplane's continuous-reconciliation advantage isn't free: adopting it means adopting Kubernetes as infrastructure for infrastructure, a bigger commitment than the reconciliation benefit alone justifies for most platforms.
• If provider coverage for a specific, load-bearing service is the actual blocker, a niche managed service only has a mature cloud-native or Terraform provider, not a Pulumi or Crossplane one yet, that single gap can decide the whole platform's tooling, since running two different IaC tools for two different resource types is its own maintenance cost.
• If the team is already writing the application in TypeScript or Python and wants infra tests to run in the same CI job with the same test framework as the app, that's Pulumi's actual differentiator, the shared tooling and testability, not "the language is nicer."
• Cloud-native templates earn a place even in a multi-cloud platform for the narrow slice of resources that are genuinely cloud-specific, where the vendor's own template format gets day-one support for a new feature before any third-party provider catches up; that's a deliberate, scoped exception, not a sign the whole platform should move to vendor templates.

Worked example
Applying the axis to one concrete requirement: the platform needs a managed Kubernetes cluster on both AWS (EKS) and GCP (GKE), plus continuous enforcement that no cluster ever drifts from its baseline node count and network policy.

• Provider coverage: both EKS and GKE have mature, long-standing Terraform providers; this doesn't rule anything out yet.
• Continuous drift correction: this is the requirement that actually narrows the field, since "no cluster ever drifts" is a continuous-reconciliation ask, not a periodic-plan ask, which points toward Crossplane, or a Terraform-plus-a-separate-reconciler-loop hybrid, over plain Terraform.
• Team skill: if the platform team already operates Kubernetes clusters as their day job, Crossplane's operational model, another set of controllers to run, is a marginal addition, not a new discipline; that tips the recommendation to Crossplane for this specific requirement, while Terraform still provisions the surrounding account-level resources, networking, IAM, that don't need continuous reconciliation.

This is the actual decision process: none of the four tools wins as a general answer, the specific requirement, continuous drift correction here, determines which axis matters most.

Trade-offs & pitfalls

• Running two tools, Terraform for foundational resources and Crossplane for continuously-reconciled workload resources, for example, is a legitimate hybrid, but it's an explicit architectural decision with its own coordination cost, not a default to reach for without a specific requirement driving it.
• Pulumi's expressiveness, real loops, conditionals, functions, is also its biggest risk: it's easy to write infra code that's hard to review because the actual resources created depend on runtime logic, not something visible from reading the file the way a Terraform for_each mostly is.
• Crossplane's continuous reconciliation means an unintended manual change gets reverted automatically, a feature until someone makes an emergency manual fix during an incident and Crossplane quietly reverts it a few minutes later; break-glass procedures need to account for that.
• Cloud-native templates being excellent for their own cloud is real, but it's also the lock-in mechanism; a platform leaning on them for anything beyond the narrow cloud-specific exception has quietly opted out of the multi-cloud parity goal the question started with.

Where you measure latency (server-side versus client-side) changes what the number actually represents, and conflating the two, or trusting either blindly, produces an SLO that looks precise but measures the wrong thing.

Structured elaboration

Server-side instrumentation measures only the time the backend itself spends processing a request; it's cheap, low-variance, and easy to instrument consistently, but it is BLIND to everything that happens on the network between the client and the server (DNS lookup, TLS handshake, queueing, actual transit time), which for many users is a large fraction of perceived latency. Client-side instrumentation captures the full, true user-perceived latency, but introduces its own noise: clock skew between client devices (a client's own clock can be seconds off, corrupting any timestamp-difference calculation done client-side without a server-synchronized reference), highly variable network conditions across users, and sampling bias if only a subset of clients (e.g. those with a certain browser or app version) report telemetry at all.

Worked example

A service reports a server-side p95 of 80ms, comfortably inside its SLO, while client-side RUM data shows real users experiencing p95 latency of 450ms; the gap is almost entirely network transit and client-side rendering time invisible to the server-side number. An SLO built only on the server-side figure would report perfect health while the actual user experience is materially worse, which is exactly why it matters to declare, explicitly, which of the two the SLO is based on, and why most mature setups track both and set the CUSTOMER-facing SLO on the client-observed number, using the server-side one as an internal diagnostic signal.

Trade-offs and pitfalls

Histogram bucket placement compounds this problem: if buckets are chosen assuming server-side-scale latencies (fine-grained near 50-100ms) but applied to client-side data with true tails out past 1 second, the histogram's percentile estimate near p95/p99 becomes inaccurate because there aren't enough bucket boundaries where the real tail actually lives. Sampling client-side telemetry (common for cost reasons at very high traffic) also needs to be checked for bias: if only users on a fast, stable network path are more likely to successfully report their client-side telemetry (e.g. because a flaky connection also drops the telemetry beacon), the sampled data will look artificially fast, systematically underestimating the true tail.

Direct answer

Frame the cut through the 99.99% error budget, not a blanket percentage. 99.99% allows about 52.6 minutes of downtime a year; anything that doesn't touch the paths that consume that budget is safe to cut aggressively, and anything that does must be modeled against it explicitly before you touch it. In practice that means chasing waste and inefficiency hard (usually most of the 30%), and treating redundancy, failover paths, and DR test cadence as protected unless you can show the specific budget impact is acceptable.

A three-bucket framework for the savings

That number is the currency every cut gets priced in.

Chase bucket 1 and 2 first and aggressively (they rarely conflict with availability); only reach into bucket 3 if 1 and 2 don't get you to 30%, and price every bucket-3 cut against the budget above.

Worked example

Suppose an audit of monthly compute spend of $300,000 finds 15% idle or overprovisioned capacity, plus another 10% recoverable through rightsizing and reserved commitments on stateless, replaceable capacity:

That's 25 of the 30 points from buckets 1 and 2 alone, with essentially zero availability risk. The remaining 5 points has to come from bucket 3, for example dropping a redundant standby from 3 independent paths to 2 on some service. Whether that's safe depends entirely on whether that service is in the 99.99% critical scope. Reusing the same series/parallel redundancy model above (an N-way redundant path's unavailability is the product of each independent unit's own unavailability, since all N units have to fail at the same time for the whole path to be down: $U_N = (1-a)^N$), for a building block at $a=0.99$:

Dropping that one path from N=3 to N=2 raises its expected downtime contribution from about half a minute a year (0.5256 min/yr) to 52.56 minutes a year, a 100x jump that would consume the entire annual budget for the 99.99% target on that one path alone. That's the concrete argument for why redundancy counts on in-scope critical services are protected regardless of the cost target: the math shows the cut doesn't save what it looks like it saves once you price the risk.

Trade-offs and pitfalls

The classic failure is applying a flat 30% cut across the board instead of segmenting critical from non-critical, which either misses easy wins in non-critical systems or, worse, quietly erodes redundancy on a critical path because nobody explicitly modeled the budget impact. Watch for Goodhart's-law style gaming too: cutting observability or alerting spend looks free on the invoice but raises mean time to detect, which inflates the effective downtime against the same budget without showing up as an "availability" line item until an incident hits. What to protect no matter what: replica or quorum counts below the tested minimum on in-scope services, cross-region failover paths, backup and DR test cadence, and on-call staffing, because all four either directly hold the redundancy math above or determine how fast you can react when it fails.

Plan overview: run hypothesis-driven tests isolating three causes (data drift, rule coverage gaps, upstream input changes) using logging, A/B controlled rollout, and targeted remediation.

Data to collect (baseline + ongoing):

• Historical model inputs, feature distributions, and prediction labels for last 90 days.
• Confusion matrices, per-class precision/recall, confidence scores, and MTTR by ticket type.
• Triage rule hits, fallback cases, and rule evaluation traces.
• Upstream metadata: client versions, ingestion schemas, parsers, and timestamps of recent deployments.
• Sample raw tickets (text/attachments) for manual review.

Controlled test design:

1. Canary vs control A/B: split incoming traffic 10/90. Canary applies current triage; control uses a safe fallback (human triage or last-known-good model/rule set). Measure misclassification rate, confidence, and MTTR over 48–72h; log full inputs.
2. Feature shadowing: run a shadow pipeline that records model outputs for both current features and features normalized to historical distributions.
3. Rule toggle experiments: isolate suspect rules by toggling them in canary to see classification changes.

Hypothesis-specific diagnostics & remediation:

• Data drift:
  • Diagnose: compare feature distribution (KS test), drop in confidence, increased error rate correlated with feature shifts.
  • Remediate: retrain model with recent labeled samples, add drift detectors with alerts, normalize/standardize inputs, schedule regular re-evaluation.
• Rule coverage gaps:
  • Diagnose: find high-error ticket types where no rule fired or multiple conflicting rules triggered; manual review of samples.
  • Remediate: add/adjust rules, prioritize rule tests in CI, add rule conflict resolution and unit tests, introduce a fallback to human triage for low-confidence cases.
• Upstream input changes:
  • Diagnose: correlate error spikes with upstream deploy timestamps, schema validation failures, parser errors, or missing fields; inspect raw payload diffs.
  • Remediate: roll back faulty upstream change or implement backward-compatible parsing, add schema validation and contract tests, add monitoring for key field presence and format.

Success criteria and rollout:

• Success = misclassification rate returns to baseline and MTTR improves within SLOs for 2 consecutive days.
• After validation in canary, rollout fixes incrementally with monitoring and automatic rollback on regressions.

Post-test: conduct blameless postmortem, label new samples for future training, and implement automated monitoring + alerting for the specific root cause metrics discovered.

Direct answer

Extending an LRU (least-recently-used) cache with per-entry time-to-live (TTL, a lifespan after which an entry is no longer considered valid) means treating expiry and eviction as two independent mechanisms layered on the same structure: TTL decides whether an entry is even valid to serve at all, LRU decides who gets removed when the cache is full, regardless of whether that entry's TTL has expired yet. Safe concurrent access comes from sharding the cache into N independently locked pieces by key hash, so unrelated keys never contend for the same lock, rather than one global lock serializing every request. The one genuine race to guard against is a reader and an expiry-driven cleanup touching the same key at the same instant, which a single lock per shard around each operation's full check-then-mutate sequence eliminates by construction.

Structured elaboration

Data structures. Per shard: an ordered map (a doubly linked list plus a hash index, or collections.OrderedDict in Python) storing key -> (value, expires_at). get first checks expires_at against the current time; if expired, the entry is deleted and treated as a miss, never served stale. If still valid, it is moved to the most-recently-used end, same as a plain LRU. put inserts or updates the entry with a fresh expires_at, then evicts from the least-recently-used end while the shard is over capacity, exactly as a TTL-unaware LRU would, an unexpired entry can still be evicted early purely for space if it is cold enough.

Locking and sharding to minimize contention. A single lock around the whole cache serializes every request regardless of which key it touches, the same contention problem as a naive shared counter. The fix is the same: partition keys across N shards by hash(key) % N, each with its own lock, so two threads touching different shards never block each other; within a shard, the critical section (a dict lookup, a move, maybe an eviction) is small enough that a simple mutex around the whole operation is both correct and fast, a lock-free linked-list-based LRU is real engineering effort that rarely pays for itself compared to just adding more, smaller shards.

Eviction rules on TTL expiry. Two independent triggers remove an entry: lazily, on access, if its expires_at has passed; and capacity-driven, on put, evicting the least-recently-used entry regardless of that entry's own TTL state once the shard is full. A background sweep thread that proactively scans for and removes expired-but-never-accessed entries is an optional memory-hygiene addition, not a correctness requirement, lazy expiry alone already guarantees an expired entry is never served, it just might sit unreclaimed in memory until LRU pressure or a sweep removes it.

Race conditions between expiry and access. Because the expiry check and the touch-or-evict mutation both happen inside the same lock acquisition as one atomic operation, no other thread on that shard can observe a half-expired state in between. Two remaining subtleties matter: use a monotonic clock (one that only ever moves forward, unaffected by wall-clock adjustments) for expiry comparisons, since a wall-clock adjustment (an NTP correction, a daylight-saving shift) can jump time backward and silently corrupt TTL math; and treat the exact boundary consistently, an access at precisely expires_at should be treated as expired, not as a coin flip.

Code

import threading, time
from collections import OrderedDict

class TTLLRUCache:
    def __init__(self, capacity, default_ttl):
        self._capacity = capacity
        self._default_ttl = default_ttl
        self._store = OrderedDict()  # key -> (value, expires_at)
        self._lock = threading.Lock()

    def get(self, key, now=None):
        now = time.monotonic() if now is None else now
        with self._lock:
            entry = self._store.get(key)
            if entry is None:
                return None
            value, expires_at = entry
            if now >= expires_at:
                del self._store[key]
                return None
            self._store.move_to_end(key)
            return value

    def put(self, key, value, ttl=None, now=None):
        now = time.monotonic() if now is None else now
        ttl = self._default_ttl if ttl is None else ttl
        with self._lock:
            if key in self._store:
                del self._store[key]
            self._store[key] = (value, now + ttl)
            self._store.move_to_end(key)
            while len(self._store) > self._capacity:
                self._store.popitem(last=False)


class ShardedTTLLRUCache:
    def __init__(self, shard_count, capacity_per_shard, default_ttl):
        self._shards = [TTLLRUCache(capacity_per_shard, default_ttl) for _ in range(shard_count)]
        self._shard_count = shard_count

    def _shard_for(self, key):
        return self._shards[hash(key) % self._shard_count]

    def get(self, key, now=None):
        return self._shard_for(key).get(key, now=now)

    def put(self, key, value, ttl=None, now=None):
        self._shard_for(key).put(key, value, ttl=ttl, now=now)


if __name__ == "__main__":
    # basic TTL expiry
    cache = TTLLRUCache(capacity=2, default_ttl=10)
    cache.put("a", 1, now=0)
    print("get a at t=5 (not expired) ->", cache.get("a", now=5))
    print("get a at t=15 (expired, ttl=10) ->", cache.get("a", now=15))

    # LRU eviction independent of TTL (long ttl so eviction is purely LRU-driven)
    cache2 = TTLLRUCache(capacity=2, default_ttl=100)
    cache2.put("x", 100, now=0)
    cache2.put("y", 200, now=1)
    cache2.get("x", now=2)          # touch x -> y becomes LRU
    cache2.put("z", 300, now=3)     # evicts y (LRU), not due to TTL (ttl=100, still valid)
    print("y evicted by LRU (not TTL) ->", cache2.get("y", now=4) is None)
    print("x survived ->", cache2.get("x", now=4) == 100)

    # concurrency check: 16 threads, 200 put-then-get round trips each, shared 8-shard cache
    shared = ShardedTTLLRUCache(shard_count=8, capacity_per_shard=64, default_ttl=1000)
    errors = []

    def worker(tid):
        for i in range(200):
            key = f"t{tid}-{i}"
            shared.put(key, (tid, i), now=0)
            val = shared.get(key, now=0)
            if val != (tid, i):
                errors.append((key, val))

    threads = [threading.Thread(target=worker, args=(t,)) for t in range(16)]
    for th in threads:
        th.start()
    for th in threads:
        th.join()
    print("concurrent put/get across 16 threads, errors ->", len(errors))

    # exact boundary: an access at precisely expires_at counts as expired
    cache3 = TTLLRUCache(capacity=2, default_ttl=1.0)
    cache3.put("k", 42, now=0.0)
    print("exactly-at-expiry (t=1.0) counts as expired ->", cache3.get("k", now=1.0) is None)

Output (executed as shown, using an injected clock so TTL behavior is deterministic and reproducible):

get a at t=5 (not expired) -> 1
get a at t=15 (expired, ttl=10) -> None
y evicted by LRU (not TTL) -> True
x survived -> True
concurrent put/get across 16 threads, errors -> 0
exactly-at-expiry (t=1.0) counts as expired -> True

The concurrency check ran 16 threads, each performing 200 put-then-get round trips against a shared 8-shard cache, and confirmed every value was read back exactly as written with zero errors, evidence the sharded locking is actually safe under real concurrent access, not just single-threaded correctness.

Trade-offs and pitfalls

• Lazy-only expiry can leave cold, expired entries resident indefinitely if the cache has generous capacity and that key is never looked up again; add a background sweep if bounding memory tightly matters more than the extra CPU cost of scanning.
• Shard count is a real tuning knob, not a fixed choice. Too few shards under-spreads lock contention; too many wastes memory on mostly-empty per-shard structures and adds bookkeeping overhead.
• Distributed sharding is the same idea one layer up. The identical hash-the-key-to-a-shard principle extends from in-process locks to independent nodes in a distributed cache: shard by consistent hashing across nodes so the cache's total capacity and throughput scale past one machine, with a routing layer (client-side or proxy-side) sending each request to the node that owns its key's shard.
• Persistence across restarts should stay proportionate to what TTL data actually needs. Since every entry already has a bounded lifespan, a periodic best-effort snapshot to disk or an external store, reloaded at startup and immediately filtered to discard anything whose expires_at has already passed relative to current wall-clock time, is sufficient; treating this like a database's durability guarantee (distributed consensus, write-ahead logs) is usually over-engineering for data that was always meant to expire.
• A common wrong turn: using wall-clock time (time.time()) instead of a monotonic clock for TTL comparisons. It works in every normal test run and fails exactly once, silently, during a real clock adjustment in production, which is precisely why it is easy to ship and hard to catch without deliberately testing for it.

Direct answer. Decompose by responsibility, not by line count: pull out one function per distinct concern (parse, validate, compute, persist, notify), give each a name that states its single job, and compose them in a thin orchestrator that reads like the original steps.

Before (one function, five jobs)

def handle_order(raw):
    order_id = raw["id"]; email = raw["email"]; items = raw["items"]      # parse
    if not items: raise ValueError("order has no items")                  # validate
    if "@" not in email: raise ValueError("bad email")
    total = sum(i["price"] * i["qty"] for i in items)                     # compute
    if total > 100: total *= 0.9
    DB[order_id] = {"email": email, "total": total}                       # persist
    OUTBOX.append(f"receipt to {email}: ${total:.2f}")                    # notify
    return total

After

def parse_order(raw: dict) -> dict:
    return {"id": raw["id"], "email": raw["email"], "items": raw["items"]}

def validate_order(order: dict) -> None:
    if not order["items"]: raise ValueError("order has no items")
    if "@" not in order["email"]: raise ValueError("bad email")

def compute_total(items, *, bulk_discount_threshold=100, bulk_discount_rate=0.9) -> float:
    total = sum(i["price"] * i["qty"] for i in items)
    return total * bulk_discount_rate if total > bulk_discount_threshold else total

def save_order(db, order_id, email, total) -> None:
    db[order_id] = {"email": email, "total": total}

def send_receipt(outbox, email, total) -> None:
    outbox.append(f"receipt to {email}: ${total:.2f}")

def handle_order_v2(raw, db, outbox) -> float:
    order = parse_order(raw)
    validate_order(order)
    total = compute_total(order["items"])
    save_order(db, order["id"], order["email"], total)
    send_receipt(outbox, order["email"], total)
    return total

Verified against two cases: a two-item order totaling 120 (discounted to 108.0) and a single-item order totaling 20 (no discount) -- both versions return identical totals.

How to confirm you didn't change behavior

• Run the OLD function and the NEW orchestrator against the same set of inputs (including edge cases: empty items, exactly-at-threshold totals, malformed email) and diff the outputs.
• If there's an existing test suite, run it unchanged against the new code first -- a passing suite that never exercised the discount boundary is itself a gap worth flagging, not just a green check mark.
• Where no tests exist yet, write characterization tests against the OLD function first, then refactor, then confirm the same tests still pass -- this way the safety net exists before you touch anything.

Trade-offs and pitfalls

• compute_total's discount threshold is now a named, testable, overridable parameter instead of a buried magic number -- a deliberate improvement, but note it as a design decision so a reviewer knows it wasn't accidental scope creep.
• Over-decomposing (a function per line) makes you trace five files to understand one flow; stop splitting when each piece maps to one clear reason to change.
• The orchestrator itself now has a job: sequencing. Keep it free of business logic so it doesn't quietly become a sixth responsibility.

Framing

The ask here is explicitly upstream of load testing: before spending money provisioning for a 3x forecast, first find out whether the forecast itself is trustworthy, and whether the answer is even "add capacity" at all, the launch might shift traffic MIX rather than volume, or existing headroom might already cover it.

Validating the forecast itself

• Compare the forecast's methodology against a comparable past launch if one exists. If a similar past feature launch predicted 2x and actually delivered 1.3x, that's real evidence to widen or discount the current 3x forecast's confidence, not just trust the point estimate at face value.
• Break the 3x forecast into its components: is it 3x total requests, 3x users making the same per-user request volume, or the same users making 3x more requests each? Each has different infrastructure implications, for example 3x users usually scales roughly linearly across the system, but 3x per-user activity might concentrate load on one specific endpoint or database table rather than scaling everything uniformly.
• Run a smaller, controlled real-world test where possible: a percentage rollout or a beta cohort exposed to the new feature ahead of full launch, to get an actual measured multiplier from real traffic instead of relying purely on a forecast model.

Measurements to build confidence

Instrument the beta or percentage-rollout cohort's actual traffic multiplier and compare it against the 3x forecast in real time as rollout percentage increases. Separately, confirm CURRENT headroom, if the system already runs at 30% average utilization with room to 3x before hitting any known ceiling, the forecast may not require new provisioning at all, though that conclusion should be checked against PEAK utilization, not average, since peak is the number that actually matters for headroom.

Deciding whether additional capacity is actually needed

Set the decision threshold explicitly, ahead of time, for example: "if the beta cohort's measured multiplier plus a safety margin exceeds current peak headroom, provision ahead of full launch; if it's within headroom, monitor and scale reactively instead." This avoids two common failure modes: provisioning for every forecast regardless of whether current headroom already covers it (wasted cost), and always waiting for reactive autoscaling on a launch-day traffic profile whose ramp speed reactive autoscaling's own reaction time genuinely can't keep up with.