Lyft Software Engineer (Mid-Level) Interview Preparation Guide

Direct answer

Map stakeholders along what they actually care about and how often you'd realistically need to sync with them, then pick a trust signal for each one you could genuinely observe, being looped in before a decision affects them, or being asked to weigh in rather than simply informed afterward, rather than something unfalsifiable like "they trust me."

Structured elaboration

Worked example

A backend engineer building a new data-export feature for enterprise customers. Product leadership cares whether it unblocks a specific renewal, so you'd sync weekly in a roadmap review. Support keeps hearing requests for exports in a specific format, and a real trust signal is a support lead pinging you directly with the exact ask instead of filing a generic ticket that gets triaged blind. Legal cares because the export could include personally identifiable information (PII, data that could identify a specific person), and the trust signal is Legal reaching out proactively when a similar project starts elsewhere, because they've learned you'll flag the risk yourself rather than needing to be chased down.

Trade-offs and pitfalls

Don't apply one sync cadence to everyone, meeting weekly with Legal when they only need you twice a year wastes everyone's time and signals you don't understand the relationship. Also, more meetings attended is not itself a trust metric, being proactively looped in before you'd otherwise have to ask is the real signal, because it means the other side has updated how they think about working with you.

Direct answer
A kth-smallest/largest function needs six edge-case tests beyond the normal case: empty input, a single element, duplicate values, k out of range (too low or too high), k equal to the length (asking for the max/min of the whole list), and a large input to confirm both correctness and reasonable performance at scale.

Structured elaboration and worked example (executed)

def kth_smallest(nums, k):
    if not nums:
        raise ValueError("empty input")
    if k < 1 or k > len(nums):
        raise IndexError("k out of range")
    return sorted(nums)[k-1]

def test_empty():
    try:
        kth_smallest([], 1)
        assert False, "should have raised"
    except ValueError:
        pass  # catches: calling the function on an empty collection at all

def test_single():
    assert kth_smallest([7], 1) == 7  # catches: off-by-one in index math when len==1

def test_duplicates():
    assert kth_smallest([5,3,5,1,5], 3) == 5  # sorted [1,3,5,5,5]; catches: dedup logic that silently drops repeats

def test_k_out_of_range():
    for bad_k in (0, 5):
        try:
            kth_smallest([1,2,3], bad_k)
            assert False
        except IndexError:
            pass  # catches: missing bounds check, which would otherwise raise a confusing IndexError from sorted[k-1] itself

def test_k_equals_length():
    assert kth_smallest([4,1,3], 3) == 4  # catches: an off-by-one that treats len(nums) as out of range

def test_large_input():
    import random
    random.seed(42)
    nums = [random.randint(-10**6, 10**6) for _ in range(50000)]
    k = 25000
    assert kth_smallest(nums, k) == sorted(nums)[k-1]  # catches: correctness diverging from a full-sort baseline at scale

test_empty()
test_single()
test_duplicates()
test_k_out_of_range()
test_k_equals_length()
test_large_input()
print("ALL kth_smallest TESTS PASSED")

Running this script produces: ALL kth_smallest TESTS PASSED.

Why each test catches a specific bug class

• Empty input: catches implementations that call sorted([])[k-1] and get an unhelpful IndexError instead of a clear, documented failure.
• Single element: the smallest interesting off-by-one target; k-1 indexing is exactly where a 0-based/1-based confusion shows up.
• Duplicates: a common bug is treating the problem as "kth DISTINCT value" by accident (e.g. deduplicating before sorting), which silently changes the answer whenever duplicates exist.
• k out of range: without an explicit bounds check, Python's own list indexing will still raise something, but the exception type/message becomes an implementation accident rather than a documented contract, which breaks callers who catch a specific exception type.
• k equals length: the boundary between "valid" and "just past valid"; an implementation using k > len(nums) - 1 instead of k > len(nums) would incorrectly reject this case.
• Large input: a quickselect-style O(n) implementation (as opposed to full-sort O(n log n)) is the realistic optimization for this problem; testing it against a full-sort baseline at scale is the only way to catch a partition-logic bug that happens to work on tiny hand-picked inputs but misbehaves on larger, more varied data.

The kth-LARGEST variant is the same test list with the selection direction reversed (sorted(nums)[-k] or an equivalent), and needs its own explicit 10-item edge-case pass covering the same six categories plus negative numbers, all-identical elements, an already-sorted-ascending input, an already-sorted-descending input, and k=1 (the single maximum) specifically, since a max-heap-based implementation can have a distinct off-by-one from a min-heap-based one even though the two problems look symmetric.

Trade-offs & pitfalls
A fast implementation (quickselect, or a heap of size k) is more error-prone at the boundaries than the simple sort-and-index version shown here, precisely because it manages partial state (a partition point, or a heap that must stay exactly size k); the sort-and-index version is the right "oracle" to test a faster implementation against on randomized large inputs, exactly as the large-input test above does.

Direct answer

Counting set bits (population count, or popcount) can be done with a naive
loop that shifts and masks one bit at a time, or with Brian Kernighan's trick,
which repeatedly clears the lowest set bit using n & (n - 1) and so only
loops once per 1-bit rather than once per bit position. The same
set/clear/toggle bit-trick vocabulary (especially XOR, exclusive-or, where
x ^ x = 0 and x ^ 0 = x) solves the "find the number that appears once
while every other number appears twice" problem in O(1) extra space, and
extends, with more bookkeeping, to the "every other number appears three
times" variant.

Structured elaboration

Naive popcount. Inspect the lowest bit, shift right, repeat until the
number is zero. This always runs a number of iterations equal to the bit
width (64 for a 64-bit integer), regardless of how many bits are actually set.

Brian Kernighan's trick. n & (n - 1) clears exactly the lowest set bit
of n and leaves every other bit untouched. Why: n - 1 flips the lowest
set bit to 0 and flips every lower bit (which were all 0) to 1; ANDing with
the original n keeps all the higher bits identical (since n and n - 1
agree above the lowest set bit) but zeroes out the lowest set bit specifically,
because that bit is 1 in n and 0 in n - 1, and every bit below it was
already 0 in n so it stays 0 after the AND. Repeating this once per set bit
until the value reaches zero means the loop runs exactly as many times as
there are 1-bits, not once per bit position, which is why it beats the naive
loop whenever the number is sparse (few set bits relative to its width).

Single number, every other appears twice. XOR all elements together.
Every pair cancels (x ^ x = 0), the running XOR of 0 with the leftover
unpaired value is that value itself (x ^ 0 = x), so the final accumulator
is exactly the one number that had no partner. This needs one pass and O(1)
extra space.

Single number, every other appears three times. XOR alone cannot solve
this because XOR only detects an odd count of exposures, and three is odd,
so a value seen three times would still contribute itself to the running XOR
instead of cancelling out. The fix is to track, per bit position, the count of
that bit across all numbers modulo 3. A bit that belongs to the answer is set
in exactly one occurrence of the unique number and in zero or three
occurrences of every tripled number, so (total ones at that bit position) mod 3
recovers the answer bit by bit. A compact constant-space way to do this uses
two accumulators, ones and twos, that together simulate a 3-state counter
per bit (seen 0, 1, or 2 times mod 3), clearing a bit from both the moment it
would reach a third occurrence.

Same vocabulary, other absorbed variants:

• Power-of-two check: a positive integer is a power of two exactly when it
  has a single set bit, so n > 0 and (n & (n - 1)) == 0 reuses the exact
  Kernighan clear-lowest-bit trick above (clearing the only set bit must
  produce zero).
• Add without using +: repeatedly compute carry = (a & b) << 1 (where
  both bits are 1, a carry is generated one position to the left) and
  a = a ^ b (bits that differ sum to 1 with no carry), then set b = carry
  and repeat until there is no carry left. This is binary addition done
  manually with bitwise ops instead of the language's + operator.
• Missing number in 1..n (an array of length n-1 holding a permutation of
  1..n with exactly one value removed, or the equivalent 0..n framing over
  an array of length n): XOR every value in the full range together with
  every value actually present in the array; every number that appears in
  both cancels, leaving only the missing one, the same pairing-cancellation
  idea as the single-number problem, just applied to a known range of values
  instead of duplicated array entries.

Worked example

def popcount_naive(n: int) -> int:
    count = 0
    while n:
        count += n & 1
        n >>= 1
    return count

def popcount_kernighan(n: int) -> int:
    count = 0
    while n:
        n &= n - 1  # clears the lowest set bit
        count += 1
    return count

def single_number(nums):
    res = 0
    for x in nums:
        res ^= x
    return res

def single_number_triplets(nums):
    ones, twos = 0, 0
    for x in nums:
        ones = (ones ^ x) & ~twos
        twos = (twos ^ x) & ~ones
    return ones

def is_power_of_two(n: int) -> bool:
    return n > 0 and (n & (n - 1)) == 0

def missing_number(nums):
    res = len(nums)
    for i, x in enumerate(nums):
        res ^= i ^ x
    return res

print(popcount_naive(11), popcount_kernighan(11))            # 3 3   (11 = 0b1011)
print(popcount_naive(255), popcount_kernighan(255))          # 8 8
print(single_number([4, 1, 2, 1, 2]))                         # 4
print(single_number_triplets([2, 2, 3, 2]))                   # 3
print(single_number_triplets([0, 1, 0, 1, 0, 1, 99]))         # 99
print(is_power_of_two(16), is_power_of_two(18))               # True False
print(missing_number([3, 0, 1]))                              # 2

Output (verified by running this exact code):

3 3
8 8
4
3
99
True False
2

For popcount_naive(11), 11 is 0b1011, so the naive loop runs 4 times (one
per bit position up to the highest set bit) while Kernighan's version runs
exactly 3 times, once per set bit, clearing them in order from the lowest:
1011 -> 1010 -> 1000 -> 0000.

Trade-offs & pitfalls

• Brian Kernighan's trick only wins when the input is sparse; for a number
  with most bits set (e.g. close to all-ones), it degenerates to roughly the
  same number of iterations as the naive loop, so it is a best-case
  improvement, not a worst-case one. In real code, a hardware popcount
  instruction (exposed in most languages as a builtin) is O(1) and should be
  preferred when the question does not specifically forbid it; both loop
  versions exist to demonstrate the underlying reasoning.
• The two-accumulator triple-occurrence trick is correct but easy to get
  subtly wrong: the update order (ones before twos, each masked against
  the other's complement) matters, and swapping the two lines silently
  produces a different, incorrect state machine.
• Negative numbers need care in the triple-occurrence and add-without-plus
  tricks specifically: languages with arbitrary-precision integers (Python)
  do not have a fixed bit width, so operations meant to model fixed-width
  wraparound (like the carry-based adder) need explicit masking to the
  intended bit width, or they silently produce a value with the wrong sign
  or an unbounded number of leading one-bits for negative numbers.
• For missing-number-style XOR tricks, be careful that the index range and
  value range actually line up as claimed (e.g. "array of length n containing
  values 0..n with one missing"); XOR-ing mismatched ranges will silently
  return a wrong answer rather than erroring.

Instrumentation

Add both client-side real user monitoring (RUM, measuring what the user actually experiences, including network and rendering, not just server time) and a server-side span breakdown of the login call chain (credential check, database lookup, token issuance, any downstream calls). Tag every measurement with client platform, geography, and (once testing starts) experiment bucket, so a regression or improvement can be attributed to a segment rather than only seen in aggregate.

A/B testing

Hold back a control group on the current login path and ship each candidate optimization to a treatment group, measuring both latency (p50/p95/p99) and the actual conversion metric (login completion rate, or downstream signup/purchase completion) with statistical significance, run long enough to smooth out day-of-week and time-of-day effects rather than reading a few hours of data.

Prioritizing fixes

Use the trace breakdown to find which single hop contributes most to the p99, and start there: an uncached lookup or a serial call that could be parallelized is usually a quick win that produces fast, low-risk signal. Reserve architectural changes (moving the auth check closer to the edge, adding a dedicated cache tier, restructuring the call chain) for after the quick wins are exhausted, since they take longer to build and are riskier to roll back.

Rollout and rollback

Ship behind a feature flag with a staged rollout (for example 1% then 10% then 50% then 100% of traffic), with automatic rollback triggers tied to BOTH latency and conversion regressions, not errors alone, since a change can be error-free and still slow down or hurt conversion.

Quantifying business impact

It's well established across the industry that added latency correlates with reduced conversion, but the actual sensitivity (how many conversions are gained per millisecond improved) is specific to this product and this funnel, so it must be measured from your own A/B data, not assumed from a general rule of thumb. From the experiment data, bucket users by observed login latency (or by control vs. treatment) and compare conversion rate across buckets; the resulting curve is your own quantified relationship, which is what should drive whether the next optimization is worth the engineering cost.

Direct answer: Dijkstra's complexity depends entirely on the priority-queue implementation: $O(V^2)$ with a plain array, $O((V+E)\log V)$ with a binary heap, and $O(E + V \log V)$ with a Fibonacci heap (the theoretically best, rarely used in practice due to large constants). A* uses the same underlying machinery as Dijkstra but adds a heuristic $h(n)$ estimating remaining distance to the goal, which - if admissible (never overestimates) and consistent (satisfies a triangle-inequality-like property) - guarantees the optimal path is still found while typically exploring far fewer nodes than Dijkstra by prioritizing promising directions.

Structured elaboration

• Array-based Dijkstra: finding the minimum-distance unvisited vertex is an $O(V)$ scan, done $V$ times, giving $O(V^2)$ - reasonable for dense graphs where $E \approx V^2$ anyway.
• Binary-heap Dijkstra: each extract-min is $O(\log V)$, and each edge relaxation may trigger a decrease-key (also $O(\log V)$, or handled via re-insertion with lazy deletion in many implementations), giving $O((V+E)\log V)$ - the standard choice for sparse graphs.
• Fibonacci-heap Dijkstra: decrease-key is $O(1)$ amortized, so the bound improves to $O(E + V \log V)$ - asymptotically best, but the large constant factors and implementation complexity of Fibonacci heaps mean binary heaps usually win in practice except at very large scale.
• A*: identical algorithmic skeleton to Dijkstra, but the priority queue orders by $f(n) = g(n) + h(n)$ (cost-so-far plus heuristic estimate to goal) instead of just $g(n)$. An ADMISSIBLE heuristic (never overestimates true remaining cost) guarantees A* still finds the optimal path; a CONSISTENT heuristic (satisfies $h(n) \le \text{cost}(n, n') + h(n')$ for every edge) additionally guarantees no node needs to be re-expanded once popped, matching Dijkstra's efficiency guarantees while exploring fewer nodes in the common case, because the heuristic actively steers the search toward the goal instead of expanding uniformly outward in all directions.

Worked example

Consider road-network routing where straight-line (Euclidean) distance to the destination is used as $h(n)$: it's admissible (straight-line distance never overestimates actual road distance, which must be $\ge$ straight-line) and consistent (the triangle inequality holds for Euclidean distance). On a grid or road network, this heuristic causes A* to expand nodes roughly in an ellipse oriented toward the goal, rather than Dijkstra's expanding circle in all directions - for a goal far from the source, this concretely means A* explores a small fraction of the nodes Dijkstra would, even though both are guaranteed to find the same optimal-cost path.

Trade-offs & pitfalls

• A* is only as good as its heuristic - a poorly-chosen or non-admissible heuristic can make it explore MORE nodes than Dijkstra (if it misleads the search) or, worse, return a suboptimal path (if it's not admissible).
• When there's no useful domain-specific heuristic available (e.g. an abstract graph with no geometric embedding), A* degenerates to Dijkstra (using $h(n) = 0$ everywhere is trivially admissible and consistent).
• Fibonacci heaps are a common "textbook-optimal, practically-never-used" answer - know the asymptotic bound but be ready to say WHY binary heaps usually win in real systems (much smaller constant factors, simpler implementation, better cache behavior).

Direct answer

A genuinely non-negotiable constraint (a legal, regulatory, or security requirement with no waiver path) changes trade-off analysis from optimizing across all options to first pruning the option set down to only what's compliant, and only then optimizing cost, performance, or time-to-market among what's left. It doesn't get a weight in a scoring matrix alongside other factors; it eliminates options before scoring starts.

Structured elaboration

Treat a hard constraint as a filter applied in a distinct first pass, before any cost or performance comparison: list every candidate architecture, remove any that violate the constraint outright (not "weight them lower", remove them), and only run the normal trade-off analysis (cost, latency, time-to-market) across what survives. This ordering matters because scoring an already-infeasible option wastes analysis effort and can create a false sense that it was seriously considered.

Two realistic examples of constraints that eliminate options outright, not just penalize them:

PCI-DSS (Payment Card Industry Data Security Standard) card-data scope. If a design stores raw card numbers to power broader analytics, that option is gone the moment PCI-DSS applies, regardless of how much better the analytics would be; the only surviving options tokenize card data (replace the real card number with a random, non-sensitive placeholder token that maps back to it only inside the certified payment vault) or route it through an already-certified payment gateway.

Regulatory data residency. A requirement that a jurisdiction's data (for example, European Union customer data under data-protection law) must remain within that jurisdiction's borders eliminates any single-region deployment outside it outright, even if that region is meaningfully cheaper or already has spare capacity; there's no scoring adjustment that makes a non-compliant region viable.

Worked example

An illustrative scenario: a new payments feature needs to store transaction detail for both fraud analytics and customer support. Three candidate designs exist: (A) store full raw card data plus transaction detail for maximum analytics flexibility, (B) tokenize card data and store only tokens plus transaction metadata, (C) tokenize card data and additionally keep only aggregated, non-identifying analytics rather than per-transaction detail. Once PCI-DSS scope is applied as a hard filter, option A is eliminated outright, not down-weighted, because storing raw card data outside a certified, PCI-scoped environment isn't a slower or costlier version of the same design, it's a design that isn't legally available. The remaining trade-off analysis, cost and analytics fidelity, runs only between B and C: B keeps more per-transaction detail at a higher tokenization and storage cost (illustratively, storing a token plus full transaction metadata for 10 million transactions/month at roughly $0.0004/record runs about $4,000/month), C is cheaper (aggregating to per-customer monthly summaries cuts that record volume by roughly 95%, to around $200/month) but sacrifices per-transaction granularity for fraud analysis. That second-stage comparison is where a normal cost-vs-capability trade-off analysis applies; the first stage had none, only elimination.

Trade-offs & pitfalls

• The most common mistake is treating a hard constraint as one more weighted factor in a scoring matrix; that understates it and risks a stakeholder pushing back with "can we just accept a bit more risk here," when the honest answer is there's no risk-acceptance path available.
• Document what was eliminated and why, not just what was chosen; a stakeholder who wasn't in the room needs to see that the more attractive option was never actually on the table, not that it lost a close call.
• Distinguish a genuinely non-negotiable constraint from a strongly-preferred one; treating a soft preference as a hard filter needlessly shrinks the option set and can be walked back once challenged, which undermines trust in the rest of the analysis.
• Residual risk still needs to be documented and mitigated even after the hard filter is applied; "compliant" doesn't mean "risk-free," it means the specific eliminated risk is off the table.

Direct answer

Lead with the fact and its consequence, up front, for every audience, don't bury it in process. Then tailor what follows to what each audience actually needs to act on: engineering leadership needs enough detail to approve a path, product needs the scope trade-off, and the customer needs a plan and a date, not internal detail.

The move: same facts, different framing per audience

• Common backbone: state the risk and its consequence directly and early in every version. Leading with the process you used to find it, before the finding itself, makes people brace through unnecessary preamble.
• Engineering leadership: keep the technical detail, and bring a recommended option, since they're the ones positioned to evaluate the trade-off and approve resourcing.
• Product: strip most of the technical detail, keep the scope trade-off (ship less versus slip the date) and a recommendation, since that's the decision that's actually theirs to make.
• External customer: no internal detail. What you know, what you're doing about it, and when they'll hear next, with genuine empathy for their stake, but without promising a date you can't actually hold.
• Keep the underlying facts identical across all three conversations. Only the framing and level of detail should change; if the facts themselves start to diverge between versions, that's what actually breaks trust later.

Worked example

A technical risk in a third-party dependency (say, a payments SDK that silently drops webhook retries once its internal queue backs up under load, discovered two weeks before a major customer's go-live) could delay a critical deliverable by several weeks. To engineering leadership, you lay out the technical root cause and two mitigation options, build a lightweight retry-and-reconciliation layer in-house within the deliverable's timeline, or delay the launch two weeks for the vendor's promised fix, with a recommendation, so they can approve resourcing. To product, you skip the technical root cause and present it as a scope decision: absorb a delay, or reduce scope to protect the date, with your recommendation. To the external customer, you say plainly that you've identified a technical issue that affects their timeline, that you're actively working it, and that you'll have a concrete update by next Friday, without speculating on root cause or an exact resolution date you can't yet commit to.

Trade-offs and pitfalls

Softening the message differently for different audiences until the facts themselves quietly diverge is the real risk, someone eventually compares notes across the three conversations and it looks like you told different stories rather than the same story at different altitudes. Giving the customer an overly precise date under pressure to reassure them, before you actually know it, produces a broken promise later, which damages trust more than an honestly vague near-term update would have.

Direct answer. A shape-mismatch error tells you two tensors disagreed in dimension somewhere in the forward pass, but the traceback often points at the operation that FAILED, not the operation that introduced the wrong shape several layers earlier, so the debugging process is really about walking the shape forward from the input until it diverges from what you expect.

Step-by-step approach.

1. Print the input shape first, and compare it against what the first layer actually expects. A surprising number of shape bugs are simply "the input isn't shaped the way I assumed," not a bug in the model at all.
2. Use a model summary tool (or manually print .shape after each layer in a quick forward pass) to see the shape at every stage in one pass, rather than binary-searching by commenting out layers one at a time.
3. Add explicit shape assertions inside custom layers, at the point where a specific shape is assumed (assert x.shape[-1] == self.expected_dim, f"got {x.shape}"). This turns a downstream, confusing shape error into an immediate, precisely-located one the next time the bug is triggered, which pays for itself the first time someone else hits a variant of the same bug.
4. Write a small unit test with a known, fixed input shape that exercises just the suspect layer or block in isolation, rather than the whole model, so you can iterate on the fix without paying the cost of a full forward pass through everything else.

A concrete example of why step 1 matters. A very common real case: a model expects batch-first input (batch, seq_len, features) but receives (seq_len, batch, features) from a data loader or a different framework's convention. The shapes are individually valid tensors, nothing crashes until several layers in when a dimension that "coincidentally" matched for a while finally doesn't, at which point the error message points at a layer far from the true cause (the data loader).

The unit test that prevents recurrence. Something as small as:

def test_encoder_output_shape():
    x = torch.randn(4, 10, 32)  # (batch=4, seq_len=10, features=32), the CONTRACT this layer expects
    out = encoder(x)
    assert out.shape == (4, 10, 64), f"expected (4, 10, 64), got {out.shape}"

run in CI on every change to the layer or anything upstream of it, catches this class of bug the moment a shape contract is violated, rather than three deploys later when someone finally notices predictions look wrong.

Direct answer

A handful of small, repeatable techniques do more for me than trying to stay calm through willpower: a deliberate pause before reacting to any new piece of information, separating what's actually urgent from what just feels urgent, and consciously not matching the emotional intensity of whoever I'm talking to, whether that's a stressed teammate or an upset client. One of these, the deliberate pause, directly changed the outcome of a real incident, and I've since taught it to more junior engineers on my team.

Structured elaboration

• Deliberate pause before reacting: when new information arrives mid-incident, an alert, a concerning message, a client escalation, I take a few seconds before responding rather than reacting to the first interpretation that comes to mind, since the first read under pressure is often the most alarming one, not the most accurate one.
• Separating actually-urgent from feels-urgent: pressure makes everything feel equally critical. I explicitly ask whether something needs action in the next minute, or whether it only feels that way because someone nearby is anxious about it, before deciding how fast to move.
• Not mirroring escalated emotion: when someone else, a teammate, a manager, or a client, is visibly stressed or upset, I deliberately keep my own tone and pace steady rather than matching theirs. Escalating emotionally in response to someone else's stress doubles the tension in the room without adding any actual information; staying level is often what lets the other person de-escalate too.
• The same techniques apply to a client escalation, not just an internal incident: when a client is angry on a call during an active incident, the pause and the steady tone matter even more, since an anxious or defensive reaction in that moment can do more damage to the relationship than the incident itself.
• Teaching it: these techniques are learnable habits, not personality traits, so I've explicitly named them out loud to junior engineers in the moment, prompting them to take a breath and check what's actually urgent before acting, rather than assuming people pick them up by osmosis from watching me.

Worked example

During an incident, an alert came in that looked, at first glance, like a second, unrelated system was also failing. My first instinct was to immediately pull in a second team to investigate that system too, doubling the number of people scrambling. I used my own pause habit, a few seconds before acting on that first read, and reread the alert more carefully. It turned out to be a downstream symptom of the same root cause I was already investigating, not a second, independent failure. Pulling in that second team unnecessarily would have split focus and added coordination overhead exactly when speed mattered most; the pause let me catch that before it happened.

I've since taught this specific habit to a junior engineer on my team during a later incident, in real time: when they went to immediately escalate on a fast-moving alert, I asked them out loud to take a breath and walk through what the alert actually said versus what it felt like it meant, the same question I'd asked myself in the earlier incident. They caught, on their own, that it was a re-alert of something already being handled rather than a new issue, and afterward told me that naming the technique explicitly, rather than just modeling it silently, was what made it stick.

Trade-offs and pitfalls

The risk with telling someone to just stay calm is that it isn't actionable; it names the desired state without giving anyone a concrete practice to get there, so it doesn't actually transfer to another person. The techniques above work because they're specific enough to name and repeat, which is also why teaching them explicitly, saying the technique out loud in the moment rather than just modeling calm behavior silently, matters: someone watching a calm person under pressure often just assumes calm is a personality trait they don't have, rather than a learnable habit.

Direct answer

The first question is whether trips are actually taking longer or whether only our predictions went up. If actual trip durations rose too, something happened on the roads (an accident, a storm, an event) and the ETA (estimated time of arrival) system is probably right. If actual durations are flat while predictions jumped, our system is wrong, and I check the usual suspects in order of how often they cause sudden, single-city jumps: a recent deployment or config change, stale or broken traffic data, a drop in driver telemetry, and routing engine errors or fallbacks. I mitigate by rolling back or switching the city to a known-good data source first and finishing the diagnosis afterwards, and I capture evidence as I go so the postmortem does not depend on memory.

Triage flow

flowchart TD
  A[ETA spike in one city] --> B{Actual trip durations up too?}
  B -->|Yes| C[Real-world event: confirm, widen ranges, no rollback]
  B -->|No| D{Deploy or config change in the last hours?}
  D -->|Yes| E[Roll back, compare replayed ETAs]
  D -->|No| F{Traffic data age above normal?}
  F -->|Yes| G[Switch city to historical speeds, fix feed]
  F -->|No| H{Telemetry ingestion rate dropped?}
  H -->|Yes| I[Check ingestion pipeline and app release]
  H -->|No| J[Routing engine errors, fallbacks, map version]

What to inspect, and what each finding means

Worked example: the stale traffic table

At 11:00 the ETA for a typical 6 km cross-town trip in one city is 20 minutes instead of the usual 12. Completed trips are still taking about 12 minutes, so predictions are wrong. No deploy happened. The traffic-data age panel shows the city's speed table was last updated at 08:55.

That explains the number exactly. The table froze during rush hour, when this corridor averaged 18 km/h. At midday it normally runs at 30 km/h:

$\frac{6 \text{ km}}{18 \text{ km/h}} \times 60 = 20 \text{ min}, \qquad \frac{6 \text{ km}}{30 \text{ km/h}} \times 60 = 12 \text{ min}$

Every route in the city is being computed at 8:55 a.m. congestion levels, two hours after the congestion cleared. Only this city is affected because each city's table is built by its own job, and that job has been failing silently (for example, an upstream file format change that its parser rejects).

Immediate mitigation (minutes, not hours)

1. Switch the city to the fallback speed source: historical average speed per road segment for this hour of the week. This brings ETAs back to roughly normal immediately.
2. If a deploy is implicated, roll it back rather than fixing forward (pushing a new patch on top of the broken change instead of reverting it) during the incident.
3. Widen displayed ETA ranges and pause "arriving in 1 minute" notifications for the city while quality is uncertain.
4. Protect downstream decisions: surge pricing (the temporary price multiplier that rises when nearby demand outstrips available drivers, meant to pull more drivers into the area) and dispatch both read ETAs, because "how far away is the nearest driver" is itself computed from predicted travel time. Inflated ETAs can make dispatch think all drivers are far away and push surge up; freeze surge at its current cap (stop the multiplier from climbing any further, but leave it wherever it already is) or switch it to a supply-based signal only (compute the demand-supply imbalance from a raw count, open ride requests versus available drivers in the area, that does not depend on ETA at all), until ETAs are fixed.
5. Then fix the root cause (restart or patch the feed job) and let the city return to live speeds once data is fresh for several consecutive refreshes.

Automated remediation, so next time it fixes itself

• Freshness watchdog per city. If the table normally refreshes every 2 minutes, page at 10 minutes of age (five missed refreshes) and automatically fall back to historical speeds at 15 minutes. An hour-plus stale table would then never be served.
• Prediction-vs-reality guard. Continuously compare the median ratio of predicted to actual duration for trips completed in the last 30 minutes. If it moves beyond a set band (for example 20% above normal for 15 minutes with enough trips), trigger the fallback and page.
• Automated rollback triggers. Any ETA model, feature or map release is rolled out city by city with the same guard; tripping it reverts the release automatically.

Capturing evidence for the postmortem

• Start an incident timeline immediately: first alert, each check, each action, with timestamps.
• Snapshot the dashboards (freshness, ingestion rate, latency, fallback share, predicted vs actual) covering an hour before onset through recovery; dashboard data may be downsampled (older points get averaged into coarser buckets to save storage, so minute-by-minute detail is lost) or expire.
• Save a sample of affected requests with their full inputs: route, traffic snapshot ID or timestamp, model and map versions, output. These let you replay the bad ETAs later.
• Preserve the failing job's logs and the offending input file before anyone retries or cleans up.
• Record the impact: number of trips shown inflated ETAs, cancellation rate vs baseline, any pricing effect.
• In the postmortem, the key question is not "why did the job fail" but "why could a stale table be served for two hours": the missing freshness check is the real defect.

Trade-offs and pitfalls

• Rolling back real events. If actual trip times are up (a storm), "fixing" the ETA back down makes it wrong. Always check actuals first.
• Over-sensitive guards. An automatic fallback that triggers on every small wobble swaps good live data for coarser history too often; require sustained deviation and minimum trip counts.
• Fixing only the ETA. Check what else consumed the bad numbers (pricing, dispatch, promised delivery windows) during the incident and whether any customers need remediation.