Debugging and Systematic Troubleshooting Questions
Diagnosing defects methodically: reproducing failures, forming and testing hypotheses, reading stack traces and logs, bisecting changes, and reasoning about error handling and edge cases. Covers a disciplined root-cause approach that applies from local bugs to production issues, distinct from embedded hardware-level debugging. A universally probed engineering-craft skill.
Provide a succinct example of a recent small bug you fixed. Include the minimal code before and after in a language of your choice, explain the root cause, why your fix works, how you tested it, and what you learned that will help you avoid similar bugs in the future.
Sample Answer
Direct answer
A recent example: a function computing a running total returned a subtly wrong result specifically when passed an empty input list, because it assumed at least one element would always be present; the root cause was an unchecked assumption about the input's shape, not a logic error in the calculation itself, and the fix was a one-line explicit guard plus a test covering the previously-unconsidered case.
Structured elaboration and worked example
Before:
def average_response_time(durations):
total = sum(durations)
return total / len(durations)
This looks correct and passes any test using a non-empty list. The bug: called with an empty list (a genuinely possible input, e.g., a time window with zero recorded requests), it raises ZeroDivisionError, which in this specific case was crashing a reporting job whenever a low-traffic service had a quiet hour with literally zero requests, a real, recurring production scenario the original code never considered.
After:
def average_response_time(durations):
if not durations:
return None # no data for this window; caller decides how to display that
total = sum(durations)
return total / len(durations)
Root cause: the function was written and tested against realistic-looking sample data, which always happened to be non-empty, so the implicit assumption "there's always at least one duration" was never challenged during development; the bug surfaced only once real production traffic included a genuinely empty window, an edge case that's easy to overlook precisely because it's uncommon rather than because it's hard to reason about once you think to check for it.
How it was tested: added an explicit unit test calling the function with an empty list and asserting it returns None rather than raising, alongside the existing non-empty-input tests, so the specific edge case that caused the production issue is now permanently covered and can't silently regress.
What was learned to avoid similar bugs: for any function that aggregates over a collection, explicitly consider and test the empty-collection case as a matter of habit, not just when a bug report forces the question; more broadly, the pattern generalizes to any input assumption implicit in code (non-null, non-negative, within some expected range) that reads as "obviously always true" during development but isn't guaranteed by the function's actual contract, and is worth deliberately challenging each such assumption with a test rather than trusting that realistic-looking sample data will happen to cover it.
Trade-offs and pitfalls
The fix here (returning None for an empty input) is a specific design choice, not the only valid one; an alternative would be raising a more informative, explicit exception (ValueError("cannot average an empty list")) if silently returning None risks a caller mishandling it further downstream without noticing. The right choice depends on what callers actually need to do with a "no data" result, and is worth deciding deliberately rather than defaulting to whichever felt fastest to write.
A recent performance patch reduced average latency but increased p99 latency. How would you investigate and resolve this regression while preserving the average-case gains? Describe your analysis steps and the kinds of code or system fixes you might apply.
Sample Answer
Direct answer
Investigate by looking at the DISTRIBUTION of latencies the patch changed, not just the two summary numbers: a change that helps the median while hurting the tail usually means the patch introduced a new, occasional slow path (a lock, a retry, a cold-cache miss, a GC pause) that wasn't present before, even though it made the common case faster. The fix should target that specific slow path, ideally without giving back the average-case win.
Structured elaboration
- Confirm the shape of the regression with a full latency histogram, not just average and p99 in isolation. Did the whole distribution shift, or did a new secondary "hump" appear at the tail while the bulk of requests got faster? These imply very different causes.
- Look for what the patch changed that could introduce occasional cost. Common culprits: added caching (fast on hit, slow on the now-rarer miss, especially a cold cache after deploy), added batching (fast on average, but a request unlucky enough to wait for a batch to fill sees added latency), a lock or synchronization primitive introduced to make the common path more efficient, or a retry/backoff path that wasn't there before.
- Correlate slow outliers with a specific condition. Pull the individual traces for requests in the new p99 tail and look for what they have in common: a specific input size, a cache miss, contention with a background job, a specific shard or partition.
- Reproduce the tail behavior in isolation once you have a hypothesis, ideally with a small, targeted load test that forces the suspected condition (a cold cache, high concurrency, a large payload) rather than waiting for it to occur naturally in production traffic.
- Fix the specific slow path, not the whole change. The goal is almost never to revert the improvement; it's to bound the cost of the new slow path (a timeout, a smaller batch window, a non-blocking fallback) while keeping the average-case gain.
Worked example
A patch that adds request coalescing (batching several concurrent identical lookups into one backend call) drops average latency by 30% but pushes p99 up by 4x. The latency histogram shows a new cluster of requests waiting close to the full batch window (say, 50ms) even when they were the only request in flight, because the coalescing logic always waits for the window to close before dispatching, even with nothing to batch against. The fix: dispatch immediately if no other request has joined the batch within the first few milliseconds, keeping the win for genuinely concurrent traffic while removing the unnecessary wait for solo requests, which is exactly the group hurting the p99.
Trade-offs and pitfalls
A tempting shortcut is to just revert the patch to restore the old p99, which throws away a real average-case improvement to fix a bug that's usually addressable with a smaller, targeted change. The other trap is chasing the p99 number without ever pulling individual slow traces: two very different mechanisms (a cold-cache path and a lock-contention path) can both show up as "p99 got worse," and the fix for one does nothing for the other.
You find a function that behaves incorrectly only for a specific input. Outline how you would build a minimal, reproducible test case that isolates the bug, including how you would reduce external dependencies so the case can run reliably in CI.
Sample Answer
Direct answer
Building a minimal, reproducible test case means repeatedly cutting away anything that isn't necessary to trigger the bug, while checking after every cut that the bug still happens, until what's left is the smallest input and the smallest code path that still fails. The goal is a case small enough that the cause becomes visible by inspection, and portable enough to run without the original production dependencies.
Structured elaboration
- Capture a known-failing instance first. Before minimizing anything, get one concrete input that reliably triggers the incorrect behavior, and record the exact observed output versus the expected output.
- Remove external dependencies one at a time. Replace a live database call with a small in-memory fixture containing only the rows the failing case touches; replace a network call with a stub returning the exact response you captured; replace "today's date" or randomness with a fixed value if the bug is date- or randomness-sensitive. Each substitution must preserve the failure; if it stops failing, that dependency was load-bearing and you put it back.
- Shrink the input itself. If the failing input is a 500-row CSV, try trimming to the smallest prefix, or the smallest subset of rows, that still fails; a systematic way to do this on large inputs is a manual or scripted bisection (delete half, check if it still fails; keep the half that still fails; repeat), the same "isolate by halving" idea that git-bisect applies to commit history.
- Shrink the code path. If the bug lives inside a large function, comment out or short-circuit branches that don't affect the failing case, or extract the smallest sub-function that reproduces it in isolation, so you're not staring at a hundred lines that are irrelevant to the actual defect.
- Verify the minimized case is still faithful. Run it once more end to end and confirm the output matches the original failure exactly, not just "something goes wrong."
Worked example
Given a report that "generating a monthly report crashes for some customers," start from one customer ID that reliably crashes. Replace the database call with a fixture containing just that customer's rows (still crashes: good, the DB wasn't the cause). Shrink from 40 line items to 2 by deleting half and re-checking each time; discover it still crashes with exactly one line item that has a null discount field. That's the minimized case: one hand-built object, no database, no network, five lines of setup, and it reliably reproduces "unhandled null in discount calculation." That five-line reproduction is now something you can paste directly into a bug report or a unit test.
Trade-offs and pitfalls
Minimizing too aggressively without re-verifying after each cut is the main trap: you can accidentally remove the actual trigger and end up "reproducing" a different, easier bug, or no bug at all, while believing you've simplified the real one. Always re-check after each individual cut, not after a batch of cuts, so that if the failure disappears you know exactly which change caused it.
Explain, with examples, how cognitive biases such as confirmation bias, anchoring, and sunk-cost fallacy can hinder a debugging investigation. Describe concrete practices, such as pair debugging, rotating investigators, hypothesis logs, and clear acceptance criteria, that you have introduced on a team to mitigate these biases.
Sample Answer
Direct answer
Confirmation bias, anchoring, and sunk-cost fallacy each distort a root-cause investigation in a specific, predictable way: confirmation bias makes you notice evidence supporting your first guess and discount evidence against it; anchoring makes an early, possibly wrong hypothesis dominate the rest of the investigation even after better evidence emerges; and sunk-cost fallacy keeps you investigating a disproven lead because of time already invested in it, rather than switching based on current evidence. Concrete team practices (pair debugging, rotating investigators, hypothesis logs, explicit acceptance criteria) counter each of these by introducing structure that doesn't rely on individual willpower to overcome the bias.
Structured elaboration
Confirmation bias: once you suspect a cause, you unconsciously interpret ambiguous evidence as supporting it and are quicker to dismiss evidence that doesn't fit. In debugging, this looks like reading a log line as "consistent with my theory" when a more neutral read would call it inconclusive, or stopping the investigation the moment ANY supporting evidence appears rather than continuing to look for disconfirming evidence too.
- Mitigation: hypothesis logs. Writing down each hypothesis and what specific evidence would DISCONFIRM it, before looking for evidence, forces a falsifiable framing up front; when you later find evidence, you check it against the pre-written disconfirmation criteria rather than retroactively deciding it "counts" as support.
Anchoring: the first plausible explanation offered (by you, or by someone else on the call) tends to dominate the rest of the investigation's framing, even as better evidence appears, because everyone's mental model has already organized around it.
- Mitigation: rotating investigators / a fresh pair of eyes. Someone joining the investigation LATE, without the accumulated anchor, will naturally form hypotheses from the current evidence rather than the initial framing, and is often the person who notices the anchor was wrong; deliberately bringing in a fresh perspective partway through a long investigation is a structural way to interrupt this.
Sunk-cost fallacy: having spent two hours pursuing one lead makes it psychologically harder to abandon, even once evidence stops supporting it, because abandoning it "wastes" the time already spent (which is, of course, already spent either way, and not actually recoverable by continuing).
- Mitigation: explicit acceptance criteria and timeboxing set in advance. Deciding, BEFORE starting to investigate a specific hypothesis, what evidence would confirm it and how much time is reasonable to spend testing it, makes the "abandon or continue" decision a pre-committed rule rather than an in-the-moment judgment call that sunk cost can distort.
Pair debugging as a mitigation for all three simultaneously: a second person, thinking independently, is less likely to share the exact same anchor or the exact same sunk-cost attachment to a specific lead, and naturally provides a real-time check on confirmation bias by asking "does that evidence actually support that, or are we reading it generously?"
Worked example
A team investigating an intermittent failure anchors early on "it's probably the recent deploy" (a reasonable first guess, given the timing). Two hours in, with the deploy's code reviewed thoroughly and nothing found, sunk cost starts to argue for continuing to scrutinize that same deploy rather than considering it possibly unrelated. A hypothesis log, written at the start, had specified "if the failure recurs on a service instance that predates this deploy, that disconfirms the deploy hypothesis"; checking that specific, pre-committed criterion shows the failure DID recur on an older instance, cleanly disconfirming the deploy theory despite two hours of sunk investigation into it. A rotating fresh investigator, brought in specifically because the original two were stuck, asks a question neither anchored investigator had considered ("what else changed around that time besides the deploy?") and identifies an unrelated infrastructure change that actually explains the failure.
Trade-offs and pitfalls
These practices have a real cost (pairing takes two people's time instead of one; hypothesis logs take a few minutes to write that could otherwise go straight into investigating), and the trade-off is worth it specifically for investigations that are ALREADY taking a long time or where the cost of a wrong conclusion is high; for a five-minute, low-stakes bug, the overhead of formal hypothesis logging isn't proportionate to the risk these biases actually pose in that context.
Tell the story of a concrete bug or production failure you found. Explain how you detected it, how you reproduced it if that was possible, the debugging tools and techniques you used, the root cause, and the permanent fix you implemented.
Sample Answer
Direct answer
A concrete story: a service occasionally returned stale pricing data to a subset of users, detected via a customer complaint rather than any internal alert (since the values were plausible-looking, just wrong, not obviously broken); the root cause traced to a caching layer that keyed its cache entries incorrectly, causing two logically-distinct pricing contexts to collide and overwrite each other's cached value, and the permanent fix corrected the cache key's uniqueness rather than just adjusting the cache's expiry time.
Structured elaboration
How it was detected: a customer support ticket reported seeing a price that didn't match what should have applied to their account tier, with no corresponding error or alert on the engineering side, since the returned value was a real, validly-formatted price, just the WRONG one; this is a useful detail because it illustrates a class of bug (returning plausible-but-wrong data) that's structurally invisible to error-rate-based monitoring, and only surfaces via a downstream consumer noticing a substantive discrepancy.
How it was reproduced: confirming the report wasn't a one-off required identifying the PATTERN, not just the single instance; checking whether other users on the same account tier around the same time window also received an unexpected price showed a small but real cluster, ruling out "one weird one-off" and confirming a systemic, reproducible mechanism worth a full investigation.
Debugging tools and techniques used: traced the pricing-lookup code path for the affected requests, and found it flows through an in-memory cache keyed, it turned out, on account tier ALONE rather than on the combination of account tier AND region (pricing legitimately varies by both); when two users on the same tier but different regions made requests close together in time, the second request's result could overwrite the first's cache entry under the shared, insufficiently-specific key, and a THIRD user (same tier, either region) arriving shortly after could then receive whichever region's price happened to be cached most recently, regardless of their own actual region.
The root cause: a cache key that didn't include every dimension the underlying value actually varied by, a classic caching-correctness bug: the cache was implicitly promising "this value is valid for anyone with this tier," when the real invariant needed was "this value is valid for anyone with this tier AND this region."
The permanent fix implemented: updated the cache key to include region alongside tier, restoring the correct invariant; also added a specific integration test that exercises exactly this scenario (two regions, same tier, interleaved requests) to catch a regression of this specific mechanism in the future, since the original bug had shipped without any test covering this particular combination of dimensions.
What you learned that helps you avoid similar bugs: whenever introducing a cache, explicitly enumerate every dimension the cached value can legitimately vary by, and verify the cache key includes ALL of them, not just the ones that happen to be obvious or top-of-mind at implementation time; a caching bug of this shape is especially dangerous specifically because it fails SILENTLY (no error, no crash, just occasionally-wrong data) and is invisible to typical error-rate monitoring, which argues for treating "does this cache key capture every dimension of variation" as a deliberate design-review question on any future caching work, not something to verify only after a bug report arrives.
Trade-offs and pitfalls
The tempting quick fix, once the symptom (stale/wrong cached price) was understood, would have been to simply reduce the cache's TTL (expiry time), which would have reduced the WINDOW during which a collision could produce visibly wrong data without fixing the actual collision mechanism at all; the permanent fix specifically addressed the cache KEY's correctness, not the expiry duration, since a shorter TTL would have masked the bug's visible frequency without removing its root cause.
Unlock Full Question Bank
Get access to all 13 Debugging and Systematic Troubleshooting interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.