Google Site Reliability Engineer (SRE) - Junior Level (1-2 years) Interview Preparation Guide

Direct answer

When an urgent security issue and a sales-critical demo land the same day, the deciding factor is exposure, not who asked more forcefully: what could go wrong if the security issue waits, and what can still be preserved for the demo without touching the risky path. Usually both can be partially served: contain or fix the security issue first, and give sales something real to show that doesn't depend on the vulnerable code.

Structured elaboration

1. Triage both in parallel, fast

Read the security bulletin and the demo request together. Identify exactly which services, data, or endpoints the vulnerability touches, and exactly what the demo needs to show.

2. Weigh exposure, not urgency of the ask

A security issue usually carries broader exposure (any affected customer, potential data risk) than a single demo (one prospective deal). That asymmetry is normally the tiebreaker, but it should be checked rather than assumed: a demo that's the last step before a major renewal can occasionally weigh more than a low-severity, well-contained finding.

3. Look for a path that serves both

A scoped hotfix with a canary rollout (releasing the fix to a small slice of traffic first, watching it closely, then rolling out to everyone once it looks clean) for the security issue, paired with a sandboxed or stubbed version of the feature for the demo, often means sales isn't actually blocked on the mainline fix landing first.

4. Communicate the decision and the reasoning immediately

Both sides need a concrete plan with timestamps, not just a priority call: what's happening, by when, and what the other side gets in the meantime.

Worked example

The patch ships to a small share of traffic first while being monitored, then rolls out fully once confirmed clean. In parallel, a second engineer builds a stubbed version of the requested feature specifically for the demo environment, so sales can present it without depending on the code currently under remediation. Both sides get an update within a couple of hours: security gets an ETA for full rollout, sales gets confirmation the demo will work and exactly how.

Trade-offs and pitfalls

• Defaulting to whichever request comes from the louder or more senior stakeholder, rather than actual exposure, is the most common failure mode here.
• Building a demo-only workaround without labeling it clearly as temporary risks it quietly becoming the real implementation, skipping the proper fix.
• Failing to give both sides a concrete timeline turns a reasonable prioritization call into a trust problem, even when the call itself was correct.
• Treating this as strictly either/or, instead of looking for a path that partially serves both, wastes an option that's usually available.

Direct answer: A single global lock is simple and correct but serializes ALL access, becoming a throughput bottleneck under contention regardless of how many cores are available. Per-bucket locks (lock striping) let unrelated buckets be accessed concurrently, dramatically reducing contention at the cost of more complex resize logic (must acquire all stripe locks, or use a clever incremental resize). A lock-free structure (like a CAS-based concurrent map) avoids blocking entirely - operations retry via compare-and-swap on contention rather than waiting for a lock - offering the best scalability under high contention but at substantial implementation complexity and subtler correctness reasoning (linearizability, ABA problems).

Structured elaboration

• Global lock: O(1) average time per operation when uncontended, but under N concurrent threads, effective throughput approaches that of a SINGLE thread, since only one thread can hold the lock at a time - contention doesn't just slow things down proportionally, it can be worse than that due to lock-acquisition overhead (context switches, cache-line bouncing on the lock variable itself).
• Lock striping: partition the hash table's buckets into K independent locks (e.g. lock[hash(key) % K]). Two operations on different stripes proceed fully in parallel; only operations landing in the SAME stripe serialize. This scales throughput roughly linearly with K (up to the number of cores) for well-distributed keys, at the cost of needing to acquire ALL K locks for whole-table operations like resize - a real complexity/correctness burden.
• Lock-free (Michael-Scott-style): operations use atomic compare-and-swap (CAS) on individual pointers rather than acquiring any lock. Under contention, a CAS can fail (another thread modified the same location first) and the operation RETRIES rather than blocking - this gives amortized O(1) enqueue/dequeue even under contention, because a failed CAS means SOME other thread's operation succeeded (made progress), so the system as a whole always makes forward progress even if individual threads occasionally retry.

Worked example (Michael-Scott queue linearizability sketch)

The Michael-Scott lock-free queue maintains head and tail pointers, both updated via CAS. Enqueue: allocate a new node, CAS it onto the current tail's next pointer, then CAS the tail pointer itself to point to the new node (a two-step process, because a single CAS can't atomically update two separate pointers). The subtlety: if a thread's first CAS (linking the new node) succeeds but it gets preempted before the second CAS (advancing tail), another thread's enqueue can "help" by noticing tail.next is non-null and advancing tail on the stalled thread's behalf - this "helping" mechanism is what preserves the lock-free progress guarantee (the SYSTEM always makes progress, even if one specific thread is starved). Under high contention (many threads racing to CAS the same tail pointer), each individual enqueue may need several retries, so the AMORTIZED cost per successful enqueue rises with contention, but never blocks - it degrades gracefully rather than serializing.

Trade-offs & pitfalls

• Lock-free code is substantially harder to get right than lock-based code - subtle bugs like the ABA problem (a pointer value gets reused after being freed and reallocated, fooling a CAS into succeeding when it shouldn't) require careful mitigation (hazard pointers, tagged pointers, or epoch-based memory reclamation).
• A single global lock is often the CORRECT engineering choice for low-to-moderate contention workloads - the complexity of lock striping or lock-free structures is only worth paying when profiling shows lock contention is a real, measured bottleneck.
• Lock striping's resize path (needing to coordinate across all stripes) is a common source of subtle deadlocks or missed-update bugs if not implemented carefully (e.g. always acquiring stripe locks in a consistent global order to avoid deadlock).

Direct answer

Kadane's algorithm tracks the best sum ending exactly at the current index, resetting it whenever it goes negative, which gives O(n) time and O(1) extra space. Extend it with a couple of extra index variables to also recover the winning subarray's start and end, and treat an empty input as an explicit error rather than guessing a placeholder value.

Structured elaboration

• State. current_sum: best sum of a subarray ending at the current index. best_sum: best seen anywhere so far.
• Reset rule. If current_sum goes negative, it can never help a future subarray (adding a negative running total only hurts the next element), so restart at the current element. This is what makes the algorithm handle an all-negative array correctly, PROVIDED you initialize best_sum and current_sum with nums[0], not 0.
• The classic initialization bug. Initializing best_sum = 0 silently treats the empty subarray as a legal candidate. For an all-negative array like [-3, -1, -4, -1, -5], the true best NON-EMPTY subarray sum is -1 (the single element -1), but a 0-initialized version would wrongly report 0, implying an empty subarray beats every real one. Most interview phrasings (this one included) require a non-empty subarray, so seed with the first element.
• Recovering indices. Track current_start (where the current running sum began) alongside current_sum; whenever current_sum resets, current_start moves to the current index. Whenever current_sum beats best_sum, copy current_start and the current index into best_start/best_end.
• Empty array. There is no subarray to return, so raise explicitly rather than returning 0 or None, which would look like a valid (and wrong) answer to a caller who doesn't check.

Worked example

def max_subarray(nums):
    if not nums:
        raise ValueError("max_subarray: input array must be non-empty")

    best_sum = current_sum = nums[0]
    best_start = best_end = 0
    current_start = 0

    for i in range(1, len(nums)):
        if current_sum < 0:
            current_sum = nums[i]
            current_start = i
        else:
            current_sum += nums[i]

        if current_sum > best_sum:
            best_sum = current_sum
            best_start = current_start
            best_end = i

    return best_sum, best_start, best_end


def max_subarray_brute_force(nums):
    n = len(nums)
    best = nums[0]
    for i in range(n):
        s = 0
        for j in range(i, n):
            s += nums[j]
            if s > best:
                best = s
    return best


cases = [
    [-2, 1, -3, 4, -1, 2, 1, -5, 4],
    [-3, -1, -4, -1, -5],
    [5],
    [2, 2, 2],
]
for nums in cases:
    best_sum, start, end = max_subarray(nums)
    brute = max_subarray_brute_force(nums)
    print(f"nums={nums} -> best_sum={best_sum}, subarray={nums[start:end+1]}, indices=({start},{end}), brute_force={brute}")
    assert best_sum == brute

try:
    max_subarray([])
except ValueError as e:
    print(f"empty input raised ValueError as expected: {e}")
print("brute-force cross-check passed for all cases")

Output (executed, python3 s62_kadane.py, also cross-checked against an O(n^2) brute force for every case):

nums=[-2, 1, -3, 4, -1, 2, 1, -5, 4] -> best_sum=6, subarray=[4, -1, 2, 1], indices=(3,6), brute_force=6
nums=[-3, -1, -4, -1, -5] -> best_sum=-1, subarray=[-1], indices=(1,1), brute_force=-1
nums=[5] -> best_sum=5, subarray=[5], indices=(0,0), brute_force=5
nums=[2, 2, 2] -> best_sum=6, subarray=[2, 2, 2], indices=(0,2), brute_force=6
empty input raised ValueError as expected: max_subarray: input array must be non-empty
brute-force cross-check passed for all cases

Trade-offs & pitfalls

• Seeding best_sum = 0 is the single most common bug on this problem; always test against an all-negative array to catch it (as above).
• Deciding whether the empty subarray is a legal answer is a real design decision, not a formality; state the assumption up front rather than let the code's default silently pick one.
• The state above is O(1) beyond the input, so it also works directly on a stream you can only scan once, but you cannot recover the ORIGINAL array's earlier indices after the fact unless you kept them as you went, which is exactly what current_start/best_start already do here at no extra asymptotic cost.
• A Java port of this same logic is a straightforward, mechanical translation (three int locals instead of Python variables, an explicit IllegalArgumentException in place of the ValueError); nothing about the algorithm changes across the two languages.

Requirements:

• Automatically rollback a deployment when correlated health degradation occurs (error rate, p50/p95 latency, CPU/memory saturation).
• Avoid false positives, flapping, or rollback loops.
• Allow human override and auditability.

High-level design:

• Canary controller watches canaries/rollouts; metrics pipeline collects signals; decision engine evaluates policies; executor triggers rollback; UI/CLI for overrides; audit/log store.

Key components:

1. Metrics pipeline: ingest metrics (Prometheus/Grafana/OTel), compute short-window and baseline (rolling 1h).
2. Decision engine: evaluates rules and multivariate logic; supports configurable thresholds, required-signal counts, and cooldowns.
3. Rollback executor: calls orchestration API (K8s, Spinnaker) and records actions.
4. Audit store: immutable logs (WAL + SIEM) with owner, timestamps, runbook link.
5. Notification/approval: PagerDuty/Slack + human approval flow.

Policy example:

• Signals monitored: error rate (>3% increase over baseline), p95 latency (>2x baseline or >500ms), CPU saturation (>85%).
• Required signals: at least 2 of 3 sustained for evaluation window (2 minutes) with confirmation window (10 minutes) aggregating multiple samples.
• Cooldowns: post-rollback cooldown 30 minutes; per-release cooldown 2 hours to prevent immediate re-deploy/rollback loops.
• Backoff: exponential backoff on repeated rollbacks; require manual approval after N automated rollbacks (e.g., N=2).

Preventing loops/thrashing:

• Require multi-signal correlation and sustained deviation across windows.
• Use deployment tags and versioned states: block rollback if last action was rollback less than cooldown.
• Implement hysteresis: thresholds relaxed after rollback for a warm-up period.
• Locking: optimistic lock per-service to prevent concurrent automated actions.

Human override & safety:

• Emergency stop: “global auto-rollback disable” guarded by RBAC and MFA.
• Manual approval flow: alerts with one-click approve/deny; manual overrides create audit entries and bypass automated cooldown only with approver justification.
• Runbook link and playbook automatically attached to alerts.

Audit & observability:

• Immutable event log with metrics snapshot, decision inputs, and executor response.
• Dashboards showing recent rollbacks, false positives, and MTTA/MTTR.
• Post-incident: automated postmortem template populated with decision data for SRE review.

Trade-offs:

• Conservative defaults reduce false positives but may delay rollback; allow teams to tune sensitivity per SLO.
• Centralized decision engine eases governance; per-service policies allow customization.

This design balances automation with human control, uses multi-signal confirmation and cooldowns to avoid thrashing, and provides full auditability for compliance and learning.

Direct answer

Eventual consistency with compensating transactions is not viable as the primary mechanism for an e-wallet's ledger. A compensating transaction undoes a wrong debit or credit after the fact, and for money that is a regulatory and trust failure, not merely a rough edge in the user experience. That leaves a real choice between a single-region serializable relational database with synchronous replicas, and a multi-region consensus database using a protocol such as Raft or Paxos (algorithms that get a group of replicas to agree on the order of operations even if some are slow or unreachable). The deciding question is the recovery point objective (RPO, the maximum amount of recently committed data the business can tolerate losing) if an entire region is lost. If losing zero committed transactions even when a whole region disappears is a hard requirement, choose multi-region consensus despite its latency cost. If a well-rehearsed failover to a synchronous standby in a nearby region is an acceptable answer to that same question, single-region-plus-synchronous-standby is simpler to operate correctly and faster on the common path.

Structured elaboration

Single-region serializable relational database with synchronous replicas. Run the ledger under the strictest available isolation level (serializable, which prevents subtle bugs such as two concurrent withdrawals both reading a balance before either has committed and both approving), with one or more synchronous standbys nearby that must acknowledge a write before it is considered committed. A primary failure that fails over successfully loses zero committed data. The remaining exposure is a disaster that takes out the primary and all synchronous standbys together, at which point failover falls back to an asynchronous, farther-away replica, and the last few in-flight transactions can be lost. That gap is the RPO this option accepts.

Multi-region consensus (Raft or Paxos). Every write replicates to a quorum (an agreeing majority of the replica set, enough that any two possible quorums must share at least one replica) of replicas spread across regions before it is acknowledged, so losing any single region, including the one currently holding the leader, cannot lose committed data. RPO is zero by construction. The cost is that every write pays the network round trip needed to reach that quorum, bounded by the slower of the required acknowledgments, not avoidable simply by adding more replicas within the same regions, and the team now operates a genuinely different topology: region-aware placement decisions and a different mental model for reasoning about failure.

Eventual consistency with compensating transactions. Appropriate for the parts of a wallet product that are not the ledger itself, for example refreshing a cached risk score or propagating a promotional balance display, but not for the record of who owns what money. Even a well-designed saga (a multi-step transfer broken into a sequence of local transactions, each with its own compensating "undo" action, instead of one all-or-nothing global transaction) accepts a window where the system is visibly in an in-progress, not-yet-reconciled state, and for a ledger that window is exactly where double-spend and overdraft bugs live. Use this pattern only to orchestrate a multi-step transfer between two internally consistent ledgers, never as the consistency model of a ledger by itself.

Worked example

Absent any further constraint, default to single-region serializable with synchronous standbys: it is simpler to operate correctly, and a correctly operated simpler system beats a theoretically stronger but easily misconfigured one. Now quantify the alternative's cost. Light in optical fiber travels at roughly 68% of the speed of light in vacuum, about 203,900 km/s. For a route of roughly 5,900 km, a rough distance between the eastern United States and western Europe, one-way propagation alone is 5,900 / 203,900 * 1000 ~= 28.9 ms, so a round trip is ~57.9 ms before routing and serialization overhead, which typically adds further tens of milliseconds. If a regulator or a board-level continuity mandate genuinely requires that no committed transaction can be lost even if an entire region vanishes, paying roughly that extra 60 or so milliseconds of write latency, turning a 15-millisecond transfer into something closer to 75 milliseconds (15 ms + ~60 ms), is the acceptable price, and multi-region consensus is the correct call.

Trade-offs and pitfalls

A pitfall is choosing multi-region consensus reflexively because "more resilient" sounds safer, without pricing the latency into the product. A wallet transfer that now takes ten times longer is a real user-facing regression, and if the team cannot name the specific business requirement that extra time buys, that is a sign the requirement was never actually stated, just assumed.

A second pitfall is treating "serializable" as a checkbox rather than a specific property to verify. Some database engines implement serializable isolation through optimistic conflict detection, meaning transactions can abort and must be retried under contention, which changes what the application's error handling has to do, not just a configuration setting.

What would flip the recommendation further: a wallet operating in a single legal jurisdiction with no cross-region regulatory driver, and a documented, signed-off tolerance for losing the last few seconds of transactions in the vanishingly rare event of a total regional disaster. That is precisely when single-region-plus-synchronous-standby is not just acceptable but the better choice, because it is the option the team can run correctly with less operational complexity.

Direct answer

Word Ladder is breadth-first search (BFS) on an IMPLICIT graph: nodes are words, and an edge connects two words that differ in exactly one letter position. Naively materializing that graph by comparing every pair of words costs $O(N^2 \cdot L)$ for $N$ words of length $L$, which is wasteful before the BFS even starts. The efficient approach preprocesses each word into $L$ wildcard "generic states" (hit becomes *it, h*t, hi*) and buckets words sharing a generic state; two words are neighbors exactly when they share at least one bucket, which turns neighbor lookup into $O(L)$ per word instead of an $O(N)$ scan of the whole dictionary.

Structured elaboration

Why the naive approach is slow. Comparing two words for "differs by exactly one letter" costs $O(L)$, and doing this for every pair among $N$ words costs $O(N^2 L)$. For a large dictionary (this problem is commonly asked with dictionaries up to roughly 1,000,000 words), $N^2$ is completely infeasible.

The wildcard-bucket preprocessing. For each word, generate its $L$ generic states by replacing one letter position at a time with a wildcard character. Group words by shared generic state in a dictionary (hash map) of pattern -> [words]. Two words are one-letter-apart if and only if they appear together in at least one bucket. This changes "find all one-letter-apart neighbors of a word" from an $O(N)$ dictionary scan into an $O(L)$ lookup of that word's own $L$ patterns, each yielding a bucket of true neighbors directly, no comparison against unrelated words required.

Why BFS, and why it terminates at the right answer. The transformation-sequence graph is unweighted (every edge, a single-letter change, costs the same "one step"), so BFS's standard guarantee applies unmodified: the first time endWord is dequeued, the number of steps taken is the shortest possible transformation sequence length.

Scaling further: bidirectional search. For a very large dictionary, a single BFS frontier from beginWord can visit an enormous number of words before reaching endWord, especially if the true shortest sequence is long. Growing two frontiers at once, one from each end, and stopping the moment they meet, bounds the search to roughly the square root of the single-direction node count for a typical branching factor, the same principle as bidirectional BFS on any unweighted graph. Preprocessing narrows the cost of finding each word's neighbors; bidirectional search narrows how many words need visiting at all. The two are complementary, not alternatives.

Worked example

from collections import deque, defaultdict
from typing import List

def word_ladder_length(begin_word: str, end_word: str, word_list: List[str]) -> int:
    # Shortest transformation sequence length (number of WORDS, LeetCode
    # convention) from begin_word to end_word via one-letter-at-a-time
    # changes, every intermediate word must be in word_list. Nodes are
    # words; the graph is implicit, materialized via wildcard buckets.
    words = set(word_list)
    if end_word not in words:
        return 0

    L = len(begin_word)
    buckets = defaultdict(list)
    for w in words | {begin_word}:
        for i in range(L):
            buckets[w[:i] + "*" + w[i+1:]].append(w)

    visited = {begin_word}
    q = deque([(begin_word, 1)])
    while q:
        word, steps = q.popleft()
        if word == end_word:
            return steps
        for i in range(L):
            pattern = word[:i] + "*" + word[i+1:]
            for nxt in buckets[pattern]:
                if nxt not in visited:
                    visited.add(nxt)
                    q.append((nxt, steps + 1))
            buckets[pattern] = []  # consumed: avoids re-scanning this bucket on every future pop
    return 0

if __name__ == "__main__":
    word_list = ["hot", "dot", "dog", "lot", "log", "cog"]
    print("hit -> cog:", word_ladder_length("hit", "cog", word_list))

    word_list_no_path = ["hot", "dot", "dog", "lot", "log"]  # no "cog"
    print("hit -> cog (unreachable, cog not in dictionary):", word_ladder_length("hit", "cog", word_list_no_path))

    def word_ladder_length_naive(begin_word, end_word, word_list):
        # Deliberately simple: compares every pair of words directly.
        # O(N^2 * L) but obviously correct, used only as a cross-check.
        words = set(word_list)
        if end_word not in words:
            return 0
        def one_letter_apart(a, b):
            return sum(1 for x, y in zip(a, b) if x != y) == 1
        visited = {begin_word}
        q = deque([(begin_word, 1)])
        while q:
            word, steps = q.popleft()
            if word == end_word:
                return steps
            for cand in words:
                if cand not in visited and one_letter_apart(word, cand):
                    visited.add(cand)
                    q.append((cand, steps + 1))
        return 0

    naive_result = word_ladder_length_naive("hit", "cog", word_list)
    print("Naive O(N^2*L) BFS agrees:", naive_result == 5)

    big_list = ["hot","dot","dog","lot","log","cog","cot","cat","bat","bad","bid","big","bog","bug","but"]
    fast = word_ladder_length("hit", "cog", big_list)
    slow = word_ladder_length_naive("hit", "cog", big_list)
    print("Larger dictionary: bucket result =", fast, "naive result =", slow, "match:", fast == slow)

Output (actually executed with python3):

hit -> cog: 5
hit -> cog (unreachable, cog not in dictionary): 0
Naive O(N^2*L) BFS agrees: True
Larger dictionary: bucket result = 4 naive result = 4 match: True

The 5-step result for the classic hit -> hot -> dot -> dog -> cog sequence matches the naive, deliberately-slow, independently-implemented $O(N^2 L)$ BFS exactly, confirming the bucket optimization changes performance, not the answer. The larger dictionary case (where the answer shortens to 4, since cat -> cot -> cog becomes available as a shorter route once more words are present) also agrees between both implementations.

Complexity

Preprocessing: $O(N \cdot L)$ to build all buckets ($N$ words, $L$ patterns each, $O(L)$ per pattern to construct the string). Traversal: each word is dequeued at most once, and each dequeue does $O(L)$ pattern lookups, each yielding a bucket that is consumed (emptied) after its first use, so no bucket is ever rescanned; total traversal work is $O(N \cdot L)$ as well. Overall $O(N \cdot L)$, versus $O(N^2 \cdot L)$ for the naive pairwise-comparison approach, a genuinely different complexity class, not just a constant-factor improvement.

Edge cases

• end_word not in word_list: returns 0 immediately, since the problem requires every transformed word, including the final one, to be a real dictionary entry.
• begin_word == end_word: the traversal discovers this the moment begin_word is first dequeued (word == end_word is checked before expanding neighbors), correctly returning 1 without needing begin_word itself to be present in the dictionary.
• No transformation sequence exists at all: the queue drains completely without ever dequeuing end_word, and the function falls through to return 0.
• Words of differing length: not handled, and not expected to be: the wildcard-bucket scheme assumes a fixed word length L throughout, matching the problem's own definition of a valid one-letter substitution.

Trade-offs and pitfalls

• Dictionary representation for fast lookup. Converting word_list to a set up front is what makes end_word not in words and any membership check $O(1)$ average case; using the raw list would make that check $O(N)$, a small-looking mistake that still meaningfully hurts performance at scale.
• The bucket-consumption trick (buckets[pattern] = [] after first use) matters for dense dictionaries where many words share a pattern: without it, every future word matching that pattern would re-scan the same, increasingly stale, bucket again, degrading the intended near-linear behavior.
• Common mistake: treating fast neighbor lookup as sufficient at any scale. For a truly massive (roughly 1,000,000-word) dictionary, a single-direction BFS can still visit a large fraction of it before finding a distant end_word, because preprocessing only fixes the cost PER STEP, not the NUMBER OF WORDS VISITED; that is exactly where bidirectional search stops being optional.
• Common mistake: assuming begin_word itself must be present in word_list. It typically is not required to be (LeetCode's own convention); the implementation seeds the traversal from begin_word regardless, while still requiring end_word to be present, since every transformed word must exist in the dictionary.

Direct answer

Treat pushback as signal, not an attack: restate the alternative back to the interviewer to confirm you understood it, name the assumption your original choice actually depends on, and check whether the pushback introduces a genuinely new constraint or is just testing your conviction. If it changes a load-bearing assumption, revise the design and say so plainly. If it does not, hold the decision and explain why the alternative loses on the axis that matters here, without getting defensive or repeating yourself louder.

Structured elaboration

Separate what kind of decision is being challenged

A useful first move, often invisible to the interviewer but doing real work for you, is classifying the decision itself:

• A reversible decision (a cache eviction policy, an index choice, a queue's retry backoff) can be tried, measured, and changed later at low cost. It is fine to say "I'd start with X, and revisit once we have real traffic data" and mean it.
• A largely irreversible decision (the primary datastore for a dataset that will grow to hold years of production data, a data-residency architecture with legal constraints attached) is expensive to unwind once built. These deserve a firmer defense, because "we'll just change it later" is not actually true for them.
  A candidate who signals which category their choice falls into is showing exactly the judgment this kind of pushback is designed to probe.

The actual steps, in order

1. Paraphrase the alternative back ("so the question is why not do X instead of what I proposed"). This confirms you understood the objection rather than reacting to a version of it you invented, and buys you a beat to think.
2. State the assumption or constraint your original choice depended on, out loud. This is the load-bearing piece: if that assumption is still true, your choice still holds; if the interviewer's follow-up just knocked it down, you now know exactly what to revise.
3. Ask, explicitly if needed, whether the pushback is introducing new information (a constraint you did not have, or did not weight correctly) or is testing whether you actually understand your own trade-off. Those call for different responses.
4. Decide: hold, revise, or partially revise (keep the core choice, adjust a parameter). Say which one you are doing and why, in one sentence.
5. Move on. Do not keep re-litigating a decision you already reopened and closed; that reads as insecurity, not thoroughness.

A worked dialogue skeleton

Interviewer: "Why would you use a queue here instead of just calling the downstream service directly?"

Candidate: "So the question is whether the extra moving part, the queue, is worth it compared to a direct synchronous call. My choice assumes the downstream service is slower and less reliable than the caller can afford to block on, so decoupling protects the caller's own latency and gives us a retry point if the downstream service is briefly unavailable."

Interviewer: "What if that downstream service is actually one of the most reliable and fast services we operate?"

Candidate: "That changes the assumption I was leaning on. If it is genuinely fast and reliable, the resilience argument for a queue weakens a lot, and a direct call with a short timeout and a couple of retries might be simpler and just as safe. I would want to know its actual latency and error behavior before committing either way, but I would not stubbornly keep the queue just because that is what I said first."

Interviewer: "And if it were the flakiest service in the system instead?"

Candidate: "Then I would hold the original call. A flaky downstream dependency is exactly the case the queue protects against, buffering the caller from its failures and giving us retry and backpressure without cascading the failure upstream."

Notice the candidate did not fold immediately in the second exchange, and did not dig in reflexively in the third; the answer changed only where the underlying assumption actually changed.

Trade-offs & pitfalls

• Caving on every objection is the most common failure mode: treating any pushback as proof you were wrong signals you did not have real conviction in the first place, and an interviewer who sees you reverse instantly on a restated version of your own design will keep pushing to find the floor.
• Stonewalling is the opposite failure and just as damaging: repeating your original justification louder, or refusing to update even when the interviewer has handed you a genuinely new constraint, reads as an inability to incorporate new information, which is the exact skill system-design interviews are trying to probe.
• Relitigating from scratch instead of anchoring on the specific new point wastes time and often talks yourself into a worse answer than the one you started with; stay anchored to the one assumption that was actually challenged.
• Treating every decision as equally reversible is a subtler pitfall: defending a cache TTL choice and defending your core datastore choice with the same intensity misses that one of them is cheap to revisit later and one is not. Senior candidates spend their conviction where it is actually load-bearing.
• The strongest signal is not being right on the first guess, it is showing a clear, repeatable process for deciding whether to hold or revise, and being transparent in the moment about which one you are doing.

Direct answer
When a third-party vendor is a likely cause but disputes fault, run the postmortem to establish facts using your own independent evidence first, treat the vendor's account as one input rather than the final word, and capture action items on both sides, focusing your own internal fixes on reducing dependence on the vendor's reliability rather than only on proving they were at fault.

Structured elaboration

• Build your own evidence-based timeline independently of the vendor's account. Use your own logs, monitoring, and error rates from calls to the vendor's service to establish what happened from your side, so the internal postmortem doesn't depend entirely on the vendor confirming anything.
• Engage the vendor through the relationship, not the postmortem meeting. Request their incident data and timeline through your normal vendor communication channel, ideally backed by a contractual SLA that entitles you to it, rather than trying to resolve the dispute inside your internal blameless review.
• Keep your internal review blameless regardless of the vendor's response. Your team's postmortem focuses on what YOUR system could have done differently: better fallback behavior, circuit breakers, more graceful degradation when a dependency misbehaves, not on winning an argument about whose fault it was.
• Capture action items on both sides, but don't make yours contingent on the vendor's cooperation. Internal action items (add a fallback path, tighten a timeout, add a circuit breaker) should proceed regardless of whether the vendor ever agrees they were at fault. If the vendor does supply corrective actions, track those too, but as a secondary, lower-confidence input.
• Preserve the relationship while being honest in your own documentation. You can state factually, based on your own evidence, what you observed from the vendor's service (elevated error rates, timeout patterns) without needing the vendor's agreement to write an accurate internal postmortem; disagreement with the vendor doesn't require softening your own factual account.

Worked example
Your platform experiences cascading failures that your monitoring clearly shows correlate with a spike in error rates and latency from a third-party payments vendor's API, but the vendor's status page shows no reported incident and their support team states their systems were healthy throughout. Your postmortem proceeds using your own evidence: request logs, timeout patterns, and error codes from calls to their API during the window are collected and documented factually, without needing the vendor to confirm anything. The internal conclusion, based on your own data: calls to the vendor's API showed a clear anomaly during this window, and regardless of the vendor's internal state, your system had no circuit breaker or fallback behavior to prevent that anomaly from cascading into a full outage on your side. Action items: add a circuit breaker and graceful degradation path for this dependency (internal, proceeds regardless of vendor response), and separately, raise the anomaly with the vendor through your account management relationship, requesting their incident data under your SLA, tracked as a vendor-side item with lower confidence it will be resolved quickly.

Trade-offs and pitfalls
The most common mistake is letting the postmortem stall while waiting for the vendor to agree on fault, which delays your own genuinely actionable fixes for no good reason, since your fallback and resilience improvements are valuable regardless of whose fault the original incident was. A second is softening your own factual account to avoid vendor relationship friction, which produces a less useful internal document than an honest one.

Direct answer
Robust alerting on requests-per-second needs to treat the metric as a monotonic counter's derived RATE, not a raw value: alert on rate-of-change with explicit counter-reset detection rather than an absolute threshold on the raw counter, use an evaluation window wide enough to smooth normal sampling jitter without hiding a real spike, and treat a missing datapoint as "no data available for this interval," never as "zero traffic." From a test-design standpoint, each of these is a distinct EDGE CASE of the metric's time series that needs its own test input, not just a design principle stated in the abstract, and the executed example below constructs exactly those edge cases and checks the alerting logic against them.

Structured elaboration

Rate-of-change vs absolute thresholds. An absolute threshold on a raw counter is wrong on two counts: it eventually fires on any long-lived healthy counter simply because the counter keeps climbing regardless of actual traffic rate, and it fails to fire on a genuine spike immediately after a counter reset, since the raw value is small again even though the RATE just jumped. A rate-of-change check (delta between consecutive samples, normalized to a per-minute or per-second rate) measures what actually matters, request rate, independent of how long the counter has been accumulating.

Counter-reset detection. A monotonic counter resets to zero whenever the process that owns it restarts (a deploy, a crash-restart, a scale-down/scale-up cycle). A naive delta calculation (current - previous) produces a large NEGATIVE number across a reset, which if not explicitly detected and handled, either falsely alerts on a huge negative "drop" or, worse, gets squared/absolute-valued somewhere downstream into a false spike alert. The fix is explicit: whenever the current value is smaller than the previous value, treat the CURRENT value itself as the new interval's contribution (the counter's own climb since restarting), not current - previous.

Avoiding flapping. An evaluation window that requires the rate condition to hold across multiple consecutive samples (not a single sample crossing the threshold) absorbs ordinary sampling jitter without requiring a much higher, less sensitive threshold; the trade-off is detection latency, a wider window is slower to alert on a real event, so the window size is itself a parameter that should be chosen from the actual noise characteristics of the specific metric, not copied from an unrelated metric's alerting config.

Sparse sampling and ingestion delay. A missing datapoint (a scrape failure, a delayed ingestion pipeline) must never be silently treated as a zero, since a rate calculation across a real gap (current - previous divided by an assumed one-interval time delta, when the actual elapsed time was several intervals) either fabricates a rate that's too low (if the gap is treated as normal time) or too high (if the delta is computed correctly but the false zero corrupts the previous value). The correct handling explicitly skips delta computation across any interval containing a missing sample and resumes cleanly once real data returns, rather than inferring a rate through the gap.

Worked example (executed)

def rate_of_change_alert(samples, rate_threshold):
    fired, prev = [], None
    for i, v in enumerate(samples):
        if v is None:
            prev = None  # missing datapoint: don't compute a delta across the gap
            continue
        if prev is None:
            prev = v
            continue
        delta = v - prev if v >= prev else v  # reset detection
        if delta > rate_threshold:
            fired.append(i)
        prev = v
    return fired

# one sample per minute; None = missing datapoint (scrape failure)
samples = [1000, 1050, 1100, None, 1160, 5, 55, 2000]
#  index:    0     1     2    3     4    5   6    7
#  index 5 is a process restart (counter dropped 1160 -> 5): true new traffic = 5/min
#  index 7 is a genuine spike: true new traffic = 2000 - 55 = 1945/min
print(rate_of_change_alert(samples, rate_threshold=500))

Actual output: [7]. The alert correctly fires only at index 7 (delta 1945, a genuine spike) and correctly does NOT fire at index 5, where the raw counter dropped from 1160 to 5; that drop is treated as a reset, contributing a delta of 5 (the counter's own climb since restarting), not a nonsensical -1155/min. It also correctly skips index 3 (the missing datapoint), never computing a delta across that gap, and resumes cleanly at index 4. For comparison, a naive absolute-threshold check on the same series (fire whenever the raw value exceeds 1500) also only fires at index 7 in this particular series, but for the wrong reason, because the raw counter happened to be large again at that point, not because it measured the actual rate; that check would ALSO eventually fire on any long-lived healthy counter purely from accumulation, and would MISS a real spike whose post-reset raw value stayed below the absolute threshold even while its rate was extreme.

Trade-offs and pitfalls
The most common mistake is choosing a single global evaluation window and rate threshold for every service's requests-per-second metric, when the right window and threshold depend on that specific metric's normal sampling noise and traffic pattern; copying a working config from one service to another is a frequent, quiet source of both false positives and missed real events. A second pitfall is detecting resets by checking for a value near exactly zero rather than any decrease, which misses a "partial reset" (a counter that restarts at a nonzero value due to some persisted state, or a metrics pipeline that occasionally re-delivers an older, smaller value out of order), so the correct test is "did the value decrease relative to the previous sample," not "did it hit exactly zero." A third pitfall, specific to the missing-datapoint case, is a monitoring pipeline that backfills a missing sample with the last-known value before the alerting logic ever sees it; that silently converts a real data gap into a fabricated flat rate, which passes every rate-of-change test built against the raw samples while actually hiding the gap from the alerting logic entirely, so the edge-case test needs to run against what the alerting logic ACTUALLY receives after any upstream backfill, not against the idealized raw time series.

What a file descriptor is

A file descriptor (FD) is a small non-negative integer a process uses as a handle for something it has open: a regular file, a directory, a pipe, a socket, a terminal, or a device. The kernel keeps a per-process table mapping each FD to an open file description, which in turn points at the underlying inode (for real files) or other kernel object. Every process is started with three descriptors already open by convention: 0 (stdin), 1 (stdout), 2 (stderr).

File descriptors vs inodes

An inode describes a file's identity and metadata on disk and exists whether or not anything currently has the file open. A file descriptor is a runtime handle that exists only for the lifetime of a process and tracks per-open-instance state, most importantly the current read/write offset into the file. Because the mapping is many-to-many, several descriptors (in the same process or different processes) can point at the same inode, each with its own independent offset; conversely, a single inode is fully identified without any process needing it open at all.

Standard descriptors and inspecting a process's open FDs

The conventional set is 0 = stdin, 1 = stdout, 2 = stderr. Beyond those, a process can hold arbitrarily many descriptors for files, sockets, and pipes it has opened itself. Two ways to inspect them:

$ ls -l /proc/<pid>/fd

Verified against a real sleep 30 process:

lr-x------ 1 root root 64 Sep 24 00:18 0 -> /dev/null
l-wx------ 1 root root 64 Sep 24 00:18 1 -> pipe:[1487422]
l-wx------ 1 root root 64 Sep 24 00:18 2 -> pipe:[1487423]

Each entry under /proc/<pid>/fd/ is a symlink whose target names what that descriptor points at (here, stdin was redirected from /dev/null and stdout/stderr were pipes, because the process was launched with output piped into another command).

$ lsof -p <pid>

lsof ("list open files") does the same enumeration but resolves the targets into a friendlier table (command name, PID, user, FD, file type, device, size, and the actual path or socket description), and can also be filtered the other way, e.g. lsof /var/log/app.log to find every process with a specific file open.

Increasing the allowed number of open file descriptors

Temporarily (current shell session only, lasts until that shell exits):

ulimit -n 65536

This raises the soft limit for the calling shell and anything it subsequently launches, but it cannot exceed the hard limit already in effect, and it reverts the moment the shell exits.

Permanently, two layers usually need to agree:

• For login/interactive sessions via PAM (Pluggable Authentication Modules, the framework Linux login programs use to plug in authentication and session policies), /etc/security/limits.conf (or a file under /etc/security/limits.d/):

  appuser soft nofile 65536
  appuser hard nofile 65536
  

• For a systemd-managed service (one started and supervised by systemd, the init system most current Linux distributions use), the PAM limits file is often not consulted at all, so the unit itself needs LimitNOFILE=:

  ini

  [Service]
  LimitNOFILE=65536
  

  followed by systemctl daemon-reload and a restart of the unit.

Either way, the effective ceiling can never exceed the kernel-wide maximum in /proc/sys/fs/file-max, which itself can be raised (temporarily via sysctl -w fs.file-max=..., persistently via /etc/sysctl.d/*.conf) if it's the binding constraint.

Trade-offs and pitfalls

• Raising a process's soft nofile limit with ulimit -n inside a shell only affects that shell and its children; it does nothing for a service already running under systemd, which is the single most common "I set ulimit but the app still hits Too many open files" mistake.
• "Too many open files" is very often a leak, not an under-provisioned limit: raising the ceiling buys time but doesn't fix a program that never closes sockets or file handles; check lsof -p <pid> | wc -l growing unbounded over time before assuming it's just a sizing problem.
• Every open file descriptor consumes a small amount of kernel memory and, for sockets, a kernel network buffer; setting nofile extremely high "just in case" is not free and can mask a real leak until it hits a different resource ceiling instead.