DoorDash Staff Software Engineer Interview Preparation Guide

Direct answer

Count every character's frequency in one pass (a hash map or Counter), then make a second pass over the string returning the first character whose count is exactly 1. This is $O(n)$ time and $O(k)$ space, where k is the number of distinct characters. If the string arrives one character at a time and you must be able to answer "what's the first non-repeating character so far" at any point, keep a queue of once-seen candidates in arrival order and evict its front whenever that character's count rises above 1.

Structured elaboration

Two-pass approach, for a string you already have in full:

from collections import Counter

def first_non_repeated(s):
    """
    Return first non-repeated character in s, or None if none exists.
    Two-pass approach: O(n) time, O(k) space (k = distinct characters).
    """
    if not s:
        return None
    counts = Counter(s)
    for ch in s:
        if counts[ch] == 1:
            return ch
    return None

Streaming variant. The two-pass approach needs the whole string up front. If characters arrive one at a time and a query can land at any point, you cannot afford to rescan everything seen so far on every query. Instead, maintain a frequency map alongside a queue (double-ended queue) of characters that are currently unique, in the order they first appeared:

from collections import deque

class StreamingFirstNonRepeated:
    """
    Streaming variant: feed one character at a time via .push(ch) and query
    .current() at any point without rescanning history. O(1) amortized time
    per pushed character (each character enters and leaves the deque at most
    once), O(k) space for k distinct characters seen so far.
    """
    def __init__(self):
        self.counts = {}
        self.q = deque()

    def push(self, ch):
        self.counts[ch] = self.counts.get(ch, 0) + 1
        if self.counts[ch] == 1:
            self.q.append(ch)
        while self.q and self.counts[self.q[0]] > 1:
            self.q.popleft()

    def current(self):
        return self.q[0] if self.q else None

The queue's front is always the earliest-arrived character that is still unique, because any character that becomes non-unique gets evicted from the front the moment its count rises above 1 (it may sit behind the front briefly until it becomes the front, but it is removed by the time it would otherwise be reported).

Worked example

s = "swiss"
print(first_non_repeated(s))

tracker = StreamingFirstNonRepeated()
running = []
for ch in s:
    running.append(ch)
    tracker.push(ch)
    print(''.join(running), '->', tracker.current())

Output:

w
s -> s
sw -> s
swi -> s
swis -> w
swiss -> w

The full-string answer is w, matching the streaming tracker's final answer. Along the way you can see the answer change: after "s", "sw", "swi" the answer is still s (unique so far); once the second s arrives ("swis") the tracker evicts s from the front and reports w; the final s in "swiss" doesn't change the answer since w is still unique.

Trade-offs & pitfalls

Key points

• The two-pass approach is the simplest correct solution when the full string is available; don't reach for the streaming version if you don't need "answer at any point in time" semantics, since it adds a queue and eviction logic for no benefit.
• The streaming approach never needs to rescan from the start, but it does need to keep counts for every distinct character seen so far, and the queue can (temporarily) hold characters that later get evicted, so peak memory is still O(k) not O(1).
• Multi-codepoint or combined Unicode characters (for example, accented characters built from a base character plus a combining mark) are treated as separate codepoints by both approaches; if the requirement is "first non-repeating user-visible character" rather than "first non-repeating codepoint," you would need a grapheme-aware library instead of iterating raw codepoints.

Complexity

• Two-pass: time $O(n)$, space $O(k)$.
• Streaming: $O(1)$ amortized time per pushed character (each character is added to and removed from the queue at most once), space $O(k)$ for the counts and queue combined.

Edge cases

• Empty string: both approaches return None.
• All characters repeated: both return None (the two-pass loop finds no count-1 character; the streaming queue empties out).
• Single character: trivially non-repeating, both return it.

High-level recommendation: For a 20-analyst team with mostly interactive analytics and occasional very heavy ad-hoc queries, I’d recommend a managed cloud OLAP service (e.g., BigQuery). It gives predictable performance for interactive workloads, elastic capacity for spikes, and far lower ops overhead. Below is a side-by-side comparison and justification.

Performance predictability

• Managed OLAP (BigQuery): High for typical queries; service auto-optimizes, separates storage/compute, fast for many workloads. Very large ad-hoc jobs can have variable latency but scale automatically.
• Self-managed (Presto+Parquet on K8s): Can be tuned for predictable latency if you size clusters and tune memory/GC/IO, but unpredictable under noisy neighbors unless you overprovision.

TCO

• Managed: Pay-as-you-go or flat-rate slots — lower upfront costs, operational savings. For mostly moderate usage with occasional spikes, managed often wins.
• Self-managed: Lower per-query engine cost on paper for massive steady workloads, but higher indirect costs (SRE time, monitoring, node replacement, storage and egress) that often make TCO higher unless sustained heavy usage justifies it.

Administrative overhead

• Managed: Minimal — no cluster management, upgrades, tuning or patching.
• Self-managed: Significant — cluster orchestration, resource tuning, backups, security patches, observability, incident response.

Query concurrency

• Managed: High concurrency natively; autoscaling handles many simultaneous analysts.
• Self-managed: Concurrency limited by cluster size; needs autoscaler + queues or more nodes to match managed concurrency, increasing cost.

Vendor lock-in

• Managed: Higher lock-in (SQL dialects, storage format, proprietary features). Mitigation: use standard SQL, Parquet/ORC for exports, and abstract access in code.
• Self-managed: Lower lock-in if you standardize on open formats (Parquet) and engines, but you still depend on chosen components.

Recommendation rationale

• Team size and pattern: 20 analysts => many concurrent interactive queries; occasional heavy jobs require elasticity. Managed OLAP provides concurrency and burst capacity without ops work.
• Risk/cost tradeoff: Managed reduces operational risk and time-to-insight; controlled costs via quotas, flat-rate commitments, and monitoring. If the organization expects sustained, very high query volume (e.g., many terabytes per day) and has strong SRE bandwidth, re-evaluate self-managed.

Practical mitigations

• For managed: use reserved slots/commitments for predictable spend, cost alerts, and export raw data in Parquet to avoid lock-in.
• For self-managed: containerize configs, automate infra-as-code, and enforce query governance to control costs.

Summary: Choose managed OLAP for faster delivery, lower ops, and elastic concurrency; switch to self-managed only if you can justify sustained, predictable heavy throughput and the SRE investment.

Direct answer. A shape-mismatch error tells you two tensors disagreed in dimension somewhere in the forward pass, but the traceback often points at the operation that FAILED, not the operation that introduced the wrong shape several layers earlier, so the debugging process is really about walking the shape forward from the input until it diverges from what you expect.

Step-by-step approach.

1. Print the input shape first, and compare it against what the first layer actually expects. A surprising number of shape bugs are simply "the input isn't shaped the way I assumed," not a bug in the model at all.
2. Use a model summary tool (or manually print .shape after each layer in a quick forward pass) to see the shape at every stage in one pass, rather than binary-searching by commenting out layers one at a time.
3. Add explicit shape assertions inside custom layers, at the point where a specific shape is assumed (assert x.shape[-1] == self.expected_dim, f"got {x.shape}"). This turns a downstream, confusing shape error into an immediate, precisely-located one the next time the bug is triggered, which pays for itself the first time someone else hits a variant of the same bug.
4. Write a small unit test with a known, fixed input shape that exercises just the suspect layer or block in isolation, rather than the whole model, so you can iterate on the fix without paying the cost of a full forward pass through everything else.

A concrete example of why step 1 matters. A very common real case: a model expects batch-first input (batch, seq_len, features) but receives (seq_len, batch, features) from a data loader or a different framework's convention. The shapes are individually valid tensors, nothing crashes until several layers in when a dimension that "coincidentally" matched for a while finally doesn't, at which point the error message points at a layer far from the true cause (the data loader).

The unit test that prevents recurrence. Something as small as:

def test_encoder_output_shape():
    x = torch.randn(4, 10, 32)  # (batch=4, seq_len=10, features=32), the CONTRACT this layer expects
    out = encoder(x)
    assert out.shape == (4, 10, 64), f"expected (4, 10, 64), got {out.shape}"

run in CI on every change to the layer or anything upstream of it, catches this class of bug the moment a shape contract is violated, rather than three deploys later when someone finally notices predictions look wrong.

Direct answer
This is a behavioral question best answered with a specific story showing you distinguished a genuine pattern from a one-off, kept the accountability conversation separate from the group postmortem, and led with coaching rather than punishment, while still being clear about the standard expected going forward.

Structured elaboration
A strong answer covers: the situation (what happened, why it warranted an accountability conversation rather than just the standard blameless process), how you determined it was a genuine pattern rather than bad luck, how you had the conversation itself (privately, focused on support and root cause rather than judgment), and the outcome, including how you protected the person's and the team's trust in future incident reporting.

Worked example
"An engineer on my team was involved in a second incident within two months, both traced to skipping the same pre-deploy verification step under time pressure. Before assuming it was negligence, I checked whether this was a genuine pattern: both incidents shared the same specific gap, so it was, rather than two unrelated issues that happened to involve the same person. I had a private conversation, not as part of the group postmortem, focused on understanding what was driving it. It turned out they'd been covering an unsustainable amount of on-call load for a teammate on leave and had started rushing to keep up. I addressed both sides: the workload issue got fixed by redistributing on-call coverage, which was the real systemic driver, and I paired them with a more experienced engineer on deploy discipline for a few weeks as direct support, framed clearly as investment in them, not punishment. Separately, the second incident's actual group postmortem stayed fully blameless and produced an automated pre-deploy gate that makes that specific step impossible to skip regardless of who's deploying, which is the fix that protects the whole team, not just this one person. Three months later there'd been no recurrence, and, just as important, the engineer's incident-reporting behavior stayed just as open as before, which told me the accountability conversation hadn't damaged their trust in the process."

Trade-offs and pitfalls
The most common weak answer either skips the distinction between pattern and coincidence (treating any second incident as automatic proof of a problem) or focuses entirely on the disciplinary angle without addressing what was actually driving the pattern, which misses the more senior insight that repeated mistakes are frequently a symptom of a systemic issue (workload, unclear process, inadequate training) rather than a character flaw.

Direct answer

Structure the middleware as a loop around the handler: an idempotency check runs first (before any retry bookkeeping, so a redelivery of an already-processed message never consumes a retry slot), then on each failure compute a capped exponential delay and sleep a random amount between zero and that cap (full jitter), and once the configured maximum retries is exceeded, push the message to a dead-letter queue (DLQ) and stop.

Structured elaboration

Where the idempotency check belongs. It runs before the retry loop, not inside it. A message redelivered after it already succeeded should short-circuit immediately, it should not be treated as attempt 1 of a fresh retry sequence, and it should not re-run the handler's side effects.

The retry loop. On each failure: if the attempt count exceeds maxRetries, escalate to the DLQ and stop; otherwise compute delay = min(maxDelayMs, baseMs * multiplier ** (attempt - 1)) and sleep a uniformly random value between 0 and that delay (full jitter) before the next attempt.

Configurable parameters. baseMs and multiplier control how fast the delay grows per attempt; maxDelayMs caps it so a message near its retry limit does not wait an unreasonably long time; maxRetries bounds total attempts before DLQ escalation.

function makeBackoffMiddleware({ baseMs, multiplier, maxRetries, maxDelayMs, rng, sendToDLQ, idempotencyCheck }) {
  return async function process(message, handler) {
    // Idempotency check runs BEFORE any retry bookkeeping: a redelivered
    // message already committed downstream short-circuits here instead of
    // consuming a retry slot or re-running side effects.
    if (await idempotencyCheck(message)) {
      return { status: 'skipped-duplicate', attempts: 0, delaysMs: [] };
    }

    let attempt = 0;
    const delaysMs = [];
    for (;;) {
      attempt += 1;
      try {
        await handler(message);
        return { status: 'processed', attempts: attempt, delaysMs };
      } catch (err) {
        if (attempt > maxRetries) {
          await sendToDLQ(message, err, attempt - 1);
          return { status: 'dlq', attempts: attempt - 1, delaysMs, error: err.message };
        }
        // Full jitter: sleep a UNIFORM random value between 0 and the capped
        // exponential delay, not the exponential value itself, this is what
        // prevents every instance that failed at the same moment from
        // retrying in lockstep (the thundering-herd failure mode).
        const capped = Math.min(maxDelayMs, baseMs * multiplier ** (attempt - 1));
        const jitterMs = Math.round(capped * rng());
        delaysMs.push(jitterMs);
        // production code awaits a real timer here: await sleep(jitterMs)
      }
    }
  };
}

Worked example

Executed with Node.js, baseMs = 100, multiplier = 2, maxRetries = 4, maxDelayMs = 5000, and a seeded pseudo-random number generator (mulberry32, seeded with 42) standing in for Math.random() purely so the printed jitter values are reproducible for this demo; production code would use Math.random() or crypto.randomInt() instead.

case1 (recovers within budget): {"status":"processed","attempts":4,"delaysMs":[60,90,341]}
case2 (exhausts retries -> DLQ): {"status":"dlq","attempts":4,"delaysMs":[67,35,211,219],"error":"downstream-500"}
dlq contents: [{"id":"evt-2","attempts":4,"error":"downstream-500"}]
case3 (duplicate redelivery skipped): {"status":"skipped-duplicate","attempts":0,"delaysMs":[]}

Case 1: a handler that fails 3 times then succeeds recovers on attempt 4, within the 4-retry budget, with 3 recorded jitter delays (one per failure before the successful attempt). Case 2: a handler that always fails exhausts all 4 retries and escalates to the DLQ, with the DLQ entry recording the event id, attempt count, and error. Case 3: redelivering the same event id from case 1 (already committed) is caught by the idempotency check and returns immediately with zero attempts, confirming the check runs ahead of the retry loop rather than inside it.

Trade-offs and pitfalls

Complexity. O(1) work per attempt (one comparison, one exponentiation, one random draw); total attempts bounded by maxRetries + 1, so worst-case work per message is O(maxRetries).

Edge cases.

• A message redelivered mid-retry (a second copy of the same event arrives while the first copy's retry loop is still running) needs the idempotency check to be safe under concurrency too, a check-then-act pattern here has the same race risk as any other idempotency check, it needs an atomic claim, not just a lookup.

• If handler itself is not safe to partially re-run (it has non-idempotent side effects), retrying at all is unsafe regardless of how well the backoff is tuned, the retry middleware assumes the handler's work is safe to repeat.

• A cap (maxDelayMs) that is too low relative to how long the downstream outage actually lasts causes retries to keep hammering a still-down dependency at the capped rate instead of backing off further.

• Sleeping the exponential value itself instead of a random value up to it (no jitter at all) reintroduces the thundering-herd risk this design exists to avoid.

• Forgetting to insert the idempotency check ahead of the retry loop (instead of, say, only checking once right before the DLQ push) lets an already-succeeded message consume retry attempts and, worse, potentially re-run a non-idempotent handler on every redelivery.

Approach: replace scalar loop with an AVX2 vectorized loop (256-bit registers, 8 floats per vector), process bulk with intrinsics or write compiler-friendly code and let auto-vectorizer handle it. Handle alignment/padding and remainder tail safely.

#include <immintrin.h>
void transform_avx2(float *a, const float *b, size_t n, float scale, float bias) {
    size_t i = 0;
    const size_t V = 8; // AVX2: 8 floats
    __m256 vScale = _mm256_set1_ps(scale);
    __m256 vBias  = _mm256_set1_ps(bias);

    // Process head until a is 32-byte aligned (optional; unaligned loads are OK but slightly slower)
    size_t addr = reinterpret_cast<uintptr_t>(a);
    size_t mis = (32 - (addr & 31)) & 31;
    size_t head = (mis / sizeof(float));
    if (head > n) head = n;
    for (; i < head; ++i) // scalar prologue
        a[i] = a[i] * scale + b[i] + bias;

    size_t bulk = (n - i) / V;
    for (size_t k = 0; k < bulk; ++k, i += V) {
        // load 8 floats from a and b
        __m256 va = _mm256_loadu_ps(a + i);       // or _mm256_load_ps if aligned
        __m256 vb = _mm256_loadu_ps(b + i);
        // compute: a = a * scale + b + bias
        va = _mm256_fmadd_ps(va, vScale, vb);     // va = va*scale + vb
        va = _mm256_add_ps(va, vBias);            // va += bias
        _mm256_storeu_ps(a + i, va);              // store back
    }

    // Remainder tail
    for (; i < n; ++i)
        a[i] = a[i] * scale + b[i] + bias;
}

Key points:

• Alignment: aligned loads/stores (_mm256_load_ps/_mm256_store_ps) are faster but require 32-byte aligned pointers; use posix_memalign/_aligned_alloc or std::aligned_alloc. Unaligned variants (_mm256_loadu_ps) work with any pointer.
• Padding: allocate arrays with extra elements so that tail/wrap doesn't access out-of-bounds.
• Remainders: handle head (prologue) and tail (epilogue) scalars or use masked loads with AVX-512 or tail processing.
• Compiler-friendly alternative: write simple, well-typed loop, add restrict-like hints (pointer attributes), use -O3 -march=native and pragmas (#pragma GCC ivdep, #pragma omp simd) to help auto-vectorizer.

Measuring & verification:

• Measure with high-resolution timers (std::chrono::steady_clock) over many iterations; warm up caches; disable turbo/thermal variability or pin CPU frequency.
• Use perf/VTune/likwid to measure vector instruction counts, SIMD lane utilization, memory bandwidth, and cycles.
• Verify correctness with unit tests vs scalar reference.
• Check assembler (objdump/clang -S) or compiler vectorization reports (-fopt-info-vec) to confirm vector instructions emitted.
• Profile different builds (aligned vs unaligned loads, with/without FMA) and report throughput (GFLOPS or elements/s) and speedup.

Direct answer

When two senior stakeholders give you contradictory direction on the same decision and both expect you to follow theirs, the right first move is not to pick a side or quietly satisfy whichever one you last spoke to, but to make the contradiction visible to both of them together and force an explicit resolution before proceeding.

Structured elaboration

1. Don't silently choose. Picking one direction without surfacing the conflict either burns the relationship with whoever you didn't follow, or produces work that gets undone when the conflict eventually surfaces anyway, at a later, more expensive point.
2. Bring the contradiction to both of them together, factually. A short message or meeting stating plainly "I've received direction X from one of you and direction Y from the other, and I need clarity on which to follow before proceeding" reframes the problem as theirs to resolve, not yours to guess at.
3. Provide the trade-off, not just the conflict. Where possible, lay out what each direction implies (cost, timeline, risk) so the conversation between them is grounded in consequences, not just preference.
4. If they can't resolve it between themselves, escalate to whoever can. A genuine stalemate between two people with equal standing over you needs a tie-breaker above both of them; naming that clearly rather than continuing to sit in the middle is the responsible move once direct resolution has been tried and failed.
5. Document the resolution. Once a direction is confirmed, write it down and share it back to both, so the same conflict doesn't quietly resurface a month later as a "misunderstanding."

Worked example

Two regional teams each push a different prioritized roadmap for the same shared platform, both expecting their preference to be honored. Rather than picking one, laying out both roadmaps side by side with their business rationale and cost of NOT doing the other, presented jointly to both regional leads, forces a real conversation about trade-offs between people who actually have standing to make that call, instead of an individual contributor guessing at organizational priorities they don't own.

Trade-offs and pitfalls

Surfacing the conflict too quickly, before doing any homework on the trade-offs, can look like you're avoiding the work of even a preliminary recommendation; where you have a well-reasoned view, offering it as input to their conversation (not as a decision you're making for them) is usually stronger than presenting a bare, unexamined conflict.

Direct answer

Numbers tell people what happened; a narrative tells them why it matters and to whom. When a data table or a business case document isn't landing, building the argument as a short story, real people, a specific conflict, stakes tied to something they already care about, anchored by evidence rather than replaced by it, can move a decision that pure data couldn't.

Structured elaboration

How this differs from the two other evidence vehicles. This is not the same move as anchoring a case in the data-table or business-case artifact (the default, and often the right choice when the audience trusts numbers on their own). It's also not the same as letting the physical prototype artifact carry the argument by itself (the approach where the thing you built does the persuading). Here the vehicle is a narrative: a sequence with a protagonist, a conflict, and stakes, with evidence anchoring the story rather than the story decorating the evidence.

Building the narrative.

1. Pick a protagonist who is actually affected by the status quo: a user, a support rep, an engineer on call. Not an abstraction.
2. Establish the conflict: what specifically goes wrong for them today, and why it keeps happening.
3. Anchor with evidence: one or two credible data points and a direct quote, not a full dashboard. The story should feel evidenced, not decorated.
4. Build to a concrete ask: a decision or, better, a small experiment, not just "please feel differently about this."

Adapting the telling by audience.

Worked example

Situation: a product org was deadlocked between funding a flashy AI onboarding feature leadership was excited about, and fixing a plain, unglamorous signup flow that was quietly losing new users.

The narrative: a short story following one new user through the existing signup flow, where she gets stuck partway through and gives up, alongside a support rep who fields the same complaint on repeat. The conflict: leadership wanted to invest in something exciting while the thing actually costing the company users was mundane. The stakes: continuing to ship novelty without fixing the leak meant the AI feature would land on a shrinking base.

Anchoring the story: a couple of real interview quotes from users who abandoned partway through, paired with the observed drop-off point in the flow, kept the story honest rather than invented.

Adapting the telling: for engineering, the story led with exactly where in the flow users got stuck and why. For product, it led with the user's journey and what the AI feature would cost in opportunity if the base kept shrinking. For the executive review, the ask came first: "approve a two-week experiment on the signup flow before committing the quarter to either option," with the story as the two-minute follow-up, not the opener.

Resolution: instead of the roadmap fight resolving by whoever argued loudest, leadership agreed to run the signup experiment first and revisit the AI feature with better information afterward. The story didn't replace the case for prioritization: it gave the room a shared, human reason to care about a decision that had been sitting in the abstract.

Trade-offs & pitfalls

• A narrative without real evidence anchoring it reads as manipulation, not persuasion, especially to an audience that already leans skeptical of "storytelling" in a business context.
• Over-tailoring the same story so heavily per audience risks contradicting yourself if two audiences compare notes; the underlying facts should stay identical even as the framing shifts.
• Narrative takes longer to build well than a table of numbers. It's worth the investment when the decision is stuck on people not caring yet, not when it's stuck on people not believing the numbers.
• Leading with story instead of the ask in front of executives is a common miscalibration; senior communicators state the ask first and let the narrative support it, not the reverse.

Start by clarifying goal and scope: define "velocity" metric you trust (story points completed, cycle time, throughput) and the time window. Then run a two-phase diagnosis: quick signals to triage, followed by a targeted deep dive.

Quick signals (fast, 1–3 days):

• Technical debt: rising bug count, increasing mean time to restore (MTTR), long-running PRs with many rework comments, frequent hotfixes, spikes in test failures, larger-than-normal diff sizes for small features.
• Unclear requirements: high percent of stories reopened/changed after development, many spec/acceptance-comments in tickets, frequent scope creep in sprint, backlog grooming backlog with vague descriptions.
• People/process: lower meeting attendance, increasing blocked days per ticket, uneven distribution of work (few authors), drop in pair-programming/mentorship activity, declining CI/CD deployment frequency.

Structured deep dive (1–2 weeks):

1. Collect metrics: cycle time, lead time, PR review latency, bug trends, deploy frequency, story reopen rate.
2. Interviews & observations: 30–60 min conversations with PMs, engineers, QA to surface friction and examples; join standups and a planning session to observe.
3. Trace sample stories end-to-end: pick 5 recent slow or failed stories and map where time was spent (design, implementation, review, QA).
4. Root-cause analysis: for each story, tag time lost to code complexity, spec changes, people blockers, or process (e.g., long review queues).
5. Synthesize and prioritize fixes: quick wins (clearer acceptance criteria, stricter DoR, dedicated bug-fix slot, short tech-debt spikes), and medium-term (refactor, CI improvements, hiring/training).

Example action plan:

• If technical debt dominates: schedule 10% sprint allocation to debt, add code health KPIs, enforce smaller PRs and better tests.
• If requirements dominate: require PM+engineer sign-off on DoR, add example acceptance tests, shorten feedback loops with prototypes.
• If people/process dominate: fix bottlenecks (rotate reviewers, reduce context switching), improve WIP limits and run a blameless retrospective.

Outcome focus: measure impact after two sprints using same metrics and iterate.

Direct answer

The transactional outbox pattern gets atomicity between a local database update and publishing an event by writing the event as a plain row in the SAME database transaction as the business change, instead of trying to atomically commit across two separate systems (the database and the message broker). A separate background publisher then reads that outbox table and delivers the events, so the hard part of the problem (getting a message onto a broker) is pushed into an at-least-once delivery loop the consumer can absorb, rather than solved by a distributed commit protocol.

Why not just use a distributed transaction directly

A two-phase commit (2PC) across the database and the broker would need the broker to act as a participant in the same commit protocol as the database: it would have to accept a "prepare" call, hold the message uncommitted, and only make it visible once a coordinator later sends "commit". Kafka's and SQS's client APIs do not expose that kind of prepare/commit participant interface (Kafka has its own separate transactional-producer API, not an XA participant interface, XA being the X/Open standard for coordinating a transaction manager with multiple resource managers), so 2PC across a database and a managed broker directly is not something you can wire up against most production message buses. Even where a broker does support it, the coordinator becomes a blocking point: each participant holds its local lock from "prepare" until it hears back, so a coordinator crash between phases can leave a participant blocked indefinitely.

Outbox table schema

Writing the event

Inside the same database transaction that updates the business tables (e.g. orders), the application also inserts a row into outbox. Both inserts commit together or not at all, so there is no window where the business change exists without a corresponding outbox row, or vice versa.

Background publisher

A worker process polls with something like SELECT * FROM outbox WHERE status = 'pending' ORDER BY id FOR UPDATE SKIP LOCKED LIMIT 100. FOR UPDATE SKIP LOCKED lets multiple publisher instances run concurrently without two of them claiming the same row: each instance simply skips rows another instance already has locked. The worker publishes each claimed row to the broker, and only after the broker acknowledges does it mark the row sent. As an alternative to polling, a change-data-capture (CDC) tool such as Debezium can tail the database's write-ahead log (WAL, the durability log the database already writes before committing) and stream outbox inserts to the broker with lower latency and no polling interval, without any application code change.

Avoiding duplicates and lost events across a crash

Trace through a concrete run for outbox row id=482:

1. Application transaction commits: orders row and outbox row id=482 (status=pending) both durable together.
2. Publisher polls, claims row 482 via SELECT ... FOR UPDATE SKIP LOCKED.
3. Publisher sends row 482's payload to the broker; broker acknowledges receipt.
4. Publisher crashes before executing UPDATE outbox SET status='sent' WHERE id=482.
5. Publisher restarts, polls again; row 482 is still status='pending', so it gets re-claimed and re-published. The broker now has two copies of the same event.
6. Because row 482's id travels with the payload as a dedup key, the consumer (or the broker's own dedup mechanism) recognizes the second delivery as a repeat and drops or no-ops it.

If the crash instead happens before step 3 (before the publish call), nothing has reached the broker at all: row 482 simply stays pending and gets picked up on the next poll, with no data loss. The pattern never produces a business change with a missing event, only occasional duplicate deliveries, which is why the consumer side still needs to be idempotent or dedup-aware; the outbox guarantees "at least once", not "exactly once", on its own.

sequenceDiagram
    participant App
    participant DB as Database
    participant Pub as Publisher
    participant Bus as Message Bus
    App->>DB: BEGIN TXN
    App->>DB: INSERT orders row
    App->>DB: INSERT outbox row (pending)
    App->>DB: COMMIT
    Pub->>DB: SELECT pending FOR UPDATE SKIP LOCKED
    DB-->>Pub: outbox row id=482
    Pub->>Bus: publish(id=482)
    Bus-->>Pub: ack
    Pub->>DB: UPDATE outbox SET status=sent WHERE id=482

Production checklist

• Alert on outbox backlog depth (rows still pending beyond an expected age), not just publisher liveness, since a stuck publisher looks alive but stops draining the table.
• Retention job to delete or archive sent rows so the table doesn't grow unbounded.
• Partition/order guarantee: publish using aggregate_id as the broker partition key so events about the same entity are delivered in the order they were written.
• Version the payload schema so a newer producer and an older consumer can coexist during a rollout.

Trade-offs & pitfalls

Outbox trades immediate, synchronous cross-system consistency for simple local atomicity plus eventual, at-least-once delivery: the business transaction commits instantly (no waiting on the broker), but there is a real (if usually short) window between the commit and the event actually reaching the broker. A common mistake is treating "broker acknowledged" as "consumer processed": the outbox only guarantees the event left the outbox table, not that the ultimate side effect happened, so end-to-end correctness still depends on consumer idempotency. Another common mistake is forgetting to key on aggregate_id for ordering, or running multiple publisher instances without SKIP LOCKED-style claiming, which produces either out-of-order delivery or duplicate publishes from two workers claiming the same row simultaneously.