DoorDash Senior Embedded Developer Interview Preparation Guide

The idea

Endianness is the order in which a multi-byte value's bytes sit in memory. To detect it, store a value whose four bytes are all different, such as 0x01020304, then look at the bytes in memory order. If the first byte is 0x04 the least significant byte comes first (little-endian). If it is 0x01 the most significant byte comes first (big-endian). Anything else is a rarer mixed order.

Here is what the four bytes of 0x01020304 look like at increasing memory addresses on each kind of machine (address 0 is the first byte):

The skill the question tests is how to look at those bytes without breaking C's rules. Two terms govern those rules. Undefined behaviour means the C standard puts no requirement on what the program does, so the compiler may assume the situation never happens, and the program can print anything, crash, or work today and fail after a compiler upgrade. Implementation-defined behaviour means the standard lets each compiler and platform choose, and the compiler must document its choice; the program is still valid, but its result can differ between platforms. Byte order is the second kind: it is a documented property of each machine, not a bug.

The code

The check copies the object's bytes into an unsigned char array with memcpy, then classifies them. CHAR_BIT (from <limits.h>) is the number of bits in a byte on this platform, and _Static_assert is a compile-time check: if its condition is false the program does not build. The classifier is a separate pure function so it can be tested with byte patterns from machines you do not own.

#include <limits.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>

_Static_assert(CHAR_BIT == 8, "probe assumes 8-bit bytes");
_Static_assert(sizeof(uint32_t) == 4, "probe assumes a 4-byte uint32_t");

typedef enum { ENDIAN_LITTLE, ENDIAN_BIG, ENDIAN_OTHER } endian_t;

/* Classify the four bytes that a host stored for the value 0x01020304. */
endian_t classify(const unsigned char b[4]) {
    if (b[0] == 0x04 && b[1] == 0x03 && b[2] == 0x02 && b[3] == 0x01) return ENDIAN_LITTLE;
    if (b[0] == 0x01 && b[1] == 0x02 && b[2] == 0x03 && b[3] == 0x04) return ENDIAN_BIG;
    return ENDIAN_OTHER;
}

endian_t host_endianness(void) {
    uint32_t probe = 0x01020304u;
    unsigned char bytes[4];
    memcpy(bytes, &probe, sizeof bytes);   /* copying bytes of an object is always allowed */
    return classify(bytes);
}

static const char *name(endian_t e) {
    return e == ENDIAN_LITTLE ? "little" : e == ENDIAN_BIG ? "big" : "other";
}

int main(void) {
    const unsigned char little[4] = {0x04, 0x03, 0x02, 0x01};
    const unsigned char big[4]    = {0x01, 0x02, 0x03, 0x04};
    const unsigned char mixed[4]  = {0x02, 0x01, 0x04, 0x03};
    printf("sample little -> %s\n", name(classify(little)));
    printf("sample big    -> %s\n", name(classify(big)));
    printf("sample mixed  -> %s\n", name(classify(mixed)));
    printf("this host     -> %s\n", name(host_endianness()));
    return 0;
}

Output

Compiled with gcc -std=c11 -Wall -Wextra -fsanitize=undefined (GCC 14.2, aarch64 Linux), it prints:

sample little -> little
sample big    -> big
sample mixed  -> other
this host     -> little

The first three lines come from the injected byte patterns and are the same on every machine. The last line is the real probe of the host it runs on. With -O2 the compiler folds host_endianness() to a constant (the generated aarch64 code is a single mov w0, 0 and ret), so the runtime check costs nothing in a hot path.

What this approach avoids

Whether the byte layout is controlled or merely observed is the key distinction. A program controls a layout when its own code decides where each byte goes. With out[0] = (uint8_t)(v >> 24) the first byte of out is always the most significant byte of v on every machine, so the layout is controlled. The probe does the opposite: it asks the machine how it stored a uint32_t and reports the answer, so it observes. Observing is the point of an endianness probe, and it is why the last line of the output can change from one machine to another.

What is not avoided is the thing being measured: the byte layout of an uint32_t is implementation-defined, which is exactly why the program can report different answers on different hardware. Reading bytes through unsigned char (or char) is explicitly allowed to inspect any object, and memcpy of an object's bytes is always allowed.

Complexity and edge cases

O(1) time and space (four bytes copied and at most eight comparisons). The classifier returns a third value, ENDIAN_OTHER, for orders like 02 01 04 03 instead of mislabelling them as big-endian. Do not return a plain boolean is_little, because a wrong "not little, therefore big" is a real bug on mixed-endian hardware.

Prefer not to branch on endianness at all when you can: reading and writing fields with shifts, as in a serialiser, behaves the same on every host.

Running the code

gcc -std=c11 -Wall -Wextra -fsanitize=undefined -o detect_endian detect_endian.c && ./detect_endian

Brief approach (step-by-step)

1. Hookup

• Connect LA ground to device ground. Probe TX line (device→PC) to a digital channel. If possible probe RX and a firmware GPIO that toggles on the event you want to correlate.

2. Sampling rate

• Set sample >= 10× baud for clean edges. For 115200 baud (bit ≈ 8.68 µs) use 1.2–2 MHz minimum; 4–10 MHz preferred for accurate timing and jitter measurement. (Higher if you need eye diagrams.)

3. Trigger configuration

• Use a falling-edge trigger on the TX line (idle = high, start bit = low) to capture start of frames.
• For intermittent garbage, set pattern trigger (e.g., start bit followed by byte mask) or use repeated/failure trigger if your analyzer supports UART framing errors.

4. Capture & decode

• Capture a buffer long enough for multiple frames. Enable built-in UART decoder with correct settings: baud=115200, data bits, parity, stop bits.
• Inspect decoded bytes, flag framing/parity errors.

5. Inspect start/stop/parity manually

• Zoom to bit level, measure bit width to confirm baud. Check start bit low duration ≈ 1 bit; stop bit(s) high; parity bit position if used. Look for mid-bit sampling violations, extra/short stop bits, or noise right after stop bit.

6. Correlate with firmware

• Record firmware event signal on another channel (GPIO toggle, ISR pin, or use SWD to toggle pin at relevant code points). Use timestamps from capture to compute delta t between firmware event and transmitted bytes.
• For repeated tests, instrument firmware to toggle pin before/after uart_write to measure blocking/non-blocking timing.

7. Additional tips

• If garbage only on host-side, probe both TX/RX. Use averaging or high sample rate to observe ringing/crosstalk. Export raw timestamps/bytes if deeper analysis needed.

This shows framing, timing, and firmware correlation to isolate baud/config or software-timing bugs.

What a watchdog timer (WDT) is
A WDT is hardware (or sometimes software) timer that forces a system reset if not periodically "kicked" within a configured window. It protects against hangs, live-locks, and certain classes of software faults.

How it improves resilience

• Forces recovery from unresponsive states when normal control flow is lost.
• Acts as last-resort fail-safe when other error handling fails.
• Combined with safe boot and diagnostics, it enables autonomous recovery in field devices.

Safe usage patterns

• Kick from a single, well-defined place (e.g., system supervisor or main control loop) — not scattered across code.
• Use a windowed watchdog when available to detect both stalls and runaway tasks.
• Feed only after completing health checks: scheduler alive, watchdog task heartbeat, memory/stack checks.
• On reset, perform minimal early initialization, check a persistent boot counter or CRCed state, and avoid automatic destructive actions until diagnostics run.
• Use non-volatile logs (with CRC) to record cause of reset and avoid blind loops; limit boot-loop retries with exponential backoff.

Example (C-style supervisor):

// called from main loop or supervisor task
void supervisor_tick(void) {
  if (health_checks_pass()) {
    watchdog_kick();
  } else {
    // log error, attempt controlled recovery
    attempt_recovery();
    // if unrecoverable, let WDT reset
  }
}

How to avoid masking real faults

• Never kick the WDT from ISRs or from many unrelated modules — this hides faults in main code.
• Ensure health_checks verify core subsystems (scheduler, comms, sensors).
• Prefer letting the WDT reset for unrecoverable faults instead of masking by naive kicks.

Common pitfalls

• Resetting without flushing persistent state: losing telemetry/config. Mitigate by atomic writes, journaling, or writing a shutdown marker before critical ops.
• Clearing watchdog prematurely in error paths: e.g., error handler that loops and keeps kicking. Instead, fail fast or escalate.
• Assuming WDT fixes logic bugs: it only recovers from stuck states; bugs causing repeated bad behavior after boot must be addressed (use counters, safe-mode).
• Boot loops: track reset count in RTC/NV and change behavior after N resets (enter safe mode or remote update).

Use WDT as a disciplined fail-safe integrated with health checks, persistent diagnostics, and boot protections to improve system resilience without hiding root causes.

Direct answer

Once you know a partner function is measured on something different from you (speed versus quality or risk, for example), you scope your asks to be small and cheap under their metric, and you change what "status" means when you talk to them: short, action-oriented signals instead of the detailed risk narrative you'd give your own stakeholders. You're not changing what you need, you're changing how you package it so it doesn't read as a tax on the thing they're rewarded for.

Structured elaboration

1. Diagnose the incentive, don't assume it. Confirm what the partner function is actually measured on (deploy velocity, ticket close time, uptime, cost) rather than inferring it from how they push back. Different sub-teams within the "same" function can be measured differently.
2. Scope the ask to the smallest unit that gets you what you need. If they're speed-measured, don't ask for a broad, standing review of everything; ask for a narrow, well-bounded check on the specific surface that carries the risk you actually care about, and let everything else pass without friction.
3. Translate the ask into their currency. Instead of framing a request around your risk language, frame it around what it costs (or saves) them in their terms: incident response hours avoided, rework avoided, a compliance gate they'd otherwise hit later and more expensively.
4. Change the shape of status, not just the ask. For a speed-measured partner, give a compact signal (blocked/not blocked, a count, a single risk flag) they can act on in seconds. Save the fuller narrative for your own stakeholders who need the detail. Sharing the same long-form update with both audiences under-serves the partner who needs to move fast.
5. Keep a floor. Adapting your ask to their incentive has a limit: there's a minimum you can't compromise below without failing your own mandate. Know that floor before the conversation so "scoping down" doesn't quietly become "giving up the requirement."
6. Revisit as trust builds. Early asks are necessarily narrow and low-trust. As the partner sees your asks are well-scoped and your status updates are reliable, you can often widen the ask (a slightly broader review surface, more lead time) because they've learned you're not going to slow them down for nothing.

Worked example

A platform team is measured on release velocity; a security-minded partner function is measured on defect and incident rates. Rather than asking the platform team to route every change through manual security review (a direct tax on their velocity metric), the ask is scoped to only changes that touch a named risk surface, such as authentication or payment code. Everything else ships without added friction. Status to the platform team is a single weekly line: "2 changes in the review queue, 0 blocking, both cleared by Thursday." The fuller write-up, with rationale and residual risk, goes to the security function's own leadership, not to the platform team, because that's not the audience that needs it to act.

Trade-offs & pitfalls

• Pitfall: scoping the ask down so far it stops actually managing the risk it exists to manage. Know your floor before you negotiate.
• Pitfall: assuming the incentive instead of confirming it. Guessing wrong (e.g., treating a team as purely speed-driven when they're also on the hook for a compliance metric) leads to asks that miss what would actually land.
• Pitfall: sending the same status update to every audience. It either over-informs the speed-measured partner (who tunes it out) or under-informs your own stakeholders (who need the detail to make decisions).
• Senior differentiator: treating the ask size and the status format as things you design deliberately around the incentive gap, and revisiting that design as trust changes, rather than a fixed communication style you use with everyone.

Approach summary
Detecting/handling stack overflow on constrained MCUs combines hardware protection (MPU), software guards (canaries/watermarks), static analysis, and careful linker placement. Use layered defenses: prevent, detect early, and recover safely.

Hardware MPU guard regions

• What: Configure MPU region adjacent to stack as no-access so any overflow causes a memory fault/HardFault.
• Pros: Immediate trap, low runtime cost, reliable even in release builds.
• Cons: Uses MPU region(s) (limited), requires care with stack growth direction and alignment; ISR stacks and multiple tasks complicate mapping.
• Example: set a 4KB no-access region 4KB below the main stack top.

Software canaries & periodic watermark checks

• What: Write known pattern at stack base (canary) or fill whole stack with 0xAA and check high-water mark at runtime or on context switch.
• Pros: Low hardware requirements, gives usage metrics and early warning.
• Cons: Detection is reactive (after corruption), cost in cycles to check, can be bypassed if overflow doesn't overwrite canary (e.g., small corruptions).
• Example: check canary in exception entry; on RTOS context switch compute remaining filled bytes.

Compile-time stack-size analysis

• What: Static analysis (call-graph, worst-case recursion) + linker reports to set stack sizes for tasks.
• Pros: Preventative; reduces runtime overhead.
• Cons: Hard to be precise with dynamic inputs, recursion, or compiler optimizations; requires conservative margins.

Linker script techniques for stack placement

• What: Place stacks in guarded sections, align to MPU boundaries, reserve small “sacrificial” gap, or place stacks in separate memory regions.
• Pros: Deterministic memory layout, simplifies MPU mapping.
• Cons: Less flexible, wastes RAM if oversized; must coordinate with startup code.

Designing safe recovery

• Minimal, deterministic handler: in fault handler disable interrupts, log minimal context (LR, PC, stack pointer) to a reserved NVM area or RAM (with checksum).
• Attempt controlled shutdown: notify peers, set outputs to safe state, kick watchdog to guarantee reset if recovery hangs.
• Avoid complex operations in handler (no malloc, no blocking I/O). If possible, perform a graceful restart of the failed task or whole system depending on safety needs.
• For safety-critical systems, prefer immediate reset after safe shutdown and use post-mortem logs to improve stack sizing.

Recommended strategy

• Use MPU guard for immediate trapping + periodic watermark checks for monitoring + static analysis/linker placement to size stacks correctly. On detection, run minimal handler that preserves diagnostics, sets safe outputs, and triggers deterministic restart.

Direct answer. Placement new (new (ptr) T(args)) runs a constructor on storage you already own, without the normal operator new allocating anything. You must get the storage's alignment right (at least alignof(T), which alignas on the buffer guarantees), you own the object's lifetime entirely yourself ("lifetime" here means the span of time during which the bytes are validly interpreted as a live T; before the constructor finishes or after the destructor runs, the bytes are just bytes), and you end it by calling the destructor explicitly (obj->~T()), never delete, because delete would also try to deallocate storage placement new never allocated.

How it works. Ordinary new T(args) does two things: calls operator new to get raw storage, then runs T's constructor on it. Placement new splits those apart: void* operator new(size_t, void* p) { return p; } is a standard overload that just returns the pointer you hand it, so new (buf) T(args) allocates nothing and runs the constructor directly into buf. This is the mechanism behind every fixed-size slab (a pre-carved block of memory divided into same-size slots), object pool (a set of reusable objects constructed and destroyed in place instead of going back to the heap), or std::vector's internal growth strategy, and behind an arena (one big block of memory handed out in pieces, used here as the name for the buffer in the example below): get raw bytes once, construct and destroy objects into the same bytes repeatedly without going back to the heap allocator.

Alignment. The storage must satisfy alignof(T), not just have enough bytes. alignas(T) unsigned char buf[sizeof(T)] (or alignas(16) for a type with a hardware alignment need, such as one built around SIMD vector members, the wide registers some CPUs use to operate on several numbers in one instruction, which require 16-byte-aligned data) tells the compiler to place buf itself on a boundary that satisfies every member's requirement; skipping this and using a plain char[] is undefined behavior (UB: the standard gives no guarantee about what the program does) the moment the constructor writes an aligned member, and on some architectures it faults immediately rather than just being slow.

Teardown. Because no operator new ran, no delete is correct: delete obj would call operator delete(void*), which is allowed to assume the pointer came from the matching operator new and will misbehave (and definitely corrupts the allocator's bookkeeping if it tries to free stack or arena memory). The only correct teardown is calling the destructor directly: obj->~T();. After that call, the object's lifetime has ended, but the storage is still valid and can be reused for a new placement-new call: the pointer you get back from the next placement new is the one to use. Between the destructor call and the new construction the old pointer is dangling and must not be used. Once a new object of the same type has been constructed in exactly that storage (and the old object was not a const complete object), the C++ standard's "transparently replaceable" rule ([basic.life]: the new object occupies exactly the old storage and has the same type) lets the old pointer refer to the new object, but relying on that is fragile: if the type changes, or the object has a const complete-object type, the old pointer is invalid and you must use the pointer returned by placement new or pass the old one through std::launder (a function that returns a pointer you can use for the object now living at that address).

Worked example (built and run, reusing the buffer twice).

#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <new>

struct Packet {
    alignas(16) float v[4];
    int id;
    explicit Packet(int i) : v{1, 2, 3, 4}, id(i) {}
    ~Packet() {}
};
alignas(Packet) static unsigned char arena[sizeof(Packet) * 2];

int main() {
    printf("sizeof=%zu alignof=%zu arena%%alignof=%zu\n",
           sizeof(Packet), alignof(Packet), (size_t)((uintptr_t)arena % alignof(Packet)));
    printf("offsetof(Packet, id)=%zu\n", offsetof(Packet, id));

    Packet *a = new (arena) Packet(1);   // construct #1 into arena
    a->~Packet();                         // destroy #1; storage still there, 'a' now dangling

    Packet *b = new (arena) Packet(2);   // construct #2, same bytes
    printf("b->id=%d at arena start: %s\n", b->id, (void*)b == (void*)arena ? "yes" : "no");
    // b, not a, is the valid pointer now; same address, different object
    b->~Packet();

    // Two live objects in one buffer, side by side:
    Packet *p0 = new (arena) Packet(10);
    Packet *p1 = new (arena + sizeof(Packet)) Packet(11);  // next aligned slot
    printf("offset of p1 = %zu\n", (size_t)((char*)p1 - (char*)p0));
    p1->~Packet();  p0->~Packet();                          // destroy in reverse construction order
}

Compiled with g++ -std=c++17 -O2 -fsanitize=address,undefined and run (GCC 14.4, Linux container, aarch64), the program prints sizeof=32 alignof=16 arena%alignof=0, then offsetof(Packet, id)=16, then b->id=2 at arena start: yes, then offset of p1 = 32. The 32 is not arbitrary: v is 4 floats at 4 bytes each = 16 bytes, id is a 4-byte int, giving 20 bytes of actual data, but alignas(16) on v forces the whole struct's alignment to 16, and C++ requires a type's size to be a whole multiple of its own alignment (so that an array of the type keeps every element aligned); the next multiple of 16 at or above 20 is 32, so the compiler pads the struct out to 32 bytes (confirmed by the printed offsetof(Packet, id)=16: id sits right after the 16-byte v, and the remaining 12 bytes from 20 to 32 are pure padding). arena%alignof=0 confirms the arena itself starts on a 16-byte boundary; a and b are constructed at the identical address (b->id=2 at arena start: yes); and the two-object case places the second Packet exactly sizeof(Packet) = 32 bytes after the first (offset of p1 = 32), with no sanitizer complaint. That silence is a useful check, not a proof: UBSan would flag a misaligned construction and ASan would flag a delete of the arena (running the example with delete b; in place of b->~Packet(); makes ASan abort with an attempt to free an address that was not malloc()-ed), but neither tool checks object lifetimes or that every constructed object was destroyed, so the reuse and teardown order are correct because of the rules above, not because the run was quiet.

Trade-offs and pitfalls. The two live mistakes are using delete on a placement-new object (corrupts allocator state) and continuing to use the old pointer after calling its destructor and placement-newing a fresh object into the same bytes (between the destructor and the new construction, any use of the old pointer is undefined; after a same-type reconstruction it happens to be valid again under the transparent-replacement rule, but code that holds many such pointers is easy to get wrong, so keep only the pointer placement new returned). If a type throws from its constructor, placement new propagates the exception and the storage is left uninitialized: you must not call the destructor on storage whose constructor never completed. On this exact pattern, prefer wrapping it in a small RAII type (a class whose constructor acquires a resource and whose destructor releases it automatically, so the compiler guarantees cleanup) or std::optional (a standard type that holds either a value or nothing, and tracks which) that tracks whether an object is currently live, rather than relying on every call site remembering to destroy before re-constructing; the manual version above is correct but has zero compile-time protection against double-construct or double-destroy.

Direct answer

A deadlock is a state where a set of threads (or processes) can never make progress because each one is waiting for a resource that another thread in the set holds. Nothing is broken in any single thread; the combination is stuck forever. Deadlock needs four conditions to hold at the same time (the Coffman conditions), so removing any one prevents it. In practice I break circular wait by giving every lock a fixed global order and always acquiring in that order, because it costs almost nothing at run time and is easy to enforce in code review.

The four conditions

All four must hold together. Deadlock is therefore prevented by making at least one of them impossible.

Which one to break, and how

Example of the ordering rule: the lock table says account_lock ranks below audit_lock. Any code that needs both takes account_lock first. Two threads can then never form the T1-holds-A-wants-B, T2-holds-B-wants-A cycle, because no thread ever holds the higher-ranked lock while asking for the lower one. For an appliance with several locks (network, storage, config), write the order down as a numbered list next to the lock declarations.

The four strategies compared

What a production system actually uses is a combination: a lock-ordering rule enforced by review and tooling (the Linux kernel's lockdep validator and ThreadSanitizer both learn lock-acquisition orders and report a potential cycle even if the deadlock never fired in that run), timeouts on lock waits that log which lock was held, and detection plus retry where a database is involved.

Worked example: Banker's algorithm

Three resource types (A, B, C), five processes, free now = (3, 3, 2). Read A, B and C as three kinds of resource that come in several identical units, for example A = database connections, B = file handles, C = memory buffers; the textbook totals are 10, 5 and 7 units, so (3, 3, 2) is what is not currently held. P0 to P4 are five tasks that each declared the most of each kind they might ever hold at once (Max) and currently hold some (Allocation), so P1's row [2,0,0] means it holds two connections and no handles or buffers. Need = Max - Allocation. The safety check repeatedly looks for a process whose need fits in the free pool, lets it finish and return everything it holds, and repeats. If all processes can finish, the state is safe.

# Banker's algorithm safety check: 5 processes, 3 resource types (A, B, C).
available = [3, 3, 2]
allocation = {"P0": [0, 1, 0], "P1": [2, 0, 0], "P2": [3, 0, 2], "P3": [2, 1, 1], "P4": [0, 0, 2]}
maximum    = {"P0": [7, 5, 3], "P1": [3, 2, 2], "P2": [9, 0, 2], "P3": [2, 2, 2], "P4": [4, 3, 3]}

def need(alloc, mx):
    return {p: [m - a for m, a in zip(mx[p], alloc[p])] for p in mx}

def safe_sequence(avail, alloc, mx):
    work, finished, order = list(avail), set(), []
    nd = need(alloc, mx)
    progress = True
    while progress:
        progress = False
        for p in alloc:
            if p not in finished and all(n <= w for n, w in zip(nd[p], work)):
                work = [w + a for w, a in zip(work, alloc[p])]   # p finishes, returns everything
                finished.add(p); order.append(p); progress = True
    return order if len(finished) == len(alloc) else None

def request(p, req):
    nd = need(allocation, maximum)
    if any(r > n for r, n in zip(req, nd[p])): return "error: exceeds declared maximum"
    if any(r > a for r, a in zip(req, available)): return "wait: not enough free now"
    avail2 = [a - r for a, r in zip(available, req)]
    alloc2 = {q: list(v) for q, v in allocation.items()}
    alloc2[p] = [a + r for a, r in zip(alloc2[p], req)]
    seq = safe_sequence(avail2, alloc2, maximum)
    return ("grant, still safe, e.g. " + "->".join(seq)) if seq else "deny: would leave an unsafe state"

print("need:", need(allocation, maximum))
print("safe sequence now:", "->".join(safe_sequence(available, allocation, maximum)))
print("P1 requests [1,0,2]:", request("P1", [1, 0, 2]))
print("P4 requests [3,3,0]:", request("P4", [3, 3, 0]))

Reading the code: need subtracts what each process holds from its declared maximum. safe_sequence keeps a pool work of free units; each pass over the processes finds one not yet finished whose whole remaining need fits in work, pretends it runs to completion and returns all it holds to the pool, and records it in order. If no unfinished process fits, it stops, and the state is unsafe when some never finished. request first rejects a request above the declared maximum or above what is free, then tentatively grants it (avail2, alloc2) and runs the same safety check on that tentative state.

Output of that exact program:

need: {'P0': [7, 4, 3], 'P1': [1, 2, 2], 'P2': [6, 0, 0], 'P3': [0, 1, 1], 'P4': [4, 3, 1]}
safe sequence now: P1->P3->P4->P0->P2
P1 requests [1,0,2]: grant, still safe, e.g. P1->P3->P4->P0->P2
P4 requests [3,3,0]: deny: would leave an unsafe state

By hand, the pool grows as each process finishes: (3,3,2) plus P1's allocation (2,0,0) gives (5,3,2); plus P3's (2,1,1) gives (7,4,3); plus P4's (0,0,2) gives (7,4,5); plus P0's (0,1,0) gives (7,5,5); plus P2's (3,0,2) gives (10,5,7), which equals the total the system started with. P4's request for (3,3,0) would leave only (0,0,2) free, and no process's remaining need fits in that, so the algorithm refuses it. An unsafe state is not yet a deadlock; avoidance refuses it because a deadlock becomes possible.

Pitfalls

• Lock ordering fails silently when the second lock is passed in from a callback or chosen at run time. Rank by a stable key and check it in debug builds.
• Calling unknown code (a callback, a virtual method) while holding a lock invites an ordering violation you cannot see from the call site.
• A timeout that merely returns an error and retries immediately is a livelock generator. Release what you hold, then back off with randomness.

Approach: freeze the behaviour first, then move the work, then prove nothing changed. Do the steps in this order; each one has a check. Steps 1 to 4 are the plan; step 5 is the evidence that the plan did not change behaviour, and in step 5 the equivalence test and the overrun check are the checks every refactor needs, while the on-target latency, stress and stack checks apply as the product's deadlines require.

Step 1. Measure what each ISR costs today. Without numbers you cannot show an improvement or find a regression. For every ISR: toggle a spare GPIO pin high at entry and low at exit and read the pulse width on a logic analyser (an instrument that records digital pin levels over time) or oscilloscope, under realistic load. Also record the worst-case stack depth (fill the stack with a known pattern and, after a stress run, find how far the pattern has been overwritten: that high-water mark is the deepest the stack ever went) and the interrupt latency of the most urgent interrupt, because a long ISR delays everything at lower priority. These become the baseline.

Step 2. Capture the behaviour as a golden test (characterization test). A characterization test records what the code does today, right or wrong, so you can detect any change. Log the real byte streams the ISRs receive (from the hardware or a logic-analyser capture) and the outputs they produce (responses, state changes, log lines). Keep the logs. Include the awkward cases: malformed input, back-to-back messages, a message split across interrupts.

Step 3. Separate the message logic from the ISR without changing it. Move the parsing, formatting and I/O into plain functions with no hardware or interrupt dependency (handle_byte() below). Keep the legacy ISR calling them. Run the golden test on the host and on the target: output must be identical. Nothing has been restructured yet, so a failure here is a refactoring mistake.

Step 4. Restructure so the ISR only captures. The ISR does the minimum that cannot wait: clear the source, read the data register, push the byte into a ring buffer (a fixed-size circular queue) and note an overflow if it is full. The main loop or an RTOS task pops the bytes and calls the same handle_byte(). The handoff by situation:

• Bare metal: a ring buffer (a fixed array used as a circular queue; the ISR advances only the write index head and the main loop only the read index tail, so each index has a single writer and needs no lock) plus a flag the main loop polls, or a set of event bits (one bit per kind of event in a shared variable that the ISR sets and the main loop tests and clears).
• RTOS: a queue (messages copied in and out in order), a stream buffer (a byte-oriented FreeRTOS channel for one writer and one reader), or a task notification (a lightweight per-task flag that wakes the task) as the wake-up, using the ...FromISR calls from an interrupt whose priority is at or below the kernel's limit.
• Replace any printf or blocking I/O in the old ISR with an enqueue to a logging task or a lock-protected log buffer (a buffer in the main context that is guarded by a mutex so only one task writes to it at a time).

Step 5. Verify equivalence and the new failure modes. The program below is a host model: the "ISR" and the main loop are plain functions called from a driver, and the legacy and restructured paths share the same handle_byte().

#include <stdio.h>
#include <string.h>
#include <stdint.h>

/* ---- shared message logic: one implementation used before and after ---- */
static char line[32]; static unsigned len;
static int setpoint = 0;
static char out[4096]; static unsigned outlen;          /* stands in for the UART TX */

static void emit(const char *s) { size_t n = strlen(s); memcpy(out + outlen, s, n); outlen += n; }

static void handle_byte(uint8_t b) {                     /* parse + format + "I/O" */
  if (b != '\n') { if (len < sizeof line - 1) line[len++] = (char)b; return; }
  line[len] = 0; len = 0;
  char reply[48];
  int v;
  if (sscanf(line, "SET %d", &v) == 1) { setpoint = v; snprintf(reply, sizeof reply, "OK %d\n", setpoint); }
  else if (strcmp(line, "GET") == 0)   { snprintf(reply, sizeof reply, "VAL %d\n", setpoint); }
  else                                 { snprintf(reply, sizeof reply, "ERR %s\n", line); }
  emit(reply);
}

/* ---- legacy: the ISR does everything ---- */
static void legacy_isr(uint8_t rx) { handle_byte(rx); }

/* ---- restructured: ISR captures, main loop processes ---- */
#define RING 64u
static uint8_t ring[RING]; static unsigned head, tail, overruns;
static void new_isr(uint8_t rx) {
  if (head - tail == RING) { overruns++; return; }
  ring[head++ % RING] = rx;
}
static void main_loop_step(void) { while (head != tail) handle_byte(ring[tail++ % RING]); }

static void reset(void) { len = 0; setpoint = 0; outlen = 0; head = tail = overruns = 0; memset(out, 0, sizeof out); }

int main(void) {
  const char *stream = "SET 5\nGET\nSET 12\nBOGUS\nGET\nSET -3\nGET\n";
  size_t n = strlen(stream);

  reset();
  for (size_t i = 0; i < n; i++) legacy_isr((uint8_t)stream[i]);
  char golden[4096]; strcpy(golden, out);

  int all_same = 1;
  for (unsigned burst = 1; burst <= 40; burst++) {      /* main loop runs after every `burst` bytes */
    reset();
    for (size_t i = 0; i < n; i++) { new_isr((uint8_t)stream[i]); if ((i + 1) % burst == 0) main_loop_step(); }
    main_loop_step();
    if (strcmp(out, golden) != 0 || overruns != 0) { all_same = 0; printf("burst %u differs (overruns=%u)\n", burst, overruns); }
  }
  printf("golden output:\n%s", golden);
  printf("restructured output identical for drain intervals 1..40 bytes, no overruns: %s\n", all_same ? "yes" : "NO");

  /* starve the main loop: only drain at the very end of a stream longer than the ring */
  reset();
  for (int rep = 0; rep < 10; rep++) for (size_t i = 0; i < n; i++) new_isr((uint8_t)stream[i]);
  printf("starved main loop, %zu bytes sent into a %u-byte ring: overruns=%u (counted, not silent)\n", 10 * n, RING, overruns);
  return 0;
}

How the program works: handle_byte accumulates characters until a newline, then parses the line (SET n, GET, or anything else is an error) and appends the reply to out, which stands in for the UART transmitter. legacy_isr calls it directly, so its output is the golden reference. In the restructured path, new_isr only stores the byte: head and tail are free-running counters (they only ever increase), so head - tail is the number of bytes waiting, head - tail == RING means the ring is full (the byte is dropped and overruns is incremented), and % RING maps a counter to a slot, wrapping at 64. main_loop_step is the main loop's drain: while head != tail it takes the oldest byte and calls the same handle_byte. In main, the burst loop varies how many bytes arrive before the main loop gets a turn (1 byte, 2, ... 40), so the equivalence is checked against every drain rhythm; the last section never drains until the end, to force an overrun. Compiled with gcc -O1 -Wall -Wextra -fsanitize=address,undefined (AddressSanitizer reports out-of-range memory accesses and use of freed memory; UndefinedBehaviorSanitizer reports operations the C standard leaves undefined, such as signed overflow) in a gcc:14 container, it prints:

golden output:
OK 5
VAL 5
OK 12
ERR BOGUS
VAL 12
OK -3
VAL -3
restructured output identical for drain intervals 1..40 bytes, no overruns: yes
starved main loop, 380 bytes sent into a 64-byte ring: overruns=316 (counted, not silent)

The first result says the restructured path produces the same replies as the legacy path however the bytes are batched between main-loop passes. The second shows the new failure mode: if the main loop starves (is kept from running), the ring overruns, and the ISR counts it (380 bytes sent, 64 fit, 316 counted) rather than losing data silently. The model runs single-threaded, so it cannot show races between a real ISR and main; those are checked on the target (below).

On the target:

• Replay the captured byte streams through the real ISR path (a loopback UART, which wires the transmitter back to the receiver so the board hears its own output, or a test harness that pends the interrupt and feeds the data register) and diff the outputs against the golden logs.
• Re-measure the GPIO pulse: the ISR width should have dropped from the legacy figure to the capture-and-enqueue cost, and the latency of the most urgent interrupt should be no worse than the baseline.
• Stress: send bursts at the maximum line rate while the main loop is busy with its slowest task. Confirm the overrun counter stays at zero, or size the ring and decide how overrun is handled.
• Check the new end-to-end latency: the reply is now produced later than before (ISR, then the main loop's next pass). If the protocol has a response deadline, measure it from the last byte received to the first reply byte transmitted, and compare it with the deadline.
• Run the stack high-water check again.

Sequence the rollout. Convert one ISR at a time, each in its own commit with its golden test, so a regression is bisectable (a failing build can be traced to one change by testing the history in halves). Keep the legacy path behind a build switch (a compile-time option such as #ifdef LEGACY_ISR that selects the old or new code) until the field-style tests have passed.

What would change the plan. If a protocol needs a reply within microseconds (a bus where the slave must answer inside a fixed turnaround), the reply must stay in the ISR or in a DMA/hardware path, and only the non-urgent work moves out.

Clarify requirements & constraints

• Constrained MCU (KB–MB flash, limited RAM), intermittent power, limited bandwidth, need auth, anti-rollback, integrity, resume, delta support. Secure boot/root key in immutable ROM.

High-level design

• Dual-bank A/B layout with bootloader in ROM/immutable flash. Bootloader enforces verification and atomic switch.
• Root key: Ed25519 public key stored in ROM (or fused). Use signatures for authenticity; use HKDF + AES-GCM for optional encrypted payloads.

Cryptographic primitives

• Signature: Ed25519 (small, fast, safe against misuse).
• Integrity/MAC for streamed ops: SHA-256 for hashes; AES-128-GCM for authenticated encryption (if confidentiality needed).
• KDF: HKDF-SHA256 for deriving ephemeral keys for delta patches.

Image metadata

• image_length
• image_version (monotonic increment)
• rollback_counter (monotonic)
• base_hash (SHA-256 of base image if delta)
• image_hash (SHA-256 of full image or post-patch)
• signature (Ed25519 over metadata || image_hash)
• flags: delta/applied/compressed
• compression type, patch algorithm id

Verification & update flow

1. Transfer image (or delta) to temporary staging region with progress checkpointing.
2. Verify signature: compute image_hash (for delta, verify base_hash matches running image) then verify Ed25519(signature, metadata || image_hash) using ROM public key.
3. Check rollback: require image_version > stored_monotonic_version (stored in secure monotonic flash or anti-rollback counter area). Only accept greater.
4. Integrity: verify SHA-256 matches.
5. Apply update:
   • For full image: atomically write to inactive bank; verify write by re-hashing.
   • For delta: verify base_hash equals active bank hash; apply patch to inactive bank while streaming and computing SHA-256; if patch fails, abort.
6. Commit: update monotonic_version atomically (use small atomic write or hardware monotonic counter), set boot flag to boot new bank, trigger watchdog reset.
7. Bootloader validates new bank at next boot; if validation fails or boot doesn't complete within watchdog timeout, auto-rollback to previous bank.

Rollback protection

• Monotonic counter stored in write-once / monotonic storage if available, or in redundant, wear-leveled flash with anti-tamper logic; require image_version > stored value before commit. Only increment after successful full verification to avoid fork.

Interrupted updates & recovery

• Chunked transfer with checkpointed offsets; keep both banks intact until new bank verified.
• If interruption, resume transfer using checkpoint; if patching interrupted, discard incomplete inactive bank and resume from last validated chunk.
• Watchdog + bootloader timeout ensures failed boots revert to known-good bank.

Delta updates safety

• Use content-addressed deltas: patch signed server-side producing patch_blob plus base_hash and resulting image_hash.
• Before applying, verify active image hash == base_hash. Reject if mismatch.
• Apply patch in streaming manner to inactive bank, computing hash on the fly. Use checksums per chunk to detect mid-patch corruption.
• Optionally use Merkle-tree or chunked hashes to allow partial verification and resume of patching, minimizing retransfer.

Bandwidth minimization

• Compress payload (zstd/DEFLATE), send binary delta (bsdiff or custom block-diff), use chunked transfers with HTTP range or CoAP block-wise transfers.

Practical considerations

• Keep bootloader minimal and well-audited.
• Test failure modes, power-loss during write, and ensure flash wear leveling.
• Log update attempts and failures to a persistent slot for diagnostics.

This design balances security (Ed25519, SHA-256, monotonic version), reliability (A/B + watchdog), and bandwidth (signed deltas + compression) suitable for constrained embedded devices.

Direct answer

I start from the assumption that the thing I pushed for is the prime suspect, not a bystander, and I root-cause it the same way I would anyone else's mistake, quickly and without defensiveness, because the instinct to protect a decision I championed is exactly the bias most likely to slow down finding the real cause. Once I have a real answer, I tell stakeholders before they find out on their own, and I choose between reconfiguring, slowing the rollout, and reversing based on evidence, not on how much I want to have been right.

Structured elaboration

• Treat the adoption decision itself as a hypothesis to test, not a given: actively look for evidence it caused the regression rather than looking for reasons it is innocent.
• Separate the possible root causes explicitly: was the tool's actual behavior different from what I understood, incomplete learning on my part, was an assumption about how it would behave simply wrong, or was the integration into the existing system the real problem, since the fix and the lesson differ for each.
• Get ahead of the disclosure: tell the people affected what is known as soon as it is known, rather than letting them discover the regression and connect it back to the decision themselves.
• Choose the remediation on evidence: reconfigure if the root cause is a fixable misconfiguration, slow or pause the rollout if the evidence is still incomplete, reverse if the tool genuinely cannot do what was needed.
• Afterward, name what evidence, available before the decision, would have caught this earlier, so the gap closes in the decision process itself, not just in this one instance.

Worked example

I had championed adopting a new caching layer to reduce database load, argued for it, and it was rolled out. Within a couple of weeks, a subset of reads started returning stale data intermittently, something nobody had seen before the change. I assumed from the start that the cache was the prime suspect rather than looking elsewhere first, and worked backward: I confirmed the caching layer's default invalidation behavior did not cover one specific write path used by a background job, meaning that path's writes were not invalidating the cache the way I had assumed. That was incomplete learning about the tool's actual invalidation model, not a wrong assumption about the business logic and not a pure integration bug. I told the team and the affected downstream stakeholder the same day the root cause was confirmed, including that it had been happening for roughly two weeks before it was caught, rather than only mentioning it once a fix was ready. Since the specific gap was fixable, adding the missing invalidation call, I chose to reconfigure and add monitoring on cache staleness rather than reverting the whole adoption, since the underlying idea was sound and the miss was narrow and now understood. Afterward, the concrete change was adding a checklist item to any future tool-adoption proposal: name every write path the new component touches and confirm invalidation or consistency behavior for each one, since that is precisely the evidence that would have caught this before rollout.

Trade-offs and pitfalls

• Defending the decision instead of investigating it as a suspect is the single most common way a champion misses or delays finding their own root cause.
• Delaying disclosure until a fix is ready, instead of disclosing as soon as the cause is known, damages trust more than the original mistake does.
• Reverting reflexively, without separating whether the cause was incomplete learning, a wrong assumption, or an integration bug, can throw away a genuinely good decision over a narrow, fixable gap, or conversely keep a genuinely bad decision alive by treating it as fixable when it is not.