Systematic Debugging and Root Cause Analysis Questions

Direct answer

Capture the scene in a GPU profiler, such as RenderDoc, PIX, or Xcode GPU Frame Capture, that reports CPU submit time and GPU execution time separately. Compare each to the frame budget: whichever is close to or over budget while the other has slack is the bottleneck.

Structured elaboration

• At 60 fps the budget is $1000/60 \approx 16.67$ ms.
• High CPU submit time near budget suggests too many draw calls or expensive game logic; batching or instancing typically helps.
• High GPU time suggests overdraw (the same pixel getting shaded more than once in a single frame, e.g. from overlapping transparent UI elements or unculled geometry hidden behind other objects), expensive shaders, or too much resolution/post-processing cost; the profiler shows which pass dominates.
• Confirm by testing a lever: reducing draw distance should help a CPU-bound frame; reducing resolution should help a GPU-bound one.

Worked example

$16.67 - 4 = 12.67 \text{ ms of CPU slack, while the GPU takes 22 ms}$

GPU (22 ms) already exceeds the 16.67 ms budget while the CPU has slack, the signature of a GPU-bound frame: the CPU finishes early and waits on the GPU queue.

Trade-offs and pitfalls

Capture several frames, not one; thermal throttling and background OS work add noise. Fixing one bottleneck can shift it to the other side, so re-profile after each change.

What the interviewer probes next

Which GPU-side techniques, like occlusion culling or LOD, versus CPU-side ones, like batching or job-system parallelism, you'd reach for.

Direct answer

Work it as a cheap-to-expensive triage tree across all three suspects. Confirm it reproduces on more than one card from the vendor, check that vendor's driver version and changelog against your own shader compiler warnings, and only suspect the asset once both are ruled out.

Structured elaboration

• Reproducing on 2+ cards from the same vendor rules out one bad install; if it reproduces on only one physical card, stop here, it's that machine, not a driver or shader problem.
• Suspect a driver bug when the vendor's shader compiler reports zero warnings, the same shader binary runs correctly on every other vendor, and the vendor's release notes or public bug tracker list a known erratum for that driver version or GPU generation; confirm by testing an older or newer driver build and seeing whether the artifact follows the driver, not the hardware.
• Suspect a shader bug when that vendor's shader compile log itself flags undefined behavior, like implicit truncation or exceeding a texture-unit limit another vendor allows silently; this is a bug in your code, not theirs, even though it only shows up on one vendor.
• If shaders compile clean everywhere and the driver is current with no known erratum, suspect the texture's compression format; an unsupported format on that GPU generation often fails silently to black or garbage rather than erroring. Swap the suspect texture for an uncompressed placeholder; if the artifact disappears, it's the asset, not the shader or driver.

Worked example

A shader relying on undefined behavior at a boundary value, a texture coordinate exactly at 1.0, can be handled differently by two vendors' compilers purely because the spec leaves that case implementation-defined, that's a shader bug. Contrast that with the same clean-compiling shader producing garbled output only on driver version 456.71 and correctly on both 456.55 and 457.09 from the same vendor, that pattern points at the driver itself, not your code or the asset.

Trade-offs and pitfalls

Confirm you're not relying on undefined behavior in your own code before writing a vendor branch or filing a driver bug report; fixing the assumption is more durable. Silently-failing formats are dangerous because nothing errors in testing, and blaming the driver first without checking your own compiler warnings wastes the vendor's and your own time.

What the interviewer probes next

How you'd set up automated cross-vendor rendering tests to catch this before release, and how you'd responsibly report a confirmed driver bug back to the vendor.

Direct answer

A single-frame glitch that worsens under load smells like a race between the sim and render threads. Most engines double-buffer so the renderer always reads a stable snapshot, and a missing or misplaced sync point lets it read a buffer mid-write.

Structured elaboration

• Check the sim-to-render handoff: sim writes buffer A while render reads buffer B, then they swap; an unfenced swap lets render start reading A before sim finishes writing it.
• A telltale symptom is a "torn" read: position updates correctly but rotation, written moments later in the same update, still shows the old value for one frame.
• Widen the race by adding artificial CPU load, since "worse under load" points at timing, not deterministic logic.
• Fix with a genuine synchronization point, an atomic buffer-pointer swap, so render never starts before sim fully publishes its writes.

Worked example

$\frac{1000}{60} \approx 16.67 \text{ ms per frame}$

An unfenced swap can let render read buffer A microseconds before sim finishes writing it, producing exactly one frame of torn state before the next swap self-corrects it.

Trade-offs and pitfalls

Locking every field eliminates the race but can stall the render thread, reintroducing frame-time problems; prefer a lock-free atomic swap. Verify the fix with the same artificial-load test, not just its apparent disappearance.

What the interviewer probes next

How you'd design the swap to avoid stalling either thread, and how you'd detect a torn read automatically.

Debugging a multithreaded C++ crash starts with reproducing under a sanitizer, since a plain crash trace from a race often just shows whichever thread happened to fault, not the real conflicting access.

Verified example: build and run under ThreadSanitizer

#include <pthread.h>
#include <stdio.h>
int count = 0;
void* t(void*_) { for(int i=0;i<1000000;i++) count++; return NULL; }
int main(){ pthread_t a,b; pthread_create(&a,NULL,t,NULL); pthread_create(&b,NULL,t,NULL);
  pthread_join(a,NULL); pthread_join(b,NULL); printf("%d\n", count); }

Build with gcc -fsanitize=thread -g -o race race.c && ./race: TSan reports a data race on count with both thread backtraces, and the program's printed total is reliably less than 2000000 because count++ is a non-atomic read-modify-write. Fix with an atomic or a mutex:

#include <stdatomic.h>
atomic_int count = 0;
void* t(void*_) { for(int i=0;i<1000000;i++) atomic_fetch_add(&count, 1); return NULL; }

Core dumps, breakpoints, thread backtraces

For a crash rather than a silent wrong-answer race: enable core dumps, attach gdb/lldb to a live hung/crashing process (or load the core), and use thread apply all bt to get every thread's backtrace at once, since the crashing thread is often a victim of another thread's earlier bad write, not the culprit itself. Watchpoints (watch <var>) catch the exact instruction that last modified a corrupted value. Helgrind (valgrind's race detector) is a slower but useful second opinion when TSan isn't available for the toolchain in use.

Trade-offs and pitfalls

Sanitizers add real overhead (roughly 2-20x depending on tool), so they run in CI/staging on representative inputs, not as the default production build; a race that only appears under specific production concurrency may need a staging load test built specifically to reproduce that concurrency level before the sanitizer can catch it.

Direct answer

A crash only after hours, never in a short session, points at accumulation rather than a one-shot bug. On a fixed-budget console with a custom allocator, the leading suspect is fragmentation: total free memory looks fine, but no single block is big enough.

Structured elaboration

• Log the allocator's largest-contiguous-free-block size, not just total free bytes, at intervals during a soak run.
• Fragmentation comes from interleaving long-lived allocations (geometry, persistent state) with many short-lived, variable-sized ones (particles, audio buffers).
• Mitigate with dedicated pools sized per allocation lifetime, or periodic defragmentation at a safe pause point like a level load.
• Validate by rerunning the soak test and confirming the largest-block metric stays stable over hours.

Worked example

$40 \text{ MiB total free} > 10 \text{ MiB requested, yet } 3 \text{ MiB largest block} < 10 \text{ MiB requested}$

Total free memory alone would have looked fine; the largest-block metric is what actually explains the failed allocation.

Trade-offs and pitfalls

Dedicated pools trade flexibility for predictability. Defragmentation by moving live memory requires every pointer to be relocatable via handles, a bigger architectural commitment if not designed in from the start.

What the interviewer probes next

How you'd size pools up front, and what it takes to support safe relocation for defragmentation.