DoorDash Applied Scientist (Staff Level) - Comprehensive Interview Preparation Guide

Direct answer

Training a 100B+-parameter transformer to finish in 2-4 weeks on commodity GPU nodes requires combining three axes of parallelism (data, tensor, pipeline) with aggressive memory optimization (ZeRO/FSDP sharding, activation checkpointing, mixed precision), and treating fault tolerance and checkpointing as first-class requirements rather than an afterthought, since a multi-week job on hundreds to thousands of GPUs will experience hardware failures during the run.

Structured elaboration

• Memory budget first: a 100B-parameter model in bf16 needs ~200GB for weights alone; with Adam's fp32 master weights, momentum, and variance, full unsharded state is on the order of 1.6TB. This is why ZeRO/FSDP-style sharding of optimizer state (and often gradients and parameters too) across the data-parallel group is not optional at this scale.
• 3D parallelism layout: tensor-parallel within a node (NVLink bandwidth), pipeline-parallel across a modest number of node groups (tolerates lower cross-node bandwidth, at the cost of pipeline bubble), data-parallel to scale out the remaining GPU count, with ZeRO stage 1 or 2 layered on the data-parallel dimension for further memory reduction without adding as much communication as ZeRO-3/FSDP would.
• Throughput target math: reaching a 2-4 week finish time for a fixed token budget requires computing the required aggregate FLOPs/sec from the model's FLOPs-per-token and target token count, then sizing the GPU count and expected model-FLOPs-utilization (MFU, typically 30-50% of peak for well-tuned large-model training) to hit that throughput.
• Fault tolerance and checkpointing: on commodity (as opposed to purpose-built, fully redundant) hardware, expect failures during a multi-week run; frequent, cheap incremental checkpoints (sharded across the same topology the model is sharded across) and automated restart-from-checkpoint are required so a single node failure costs minutes, not days.
• Activation checkpointing: recomputing activations during the backward pass instead of storing all of them trades extra compute (roughly 20-30% more FLOPs) for a large reduction in activation memory, which is usually the right trade at this model scale since memory, not compute, is the binding constraint.

Worked example

Target: 100B parameters, 300B training tokens, transformer FLOPs-per-token approximation of 6 x parameters (forward + backward). Total FLOPs $\approx 6 \times 100\times10^9 \times 300\times10^9 = 1.8\times10^{23}$ FLOPs. An A100 delivers roughly 312 TFLOPS of bf16 peak; at 40% MFU that's $\approx 1.25\times10^{14}$ effective FLOPS per GPU. To finish in 21 days (1.8144 x 10^6 seconds): required aggregate FLOPS = $1.8\times10^{23} / 1.8144\times10^6 \approx 9.9\times10^{16}$ FLOPS, requiring roughly $9.9\times10^{16} / 1.25\times10^{14} \approx 793$ GPUs at that utilization, so a cluster in the 800-1000 GPU range with headroom for failures and imperfect scaling efficiency.

Network topology, storage tiering, orchestration, and monitoring

• Network topology: within a node, the 8x A100s connect via NVLink/NVSwitch for tensor-parallel traffic; across nodes, a rail-optimized InfiniBand (or RoCE) fat-tree fabric with GPUDirect RDMA is standard so pipeline- and data-parallel collectives bypass the CPU and avoid becoming the bottleneck at 800-1000 GPU scale, since a flat, oversubscribed Ethernet fabric would leave cross-node all-reduce badly bandwidth-starved.
• Storage tiering: training data and checkpoints don't fit (or shouldn't live) on one tier. Hot, frequently-read shuffled shards of the token dataset live on local NVMe per node for the fastest, lowest-latency reads during data loading; a shared parallel filesystem (e.g. Lustre/GPFS) or high-throughput object store holds the full dataset and is the durable target for periodic sharded checkpoints; and cold, long-term artifact storage (final checkpoints, logs) goes to cheaper object storage (S3-class) once a run's active window has passed.
• Orchestration: a cluster scheduler (Kubernetes with a GPU-aware scheduler, or Slurm, common in HPC-style GPU clusters) handles job placement, GPU allocation, and automatic restart-from-checkpoint on node failure; the orchestration layer needs to cooperate with the training framework's own elastic/fault-tolerant world-size handling so a restart after a node failure resumes at the correct sharded checkpoint rather than restarting the whole job from scratch.
• Monitoring: production runs at this scale track GPU utilization and model-FLOPs-utilization (MFU) per node to catch degraded or straggling hardware early, loss/gradient-norm curves to catch training instability, and per-node health checks (ECC errors, thermal throttling, NVLink/IB link errors) so a failing GPU is drained before it corrupts a step rather than discovered only after a collective hangs.

Trade-offs & pitfalls

The single most common planning mistake at this scale is sizing the cluster off peak FLOPS instead of realistic MFU; a well-tuned large-model training run typically only achieves 30-50% of theoretical peak once communication overhead, pipeline bubbles, and memory-bound operations are accounted for, so plans that assume near-100% utilization consistently under-provision.

Direct answer

Batch inference requests onto a GPU by accumulating events until either a batch-size threshold or a maximum wait time is reached, whichever comes first, so you get GPU-efficient batch sizes without letting any single event wait indefinitely during a quiet period.

Structured elaboration

GPU inference is generally far more efficient per-item at larger batch sizes (better hardware utilization), which pulls toward waiting to accumulate a big batch. But a pure size-based trigger means, during low-traffic periods, an event could wait a very long time for the batch to fill, which is unacceptable if there's a latency SLA (service-level agreement) on the prediction. The standard resolution is a dual trigger: flush whichever of "batch size N reached" or "time window T elapsed since the oldest buffered item" happens first, bounding worst-case added latency to T regardless of traffic volume.

Worked example

For a scoring pipeline needing predictions within roughly 200 milliseconds end to end, you might set batch size to 64 and the time window to 50 milliseconds: during high traffic, batches fill to 64 quickly and inference runs at full GPU efficiency; during a lull, even a half-full batch flushes after 50 milliseconds rather than waiting indefinitely, keeping worst-case added latency bounded and predictable regardless of how bursty the traffic is.

Trade-offs and pitfalls

A batch size set too small forfeits GPU efficiency gains (defeating the purpose of batching at all); a window set too long violates the latency SLA during quiet periods even if the size threshold is fine. The added latency this introduces is a genuine, board-line design cost that has to be weighed against the throughput and cost efficiency GPU batching buys you; a design that ignores the latency side of that trade will look fine in a load test and fail an actual production SLA the first time traffic is uneven.

Direct answer

Data leakage in production means a feature accidentally contains information from the future relative to when the prediction is actually made: catching it needs both automated checks built into the feature pipeline and a validation mindset that actively looks for suspiciously-good performance rather than accepting it.

Structured elaboration

1. Automated checks: for every feature, verify its computation timestamp is strictly BEFORE the prediction's target/label timestamp, enforced programmatically rather than trusted to careful coding: a feature-computation framework that requires an explicit "as-of" timestamp for every lookup, and rejects a query that would pull data from after that timestamp, prevents an entire class of leakage structurally rather than relying on someone remembering to check.
2. Metrics and heuristics that flag potential leakage: a single feature with unusually high individual predictive power (far higher than domain knowledge would suggest is plausible) is a classic leakage tell: if one feature alone nearly solves the prediction task, it's worth specifically investigating whether that feature could only be known AFTER the outcome. Similarly, a sharp, unexplained jump in offline evaluation metrics after adding a new feature deserves scrutiny before celebrating the improvement.
3. Validating true positives vs. false positives: a flagged high-predictive-power feature ISN'T automatically leakage: some features genuinely are strong, legitimate predictors. Validate by tracing the feature's actual computation logic and confirming its timestamp discipline, or by testing the model's performance with that feature computed using ONLY data genuinely available at prediction time in a realistic backtest: if performance collapses when the feature is computed honestly (as-of the real prediction time) versus computed with hindsight, that's confirmed leakage, not a coincidentally strong signal.

Worked example

A classic example this catches: a churn-prediction model where a feature is "number of customer-support tickets in the following 30 days": mistakenly computed using a lookback window that actually included data AFTER the prediction date due to an off-by-one error in the pipeline's date handling. This feature would show extremely high predictive power (customers who are about to churn often file more support tickets right before leaving) purely because it's leaking the future, and a backtest recomputing the SAME feature honestly (using only data genuinely available at prediction time) would show the feature's real, much weaker predictive power, revealing the leak.

Trade-offs & pitfalls

The instinct when a new feature dramatically improves offline metrics is to treat it as a genuine win: the discipline that catches leakage is treating an unusually large improvement as a signal worth SPECIFICALLY investigating before trusting it, rather than as unambiguous good news, since the single most common real-world leakage incidents are discovered exactly this way: a suspiciously good result that turned out too good to be true.

Situation & Task
I discovered a production-quality drop when a fraud-detection model's precision fell 18% two weeks after deployment. Business alerted because false positives spiked and operations saw increased manual reviews.

Investigation & Isolation (Action)

• Reproduced issue locally with a recent snapshot of production features.
• Ran feature-distribution checks and found a categorical feature’s cardinality had increased — a new payment-provider code appeared and was mapped to the default “other” bucket during featurization.
• Traced upstream: a partner changed their API response (new enum value) so our ETL parsed it as null. I confirmed via logs and raw message samples in S3 and by comparing schema evolution in the data catalog.

Remediation (Action)

• Short-term: rolled back model to the previous version and applied a hotfix in the featurizer to treat unseen enums explicitly, reducing misclassification immediately. Deployed a backfill job for the last 7 days to rebuild affected features and retrained the model with corrected data.
• Long-term controls: implemented schema validation with Great Expectations checks at ingestion, added an alert for categorical cardinality drift, and added a CI test that simulates unseen enums. Created a data contract with the partner and set up weekly contract-validation jobs.

Communication & Outcome (Result)

• Sent a concise incident summary and remediation plan to product, ops, and the partner within 2 hours, followed by a postmortem with timeline, root cause, and action items. Precision recovered to pre-incident levels; manual reviews dropped 40%. Lessons: enforce schema contracts, add automated checks, and include robustness for unseen categories in model inputs.

Situation & high-level approach
I split team capacity into three lanes: 60% delivery (short-term product work), 25% exploratory research, 15% infrastructure/ops and skill development. I adjust the split quarterly based on product needs.

Decision criteria

• Impact: expected improvement in core metric (CTR, revenue, latency).
• Risk & uncertainty: technical feasibility and data readiness.
• Time-to-value: how long to a usable prototype.
• Strategic learning: unique IP or long-term differentiation.

Estimating experiment ROI
I estimate ROI with a simple expected-value model:

Expected ROI = Probability(success) * Net benefit per unit time - Cost per unit time

Intuition: multiply likelihood of success by benefit (e.g., revenue uplift or latency reduction), subtract resource costs. I quantify benefits in product KPIs or engineering savings.

Aligning priorities with stakeholders

• Monthly reviews with PMs/engineers to map experiments to KPI owners.
• Define success criteria and fallback plans before kickoff.
• Use a lightweight RACI and show expected ROI and timeline to inform trade-offs.

Measuring progress & outcomes

• Short checkpoints: weekly status + quantitative metrics (validation loss, offline A/B estimates).
• Gate reviews: prototype -> pilot A/B -> full rollout with predefined metrics and significance thresholds.
• Post-mortem: document learnings, negative results, reproducible code and next steps.

Example: For a recommender improvement, I ran a 6-week probe (25% capacity), estimated 10% uplift with 40% success probability, ran an A/B pilot, and only promoted models that passed 95% CI on CTR and retention.

Direct answer

Pick based on two properties of the graph and one property of the query: if every edge has the same weight, breadth-first search (BFS) alone gives shortest paths in linear time; if weights differ but are never negative, Dijkstra's algorithm is the standard single-source choice; if a negative weight is possible (but no negative cycle), Dijkstra can give a wrong answer and Bellman-Ford is required instead; and if you need distances between every pair rather than from one source, Floyd-Warshall (a dynamic-programming algorithm that considers every node in turn as a possible shortcut between every pair) is the natural fit for all-pairs, or equivalently running Dijkstra from every node. A* only changes single-source, non-negative-weight search: it adds a heuristic estimate of remaining distance to prioritize expansion toward a specific goal, and it only helps, versus plain Dijkstra, when that heuristic is admissible (it never overestimates the true remaining cost), since an inadmissible heuristic can cause A* to return a path that is not actually shortest.

Structured elaboration

Decision order:

1. Are all edge weights equal (or is the graph unweighted)? Use BFS: $O(V + E)$, no priority queue needed at all.
2. Do you need distances between every pair of nodes, not just from one source? Use Floyd-Warshall, $O(V^3)$, or repeat Dijkstra from every source if the graph is sparse and there are no negative weights.
3. Otherwise, single-source with weights: are negative edge weights possible? If yes, use Bellman-Ford, $O(V \cdot E)$, which also detects a negative cycle if one exists (a cycle whose total weight is negative, which makes "shortest path" undefined, since you could loop it forever to keep decreasing the cost). If no, use Dijkstra, $O((V+E) \log V)$ with a binary heap.
4. If you additionally have a single, known goal node (not "distances to everywhere"), and a decent estimate of remaining distance, layer A* on top of Dijkstra's non-negative-weight assumption to reduce how much of the graph gets explored.

Why Dijkstra breaks under negative weights: once Dijkstra pops a node off its priority queue, it treats that node's distance as final and never revisits it, on the assumption that nothing already queued could possibly offer a shorter path, since all remaining edges only add non-negative weight. A negative edge violates that assumption directly: a longer-looking path discovered later can still turn out shorter once a negative edge is added to it.

What A's heuristic must guarantee*: admissibility, never overestimating the true remaining cost to the goal. Given an admissible heuristic, A* is guaranteed to still find a shortest path, exactly like Dijkstra, but it explores fewer nodes when the heuristic is informative, because it prioritizes nodes that look closer to the goal rather than merely closer to the start. (A stronger property, consistency, additionally guarantees a node is never re-expanded after being finalized, which is what lets A* implementations skip the "revisit and relax an already-closed node" bookkeeping that a merely admissible-but-inconsistent heuristic would otherwise require.) A straight-line (Euclidean) distance heuristic on a road network or grid is a classic admissible choice, since no real route can be shorter than the straight line.

Worked example

Bellman-Ford correctness under a negative edge, where Dijkstra gets it wrong:

import math

def dijkstra(adj, src):
    import heapq
    dist = {src: 0}
    finalized = set()
    pq = [(0, src)]
    while pq:
        d, u = heapq.heappop(pq)
        if u in finalized:
            continue
        finalized.add(u)
        for v, w in adj.get(u, []):
            if v in finalized:
                continue
            nd = d + w
            if nd < dist.get(v, math.inf):
                dist[v] = nd
                heapq.heappush(pq, (nd, v))
    return dist

def bellman_ford(edges, nodes, src):
    dist = {n: math.inf for n in nodes}
    dist[src] = 0
    for _ in range(len(nodes) - 1):
        for u, v, w in edges:
            if dist[u] + w < dist[v]:
                dist[v] = dist[u] + w
    for u, v, w in edges:
        if dist[u] + w < dist[v]:
            raise ValueError("negative cycle detected")
    return dist

# True shortest A->B is via C: 1 -> 4 + (-10) = -6, but Dijkstra finalizes
# B at distance 1 (direct edge) before C is even processed.
adj = {"A": [("B", 1), ("C", 4)], "C": [("B", -10)]}
edges = [("A", "B", 1), ("A", "C", 4), ("C", "B", -10)]
nodes = ["A", "B", "C"]

print("dijkstra:", dijkstra(adj, "A"))
print("bellman_ford:", bellman_ford(edges, nodes, "A"))

Running this prints:

dijkstra: {'A': 0, 'B': 1, 'C': 4}
bellman_ford: {'A': 0, 'B': -6, 'C': 4}

Dijkstra reports B at distance 1 (wrong: it finalized B via the direct edge before discovering the cheaper route through C), while Bellman-Ford correctly finds -6 via A to C to B.

A exploring fewer nodes than Dijkstra given an admissible heuristic*, on an open 20x20 grid from (0,0) to (5,5), using Manhattan distance as the heuristic:

import heapq

def neighbors(pos, size):
    x, y = pos
    for dx, dy in ((1, 0), (-1, 0), (0, 1), (0, -1)):
        nx, ny = x + dx, y + dy
        if 0 <= nx < size and 0 <= ny < size:
            yield (nx, ny)

def manhattan(a, b):
    return abs(a[0] - b[0]) + abs(a[1] - b[1])

def search(start, goal, size, use_heuristic):
    h = (lambda n: manhattan(n, goal)) if use_heuristic else (lambda n: 0)
    g = {start: 0}
    pq = [(h(start), start)]
    visited = set()
    expansions = 0
    while pq:
        _, node = heapq.heappop(pq)
        if node in visited:
            continue
        visited.add(node)
        expansions += 1
        if node == goal:
            break
        for nb in neighbors(node, size):
            ng = g[node] + 1
            if ng < g.get(nb, float("inf")):
                g[nb] = ng
                heapq.heappush(pq, (ng + h(nb), nb))
    return g[goal], expansions

size = 20
start, goal = (0, 0), (5, 5)
dist_astar, exp_astar = search(start, goal, size, use_heuristic=True)
dist_dij, exp_dij = search(start, goal, size, use_heuristic=False)
print("A* distance:", dist_astar, "nodes expanded:", exp_astar)
print("Dijkstra distance:", dist_dij, "nodes expanded:", exp_dij)

Running this prints:

A* distance: 10 nodes expanded: 36
Dijkstra distance: 10 nodes expanded: 61

Both find the same correct shortest distance (10), but A* reaches it having expanded 36 nodes against Dijkstra's 61, because the heuristic steered expansion toward the goal instead of outward in every direction equally.

Trade-offs & pitfalls

A common mistake is treating A* as "a different algorithm" from Dijkstra rather than as Dijkstra with a heuristic added to the priority; with a heuristic of zero everywhere (as in the comparison above), A* degenerates to exactly Dijkstra, which is a good way to check an A* implementation for bugs. A second pitfall is picking a heuristic that overestimates in some region "because it prunes more nodes": an inadmissible heuristic can make A* return a path that is not actually shortest, so any heuristic must be checked against the admissibility property, not just judged by how much it speeds things up. Practically, Dijkstra's own implementation constant matters too: with a binary heap, each edge relaxation that improves a distance costs $O(\log V)$ to sift, which under many decrease-key-style relaxations is the dominant cost; the common fix is not switching to a Fibonacci or pairing heap (both have amortized $O(1)$ or near-$O(1)$ decrease-key but carry higher constant factors and more complex implementations) but instead allowing duplicate, stale entries in a plain binary heap and lazily discarding them on pop, which is what the Dijkstra implementation above already does and is the standard practical choice.

Direct answer: Churn and segmentation feature design at scale needs to combine behavioral counts/rates/recency, cross-source interactions, and time-based signals computed from usage logs, support tickets, and demographics, with an explicit leakage check before trusting any offline lift, since churn labels are especially prone to accidentally encoding future information.

Structured elaboration: Eight concrete engineered features, with intuition and how each is computed at scale (roughly two million rows of mixed numeric, high-cardinality categorical, timestamp, and free-text data):

1. Days since last login/usage event (recency) - intuition: a lapsing user's engagement gap widens before they formally churn. Computed via a vectorized groupby-max on the timestamp column per customer, not a row-by-row loop, which is essential at two million rows.
2. Login/event frequency over trailing 7/30/90-day windows - intuition: a declining frequency trend is often more predictive than any single-point count. Computed via grouped, windowed aggregation (e.g. groupby(customer_id).rolling() or a windowed SQL aggregate).
3. Usage-intensity trend (slope of a simple linear fit or week-over-week ratio of event counts) - intuition: distinguishes "still active but declining" from "steady at a low level," which behave differently for churn.
4. Support ticket count and recency - intuition: a recent spike in support contact correlates with dissatisfaction, but must be computed carefully to avoid the leakage case below.
5. Support ticket sentiment or category distribution (from free-text ticket bodies, e.g. via a lightweight TF-IDF or keyword-tagging pass) - intuition: ticket TONE (frustrated vs. neutral) often separates at-risk from routine contacts even at similar ticket counts.
6. Plan/product tenure (time since signup or since last plan change) - intuition: churn risk is often U-shaped or front-loaded relative to tenure.
7. High-cardinality categorical encoding of plan or product identifier - using target encoding (with proper out-of-fold computation to avoid leakage) or frequency encoding rather than naive one-hot, since a high-cardinality plan/product ID field would otherwise blow up the feature space.
8. Cross-source interaction: declining usage AND a recent support contact (an explicit interaction feature, e.g. a flag or product of the two normalized signals) - intuition: the combination is a stronger churn signal than either usage decline or ticket volume alone, since users who complain but keep using the product, or who go quiet without complaining, behave differently than users doing both.

At roughly two million rows, computing these efficiently means vectorized, grouped aggregation (pandas groupby/window functions, or an equivalent SQL/Spark aggregation) rather than row-by-row processing, and encoding the high-cardinality categoricals (plan or product identifiers) with target or frequency encoding rather than naive one-hot encoding, which would otherwise create an unmanageably wide sparse matrix.

Whether to standardize or bin a behavioral signal depends on the downstream model and the signal's distribution: a heavily skewed count (most customers have very few support tickets, a small tail has many) often benefits from binning or a log-style transform for a linear model, while a tree-based model is largely indifferent to the raw scale and doesn't need standardization at all.

Detecting leakage specifically for churn: check whether any candidate feature (feature 4 and 5 above are the highest-risk ones) is computed using data from AFTER the point the churn decision would have actually been observable. A support ticket opened in response to a cancellation the customer had already decided on is a classic hidden leak, since it looks predictive but is really a symptom of the outcome, not a cause.

Worked example: A "days until account closure" style feature (however indirectly encoded, e.g. a support-ticket category that only exists post-cancellation) would show enormous offline lift and be completely unusable in production, since at prediction time you don't yet know whether the customer will churn; catching this requires explicitly auditing what information was genuinely available BEFORE the churn decision point for every candidate feature, not just checking whether the feature "looks reasonable."

Trade-offs and pitfalls: A common trap at this scale is trusting an offline metric improvement without an out-of-time evaluation; a feature that looks powerful on a random split can be capturing something that's specific to the historical period (a promotion that ran during part of the data) rather than a durable churn signal.

Redesign the input path
I would separate expensive preprocessing from the training loop and make the input stream sharded, cached, and deterministic.

Main changes

• Compact many small object-store files into larger shards so reads are sequential instead of chatty.
• Precompute expensive transforms once, then store the transformed output or an intermediate cache.
• Use a manifest that records shard order, sample IDs, and versioned transform code.
• Prefetch data to local disk or memory so GPUs do not wait on network reads.
• Keep lightweight, deterministic transforms in the training worker so results are reproducible.

Worked example
If the dataset is 10,000 files of 2 MB each, that is 20 GB spread across many requests. Repacking into 200 shards of 100 MB keeps the same 20 GB, but cuts file open and listing overhead dramatically.

Why this helps
The GPUs stay busy because decoding and network latency happen ahead of time. Reproducibility comes from versioned manifests and fixed seeds. Debuggability comes from keeping raw sample IDs so a bad example can be replayed exactly.

This is usually the highest leverage change when preprocessing time dominates training time.

Direct answer

I'd separate the timeline question from the model-complexity question: propose a simple analytics baseline that can ship inside the deadline, while being explicit about what it will and won't deliver, and frame the more complex model as a follow-up investment once the baseline's results are in.

Structured elaboration

1. Quantify the time-versus-value trade-off concretely. How much longer would the complex model realistically take, and what specific additional value would it capture that the simpler approach can't? Vague statements like "the ML model would be better" aren't useful to a stakeholder making a timeline call.
2. Estimate resourcing for both paths honestly. The simpler analytics approach's true cost (including validation, not just a quick query) versus the ML path's true cost (data preparation, training, evaluation, and the ongoing maintenance it creates).
3. Propose a phased plan. Ship the simpler approach on the stakeholder's timeline, instrument it well enough to measure its actual impact, and use that measurement to make the case (or not) for investing in the complex model afterward.
4. Keep the stakeholder's actual goal in view. The tight timeline usually reflects a real business pressure (a launch date, a reporting deadline); the plan needs to respect that pressure rather than just arguing it away.

Worked example

For an analysis that needs to inform a go or no-go decision in two weeks, a simple cohort-based analysis with clearly-stated assumptions and confidence bounds, delivered on time, is worth more to the business than a more rigorous model that arrives after the decision has already been made without it. Framing this honestly to the stakeholder (here's what the fast approach gives you, here's what it doesn't, here's what a slower approach would add) turns a potential conflict into a shared decision.

Trade-offs and pitfalls

The risk of always deferring to the tight timeline is under-delivering on decisions that genuinely need more rigor; the risk of insisting on the complex approach regardless of the deadline is missing the actual business window entirely, which can make even a perfect analysis worthless. The honest phased-plan framing avoids both traps by making the trade-off explicit rather than making the call unilaterally.

Direct answer

Go to the colleague privately first, before doing anything more public, and frame the concern as a question about the sampling method rather than a conclusion about their competence. Bring the specific evidence, propose a joint, falsifiable check that would settle whether the bias is real, and only then decide together how to handle the already-circulated report.

Structured elaboration

1. Verify before you raise it. Confirm the specific gap yourself (which source, what kind of gap) so you are not escalating a hunch. Raising a vague suspicion is more likely to read as an attack than raising a concrete, checkable one.
2. Private channel first. Do not raise it in the stakeholder meeting or a public thread. The goal at this stage is a shared understanding between the two of you, not a public correction.
3. Lead with evidence, not the conclusion. Ask how the sample was chosen and show what you noticed, rather than opening with "your report is biased." The evidence does the work; you are not the one delivering a verdict.
4. Propose a joint, falsifiable test. Agree in advance on a specific check that would settle the question either way, for example, re-running the analysis with a more complete data source and comparing results. If the two produce materially different conclusions, that is evidence of the bias; if not, the original report holds and nothing was lost.
5. Handle the stakeholder-facing correction together. If the test confirms the bias, present the fix as a normal part of the quality process, credit the colleague's original work, and avoid framing it as catching an error.

Worked example

A colleague circulated a cohort analysis to stakeholders built from a single data source you know has intermittent collection gaps. Rather than flagging it in the stakeholder thread, you ask to talk privately: "I noticed this cohort uses source A, do you know if that source had full coverage this quarter?" You show the specific evidence (gap periods, affected date ranges) and ask how the sample was chosen. Together you agree on the joint test: re-run the same cohort analysis using a second, more complete source and compare the two results. If the numbers move meaningfully, you have confirmed a real bias and both go to stakeholders together with an updated report and a data-quality caveat; if the numbers hold steady, the original report stands and the check cost an afternoon, not a reputation.

Trade-offs & pitfalls

• Raising it directly in the stakeholder meeting "to protect the org from a bad decision" scores a point in the moment but damages the working relationship and makes the colleague defensive on the next collaboration.
• Staying silent because raising it feels confrontational lets a real bias ship into decisions uncorrected, which is a worse outcome for the partnership than a slightly awkward private conversation.
• The senior move here is designing the joint test so the evidence settles the question, rather than relying on how persuasively you phrase the concern. A well-chosen test does the convincing; the conversation itself does not have to.
• A remaining pitfall: proposing a fix without proposing how to verify it worked. A joint investigation that ends without a joint, agreed check on the outcome tends to resurface as the same disagreement later.