DoorDash Applied Scientist (Mid-Level) Interview Preparation Guide

Direct answer: An interaction feature is the product (or another combination) of two existing features, and a polynomial feature raises a single feature to a higher power; both aim to let a model that can only represent additive, linear relationships capture a joint or curved effect it otherwise couldn't, but both can also actively hurt generalization if applied without restraint.

Structured elaboration:
An interaction term (like price times discount rate) helps when the TRUE relationship genuinely depends on the combination of two variables together, not either alone: a discount's effect on purchase likelihood might be completely different at a high price point than a low one, a relationship a purely additive model (with price and discount as separate, unrelated inputs) structurally cannot represent. A high-degree polynomial feature can similarly let a model fit a curved relationship a linear model alone would miss.

Worked example: Adding price × discount_rate as an explicit feature lets a linear model represent that a 20% discount matters very differently for a $10 item versus a $10,000 item, an effect the two features alone (fed in separately) cannot capture no matter how they're weighted individually. Conversely, adding high-degree polynomial terms (price squared, price cubed, up through a high order) on a modest dataset commonly causes overfitting: the model fits increasingly wiggly curves that chase noise in the training data rather than the true underlying relationship, and generalizes poorly to new data specifically because there wasn't enough data to reliably estimate all those additional polynomial coefficients.

Trade-offs and pitfalls: The failure mode in both cases is the same at its root: adding flexibility (whether via an interaction or a higher-degree polynomial) without enough data to support estimating the added parameters reliably trades a small chance of capturing real additional signal against a much larger risk of fitting noise; regularization, feature selection after expansion, and simply testing whether the added terms actually help on a genuine holdout (rather than assuming more flexibility is free) are the standard guardrails.

Approach summary
Start by treating each model as a probabilistic scorer (calibrate if needed), represent predictions plus uncertainty, then fuse using ensemble methods chosen by downstream decision objective (expected utility).

Probabilistic calibration

• Calibrate each model separately: isotonic regression or Platt scaling for classifiers; temperature scaling or beta calibration for NN; histogram/quantile mapping for tree ensembles.
• Validate calibration with calibration curve, Brier score, and expected calibration error (ECE).

Uncertainty propagation

• Capture aleatoric + epistemic: NN: MC dropout or ensembles; trees: predictive distributions via quantile regression forests or bootstrapping.
• Convert outputs to a common probabilistic form (e.g., predictive mean and variance). Propagate through decision function via analytic formulas if linear, otherwise Monte Carlo sampling to estimate downstream expected utility and risk.

Fusion methods

• Stacking: meta-learner trained on out-of-fold predictions; include model uncertainties and features; optimize directly for decision metric (e.g., cost-weighted loss).
• Bayesian Model Averaging (BMA): weight models by posterior model probability; use marginal likelihood or variational approximation; naturally accounts for uncertainty but higher compute.
• Hybrid: use BMA priors to regularize stacking weights or Bayesian stacking (optimize weights with Dirichlet prior).
• When decision utility is asymmetric, train fusion to maximize expected utility or CVaR.

Computational trade-offs

• Stacking (deterministic) is cheap at inference; requires O(N_meta) extra features and meta-model cost.
• BMA/MC sampling gives better uncertainty but costs M times base-model inference; mitigate with distillation or low-rank approximations.
• Use adaptive evaluation: cheap model first, run expensive model only on uncertain instances (gating).

Validation under covariate shift

• Simulate shifts via importance weighting, label shift correction, or create out-of-distribution holdouts. Use importance-weighted metrics to estimate real-world performance.
• Validate decisions with counterfactual or A/B tests where possible; measure downstream KPIs (expected utility, regret, cost savings), calibration under shift, and robustness metrics (worst-case loss).
• Monitor online: drift detectors on input features and on predicted uncertainty; recalibrate or reweight models when drift detected.

Example

• Tree (quantile forest) + NN: calibrate both, extract mean/var, run Bayesian stacking with Dirichlet prior, validate via importance-weighted expected utility under simulated shifts, deploy with gating to save compute.

This pipeline balances reliable probabilities, propagated uncertainty, computational cost, and real decision improvement under shift.

Start by clarifying objectives: preserve the teacher’s cross-modal representations and task behaviour in a smaller/multimodal student. You want the student to (a) match teacher outputs (task-level), (b) align intermediate cross-modal embeddings (representation-level), and (c) keep alignment between modalities (cross-modal signals).

Recommended loss terms and signals:

• Task (logit) distillation: softened cross-entropy between teacher logits zT and student logits zS with temperature T. L_task = KL(softmax(zT/T) || softmax(zS/T)).
• Feature alignment: L_feat = ||ProjT(fT) − ProjS(fS)||^2 applied at key layers. Use small projection heads to map different dims.
• Cross-modal contrastive loss: InfoNCE between teacher’s vision embeddings and student’s language (or fused) embeddings to preserve cross-modal retrieval/alignment capabilities.
• Attention/activation matching: L_att = MSE between normalized attention maps or per-head activations to convey where the teacher “looks.”
• Token-level / patch-level distillation: for vision→multimodal, match per-patch/class-token embeddings so fine-grained spatial cues transfer.
• Auxiliary reconstruction: decode student embeddings back to teacher modality (e.g., reconstruct teacher image features) to retain modality-specific info.
• Ground-truth supervision: combine with original loss L_gt so student doesn’t overfit teacher mistakes. Total loss = αL_gt + βL_task + γL_feat + δL_contrastive + εL_att + ζL_recon.

Practical notes:

• Use projection heads and temperature tuning; weight terms via validation. If teacher and student modalities differ, train cross-modal adapters that translate embeddings before matching. For transfer-across-tasks, include task-specific heads and multi-task KD: distill logits and representations for each task, or use task-agnostic contrastive signals to keep general features.
• Curriculum: start with representation/contrastive losses, add task logits later to stabilize.
• Watch for negative transfer: monitor teacher errors, allow student to rely on ground truth when teacher is wrong (e.g., label smoothing or confidence-thresholded distillation).

This combination preserves both task performance and cross-modal alignment during compression.

Three production-friendly encodings:

1. Hashing trick (feature hashing): map high-cardinality IDs into a fixed-size vector (e.g., 2^20 buckets). Pros: constant memory, simple, fast to serve. Cons: collisions may reduce signal; lacks learned similarity.
2. Learned embeddings (embedding table): train low-dim embeddings per ID (embedding size 32–128) used by models (FFN or MLP). Pros: captures semantics, high accuracy. Cons: large memory for millions of IDs—requires sharding, memory store (Redis/Key-Value), or parameter servers; increased serving latency unless cached.
3. Target encoding / frequency encoding: replace ID with statistics (mean label, count, smoothed) computed from historical data. Pros: compact numeric features, low latency. Cons: leakage risk—must use out-of-fold encoding and smoothing; may not capture complex relations.
   Trade-offs:

• Model complexity: embeddings and target encodings enable richer models (neural nets) while hashing fits linear/GBDT with less complexity.
• Serving latency: hashing and target encodings are fastest. Embeddings need an efficient store or on-device cache to meet low latency; consider quantized embeddings or approximate lookups.
• Memory and operational cost: embeddings require memory/ops to manage updates and cold-starts; hashing is low maintenance.
  Choose based on accuracy vs operational constraints; hybrid patterns (hashing for rare IDs, embeddings for top-K heavy hitters, target stats for others) often work well.

Direct answer. Detecting whether a production sample has drifted from a baseline distribution requires a test matched to each feature's TYPE, a numeric distributional test like Kolmogorov-Smirnov for continuous features and a chi-square test for categorical ones, run per feature, with a correction for the fact that testing many features at once inflates the chance of a false alarm somewhere.

Structured elaboration. For numeric features, the Kolmogorov-Smirnov test compares the full empirical distributions of the baseline and the sample without assuming any particular shape, which makes it a good general-purpose choice when you don't know in advance what kind of shift (mean, spread, skew) might occur. For categorical features, a chi-square goodness-of-fit test compares the sample's category proportions against the baseline's expected proportions. Missingness should be tracked SEPARATELY from the distributional test on the non-missing values, since a feature can have a stable distribution among the values it DOES have while its missing-rate silently climbs, a distinct and equally important failure mode a bare Kolmogorov-Smirnov (KS) or chi-square test on non-null values alone would miss entirely. With many features tested at once, apply a multiple-testing correction (Bonferroni divides the significance threshold by the number of tests) so the overall false-alarm rate across the whole feature set stays controlled, rather than letting a handful of features flag purely by chance.

Worked example (executed in Python with numpy/scipy). Comparing a numeric feature (avg_order_value, baseline mean 50, sample mean 58, 5,000 baseline rows vs 1,200 sample rows) and a categorical feature (region, unchanged mix in both): the KS test on avg_order_value returns statistic 0.309, p-value effectively 0 (well under a Bonferroni-corrected threshold), correctly flagging real drift; the chi-square test on region returns p=0.072, correctly NOT flagging it even without correction, since the region mix genuinely didn't shift in this synthetic example. Missingness delta for both features came back at 0.0, confirming the drift here is purely distributional, not a completeness issue.

Trade-offs and pitfalls. KS and chi-square are both sensitive to sample size: with a very large sample, even a practically trivial shift becomes statistically significant, so pair the p-value with an effect-size measure (like the KS statistic's magnitude itself, or a simple difference-in-means relative to the baseline's spread) rather than treating statistical significance alone as evidence of a PRACTICALLY important drift. For a streaming or very-large-feature-count setting, exact KS computation can be too slow; approximate quantile sketches or chunked/streaming variants trade a small amount of precision for the ability to run continuously rather than only on demand.

Direct answer

With limited labeled data to validate a new fraud model, a small canary rollout is almost always the safer starting point over a full production launch, since it lets you gather real production evidence before committing to broad exposure, as long as the canary's blast radius is genuinely small and monitored closely.

Structured elaboration

1. Canary approach. Ship to a small percentage of live traffic, closely monitor both model-level metrics (precision, recall against whatever confirmed labels arrive) and business-level guardrails (customer complaints, transaction friction), and expand gradually as confidence grows.
2. Full-rollout approach. Faster to reach full business impact if the model works, but any undetected problem (a mis-calibrated threshold, a data issue that only shows up at scale) affects the whole business at once, which is a much larger risk given how little validation data exists.
3. Metrics to track during the canary. Both the model's own precision and recall as confirmed labels trickle in, and the business cost signals (false-positive complaint volume, estimated fraud losses in the canary segment versus a comparable control segment) that would catch a real problem quickly.
4. Risk of each path. The canary's main risk is a slower path to full value and the extra engineering work of running two populations differently; the full rollout's main risk is a costly, hard-to-reverse mistake at scale if the model behaves unexpectedly outside of what the limited validation data could show.

Worked example

Starting the canary at a small percentage of transactions, holding a comparable control segment on the existing system, and setting explicit graduation criteria (a defined number of days with no anomalous complaint spikes and precision holding within an acceptable range against confirmed labels) turns "when do we go full rollout" from a gut call into a pre-agreed, evidence-based decision.

Trade-offs and pitfalls

The common mistake is running a canary without a real control group or explicit graduation criteria, which leaves the team unable to say confidently when it's safe to expand. The other risk is treating the canary phase as a formality and expanding on a fixed timeline regardless of what the early data shows, which defeats the purpose of testing with limited validation data in the first place.

Direct answer

Dropping rows, simple statistical imputation, model-based imputation, an explicit "missing" category, and leaving values as-is each introduce a different kind and amount of bias into downstream aggregates, and the right choice depends on how much data would be lost, whether the missingness itself carries information, and whether the consuming model or report can handle an explicit missing indicator natively.

Structured elaboration

• Dropping rows: unbiased only if missingness is completely random (MCAR); if missingness correlates with anything else in the data (MAR) or with the missing value itself (MNAR), dropping rows systematically removes a non-representative subset and biases every downstream aggregate, not just the missing column's own statistics.
• Simple statistical imputation (mean/median/mode): preserves sample size but artificially shrinks variance (every imputed value is identical, collapsing the natural spread) and can distort correlations between the imputed column and everything else, since the imputed value carries no relationship to the other fields in that row.
• Model-based imputation (KNN, MICE): KNN (K-nearest-neighbors imputation) fills a missing value using the average of that same field across the most similar other rows, similar meaning close on the other, non-missing features; MICE (Multiple Imputation by Chained Equations) iteratively models each incomplete column as a function of all the other columns, cycling through columns and refitting until the imputed values stabilize. Both better preserve relationships between features than simple imputation, since the imputed value is informed by similar records, but are more expensive to compute, harder to explain to a non-technical stakeholder, and can still introduce circularity risk if not done carefully.
• Explicit "missing" category: for categorical features, adding "missing" as its own valid category rather than imputing a value preserves the actual information (that it was missing) rather than manufacturing a value; for numeric features the closest analog is a paired missing-indicator flag alongside an imputed value. The bias this introduces into aggregates is different in kind from the other strategies: instead of shifting a mean or proportion, it changes what the aggregate is computed over. A "percentage of customers in category X" computed with missing kept in as its own bucket correctly reflects the true unknown rate, while the same statistic computed after silently excluding or imputing the missing rows will read differently, so any downstream aggregate has to decide explicitly whether "missing" counts as its own segment or gets excluded from the denominator, and getting that decision wrong is itself a source of bias.
• Leaving values as-is: only viable for models that natively handle NaN (some tree-based models do); for reporting and most other modeling contexts, this simply propagates missingness downstream unresolved. The bias risk here is not inside the model itself (a tree-based model can legitimately split on "is this value missing", which is often genuinely informative under MNAR) but in any aggregate computed outside that model: a plain AVG() or SUM() over a column with NaNs left in typically skips those rows by default in most SQL and pandas implementations, which is silently equivalent to a row-drop strategy for that one aggregate even though no row was actually removed from the dataset, so "leaving as-is" defers the bias decision to whatever tool happens to compute the next aggregate, rather than eliminating it.

Worked example

Under MCAR (missing completely at random), mean imputation on a numeric column leaves the mean of the column mathematically unchanged (since the missing values were a random subsample, imputing the mean does not shift it) but understates the true variance, since every imputed value contributes zero deviation from the mean. Under MNAR, for example high-income respondents disproportionately skip an income question, mean imputation using only the observed (systematically lower-income) values produces a biased-downward mean estimate for the full population, and no amount of more sophisticated imputation on the observed data alone can fully correct this without an external signal about the true relationship between income and the propensity to skip the question. To make this concrete with an illustrative (invented for this example, not measured) population: suppose 100 survey respondents have a true full-population mean income of $60,000, and the 20 who skip the income question happen to average $92,000 while the 80 who answer average $52,000; the underlying arithmetic is internally consistent, (80 x $52,000 + 20 x $92,000) / 100 = ($4,160,000 + $1,840,000) / 100 = $60,000. Mean imputation only ever sees the 80 observed rows, so it fills each of the 20 missing values with $52,000, the observed mean, and the reported overall mean stays at $52,000, $8,000 short of the true $60,000, with nothing in the observed data alone signaling that a gap exists.

Trade-offs and pitfalls

The critical, easy-to-skip step is diagnosing which missingness mechanism (MCAR, MAR, MNAR) is actually present before picking a strategy, since the same imputation method that is nearly harmless under MCAR can introduce serious, hard-to-detect bias under MNAR. Treating imputation as a purely mechanical, one-size-fits-all step, applied without first checking whether missingness correlates with anything else in the data, is the single most common way this class of decision goes wrong.

Brief framing
With 1% positives, goal is robust recall on positives while controlling false alarms and calibration for production decisions.

Data-level strategies

• Resampling: use stratified mini-batches and combine SMOTE or ADASYN for augmentation of minority class, but validate on original distribution.
• Targeted feature engineering: create signals that separate rare events (e.g., time-windowed aggregates, anomaly scores).
• Cost-aware cross-validation: use stratified k-fold with repeated runs to ensure rare positives appear in validation.

Algorithm-level strategies

• Class-weighting / focal loss: upweight positives or use focal loss to focus training on hard positives.
• Ensemble methods: stacked ensembles or calibrated bagging (e.g., balanced random forest, XGBoost with scale_pos_weight).
• Threshold optimization & calibration: tune decision threshold on precision-recall tradeoff and calibrate probabilities (Platt/ isotonic).

Evaluation & tuning

• Use PR-AUC, recall@fixed-false-positive-rate, and F2 (or business-weighted loss) for model selection; avoid accuracy and ROC-AUC alone.
• Use time-based holdout and conduct uplift / stability tests.

Production metrics to monitor

• Real-time: precision, recall, false positive rate, and alert volume.
• Calibration drift: Brier score or reliability diagram changes.
• Data drift: feature distribution and population shifts.
• Business KPIs: cost per true positive, downstream conversion, and latency.

These combined steps ensure robust training, appropriate risk trade-offs, and operational monitoring for a reliable production model.

Direct answer

For this kind of general, range-unknown input, I would implement randomized in-place quicksort: pick a uniformly random pivot at each partition step (to defeat adversarial or already-sorted inputs), partition with Lomuto's scheme, and always continue on the smaller of the two resulting partitions while deferring the larger one, which bounds recursion/stack depth to O(log n). Quicksort is the right default here over merge sort (which needs O(n) extra space) or counting sort (which needs a small, known key range).

Structured elaboration

Why quicksort for this input: it sorts in place (O(log n) extra memory instead of merge sort's O(n)), its sequential partition scan is cache-friendly and typically fastest in practice, and it makes no assumption about the range of the keys, unlike counting sort.

Partition step (Lomuto): move the pivot to the end, then walk the range once, swapping any element smaller than the pivot into a growing "less-than" prefix; finally swap the pivot into its correct final position.

Random pivot selection removes the classic O(n^2) worst case that a fixed first/last-element pivot suffers on already-sorted or reverse-sorted input.

import random

def quicksort_inplace(a: list[int], seed: int = 0) -> None:
    """
    Randomized in-place quicksort with Lomuto partitioning.
    Always recurses on the smaller side and loops on the larger side
    (via an explicit stack) to bound auxiliary depth at O(log n).
    """
    rng = random.Random(seed)

    def partition(lo: int, hi: int) -> int:
        pivot_idx = rng.randint(lo, hi)
        a[pivot_idx], a[hi] = a[hi], a[pivot_idx]
        pivot = a[hi]
        store = lo
        for i in range(lo, hi):
            if a[i] < pivot:
                a[store], a[i] = a[i], a[store]
                store += 1
        a[store], a[hi] = a[hi], a[store]
        return store

    lo, hi = 0, len(a) - 1
    stack = [(lo, hi)]
    while stack:
        lo, hi = stack.pop()
        while lo < hi:
            p = partition(lo, hi)
            left_size = p - lo
            right_size = hi - p
            if left_size < right_size:
                stack.append((p + 1, hi))
                hi = p - 1
            else:
                stack.append((lo, p - 1))
                lo = p + 1


data = [5, 2, 9, 1, 5, 6, 3, 8, 2, 0, -4, 7]
quicksort_inplace(data, seed=42)
print(data)

data2 = []
quicksort_inplace(data2, seed=42)
print(data2)

data3 = [1]
quicksort_inplace(data3, seed=42)
print(data3)

Output:

[-4, 0, 1, 2, 2, 3, 5, 5, 6, 7, 8, 9]
[]
[1]

Key points: continuing inline on the smaller partition (via the loop) while pushing only the larger one onto the explicit stack is what bounds the stack's maximum size to O(log n), since each level you push at least halves relative to its parent range.

Worked example

The input [5, 2, 9, 1, 5, 6, 3, 8, 2, 0, -4, 7] (note the repeated values 5 and 2) sorts correctly to [-4, 0, 1, 2, 2, 3, 5, 5, 6, 7, 8, 9] with seed 42, as shown in the run above; the duplicate values are handled fine by Lomuto's plain "<" comparison, since equal elements simply land in the "not less than pivot" partition without needing any special casing.

Complexity

Direct answer

A 7% conversion gap between two arms of an A/A test with p = 0.04 is surprising but not automatically evidence of a broken pipeline: with a conventional alpha, roughly 1 in 20 A/A tests will cross p < 0.05 by chance alone even with perfect randomization, so a single significant A/A result sits right at the boundary between noise and something worth investigating. The right response is a fixed sequence of diagnostics, cheapest and most mechanical first, that can positively confirm or rule out each concrete failure mode: broken hashing or salting in the assignment logic, a covariate imbalance introduced by a logging bug, or ordinary sampling variation.

Structured elaboration

Ordered diagnostics

1. Reproduce on raw logs with a fixed seed. Re-run the exact analysis against raw exposure and outcome events, not a downstream aggregated table, to rule out an analysis or pipeline bug before suspecting randomization itself.
2. Check the allocation ratio. Confirm the number of users landing in each arm matches the intended split; a large deviation from the intended ratio is one of the strongest and cheapest signals that the hashing or salting scheme is broken, and it is a distinct check from the 7% conversion gap itself. An experiment can have a balanced allocation and still show a spurious metric difference, or a skewed allocation and a metric difference that is actually the real symptom.
3. Audit covariate balance, not just the topline metric. Compare the two arms on pre-experiment covariates that should be identical under correct randomization: browser type, geography, device, and signup cohort. If, for example, one arm has a meaningfully different browser or region mix, that points directly at the hashing or salting scheme, perhaps it is keyed on something correlated with browser or region rather than a clean user id, rather than at chance.
4. Audit instrumentation and logging for the affected population, not just in aggregate. Check whether the difference is spread evenly across all users or concentrated in a specific segment, for example a subset of power users whose events are logged differently or who hit a code path that assigns or tags them incorrectly; a difference concentrated in one segment is far more consistent with a logging bug than with chance, since chance-driven noise does not usually cluster by user type.
5. Check for multiple looks and multiple metrics. Confirm this is the primary metric and the first pre-registered look at the data; if the gap emerged after peeking at several days or several metrics, the effective false-positive rate is higher than the nominal 5% used to compute that p = 0.04.
6. Re-run on an independent window if steps 1 to 5 find nothing. If every mechanical check comes back clean, run a fresh A/A on a new time window, not the same data re-sliced, before concluding the original result was real noise; a second clean A/A is much stronger evidence than reasoning about a single result in isolation.

Practical thresholds for concern

A single A/A test crossing the standard p < 0.05 threshold is expected roughly 1 in 20 times under correct randomization and is not, by itself, cause for alarm. Because allocation-ratio checks tend to run on every experiment in a mature program, many teams apply a stricter threshold specifically to that check, for example requiring p < 0.001 before treating a deviation as a real problem, because running a 0.05-threshold check on every one of hundreds of experiments would itself generate frequent false alarms. This is a practitioner convention for managing check-your-own-checks noise, not a statistical property being claimed as derived; teams calibrate the exact cutoff to their own experiment volume.

Worked example

Take the power-user logging-bug scenario concretely, with stated illustrative inputs. Suppose the population is 1,000,000 users, correctly randomized 50/50 by the underlying hashing scheme. Power users are 10% of the population (100,000 users), and a logging bug causes every power user's exposure event to be recorded under the control label regardless of which arm they were actually assigned to.

Non-power users (900,000) log correctly: 450,000 recorded as control, 450,000 recorded as treatment. Power users (100,000) all log as control regardless of true assignment.

$\text{Recorded control} = 450{,}000 + 100{,}000 = 550{,}000, \qquad \text{Recorded treatment} = 450{,}000$
$\text{Recorded split} = 55\% : 45\%$

A 55/45 recorded split against an intended 50/50 is a large, easily detectable sample-ratio deviation, exactly the kind of signal step 2 is designed to catch, and it would also explain a spurious conversion difference: if power users convert at a different rate than the general population, mislabeling all of them into "control" pulls the control arm's average toward the power-user rate and creates a gap that has nothing to do with the treatment itself.

Trade-offs and pitfalls

• Investigating every marginal A/A result exhaustively is expensive; teams running many experiments generally accept a small, known false-alarm rate rather than deeply diagnosing every borderline case, but should still run the cheap mechanical checks (steps 1 to 3) before shipping a launch decision built on top of infrastructure that produced a suspicious A/A.
• A clean allocation ratio does not prove randomization is unbiased end to end; a hashing scheme can produce a correct overall 50/50 split while still being correlated with a covariate, for example alternating by the last digit of a sequential user id, which can correlate with signup cohort. The ratio check and the covariate-balance check are complementary, not substitutes for each other.
• Re-running on a new window is the strongest single diagnostic but takes real calendar time, so it should be the last check, not the first, when cheaper log-based checks can often localize the problem faster.
• Concluding "just noise" without running the mechanical checks at all is itself a pitfall: an unresolved instrumentation bug found only after it has already contaminated a live experiment is far more costly than the time it takes to check allocation ratio and covariate balance up front.