DoorDash AI Engineer (Junior Level) Interview Preparation Guide

Direct answer

Don't build the dense matrix at all. Keep the data long-form (one row per product-date-metric) as the source of truth, and only materialize a wide view for a bounded subset of metrics you actually need at once, using pandas' SparseDtype so the empty cells never occupy real memory. A full dense pivot allocates a cell for every (product-date, metric) pair even though any single product-date only touches a handful, so a thousand-metric catalogue turns a few thousand real values into millions of stored NaN (not-a-number, pandas' missing-value marker) cells.

Approach

1. Aggregate to a compact, sparse-friendly representation: for each product-date, keep only the metrics that are actually present, as a dict/JSON payload rather than a row of mostly-empty columns.
2. If a tabular wide view is genuinely needed (for a model or a dashboard), pivot only the metrics of interest and store the result with pandas.SparseDtype, which keeps non-fill values plus their positions instead of one cell per (row, column) pair.
3. For downstream analytics or machine learning that wants a full matrix, build a scipy.sparse coordinate (COO, coordinate-format sparse matrix storing (row, column, value) triples instead of a dense grid) matrix directly, never a dense pandas pivot.

import pandas as pd
import numpy as np
import json
from scipy import sparse

def to_sparse_long(df, id_cols=("product_id", "date"), metric_col="metric", value_col="value"):
    """One row per (product_id, date) holding only the metrics present that day, as JSON."""
    grouped = (
        df.groupby(list(id_cols))
          .apply(lambda g: dict(zip(g[metric_col], g[value_col])), include_groups=False)
          .rename("metrics_dict")
          .reset_index()
    )
    grouped["metrics_json"] = grouped["metrics_dict"].apply(json.dumps)
    return grouped[[*id_cols, "metrics_json"]]

def to_sparse_wide(df, metrics_of_interest, id_cols=("product_id", "date"),
                    metric_col="metric", value_col="value"):
    """Wide pivot restricted to a bounded metric set, stored as pandas SparseDtype."""
    subset = df[df[metric_col].isin(metrics_of_interest)]
    wide = subset.pivot_table(index=list(id_cols), columns=metric_col,
                               values=value_col, aggfunc="first")
    for c in wide.columns:
        wide[c] = wide[c].astype(pd.SparseDtype(np.float64, np.nan))
    return wide

def to_coo_matrix(df, id_cols=("product_id", "date"), metric_col="metric", value_col="value"):
    """Full (product-date x metric) matrix as a scipy COO sparse matrix, no dense pandas pivot."""
    row_key = list(df[list(id_cols)].itertuples(index=False, name=None))
    row_map = {k: i for i, k in enumerate(dict.fromkeys(row_key))}
    metric_map = {m: i for i, m in enumerate(df[metric_col].unique())}
    rows = [row_map[k] for k in row_key]
    cols = [metric_map[m] for m in df[metric_col]]
    return sparse.coo_matrix((df[value_col].to_numpy(), (rows, cols)),
                              shape=(len(row_map), len(metric_map)))

Key points:

• The long-form JSON table (to_sparse_long) is O(non-zero entries) in storage and is the right form to persist to parquet for a BI tool that can unpack a JSON column, or serve on demand via an API.
• pandas.SparseDtype is for when you genuinely need a small number of columns as real DataFrame columns (feature selection for a model, a report) without paying for every empty cell.
• scipy.sparse.coo_matrix is the right structure for the FULL matrix when something downstream (a model, a similarity computation) needs the whole (product-date x metric) space at once.

Worked example

df = pd.DataFrame([
    {"product_id": "A", "date": "2025-01-01", "metric": "m1", "value": 10},
    {"product_id": "A", "date": "2025-01-01", "metric": "m20000", "value": 5},
    {"product_id": "B", "date": "2025-01-01", "metric": "m2", "value": 7},
])

print(to_sparse_long(df))
print()
wide = to_sparse_wide(df, metrics_of_interest=df["metric"].unique())
print(wide)
print(wide.dtypes)
print()
print(to_coo_matrix(df).toarray())

Output (verified against pandas 3.0.3):

  product_id        date                     metrics_json
0          A  2025-01-01  {"m1": 10, "m20000": 5}
1          B  2025-01-01               {"m2": 7}

metric                   m1   m2  m20000
product_id date
A          2025-01-01  10.0  NaN     5.0
B          2025-01-01   NaN  7.0     NaN
metric
m1        Sparse[float64, nan]
m2        Sparse[float64, nan]
m20000    Sparse[float64, nan]
dtype: object

[[10  5  0]
 [ 0  0  7]]

Only 3 values exist across 2 product-dates and 3 metrics, but a naive pivot over the real thousands-of-metrics catalogue would allocate a dense (product-dates x metrics) grid; here it is a 2x3 grid to illustrate the shape, and every metric cell not actually observed for that product-date is absent from storage in the long form and only a lightweight fill-value entry in the sparse-dtype and COO forms.

Complexity and edge cases

Complexity: building the long-form grouped table is O(n) time and O(k) storage where k is the number of non-zero (product-date, metric) entries, versus O(rows x thousands-of-metrics) for a naive dense pivot. The SparseDtype wide view and the COO matrix are both O(k) storage for the values plus O(k) for their position indices, a constant-factor overhead over the long form, still far below dense.

Edge cases: duplicate (product-date, metric) rows need an explicit aggregation choice (aggfunc="first", "last", or "mean") or pivot_table silently picks one; a metric present for one product-date but never elsewhere still gets its own sparse column once selected into metrics_of_interest; extremely large per-row JSON payloads (a product-date with hundreds of metrics present) should go to parquet/object storage rather than staying as an in-memory string column; a product_id/date combination with zero rows never appears at all in the long form, so any downstream join needs an explicit outer join against the full key space if "no data that day" must be distinguished from "value is genuinely zero".

Trade-offs and pitfalls

Long-form JSON storage is the cheapest and most flexible (any BI tool or API can read a subset of keys), but it pushes the reshape work downstream to whoever consumes it. SparseDtype gives you a real, indexable DataFrame with column-level operations, at the cost of only being practical for a bounded metric subset, not the full thousands-wide catalogue, since selecting and materializing that many sparse columns still carries per-column overhead. The COO matrix is the most memory-efficient full-matrix representation and is what you want for downstream linear algebra or ML, but it loses the readable row/column labels unless you keep the row_map/metric_map alongside it. The pitfall to avoid is reaching for pivot_table directly on the full metric set out of habit: pandas will build the dense matrix in memory before you ever get a chance to convert it to a sparse dtype, so the peak memory spike happens regardless of what dtype the result ends up with.

Direct answer
Overfitting is when a model gets extremely good at the exact examples it practiced on, but hasn't actually learned the general pattern, so it struggles on anything new. A useful analogy: imagine a student who memorizes the answers to last year's practice exam questions word for word, rather than learning the underlying concepts. They'd score perfectly if given that exact practice exam again, but do poorly on this year's real exam, which asks the same concepts in slightly different ways.

Structured elaboration
That's exactly what's happening when a model shows excellent training performance but poor performance on new data: it has effectively memorized quirks and noise specific to the training examples, rather than learning the broader pattern that would actually transfer to new, similar situations. The training metrics look great because the model is being tested on the same "practice questions" it studied; the real-world performance is poor because new data is like a slightly different, real exam.

Worked example
A simple, concrete step: hold back some data the model never gets to train on (this is the equivalent of a genuinely new practice exam the student hasn't seen), and check performance there instead of only on the training data. If performance on that held-back data is much worse than on the training data, that gap is itself evidence of overfitting, similar to how the student's real exam score being much lower than their practice-exam score would be the tell that memorization, not real understanding, was happening.

Trade-offs and pitfalls
A second simple, concrete step: make the model somewhat "simpler" or less free to memorize fine details, similar to how you might encourage a student to focus on understanding a few core concepts well rather than memorizing every specific example, which usually generalizes better to questions they haven't seen before. The trade-off worth flagging honestly to a non-technical stakeholder is that a small amount of "forgetting" specific training examples is actually the price of better real-world performance, and chasing perfect training performance is often exactly the wrong goal.

Direct answer: Real-time streaming features (a live ETA, a rolling fraud window, a session aggregate) need a state store that can hold per-key running state efficiently, a defined windowing semantic for how that state is aggregated over time, and an explicit policy for out-of-order and late-arriving events, since none of these three decisions can be deferred until after the system is built.

Structured elaboration:

• State-store choice: an in-memory or embedded state store (like the state backend built into a streaming framework) for per-key aggregates that need to survive across events without a round-trip to an external database on every update; the choice of backend trades raw speed against durability and how large the total keyed state can grow.
• Windowing semantics: whether the aggregate is a fixed-size sliding window (last 24 hours, continuously updated), a fixed tumbling window (this calendar hour, reset at boundaries), or a session window (grouped by activity gaps); the choice depends entirely on what the downstream model actually needs to reason about.
• Low-latency reads: the serving path needs to read this continuously-updated state with a tight latency budget, which usually means the state store IS the online feature store for this feature, not a separate system requiring an extra hop.
• Fault tolerance: the state needs to survive a job restart without losing in-flight aggregates, via checkpointing (see the exactly-once discussion elsewhere in this topic).

Worked example: A driver's live ETA to a restaurant, recomputed every 10 seconds, keeps per-driver state (current estimated position, recent speed) in the streaming job's state store, updates it continuously as new location events arrive, and serves the current estimate directly from that same state store with no separate materialization step, since any added latency between "state updates" and "state is servable" would make the feature stale before it's even used. A fraud feature keyed on the past 24 hours of a user's activity uses a genuinely sliding (not tumbling) window, since fraud detection needs to reason about "the last 24 hours as of right now," not "this calendar day."

Trade-offs and pitfalls: Choosing a tumbling window when a sliding window was actually needed (or vice versa) is a common, subtle correctness bug: a tumbling window resets sharply at its boundary, which for a security-sensitive feature like fraud can create an exploitable blind spot right at the reset instant, whereas a genuinely sliding window has no such artificial discontinuity.

Situation: In production we have an unsupervised anomaly detector but very few real labeled anomalies, so we need reliable offline evaluation and a principled way to set alert thresholds that balance precision/recall and operational cost.

Evaluation strategies:

• Synthetic anomaly injection: create realistic anomalies by perturbing features based on domain knowledge (e.g., spike/drop, drift, missingness). Inject these at controlled rates into historical clean data to measure recall across magnitudes and types. Vary SNR to test robustness.
• Proxy signals / weak labels: derive proxies from related indicators (e.g., incident tickets, rollback events, customer complaints) and align timestamps to produce noisy labels. Treat them as lower-confidence validation sets and weight metrics accordingly.
• Recent labeled incidents: curate and augment the small set of true incidents via time-window expansion and feature augmentation; use them as high-confidence test cases.
• Cross-validation on time slices: use rolling-origin evaluation to preserve temporal structure; test detector stability across seasons and load patterns.
• Stress tests and adversarial scenarios: simulate correlated failures, concept drift, and missing telemetry to observe false positive bursts.

Choosing thresholds:

• Define cost model: quantify cost of false positives (ops time) and false negatives (business impact). Convert into a single loss: Loss = C_FN * FN + C_FP * FP.
• Use ROC/PR curves from injected + proxy labels to find threshold minimizing expected loss. For highly imbalanced tasks, prefer precision-recall curves and optimize F_beta (beta set by relative cost).
• Calibrate thresholds per segment/season: set adaptive thresholds by baseline variance (e.g., z-score with dynamic window) or percentile-based thresholds per entity to equalize alert rates.
• Implement operational controls: cap alerts per time-window, require multi-signal corroboration (logical AND or scoring ensemble), and introduce an auto-snooze for noisy periods.
• Monitor and iterate: deploy with conservative thresholds, track real-world precision/recall, update cost estimates, and retrain using any newly labeled incidents.

This approach combines simulated ground truth, weak supervision, principled cost-based thresholding, and operational safeguards to balance accuracy and costs in a production setting.

Direct answer

The two-sample KS statistic is the maximum absolute distance between the two samples' empirical CDFs; scipy's implementation returns both the statistic and an asymptotic p-value, and for small samples you fall back to bootstrapping to get a trustworthy p-value.

Structured elaboration

from scipy import stats
import numpy as np

def detect_drift_ks(reference: np.ndarray, current: np.ndarray, alpha: float = 0.05):
    statistic, p_value = stats.ks_2samp(reference, current)
    return {"statistic": float(statistic), "p_value": float(p_value), "drift_detected": bool(p_value < alpha)}

Assumptions: KS assumes both samples are drawn i.i.d. and, in its classic form, that the underlying distribution is continuous. Ties: with a discrete or heavily-rounded feature, many observations land exactly on the CDF's jump points, which biases the exact p-value; scipy's method='asymp' still runs but the p-value is approximate under ties, and the honest fix is either to use method='exact' for genuinely small samples without ties, or to accept the statistic (D) as directional evidence and not over-trust the p-value's precision when ties are common. Small samples: the asymptotic p-value formula assumes large-sample behavior; below a few dozen points per side, bootstrap instead: resample both samples with replacement many times, recompute D each time under the null (pool and re-split), and use the empirical distribution of D to get a p-value that doesn't rely on the asymptotic approximation.

Worked example

Independently executed in a Python sandbox (scipy 1.18): baseline and current drawn from the identical Normal(0,1) (n=500 each, seed=7) gave D=0.048, p=0.61: correctly not significant. A 0.5-sigma shift (current ~ Normal(0.5,1)) gave D=0.24, p≈0.0000: correctly and strongly significant, matching the expectation that a half-standard-deviation mean shift at n=500 is easily detectable.

Trade-offs & pitfalls

The most common production mistake with KS at scale is treating a tiny, business-irrelevant p-value as itself the alert signal: at high daily prediction volume, even a genuinely trivial distributional wobble (D=0.02) will produce p≈0 once your sample is large enough. The statistic D: the actual magnitude of the CDF gap: is the more stable, more interpretable number to threshold against for alerting; the p-value is best used to confirm "is there SOME real difference at all" on a modest, fixed-size sample rather than as your primary threshold at full production scale.

Direct answer

Make the mismatch visible the moment you see it, whether that is after the upstream team is already running behind or as soon as it surfaces during planning itself, and look first for a way to decouple your own delivery from their exact finish order, such as a stub, an adapter, or a feature flag, so you have room to negotiate re-sequencing or reduced scope instead of just waiting to see if the date slips.

Structured elaboration

Surface the mismatch immediately, not once it is a crisis

Whether you discover it because the other team is visibly behind, or because it becomes obvious during a shared planning session, name it out loud right away: here is what we committed to, here is what we now depend on, here is the gap.

Look for a decoupling option before assuming you have to slip

A mock interface, a stubbed API, or a feature flag lets your work continue against a placeholder while the real dependency finishes in parallel, with a defined swap-in point once it is ready.

Negotiate re-sequencing with a concrete ask, not just a complaint

Pointing out that another team is behind invites defensiveness. Proposing a specific way both teams can still hit their dates if two pieces are resequenced invites problem-solving instead.

Communicate consistently to everyone downstream of the decision

Use the same explanation each time: what changed, what the new plan is, and what happens if it changes again.

Set escalation triggers before you need them

Agree upfront on the specific checkpoint, a date or a milestone, at which, if the upstream work still is not ready, the issue escalates automatically to both leads, rather than waiting for the final deadline to find out.

Worked example

Base case: discovered after the upstream team is already behind. A team is building a feature on top of a platform capability, and the platform team is now behind schedule on it. Rather than waiting to see if the platform team catches up, the team builds a lightweight adapter against a mocked version of the interface, so its own work continues. They set an explicit go or no-go checkpoint a week before their real deadline: if the real dependency is not ready by then, they ship against the mock with a manual fallback, and swap in the real dependency once it lands.

Planning-time discovery variant. During a multi-team sprint-planning session, it becomes clear in the room that one team's planned start date for a shared integration depends on another team's work, which is not scheduled to finish until after the first team's own committed date, a mismatch nobody had caught before that meeting. The engineer facilitating the session, in this scenario a DevOps engineer coordinating the shared infrastructure both teams touch, flags the conflict on the spot and proposes re-sequencing right there: the first team starts against a stubbed interface while the second team's work continues in parallel, with the real dependency swapped in once ready. Right after the session, the facilitator sends a short written summary to both team leads and stakeholders using a repeatable communication template: what was found, what was agreed, and what happens if either date slips again. The summary also sets an explicit escalation trigger: if the second team's work is not ready by a named checkpoint date, it escalates automatically to both leads instead of surfacing again only at the final deadline.

Trade-offs and pitfalls

Building a decoupling layer, such as an adapter, a mock, or a flag, costs real engineering time that is wasted if the upstream team finishes on schedule after all. It is worth it when the downside of waiting and being wrong is worse than the cost of building it and not needing it, which is usually true for anything on a hard external deadline.

Escalating too early, before giving the upstream team a real chance to communicate a plan, burns trust and can look like an attempt to shift blame preemptively. Escalating too late removes any options besides slipping the date. Pre-agreed, specific escalation triggers tied to a date rather than a feeling are what keep this from being a judgment call made under pressure.

Direct answer

Choosing between two large language model (LLM) vendors comes down to mapping each vendor's technical capabilities and contract terms onto the specific outcomes your product needs, then scoring them on a shared scale rather than picking based on brand reputation or a single benchmark.

Structured elaboration

1. Technical capabilities to compare. Latency for your typical request size, whether fine-tuning or customization is supported and how, how the vendor handles your data (training on it, retention, deletion), and the range of model sizes or tiers available for cost-versus-quality trade-offs.
2. Contractual terms. Pricing structure and how it scales with volume, any minimum commitments, data-processing terms, and how easy the contract makes it to leave.
3. Mapping to business outcomes. For each capability, ask what it actually changes for the product: does lower latency meaningfully improve the user experience, does better fine-tuning support reduce the need for costly prompt engineering, does the pricing model fit your expected usage pattern.
4. Scoring and weighting. Assign each criterion a weight based on how much it matters for THIS product (a latency-sensitive real-time feature weighs latency heavily; a batch-processing use case might weight cost more heavily instead), then score each vendor and compare the weighted totals, while flagging any hard disqualifiers (a data-handling term you can't accept) separately from the weighted score.

Worked example

For a real-time customer-support chat assistant, latency and fine-tuning support might be weighted heavily (say, together worth half the total score), with cost weighted less because the volume is modest; for a nightly batch-summarization job, cost per token might dominate the weighting instead, since latency is nearly irrelevant to a job that already runs asynchronously. The same two vendors could rank differently on these two use cases even with the same raw technical capabilities.

Trade-offs and pitfalls

The common mistake is using one generic scorecard for every use case, when the right weighting genuinely depends on what the product needs; a vendor that wins for a latency-sensitive feature might lose for a cost-sensitive batch job. The other risk is under-weighting contractual exit terms, which matter far more once you're locked into fine-tuned models or embedded workflows than they seem to at signing time.

Direct answer
Grid search is exhaustive and simple but wastes trials in high dimensions; random search covers more of the space per trial; Bayesian optimization is sample-efficient for expensive, low-to-moderate-dimensional, low-noise objectives; Hyperband trades a small risk of discarding a late bloomer for large speedups by exploiting cheap partial evaluations; population-based training is the right tool when hyperparameters should themselves change during a single training run.

Structured elaboration

• Grid search: fully parallel (every point is independent), degrades sharply as dimensionality grows (a 5-value grid over 5 hyperparameters is already 3,125 combinations), and handles noisy objectives poorly since it never revisits or refines a promising region.
• Random search: also fully parallel, scales much better with dimensionality (Bergstra & Bengio's classic result: it finds comparably good configurations in a fraction of grid search's trials when only a few hyperparameters actually matter), still doesn't adapt based on what it's already learned.
• Bayesian optimization: sequential by nature (each new point depends on the surrogate fit to all previous points), which limits parallelism (though batched/async variants exist); handles noisy objectives by modeling noise explicitly in the surrogate, but the surrogate model itself degrades in high dimensions (roughly beyond 15-20 continuous hyperparameters, the standard Gaussian-process surrogate stops being reliable).
• Hyperband/Successive Halving: highly parallel within each rung, and its core trick is spending most of the budget only on configurations that already look promising at a cheap fidelity; the real risk is discarding a configuration whose LEARNING CURVE is slow to start but eventually wins, a genuine failure mode when candidate configurations have very different convergence speeds.
• Population-based training: unlike the others, it doesn't pick hyperparameters once, it evolves them DURING training, which is the right fit when the ideal hyperparameter schedule genuinely changes over the course of training (e.g. a learning rate that should decay differently depending on how training is progressing) rather than being one fixed best value.
  When to prefer which: grid for a tiny, cheap, low-dimensional space where exhaustiveness itself has value (e.g. regulatory documentation); random as a solid, nearly cost-free default upgrade over grid; Bayesian opt when each trial is genuinely expensive (hours) and you have a modest number of hyperparameters; Hyperband/ASHA (Asynchronous Successive Halving) when trials are cheap to partially evaluate (most neural network training) and you want the search wall-clock time down; population-based training (PBT) when you're training one long run and want the hyperparameter schedule itself to adapt.

Worked example
Tuning a transformer with 6 continuous/discrete hyperparameters where a single full training run takes 8 hours: pure grid search over even 3 values per hyperparameter is 729 runs, infeasible. Bayesian optimization over ~30-50 full runs is a realistic, sample-efficient choice here. If instead the model trains in 20 minutes and you can afford thousands of partial runs, ASHA lets you explore far more configurations for the same total compute by killing bad ones early.

Trade-offs & pitfalls
It's tempting to always reach for the most sophisticated method (Bayesian or Hyperband); for a very cheap, very low-dimensional search, plain random search with a healthy trial budget is often just as effective and far simpler to implement and debug.

Direct answer
A symptom is the observable effect users or dashboards notice, such as elevated error rates. A root cause is the underlying condition that, if it had been different, the incident would not have happened. A contributing factor made the incident more likely, larger, or slower to resolve, but would not by itself have caused it. Root cause and contributing factors are also sometimes described as systemic versus proximate: the proximate cause is the immediate trigger right before the failure, while the systemic cause is the deeper condition that made that trigger dangerous in the first place.

Structured elaboration
These distinctions matter because they point to different fixes. Fixing a symptom (restarting a crashed process) restores service but changes nothing about recurrence. Fixing the root cause prevents the class of failure from happening again. Fixing only a contributing factor reduces the odds or blast radius without eliminating the underlying risk.

A practical way to separate them: ask 'if I undo only this one thing, does the incident still happen?' If removing it would have prevented the incident outright, it's a strong candidate for root cause. If removing it would have made the incident smaller, shorter, or less likely, but the incident could still have happened some other way, it's a contributing factor. Multiple contributing factors lining up is far more common in real incidents than a single clean root cause, which is exactly why a rigid, single-cause framing (like a naive application of Five Whys) can mislead you into stopping the investigation too early.

Worked example
An e-commerce checkout service goes down for two hours. Symptom: checkout returns 500 errors and cart-abandonment spikes. Proximate cause: a database connection pool exhausted because a recently deployed feature opened a connection per request without releasing it. Systemic (root) cause: the codebase has no linting or code-review checklist item that catches unreleased database connections, so this class of bug can ship again in a different service tomorrow. Contributing factors: (1) the connection pool's exhaustion alert had a threshold set so high it fired only after service was already degraded, delaying detection by 20 minutes; (2) the on-call engineer was unfamiliar with this particular service's connection-pooling library, adding another 10 minutes to diagnosis. None of the contributing factors alone caused the outage, and even a perfectly-set alert would not have prevented the leak, but each one made the incident worse or longer, and each deserves its own action item.

Trade-offs and pitfalls
The most common mistake is treating the first plausible-sounding cause as THE root cause and closing the investigation, especially under time pressure to produce a tidy narrative. A second is conflating 'the last thing that changed before the incident' (the trigger) with the systemic cause: the trigger is often just the first domino, and stopping there produces a fix that only prevents that exact trigger, not the underlying fragility.

Direct answer: Allocating a fixed GPU-hour budget across continued pretraining, supervised fine-tuning, preference collection, reward-model training, and RLHF should maximize expected human-preference gain per GPU-hour, modeled with an explicit (if approximate) marginal-return function per stage, then validated and corrected with cheap real pilot experiments rather than trusted blindly.

Structured elaboration:

• Objective and formalization: define the goal explicitly as maximizing total incremental preference gain across stages divided by the fixed total GPU-hour budget, subject to the constraint that GPU-hours across all stages sum to the budget, which turns "how should we split the budget" into a concrete constrained-optimization problem rather than an intuition call.
• Approximate marginal-return model per stage: continued pretraining and reward-model training both tend to show strong early gains with sharply diminishing returns (a logarithmic-shaped return curve is a reasonable approximation), supervised fine-tuning and RLHF policy optimization tend to saturate faster once enough structure has been learned (an exponential-saturation shape is a reasonable approximation there); and because reward-model quality and RLHF gains both depend on how many labeled preference pairs exist, those two stages' effective returns should be modeled as scaling with the annotation volume, not treated as independent of it.
• Allocation procedure: fit the model's parameters from historical runs or a small pilot grid search, then solve the constrained allocation (a greedy hill-climb that allocates each additional GPU-hour to whichever stage currently shows the highest estimated marginal return, or a more principled Bayesian-optimization search over the full allocation vector), while respecting minimum viable investments (for example, a reward model needs some minimum labeled-pair count before it is trainable at all, regardless of what the marginal-return curve suggests in isolation).
• Validation and iteration: treat the fitted model's predictions as a prior, not ground truth, run small, cheap proxy experiments (a short continued-pretraining snapshot, a small SFT run, a few annotation-budget variants) specifically to estimate the model's parameters empirically; reserve a genuine exploration fraction of the total budget (for example around 10-15%) to test allocations the current model considers suboptimal, since the model's own estimate could be wrong; and update the marginal-return model with observed real results as they come in, re-solving the allocation as better estimates arrive rather than committing the entire budget to the first fitted plan.

Worked example: If the fitted model initially estimates continued pretraining has the highest marginal return per GPU-hour early in the budget, a hill-climbing allocation would spend GPU-hours there first, but only up to the point where its estimated marginal return, now diminished by the logarithmic-return assumption, drops below what supervised fine-tuning or annotation collection would yield for the same GPU-hour, at which point the allocation shifts; the 10-15% exploration reserve specifically tests whether the model's assumed shape is wrong, for example if continued pretraining's real marginal return diminishes FASTER than the fitted logarithmic curve assumed, that would be visible in the exploration results and should correct the next allocation round rather than being ignored.

Trade-offs and pitfalls: Committing the entire GPU-hour budget to a single allocation plan based purely on the fitted marginal-return model, without any real-world exploration budget reserved, risks over-investing in whichever stage the (necessarily approximate) model happens to favor, a mistake that is expensive to discover only after the budget is spent. A second pitfall is treating annotation and reward-model training as independent of each other in the allocation model, they are coupled (reward-model and RLHF gains both scale with how many preference pairs exist), and modeling them as independent risks under-allocating to annotation relative to what the coupled reality actually requires.