Model Evaluation and Validation Questions

Precision, recall, specificity, F1, and accuracy (binary classification): definitions:

• Precision = TP / (TP + FP). Of predicted frauds, fraction that are actual fraud. Measures false-alarm rate.
• Recall (a.k.a. sensitivity) = TP / (TP + FN). Of actual frauds, fraction we detect. Measures missed-fraud rate.
• Specificity = TN / (TN + FP). Of actual non-fraud transactions, the fraction correctly left alone. Prioritize it when a false positive on a legitimate customer is expensive: e.g. a system that auto-freezes flagged accounts, where wrongly freezing a real customer's account triggers a costly support escalation and risks losing that customer, so specificity (not just recall on the fraud side) becomes the number the fraud-ops team watches.
• F1 = 2 * (precision * recall) / (precision + recall). Harmonic mean balancing precision and recall; useful when classes are imbalanced.
• Accuracy = (TP + TN) / (TP + FP + TN + FN). Overall fraction correct across both classes. Prioritize it when classes are roughly balanced and the two error types cost about the same, e.g. an internal audit-sampling task where a false positive and a false negative both just cost one analyst-hour to re-check, unlike this heavily imbalanced fraud case where accuracy is the misleading headline number.

Business priority for fraud detection:

• If false negatives (missed fraud) are much more costly than false positives, prioritize recall. Catching fraud reduces direct monetary loss; higher recall may raise false positives (lower precision), which increases investigation cost but is acceptable if business tolerates it.

Example confusion matrix (actual vs predicted):

• True Fraud (positive): 100 cases
• Non-Fraud (negative): 9,900 cases
  Scenario A (conservative threshold → high recall):
• TP=95, FN=5, FP=800, TN=9,100
  Precision = 95/(95+800)=10.6%; Recall=95/100=95%; Specificity = TN/(TN+FP) = 9,100/9,900 ≈ 91.9%; Accuracy = (TP+TN)/Total = (95+9,100)/10,000 = 9,195/10,000 ≈ 92.0%
  Scenario B (strict threshold → high precision):
• TP=60, FN=40, FP=100, TN=9,800
  Precision = 60/(60+100)=37.5%; Recall=60%; Specificity = TN/(TN+FP) = 9,800/9,900 ≈ 99.0%; Accuracy = (TP+TN)/Total = (60+9,800)/10,000 = 9,860/10,000 = 98.6%
  Notice specificity and accuracy stay high (>90%) in BOTH scenarios here: with only 100 actual frauds against 9,900 actual non-frauds, the negative class dominates the denominator for both metrics, so neither one moves much even as precision swings from 10.6% to 37.5% and recall swings from 95% to 60%. That is exactly why specificity and accuracy are the wrong metrics to lead with on this imbalanced a problem, and why the scenario above prioritizes recall/precision instead.

Trade-offs and thresholding:

• Lowering decision threshold increases predicted positives → raises recall, lowers precision.
• Raising threshold decreases predicted positives → raises precision, lowers recall.
• Use precision-recall curve and choose threshold to meet a target recall (business SLA), possibly optimizing a cost function: Cost = C_FN * FN + C_FP * FP.
• Monitor production metrics (precision at required recall, alert volume) and consider using ranked outputs + manual review for top-K risky transactions to balance operational load.

Additional points:

• F1 useful if you want a single balanced metric, but when costs are asymmetric use recall or a weighted metric (Fβ with β>1 emphasizes recall).

Random train/test splits:

• Randomly sample rows into train and test. Assumes i.i.d. data and exchangeability. Good for static datasets without time dependence but invalid for time-series because it mixes past and future and can produce optimistic estimates.

Holdout test set:

• Reserve a contiguous portion of data (often the most recent period) as a final unseen evaluation set. Used to simulate production performance once models and hyperparameters are fixed. Prevents repeated peeking at test data.

Temporal (rolling / forward-chaining) splits:

• Respect time order. Examples:
  • Holdout: train on t0..tN, validate on tN+1..tM, test on tM+1..tK.
  • Rolling / walk-forward CV: repeatedly train on [t0..ti], validate on [ti+1..ti+h], advance window. Provides robust estimate of performance over time and for model stability.

For next-day demand forecasting I would choose temporal splits (walk‑forward / forward‑chaining) with a final contiguous holdout of the most recent period. Reason: the prediction task is inherently temporal: using future data to predict past inflates performance. Walk‑forward CV mirrors production retraining cadence and reveals degradation over time; the final holdout simulates deployment.

Concrete calendar walkthrough for 3 years of daily demand data (2021-01-01 through 2023-12-31): require at least 12 months of training history per fold so every fold's model has seen a full seasonal cycle (weekly pattern plus the yearly holiday peak) at least once, then advance the window one quarter at a time:

• Fold 1: train 2021-01-01 to 2022-06-30 (18 months), validate 2022-07-01 to 2022-09-30 (Q3 2022).
• Fold 2: train 2021-01-01 to 2022-09-30, validate 2022-10-01 to 2022-12-31 (Q4 2022, which includes the holiday-season demand spike, an important seasonal stress test).
• Fold 3: train 2021-01-01 to 2022-12-31, validate 2023-01-01 to 2023-03-31 (Q1 2023).
• Fold 4: train 2021-01-01 to 2023-03-31, validate 2023-04-01 to 2023-06-30 (Q2 2023).
• Final holdout: 2023-07-01 to 2023-12-31, evaluated once, after all fold-based tuning is frozen.

What a degrading model's walk-forward curve looks like: suppose validation MAPE comes back as 8% (fold 1), 9% (fold 2), 12% (fold 3), 15% (fold 4). A single train/test split would only ever have shown one of these numbers; the walk-forward sequence shows the error creeping from 8% to 15% across four folds, which is the detection mechanism itself; a model whose error is flat or improving across folds is stable, while one that climbs the way this one does is quietly losing relevance to the current data (e.g. because demand patterns have shifted since the earliest training data) and should trigger a retrain or a feature review before it reaches production.

Key leakage risks to avoid:

• Target leakage: including features that contain information that would only be available after the prediction time (e.g., next-day promotions, returns aggregated including the target day).
• Temporal aggregation leakage: engineering aggregates (rolling means, cumulative sums) computed using the full series instead of cutoff at prediction time.
• Leakage via preprocessing: scaling, imputation, or feature selection fit on full dataset rather than within each training fold.
• Data duplication: identical observations across splits (e.g., user IDs repeated) that connect train and test.
• Label propagation: joining external data indexed by future timestamps.

Best practices:

• Always compute features using only past information up to prediction cutoff.
• Use pipeline-aware preprocessing (fit on train fold, apply to validation/test).
• Keep a final chronological holdout for unbiased evaluation.
• Monitor backtests by time slice and track performance drift; retrain frequently if nonstationarity detected.

When teams optimize proxies (e.g., click-through rate, predicted relevance score) instead of the true objective (e.g., lifetime value, user retention), Goodhart’s law kicks in: the proxy ceases to be informative because models and people learn to exploit quirks of the proxy rather than driving real value. In AI systems this appears as reward-hacking (models generating superficially high-scoring outputs), dataset-specialization, or surface-level UI changes that boost metrics but harm retention.

Evaluation regime to detect & mitigate gaming:

1. Define hierarchical objectives

• Primary business KPIs: retention, LTV, revenue per user, safety incidents.
• Secondary proxies: CTR, predicted relevance: explicitly tagged as proxies.

2. Adversarial holdout sets

• Maintain holdout sets crafted adversarially (edge cases, manipulated inputs, distribution shifts).
• Periodically evaluate new models on these sets to reveal brittle optimizations.

3. Randomized audits

• Run randomized, blind human audits across outputs (A/B tests with hidden evaluation teams).
• Include synthetic adversarial prompts designed to reveal overfitting to proxies.

4. Long-term business metrics & cohort analysis

• Measure downstream outcomes (30/90-day retention, churn, refund rates) per experiment cohort.
• Use uplift modeling to attribute long-term impact vs short-term proxy gains.

5. Cross-team guardrails & incentives

• Align incentives: tie engineering/product goals partly to long-term KPIs.
• Require evaluation checklists and sign-offs from analytics, trust/safety, and business stakeholders.

6. Continuous monitoring and anomaly detection

• Instrument metric correlations: monitor cases where proxy improves but business KPIs stagnate or decline.
• Build automated detectors for suspicious patterns (sudden localized improvements, distributional changes, surge in edge-case failures) and trigger rollback/hold.
• Use feature-importance drift, calibration checks, and distributional tests (KS, population stability index).

7. Investigations & remediation workflow

• On anomaly: run forensic tests (batch vs online, example-level inspection, adversarial inputs), freeze rollout if confirmed, and deploy fixes (regularization, adversarial training, reward reshaping, or revised proxy).

Example: A reranker increases CTR by 15% but cohort retention drops. Investigation reveals click-bait snippets; mitigation included adding a downstream retention objective to training, adversarial holdout of click-bait examples, and human audits: restoring aligned improvements.

This regime blends offline adversarial tests, randomized human checks, long-horizon measurement, organizational incentives, and continuous automated monitoring to detect and reduce metric gaming.

Situation: Timestamp-derived features (e.g., "days since last purchase") are powerful but easy to compute incorrectly for validation sets, causing evaluation leakage and overly optimistic metrics.

Problem (how leakage happens)

• Using a global reference time (e.g., "today") when computing features for validation instead of using each example’s cutoff time.
• Computing features over the full dataset (including events after the validation cutoff) so "last purchase" uses future events.
• Recomputing features with label timestamps (target-time leakage).

Traced example: a customer whose validation cutoff_time is 2024-01-15, and whose last real purchase before that date was on 2023-12-01. The correct days-since-last-purchase feature, computed only from events strictly before cutoff_time, is (2024-01-15 minus 2023-12-01) = 45 days. Now suppose this customer also placed a purchase on 2024-02-01, which is after cutoff_time and should never have been visible to a feature computed for a 2024-01-15 prediction. If the feature pipeline used a global reference time (today, at pipeline-run time) instead of each row's own cutoff_time, it would pick up the 2024-02-01 purchase as the last one and compute (2024-01-15 minus 2024-02-01) = -17 days: a negative, nonsensical value that some pipelines silently clip to 0, while others pass the negative number straight through. Either way the model is being handed information it should not have: the fact that this customer purchases again soon after the prediction date, which is exactly the label-correlated signal validation is supposed to prevent from leaking in. The correct feature value for this row is 45, not -17 or 0.

Safeguards / best practices

• Define a clear per-row cutoff time (prediction time) and compute features using only events with event_time < cutoff_time.
• Build time-aware feature transformers that accept cutoff_time and operate via windowed joins or incremental aggregations.
• Use temporal cross-validation (time-based folds) instead of random splits.
• Use an offline feature store or frozen snapshot versioned per date so training and validation use identical code and data slices.
• Enforce immutable pipelines: feature code should be deterministic and not depend on current system time.

Concrete unit / integration tests (examples)

1. Sanity assertion after feature build:

# features_df has columns: entity_id, cutoff_time, last_event_time
assert (features_df['last_event_time'] < features_df['cutoff_time']).all()

2. Maximum-event-time test:

def test_no_future_events(features_df):
    bad = (features_df['last_event_time'] >= features_df['cutoff_time']).any()
    assert not bad, "Feature used events at or after cutoff_time"

3. Replay test with synthetic clock:

• Create synthetic event history, compute features with known cutoff; compare to expected hand-calculated values.

4. Snapshot reproducibility:

• For a fixed snapshot date, run feature pipeline twice and assert identical outputs (idempotency).

5. Integration fold test:

• For each temporal fold, compute metrics and assert model performance does not drastically exceed performance on later real-time evaluation (detects leakage if train metrics >> production).

Why these work

• They ensure feature calculations are strictly causal relative to cutoff_time and make leakage explicit and automatable. Combining time-aware pipelines, temporal CV, and the shown assertions prevents subtle future-data contamination during validation and deployment.

Requirements & goals:

• Evaluate ranking quality (relevance), business outcomes (CTR, revenue, retention), and long-term user satisfaction at 100M DAU with low risk.

Offline metrics & protocol:

• Relevance: NDCG@k (Normalized Discounted Cumulative Gain: rewards a ranked list for placing relevant items near the top, discounts relevant items that appear further down, and normalizes against the best possible ordering so scores are comparable across queries), Recall@k (of all the relevant items that exist, the fraction that showed up anywhere in the top k), MAP@k (Mean Average Precision: the average precision computed at each rank position where a relevant item appears in the top k, then averaged across queries) computed on holdout sessions; use session-level aggregation and per-user temporal splits (train on t, test on t+delta).
• Calibration & confidence: compute confidence intervals via bootstrapping by user.
• Diversity & novelty: catalog-based measures (intra-list diversity, coverage).

Correcting position/exposure bias:

• Propensity scoring via logged exposure probabilities (from serving logs): use IPS (inverse propensity scoring) and SNIPS (Self-Normalized IPS: the same reweighting idea as IPS, but rescaled by the sum of the propensity weights rather than by the raw sample count, which reduces the wild variance plain IPS can have when some items had very low exposure probability) to unbiasedly estimate CTR and NDCG.
• Train position-bias models (e.g., an examination model / PBM) to estimate propensities when they aren't logged directly.
• Use doubly robust estimators combining IPS with outcome models to reduce variance.

Online metrics:

• Immediate: raw CTR, conversion rate, revenue per thousand impressions.
• Short-term engagement: session length, day-over-day retention.
• Long-term value: 7/30/90-day retention, LTV, churn rate, downstream purchases.

Logging schema (must be complete & immutable):

• Event id, user_id (hashed), timestamp, session_id, request_id, placement_id, rank_list (item_ids + positions), served_probabilities (model score, softmax prob), exposure_flag per item, click/engagement events with timestamps, item metadata (owner, category), context (device, region), policy_version, experiment_id, traffic_bucket, reward signals (purchase, watch_time), prior user-state feature snapshot. Ensure deterministic replay keys and a sampling indicator for subsampling.

Counterfactual eval & offline simulator:

• Offline simulator: replay logged requests, simulate alternative policies using logged propensity or importance weights. Include synthetic user-response models learned from logs for stress tests (e.g., adversarial content).
• Use IPS/SNIPS/doubly robust for policy evaluation. Validate simulators by backtesting on historical A/B tests.

A/B testing & system components:

• Experiment platform: traffic allocation, randomization (user-level), exposure logging, kill switch.
• Metrics pipeline: near-real-time aggregator for guardrail metrics, weekly cohort analyses for long-term metrics.
• Policy rollout: staged (canary to ramp), automatic risk checks (statistical significance and business bounds).
• Analysis tools: automated uplift estimation, sequential testing with alpha-spending (a rule for how much of your total false-positive budget you're allowed to spend by peeking at a running experiment's results early, so repeated interim looks don't silently inflate the overall false-positive rate the way naive repeated significance testing would), and variance reduction via stratification/ANCOVA (Analysis of Covariance: adjusting an experiment's outcome metric for a pre-experiment covariate, like each user's prior engagement level, to strip out predictable noise before comparing groups, which tightens confidence intervals without needing more traffic).

How offline evaluation, online A/B tests, and champion-challenger deployment fit together:

• Offline metrics are the fast, cheap FILTER: any new policy must beat the current champion on the offline holdout and the offline simulator (using IPS/SNIPS/DR) before it is allowed anywhere near real traffic. This is the stage that screens out most bad candidates for near-zero cost.
• A/B testing is the causal VALIDATION step: a policy that clears the offline bar gets a randomized, low-traffic online test against the current champion, because even debiased offline estimators can miss position-bias or feedback-loop effects that only appear with real exposure.
• Champion-challenger is the ONGOING PRODUCTION pattern once a challenger has won its A/B test: instead of a full one-shot replacement, the challenger is promoted to serve a small, sustained slice of live traffic (e.g. 5-10%) permanently alongside the incumbent champion, with the same online metrics tracked continuously rather than for a fixed test window. This catches slow drift, seasonality, and small regressions a short A/B window would miss, and gives an instant, no-redeploy rollback (shift traffic back to the champion) if the challenger degrades later. Only after a sustained period of the challenger matching or beating the champion does it get promoted to be the new champion, at which point a fresh challenger can be tested against it.
• Together the three form a funnel of increasing cost and decreasing risk: offline (cheap, many candidates screened, imperfect signal) -> A/B (moderate cost, causal, time-boxed) -> champion-challenger (small ongoing cost, the steady-state safety net that a time-boxed test can't provide).

Long-term impact tracking:

• Cohort-based LTV and retention dashboards, causal impact analyses (difference-in-differences, synthetic controls), monitor content-provider effects and feedback loops (popularity bias).
• Periodic offline retraining with debiased labels and causal features to prevent feedback loops.

Trade-offs & operational notes:

• Logging volume: sample some heavy fields but keep deterministic keys for replay.
• Bias-variance: IPS is unbiased but high variance; prefer doubly robust estimators in production.
• Privacy: hash/anonymize PII; consider differential privacy for aggregate dashboards.

This framework provides unbiased offline evaluation, safe online experimentation, and a champion-challenger steady state to iterate recommendation policies at scale with continuous, low-cost safety monitoring.