Data Preparation and Class Imbalance for ML Questions

Direct answer

Split the label vector into folds so each test fold's class proportions approximately mirror the overall dataset, then add a fallback for the case where a class has fewer examples than there are folds, since perfect stratification is impossible when a class's count is smaller than the fold count.

Structured elaboration

import numpy as np
from sklearn.model_selection import StratifiedKFold

def stratified_cv_indices(y, n_splits=5, random_state=None):
    """Yield (train_idx, test_idx) with every test fold containing at least one positive,
    when possible given the label counts."""
    y = np.asarray(y)
    n_positive = int(y.sum())
    effective_splits = min(n_splits, max(n_positive, 1))  # never ask for more folds than positives available
    skf = StratifiedKFold(n_splits=effective_splits, shuffle=True, random_state=random_state)
    for train_idx, test_idx in skf.split(np.zeros(len(y)), y):
        yield train_idx, test_idx

y = np.array([0]*95 + [1]*5)
for i, (train_idx, test_idx) in enumerate(stratified_cv_indices(y, n_splits=5, random_state=0)):
    print(f"fold {i}: test positives = {y[test_idx].sum()}, test size = {len(test_idx)}")

When positives are fewer than the requested n_splits, StratifiedKFold itself would raise an error asking for more folds than the rarest class has members; capping effective_splits at the positive count (with a floor of 1) sidesteps that by never requesting more folds than can genuinely each get at least one positive.

Worked example

Running the code above on 100 rows with 5 positives and n_splits=5 gives 5 folds, each with exactly 1 positive test example (since 5 positives split evenly across exactly 5 folds); reducing to only 3 positives with the same n_splits=5 request automatically falls back to effective_splits=3, still guaranteeing at least one positive per test fold rather than erroring out or silently producing an empty-positive fold.

Trade-offs and pitfalls

Falling back to fewer folds than requested is a real trade-off, not a free fix: fewer, larger folds mean less granular cross-validation and a slightly less reliable estimate of variance across folds, so if the positive count is small enough that this fallback triggers, it's worth treating the resulting metric estimates with extra caution (or supplementing with a bootstrap confidence interval) rather than trusting the standard cross-validation summary statistics at face value.

Direct answer

Favor an unsupervised anomaly detector when you have little or no reliable labeled data for the rare class and are trying to catch NOVEL patterns you haven't seen before; favor a supervised classifier when you have enough labeled examples of the specific pattern you're targeting and want to optimize directly for catching more of exactly that pattern.

Structured elaboration

Decision criteria:

• Label availability and quality: supervised classification needs enough labeled positives to learn a real decision boundary; below some threshold (a handful to a few dozen confirmed examples), a supervised model has too little signal, and an unsupervised "what does normal look like" approach makes better use of the abundant unlabeled negative data instead.
• Novelty of the pattern: if attackers, fraudsters, or failure modes evolve and you need to catch things you've never labeled before, unsupervised approaches generalize to novel patterns better, since they're not fit to any specific known positive shape; a purely supervised classifier trained only on past labeled fraud will systematically miss fraud that looks different from what it learned.
• Operational constraints: a supervised classifier gives you a single, directly-optimized precision/recall trade-off you can tune; an unsupervised anomaly score requires you to separately calibrate what threshold on "anomalousness" corresponds to an acceptable alert volume, which is less direct but doesn't require labels to set up in the first place.

In production, these are rarely either/or: run both and combine, using the unsupervised score as an additional FEATURE fed into the supervised model (so the supervised model gets to learn how much to trust the anomaly signal), or run the unsupervised detector as a first-pass filter over a huge volume of events before a more expensive supervised model or human reviewer looks at the smaller flagged subset.

Worked example

For a new fraud pattern with only 3 confirmed cases so far, an unsupervised Isolation Forest trained on the abundant "normal" transaction data can flag transactions that look statistically unusual along any dimension, without needing to have seen this exact pattern before; a supervised classifier trained on those same 3 examples would essentially be memorizing 3 points, with almost no ability to generalize to a fourth, differently-shaped case.

Trade-offs and pitfalls

The common mistake is picking one approach permanently rather than revisiting the choice as labels accumulate: what starts as an unsupervised-first system, appropriate when labels are scarce, should graduate toward a supervised or hybrid approach once enough confirmed labels exist to make supervised learning worthwhile, since a well-tuned supervised classifier will generally outperform an unsupervised one on the SPECIFIC pattern it has plenty of labels for.

Direct answer

Frame it as a dial, not a switch: every extra fraud case you catch by loosening the model comes with more false alarms for the review team to work through, and there's a point past which the added review burden costs more than the fraud you're preventing. My job is to help you find where that point should sit, not to promise "catch everything" is free.

Structured elaboration

Translate the trade-off into the vocabulary the stakeholder already uses:

• Recall (the fraction of real cases you catch) is what "catch every case" is asking for.
• Precision (the fraction of your alerts that are real) is what determines how much wasted reviewer time, customer friction, or false-decline revenue loss you generate along the way.

A concrete way to make it tangible: state the current operating point in business units ("today we catch 80% of fraud and our reviewers close 40% of alerts as false positives, costing about 6 reviewer-hours per confirmed fraud case"), then show what pushing recall to 95% would do to that ratio using the model's actual precision-recall curve, not a hypothetical.

Then propose a concrete plan rather than leaving it as an abstract trade-off: agree on a target operating point tied to a resource constraint (how many alerts the review team can actually process per day), commit to it as an SLA, and set up a monitoring dashboard so both sides can see recall, precision, and reviewer load drift over time and revisit the threshold deliberately instead of by surprise.

Worked example

If the review team can handle 200 alerts a day and the model's precision at "catch everything" settings drops to 10%, that's 200 alerts producing only about 20 real cases and 180 false alarms consuming the entire day's capacity, versus a calibrated threshold that might catch 85% of fraud at 45% precision, producing far fewer false alarms per real case caught.

Trade-offs and pitfalls

Three scenarios worth naming explicitly to make the trade-off concrete for a stakeholder: recall should usually win for disease screening (missing a real case is far worse than a false alarm that a follow-up test resolves) and for safety-critical detection; precision should usually win for a marketing-offer trigger or a low-stakes automated action (annoying a customer or wasting a small amount of budget on a false positive adds up fast at scale). The pitfall is presenting this as a purely technical choice: the right operating point is a business decision informed by the technical trade-off curve, and the stakeholder should own the final call once they can see the real numbers.

Direct answer

Combine rule-based normalization for the common, predictable variants with fuzzy matching for the long tail, route genuinely ambiguous cases to human review, and store the resulting canonical mapping somewhere versioned and reusable so the same standardization applies consistently across every pipeline that touches this field, not just the one that happened to build it.

Structured elaboration

Rule-based normalization first: lowercase, strip whitespace and punctuation, expand known abbreviations via an explicit lookup table (NY → New York, if that direction is the canonical one you've chosen), catching the majority of common variants cheaply and deterministically.

Fuzzy matching for what rules miss: for values that don't match any known rule, score them against the existing canonical values using a string-similarity metric, and auto-accept matches above a high-confidence threshold.

Human-in-the-loop for ambiguous cases: values that fuzzy-match multiple canonical candidates similarly well, or fall in a middle-confidence band, go to a reviewer rather than being auto-resolved by a coin-flip-equivalent threshold decision; each reviewer decision should feed back into either the rule table (if it reveals a new common pattern worth automating) or the canonical mapping directly.

Storing and versioning the mapping: keep the canonical value table (raw variant to normalized value) in a shared, version-controlled location (a database table with an audit log, or a versioned config file) rather than embedded logic scattered across multiple pipelines, so every downstream consumer of this field, not just the pipeline that built the mapping, applies the exact same standardization, and any future correction to the mapping is automatically reflected everywhere it's used rather than requiring a hunt through every pipeline that independently reimplemented the same logic.

Worked example

"NY", "N.Y.", "new york", and "NewYork" all normalize, via a mix of a rule (case-insensitive, punctuation-stripped exact match against a known abbreviation table) and fuzzy fallback, to a single canonical "New York"; a genuinely ambiguous input like "Georgia" (the US state, or the country) would fall outside straightforward automated resolution and route to a reviewer with the surrounding context (other fields in the same record) to disambiguate.

Trade-offs and pitfalls

The most common failure mode in a standardization pipeline like this isn't the matching logic itself, it's DRIFT: a canonical mapping built once and never revisited slowly falls out of sync as new source systems introduce new variants, so treating the mapping table as a living, versioned artifact with an owner and a process for adding new entries (rather than a one-time script output) is what keeps the standardization actually correct over time.

Direct answer

Behavioral answer, using the STAR structure: describe a specific situation where a preprocessing or data-quality decision (an imputation choice, a resampling strategy, a fix to a broken date-parsing routine) turned out to materially change a model's behavior or a business outcome, and walk through the reasoning, the validation, and the lesson learned.

Structured elaboration

Situation: set up the concrete context, what model or report was affected and why it mattered (a churn model whose predictions were being used to prioritize a retention team's outreach).

Task: what you were responsible for, specifically (diagnosing why the model's predictions seemed off, or improving a pipeline you inherited).

Action: the actual preprocessing decision and the reasoning behind it, described concretely (discovering that a "days since last purchase" feature was being computed with a bug that included future purchases for churned customers, a leakage bug; or deciding that median imputation on an income field was distorting predictions for a specific segment and switching to a grouped, segment-aware imputation instead).

Result: the measurable outcome, tied to real numbers where possible (validation AUC improved from X to Y, or the retention team's outreach precision improved by Z%, or a specific downstream business metric moved), plus what you validated to confirm the improvement was real (an A/B test, a time-based holdout, not just a training-set number).

Worked example

"I inherited a churn model whose predictions correlated suspiciously well with a 'days_to_cancel' feature. Tracing the feature's derivation, I found it was computed from the FULL customer history including records after cancellation, a leakage bug. I removed the feature, retrained, and validation AUC dropped from an unrealistic 0.94 to a more modest but honest 0.79. I then worked with the team to build genuinely predictive features from PRE-cancellation behavior only, recovering AUC to 0.83 over the following two weeks, a number the team could actually trust for prioritizing outreach, versus the original number which would have looked great in a demo and failed in production."

Trade-offs and pitfalls

The strongest version of this answer names a decision that could have gone WRONG, and shows the reasoning and validation that confirmed it was right, rather than a story where the "right" choice was obvious in hindsight; interviewers are listening for genuine judgment under uncertainty, not just a happy ending.