Data Preparation and Class Imbalance for ML Questions

Direct answer

Use blocking to avoid comparing every pair of records, a string-similarity metric to score candidate pairs within each block, a threshold to decide auto-merge versus manual review, and measure precision and recall of the merges against a labeled sample to know how well the whole pipeline is actually working.

Structured elaboration

Blocking: instead of comparing every record against every other record (an $O(n^2)$ comparison that's infeasible past a few tens of thousands of rows), group records into smaller candidate BLOCKS likely to contain true duplicates (same first 3 letters of last name, same zip code, same phone-number prefix), and only compare within each block, cutting the number of comparisons dramatically at the cost of possibly missing a true duplicate that landed in different blocks due to a genuinely different key.

Similarity metric: for name and address fields specifically, a string-edit-distance metric like Levenshtein distance handles typos well, while a token-set-based metric (token-set ratio, comparing the SET of words regardless of order) handles reordering better ("John Smith" versus "Smith, John"); combining both, or a weighted blend, tends to catch more real duplicates than either alone.

Threshold and manual review: set a similarity threshold above which pairs are auto-merged, a lower threshold below which pairs are confidently NOT duplicates, and a middle band routed to manual review, since forcing every borderline case into an automated auto-merge-or-reject binary decision is where most of the real error comes from.

Measuring precision and recall of merges: on a labeled sample of pairs (some true duplicates, some true non-duplicates, ideally including hard cases near the threshold), precision tells you what fraction of AUTO-merged pairs were actually correct, and recall tells you what fraction of TRUE duplicates the pipeline actually caught (including any it missed due to blocking or a too-conservative threshold), and you tune the threshold against this measured trade-off rather than guessing a value.

Worked example

For "Jon Smith, 123 Main St" and "John Smith, 123 Main Street," a Levenshtein-based comparison catches the "Jon"/"John" typo but might score the address pair only moderately (since "St" versus "Street" is a real edit distance), while a token-set comparison on the address handles that abbreviation difference more gracefully; combining both signals into a blended score is often more robust than trusting either metric alone.

Trade-offs and pitfalls

The blocking strategy itself is a real accuracy lever, not just a performance optimization: an overly narrow block key (requiring an EXACT zip-code match) can silently drop true duplicates that differ on the blocking key itself (a customer whose zip code was mistyped in one of the two records), so validating recall specifically includes checking whether the chosen blocking strategy itself is excluding true duplicate pairs before they ever reach the similarity-scoring step.

Direct answer

Standard k-fold shuffles rows randomly across folds, which for time-series data means training on the future to predict the past, information a real deployment would never have. Walk-forward (rolling-origin) validation and an expanding window both fix this by always keeping training data strictly earlier in time than validation data.

Structured elaboration

Walk-forward / rolling-origin: train on a fixed-size window of the most recent past, validate on the next period, then slide the whole window forward and repeat. Expanding window: keep the training start fixed but grow the training window forward each round, so each successive fold trains on more history than the last while validation always stays just ahead of it.

def walk_forward_splits(n, n_splits, val_size):
    """Yield (train_idx, val_idx) using an expanding training window."""
    fold_size = (n - val_size) // n_splits
    for i in range(n_splits):
        train_end = fold_size * (i + 1)
        val_start, val_end = train_end, min(train_end + val_size, n)
        if val_end <= val_start:
            break
        yield list(range(0, train_end)), list(range(val_start, val_end))

for train_idx, val_idx in walk_forward_splits(n=1000, n_splits=5, val_size=100):
    print(f"train=[0,{train_idx[-1]}]  val=[{val_idx[0]},{val_idx[-1]}]")

If the target is ALSO rare within this time-ordered setting (equipment failures, fraud), preserving an approximate positive-class ratio per fold matters too: rather than a purely chronological cut that might leave one window with almost no positives, you can widen the training window for that fold until it captures a reasonable minimum count of positive examples, trading strict recency for a usable minority sample.

Worked example

Running the code above prints five expanding windows: train=[0,179] val=[180,279], train=[0,359] val=[360,459], and so on through five folds, each one training on strictly earlier data than the validation window that follows it, with training set size growing each round while validation size stays fixed at 100.

Trade-offs and pitfalls

Common pitfalls: allowing ANY feature that uses future information (a rolling average computed with a centered window instead of a trailing one, or a label-derived aggregate) to leak across the train/validation boundary even though the ROW split itself is chronologically correct; and forgetting that walk-forward validation naturally produces fewer, larger folds than standard k-fold for the same dataset, so per-fold metric variance needs to be interpreted with that in mind.

Direct answer

Naive random oversampling copies existing minority examples exactly, so the model can end up memorizing those specific duplicated points rather than learning the general pattern behind them, especially when a model with enough capacity is trained for many epochs over the same small set of unique minority examples repeated many times.

Structured elaboration

Three mitigation strategies and the mechanism behind each:

1. Synthetic oversampling (SMOTE and variants) instead of exact duplication: interpolating between minority points generates NEW feature combinations rather than exact repeats, so the model can't simply memorize a fixed set of duplicate rows, it has to generalize across a genuinely larger effective set of distinct points.
2. Stronger regularization: increasing L2 penalty, adding dropout, or reducing model capacity directly counteracts the model's tendency to fit training data too closely, which matters more when duplicated points give the optimizer repeated, easy opportunities to overfit to them.
3. Careful validation-set construction: ensuring the validation set is drawn from the ORIGINAL, non-oversampled distribution (never itself oversampled) is what actually lets you DETECT this overfitting in the first place; a validation set built the same duplicated way as training would mask the exact problem you're trying to catch.

Worked example

If a minority class has only 20 unique examples and you oversample to 2,000 total minority rows via exact duplication, each unique point is now repeated 100 times; a sufficiently flexible model (a deep tree, a large neural network) can, given enough training passes, essentially memorize those 20 specific points and their labels rather than learning a decision boundary that generalizes to a 21st, unseen minority example.

Trade-offs and pitfalls

The most reliable signal that this is happening is a large gap between training-set minority performance (near-perfect) and validation-set minority performance (much worse), specifically for the minority class rather than overall accuracy, since overall metrics can look fine even while the model has essentially memorized the tiny duplicated minority set.

Direct answer

PCA finds directions of maximum VARIANCE, and variance is measured in the feature's own units, so an unscaled feature with a naturally larger numeric range (income in dollars versus age in years) will dominate the principal components purely because of its units, not because it's actually more informative.

Structured elaboration

PCA computes eigenvectors of the feature covariance matrix, and covariance/variance scale with the SQUARE of a feature's units; a feature measured in the thousands (like income) has a variance orders of magnitude larger than a feature measured in single digits or tens (like age or years of tenure) purely from the unit choice, with no relationship to which feature actually carries more useful signal.

Worked example

Consider two features, "age" (values in the tens, say variance $\approx 100$) and "income" (values in the tens of thousands, say variance $\approx 10^{8}$): without scaling, the first principal component will be overwhelmingly dominated by income's variance almost regardless of the actual correlation structure between the two features, since income's raw variance is roughly a million times larger than age's; after standardizing both features to unit variance, PCA can find the genuinely most-informative combination of the two based on their correlation structure, not their arbitrary original units.

Trade-offs and pitfalls

Acceptable to skip scaling before PCA specifically when every feature is ALREADY on a comparable, meaningful scale (all features are, say, normalized pixel intensities in the same range, or all are already z-scored counts of the same type of event), where the relative variance differences genuinely reflect differences in variability rather than an artifact of differing units; in that specific case, scaling again could actually remove real signal (a feature that's LEGITIMATELY more variable, in comparable units, is exactly what PCA should be allowed to notice).

Direct answer

Class imbalance means one class (usually the outcome you actually care about, like fraud or churn) makes up a small fraction of the data. It matters because accuracy rewards the model for predicting the majority class every time, so a model that never fires can still score very high while being useless.

Structured elaboration

Take a binary problem with a 1% positive rate. A model that predicts "negative" unconditionally, with zero understanding of the data, gets 99% accuracy. That number tells you nothing about whether the model can find the positives you actually built it to find. Signs that imbalance is hurting you in practice: recall on the positive class near zero while accuracy looks fine, a model that converges to predicting a single class, or a confusion matrix with almost no entries in the true-positive cell.

It also helps to separate three related but distinct problem shapes, because they call for different remediation:

• Class imbalance: both classes are well-defined and you have real labeled examples of each, just in very different proportions.
• Rare-event forecasting: you are trying to predict something that happens infrequently in time (equipment failure, a spike), where the temporal structure matters as much as the class ratio.
• Anomaly detection: you may not even have labeled examples of the rare class, and are instead modeling what "normal" looks like and flagging deviations.

Naturally-imbalanced problems are common and mostly follow this rough shape: fraud detection (well under 1% positive), churn prediction (single-digit percent), and rare-disease screening (often far below 1%).

Worked example

With 10,000 examples and a 1% positive rate (100 positives, 9,900 negatives), a model that predicts every example negative gets $TP=0,\ FN=100,\ TN=9900,\ FP=0$, so accuracy is $9900/10000 = 0.99$, a 99% accuracy score, while recall on the class you care about is $0/100 = 0$.

Trade-offs and pitfalls

The immediate remediation to try first is almost never a fancy technique. It's swapping your primary metric away from accuracy (to recall, precision-recall AUC, or a business-cost-weighted metric) so you can actually see the problem, before touching the data or the loss function at all. The common mistake is jumping straight to resampling or a fancy loss function without first confirming, via the right metric, that imbalance is the actual bottleneck rather than a genuinely hard signal-to-noise problem.