Handling Class Imbalance & Special Modeling Scenarios Questions

Techniques for building and evaluating machine learning models when confronted with imbalanced datasets and other specialized modeling scenarios. Includes data-level methods (oversampling, undersampling, SMOTE and variants), algorithmic approaches (class weights, focal loss, cost-sensitive learning), evaluation strategies and metrics suited for imbalanced problems (precision-recall AUC, F1, balanced accuracy), threshold tuning, calibration, and robust validation (stratified cross-validation). Also covers anomaly/rare-event detection, multi-class and multi-label considerations, and practical production considerations such as model monitoring, fairness implications, and deployment trade-offs in skewed data settings.

EasyTechnical

27 practiced

Explain SMOTE (Synthetic Minority Over-sampling Technique) conceptually. Describe common pitfalls when applying SMOTE (e.g., creating samples across class boundaries, handling categorical or sparse features) and practical mitigations (SMOTENC, combining with cleaning methods, pipeline placement).

MediumTechnical

28 practiced

Given numpy arrays y_true (binary) and y_prob (predicted probability), implement Python code to find the probability threshold that maximizes F1 while ensuring precision >= 0.9. If no threshold satisfies the precision constraint, return None. Explain algorithmic complexity and how to scale this to large datasets.

Sample Answer

Approach: sort predictions by decreasing probability and compute cumulative true positives and false positives to get precision/recall for every distinct threshold (unique probabilities). Compute F1 where precision >= 0.9 and pick the threshold that yields maximum F1. This is exact, vectorized, and O(n log n) due to sorting.

python

import numpy as np

def best_threshold_max_f1(y_true, y_prob, min_precision=0.9):
    """
    Return threshold (float) that maximizes F1 while precision >= min_precision.
    If no threshold meets precision constraint, return None.
    """
    y_true = np.asarray(y_true).astype(int)
    y_prob = np.asarray(y_prob).astype(float)
    if y_true.size == 0:
        return None

    # Sort by probability descending
    order = np.argsort(-y_prob)
    probs_sorted = y_prob[order]
    labels_sorted = y_true[order]

    # Cumulative true positives at each cut (predict positive for prob >= threshold)
    tp_cum = np.cumsum(labels_sorted)               # TP if threshold just below current prob
    preds_cum = np.arange(1, len(labels_sorted) + 1)  # number predicted positive at each cut
    fp_cum = preds_cum - tp_cum

    # Total positives to compute recall
    total_pos = tp_cum[-1]
    if total_pos == 0:
        return None  # no positive labels => undefined F1

    precision = tp_cum / preds_cum
    recall = tp_cum / total_pos
    # F1: avoid division by zero
    denom = precision + recall
    f1 = np.where(denom > 0, 2 * precision * recall / denom, 0.0)

    # For thresholds use the current prob value (predict >= prob)
    # But only consider unique probability values to avoid redundant thresholds
    unique_probs, first_idx = np.unique(probs_sorted, return_index=True)
    # first_idx gives first position where that prob appears in descending-sorted array
    candidate_prec = precision[first_idx]
    candidate_f1 = f1[first_idx]
    candidate_probs = unique_probs  # descending order because probs_sorted was descending

    # Filter by precision constraint
    mask = candidate_prec >= min_precision
    if not np.any(mask):
        return None

    best_idx = np.argmax(candidate_f1 * mask)  # masked argmax: non-qualified become zero
    return float(candidate_probs[best_idx])

Key points:- Time: O(n log n) dominated by sorting. Space: O(n).- Edge cases: no positives in y_true -> return None; ties handled by taking unique probabilities.- Scaling: for very large datasets, avoid full sort—use binning (e.g., 1000 quantile buckets) or streaming histograms of predicted probabilities to approximate thresholds; or compute precision-recall curve with distributed aggregations (map-reduce) by bin counts per probability bucket. If exact high resolution needed, sort can be parallelized (external or distributed sort).

MediumTechnical

27 practiced

Discuss ethical and fairness implications of oversampling underrepresented demographic groups by generating synthetic examples. When might synthetic oversampling help fairness, when could it worsen harms, and what safeguards (auditing, subgroup evaluation, provenance tracking) should be implemented before deploying such a model?

Sample Answer

Situation: As an ML engineer deciding whether to synthetic-oversample underrepresented demographic groups, weigh both potential fairness gains and ethical risks before deployment.

When synthetic oversampling can help fairness:- Improve representation for rare groups so models learn relevant patterns rather than defaulting to majority behavior (e.g., medical imaging where few samples from a demographic lead to misdiagnosis).- Reduce sampling bias that causes systematic underperformance, enabling better subgroup calibration and lower disparate error rates.- Enable privacy-preserving augmentation when raw minority data is sensitive and cannot be shared.

When it can worsen harms:- Synthetic examples that imperfectly capture causal structure or social context can introduce spurious correlations—models may learn artifacts, amplifying bias.- If synthetic data overrepresents synthetic patterns relative to real-world prevalence, it may produce unrealistic decisions or mask real-world distributional risks.- Poorly generated samples can encode and magnify existing labeling errors or stereotypes present in the minority subset.

Safeguards before deployment:- Subgroup evaluation: report metrics (accuracy, precision/recall, calibration, false positive/negative rates) per demographic and intersectional subgroups; track uncertainty and confidence shifts.- Robust auditing: run counterfactual and stress tests (distribution shifts, adversarial examples), causal sensitivity analysis, and fairness metrics (equalized odds, predictive parity) to detect regressions.- Provenance & versioning: record data lineage (source, generation method, generator model version, hyperparameters), and link synthetic samples to original records where appropriate (and privacy-preserving).- Human review & domain validation: have domain experts assess realism and ethical suitability of synthetic data and downstream decisions.- Conservative deployment: pilot in low-risk settings, monitor real-world performance and user impact, and include rollback triggers if harms emerge.- Transparency & consent: disclose use of synthetic data to stakeholders and ensure compliance with legal/privacy requirements.

Trade-offs: weigh improved statistical parity vs. risk of artifact-driven harm. Use synthetic oversampling as one tool within a larger fairness engineering process, not as a silver bullet.

HardTechnical

46 practiced

Propose how to use tabular GANs (e.g., CTGAN) to generate synthetic minority-class records to augment training. Describe preprocessing steps, conditioning strategies, evaluation metrics for fidelity and utility (e.g., classifier two-sample tests, downstream performance), and privacy risks including membership inference.

Sample Answer

Approach summary: use a tabular GAN (e.g., CTGAN) to oversample the minority class by training the generator to model the joint distribution of features conditioned on class label, then synthesize minority-class records to augment training. Key goals: fidelity (synthetic looks real), utility (improves downstream model), and preserve privacy.

Preprocessing- Clean and normalize: impute missing values (or flag them), apply consistent categorical encoding (CTGAN uses one-hot/embedding internally) and scale continuous features (QuantileTransformer or Gaussianize helps CTGAN).- Preserve data types and metadata: mark categorical vs continuous, ordinal info, ranges, and constraints (e.g., sums).- Stratified split: hold out a test set of real minority and majority records not seen by GAN for evaluation and downstream training.

Conditioning strategies- Class-conditional GAN: condition generator and discriminator on class label to directly sample minority-class distributions.- Hierarchical conditioning: condition on coarse strata (e.g., class + demographic bucket) to capture multimodality.- Auxiliary losses: add feature-specific reconstruction or mutual information losses to preserve important correlations.- Balanced training: oversample minority real examples per epoch to avoid mode collapse; use class-aware sampling in discriminator batches.

Evaluation: fidelity and utility- Two-sample tests: classifier two-sample test (train a classifier to distinguish real vs synthetic)—AUC near 0.5 implies high fidelity. Also use MMD and KS tests per-feature.- Distributional checks: compare marginal and pairwise joint distributions, conditional distributions P(feature|class).- Statistical metrics: Chi-square for categoricals, Wasserstein or sliced-Wasserstein for numericals.- Downstream utility: train target model on (real+synthetic) vs real-only and compare metrics (precision/recall, F1, AUROC), especially for minority-class recall.- Robustness checks: calibrate and test on held-out real minority data; check for introduced bias or spurious correlations.

Privacy risks and mitigations- Risks: memorization leading to membership inference or direct leakage of rare records. Overfitting GANs and small minority sets amplify risk.- Detection: membership inference attacks (train shadow models), nearest-neighbor overlap between synthetic and training data.- Mitigations: differential privacy (DP-SGD) during training with tuned epsilon, gradient clipping; limit epochs; enforce sampling thresholds (reject near-duplicates), post-hoc k-anonymity or synthetic record filtering, and audit with MI attack simulations.- Trade-offs: DP reduces fidelity/utility—tune privacy budget and evaluate downstream impact.

Operational considerations- Monitor drift between synthetic and live data; version synthetic datasets; keep human-in-the-loop validation for rare-case plausibility; document provenance and privacy guarantees.

HardTechnical

30 practiced

Implement class-balanced focal loss in PyTorch using the 'effective number' weighting scheme where class weight = (1 - beta) / (1 - beta^n). Your implementation should accept per-class counts, beta hyperparameter, gamma, and support logits input. Provide a numerically stable implementation and note complexity considerations.

Sample Answer

Approach: compute per-class weight using the "effective number" formula w_c = (1 - beta) / (1 - beta^n_c), normalize or scale optional; compute standard focal loss but operate on logits via log-softmax for numerical stability. Multiply per-example loss by class weight. Support multi-class (C-way) classification with integer class labels.

python

import torch
import torch.nn.functional as F
from torch import nn

class ClassBalancedFocalLoss(nn.Module):
    def __init__(self, class_counts, beta=0.9999, gamma=2.0, reduction='mean', eps=1e-8):
        """
        class_counts: iterable of length C with class sample counts (ints or floats)
        beta: hyperparameter for effective number (near 1)
        gamma: focal loss gamma
        reduction: 'mean', 'sum', or 'none'
        eps: small constant for numerical stability
        """
        super().__init__()
        counts = torch.tensor(class_counts, dtype=torch.float32)
        if (counts <= 0).any():
            raise ValueError("class_counts must be positive")
        effective_num = 1.0 - torch.pow(beta, counts)
        weights = (1.0 - beta) / (effective_num + eps)
        # Normalize weights to sum to C (optional but often helpful)
        weights = weights / weights.sum() * counts.numel()
        self.register_buffer('class_weights', weights)  # shape (C,)
        self.gamma = float(gamma)
        self.reduction = reduction
        self.eps = eps

    def forward(self, logits, targets):
        """
        logits: (N, C) raw model outputs
        targets: (N,) long with class indices in [0, C-1]
        """
        if logits.dim() != 2:
            raise ValueError("logits must be shape (N, C)")
        N, C = logits.shape
        if targets.dim() != 1 or targets.shape[0] != N:
            raise ValueError("targets must be shape (N,)")

        # log-probs for numerical stability
        log_probs = F.log_softmax(logits, dim=1)  # (N, C)
        probs = torch.exp(log_probs)  # (N, C)

        # Gather per-sample log_prob and prob of the true class
        t = targets.view(-1, 1)
        pt = probs.gather(1, t).clamp(min=self.eps)          # (N,1)
        log_pt = log_probs.gather(1, t)                      # (N,1)

        # focal modulation: (1 - p_t)^gamma
        focal_factor = (1.0 - pt) ** self.gamma

        # per-class weight for each sample
        cw = self.class_weights.view(1, -1).gather(1, t)     # (N,1)

        # loss per sample
        loss = -cw * focal_factor * log_pt                   # (N,1)
        loss = loss.view(-1)

        if self.reduction == 'mean':
            return loss.mean()
        elif self.reduction == 'sum':
            return loss.sum()
        else:
            return loss  # shape (N,)

Key points:- Uses log_softmax + gather for numerical stability; avoids computing (1 - softmax) from logits directly.- Class weights computed with effective number; optional normalization prevents extremely large/small magnitudes.- Complexity: O(N*C) time and O(C) extra memory for weights. If C is large, compute log_softmax still costs O(N*C). For sparse targets or very large C, consider sampled softmax or hierarchical softmax.- Edge cases: zero counts disallowed; beta close to 1 can cause numerical cancellation so eps added; supports reduction modes.

Unlock Full Question Bank

Get access to hundreds of Handling Class Imbalance & Special Modeling Scenarios interview questions and detailed answers.

Join thousands of developers preparing for their dream job.