Data and Feature Strategy Questions

Explain how you reason about data and feature choices that drive model outcomes. Cover data requirements and sources, annotation and labeling strategies, quality control, and ethical considerations. Discuss feature engineering and representation choices, approaches to handle class imbalance, scarcity, and noisy labels, and techniques such as data augmentation and synthetic data. Describe how you would validate feature usefulness with ablation studies, cross validation, and robustness checks, and how feature and data decisions interact with model selection and evaluation.

MediumTechnical

27 practiced

You added ten new engineered features to a model and accuracy went up. How would you figure out which of those features actually deserve the credit?

Sample Answer

**Approach: Ablation Study**Start from the full feature set and remove one feature (or one logical group) at a time, retraining and re-evaluating each time, to isolate each feature's marginal contribution rather than trusting a single combined accuracy jump.

python

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

def ablation_score(X, y, feature_names, drop_feature, cv=5):
    cols = [c for c in feature_names if c != drop_feature]
    model = RandomForestClassifier(random_state=0)
    scores = cross_val_score(model, X[cols], y, cv=cv, scoring='roc_auc')
    return scores.mean()

baseline = cross_val_score(
    RandomForestClassifier(random_state=0), X, y, cv=5, scoring='roc_auc'
).mean()
drops = {f: baseline - ablation_score(X, y, X.columns, f) for f in new_features}

**Key Points**- A large drop when a feature is removed means it was doing real work; a near-zero drop means another feature was already covering the same signal.- Also check permutation importance as a cheaper complement, since full ablation retrains a model per feature and gets expensive with many features.

**Complexity**Full ablation is O(n) retrains for n features (or feature groups), so group correlated features together to keep it tractable on large feature sets.

**Edge Cases**Two correlated new features can each show a small individual ablation drop but a large joint drop if removed together, so also test removing correlated pairs as a group, not just one at a time.

MediumTechnical

31 practiced

You're doing feature selection on a dataset with a strong seasonal, time-based pattern. Why might standard k-fold cross-validation mislead you here, and what would you do instead?

HardSystem Design

28 practiced

You're asked to design the data and feature strategy for a recommendation model in a brand-new product vertical, where you have almost no historical interaction data and the item catalog changes weekly. How would you approach it?

Sample Answer

**Framing the Constraints**Two compounding problems: cold start (new users and new items have no interaction history) and non-stationarity (the catalog itself changes weekly, so any learned item-specific pattern goes stale fast).

**Data Strategy**- Lean on content-based features that exist before any interaction happens: item metadata (category, price, text description embeddings), user profile attributes, and context (time of day, device), since these don't require interaction history to compute.- For items with zero interactions, fall back to a rules-based or popularity-within-category recommender until enough interaction data accumulates, and log which recommendations were served this way so it's measurable, not invisible.

**Feature Engineering**- Represent items via content embeddings (e.g. from item descriptions) rather than learned ID-based embeddings, since ID embeddings need repeated exposure to train and are exactly what's missing for new items.- Build user features from cross-vertical behavior if the user has history elsewhere in the product, treating this as a transfer-learning problem rather than starting from zero.

**Validation Approach**- Standard offline metrics like AUC on historical interactions are misleading here, since the catalog changes weekly and old interactions describe items that may no longer exist.- Validate primarily through time-respecting backtests on a rolling window, and treat the first few weeks of real online experimentation as part of the validation plan, not just a launch.

**Trade-offs and What Would Change My Answer**Content-based cold-start features are lower variance but often lower ceiling than collaborative filtering once enough interaction data exists; I would plan an explicit transition point (e.g. once an item has N interactions or M weeks of exposure) to blend in collaborative signal, and monitor whether that transition point is well calibrated rather than assuming it upfront.

**Common Pitfall**Optimizing purely for offline accuracy on the sparse historical data, which overfits to whichever few items happen to have interactions, instead of accounting for the catalog's constant churn.

EasyTechnical

24 practiced

Before you let a new dataset feed into model training, what quality checks do you actually run on it?

MediumTechnical

26 practiced

A stakeholder suggests generating synthetic data with a GAN or a large language model to solve your data scarcity problem. How do you evaluate whether that's actually a good idea?

Unlock Full Question Bank

Get access to hundreds of Data and Feature Strategy interview questions and detailed answers.

Join thousands of developers preparing for their dream job.