Feature Engineering and Selection Questions

Covers the end to end process of transforming raw data into predictive and stable model inputs and choosing the most useful subset of those inputs. Topics include generating features from domain signals and timestamps, numerical transformations such as scaling binning and logarithmic transforms, categorical encodings including one hot and target encoding, creation of interaction and polynomial features, construction of dense feature vectors for model consumption, handling missing values and outliers, and strategies for class imbalance. Also includes feature selection and dimensionality reduction methods such as filter techniques statistical tests wrapper methods embedded model based selection mutual information analysis and tree based importance measures. Emphasis is placed on avoiding data leakage validating feature stability over time interpreting feature contributions and documenting rationale for feature creation or removal. For senior roles include designing feature engineering best practices mentoring others and considering feature impact on model interpretability and business metrics.

MediumTechnical

27 practiced

Given an events table in PostgreSQL:

Write a SQL query (Postgres) that computes per-user features for the last 30/7/90 day windows: total_events_30d, avg_amount_7d, days_since_last_event, and count_distinct_event_types_90d as of a given reference timestamp :ref_ts. Explain indexes and performance considerations to run this as a daily batch job for millions of users.

Sample Answer

Approach: aggregate events in three rolling windows relative to :ref_ts using filtered aggregates. Use a single scan of recent events (occurred_at >= :ref_ts - interval '90 days') and GROUP BY user_id to compute all features.

sql

WITH recent AS (
  SELECT user_id, event_type, amount, occurred_at
  FROM events
  WHERE occurred_at >= :ref_ts - interval '90 days'
    AND occurred_at <= :ref_ts
)
SELECT
  u.user_id,
  COUNT(*) FILTER (WHERE occurred_at >= :ref_ts - interval '30 days')            AS total_events_30d,
  AVG(amount) FILTER (WHERE occurred_at >= :ref_ts - interval '7 days')          AS avg_amount_7d,
  DATE_PART('day', :ref_ts - MAX(occurred_at))                                   AS days_since_last_event,
  COUNT(DISTINCT event_type) FILTER (WHERE occurred_at >= :ref_ts - interval '90 days') AS count_distinct_event_types_90d
FROM recent r
RIGHT JOIN (SELECT DISTINCT user_id FROM users) u USING (user_id) -- include users with 0 events
GROUP BY u.user_id;

Key points:- Single scan of events within 90 days reduces I/O.- Use FILTER to compute windowed aggregates in one GROUP BY.- RIGHT JOIN with users ensures zero-event users appear (adjust if not needed).

Indexes & performance for daily batch over millions of users:- Primary index: CREATE INDEX ON events (occurred_at) or (occurred_at DESC) for range scan.- Composite index: CREATE INDEX ON events (user_id, occurred_at) to accelerate grouping and filtering by user/time.- If selective by event_type or amount predicates, consider INCLUDE columns: CREATE INDEX ON events (occurred_at) INCLUDE (user_id, event_type, amount).- Partition events by time (monthly) to prune older partitions quickly.- Use parallel query (increase max_parallel_workers_per_gather) and work_mem tuned for aggregations.- Consider incremental materialized view or nightly incremental aggregation: pre-aggregate daily per-user buckets and roll them up for 7/30/90 windows to avoid scanning 90 days each run.- Monitor vacuum/analyze; keep stats updated.

Edge cases:- NULL amounts handled by AVG (returns NULL if no events in 7d) — COALESCE if you need 0.- If you need last event even beyond 90d, compute MAX(occurred_at) separately without 90d filter.

MediumTechnical

22 practiced

Explain how tree-based models compute feature importance: gain (split improvement), split count, and permutation importance. Describe common pitfalls (bias toward high-cardinality features, correlated features splitting importance) and, as an MLE, propose methods to obtain more reliable importance estimates (e.g., permutation importance, SHAP, conditional importance).

Sample Answer

Tree-based libraries expose several ways to quantify feature importance; each has strengths and pitfalls.

How they work- Gain (split improvement): sum of reduction in impurity (e.g., Gini, MSE) attributed to splits on a feature across all trees. Interprets importance as contribution to training loss reduction.- Split count (frequency): number of times a feature is used for a split (optionally weighted by node samples). Simple proxy for usage.- Permutation importance: measure drop in validation metric after randomly permuting a feature’s values (breaking its relationship with target). Evaluated post-fit on held-out data.

Common pitfalls- Bias to high-cardinality features: gain and split count prefer variables with many distinct values because they provide more potential split thresholds.- Correlated features: importance can be split across correlated predictors; a model may arbitrarily prefer one, underestimating others.- Training-data / model dependence: gain reflects training split choices and can overfit; permutation depends on evaluation set distribution and metric.- Interaction effects: univariate importance misses interaction contributions.

As an MLE — practical recommendations for more reliable estimates- Use permutation importance on a held-out validation set and repeat with multiple random seeds to estimate variance.- Compute conditional permutation importance (shuffle within strata or use conditional sampling) to control for correlated features.- Use SHAP (TreeSHAP) for consistent, local and global attributions that fairly distribute interaction effects; aggregate SHAP values for global importance and report dispersion.- Combine methods: report gain/split for training diagnostics, permutation/SHAP for production interpretation, and check sensitivity with feature removal / retraining experiments.- Monitor stability: bootstrap feature-importances across data folds; flag features with high variance before acting on them.

Present importance with confidence intervals and document dependence on data preprocessing (encoding, binning) to make decisions robust for production.

MediumTechnical

23 practiced

Evaluate the pros and cons of using automated feature engineering tools (e.g., FeatureTools, tsfresh, autofeat) versus manual, domain-driven feature creation. As an MLE, when would you adopt AutoFE, how would you validate the generated features, and what guardrails would you put around automatic feature expansion to avoid feature explosion and instability?

Sample Answer

Automated FE tools speed up feature discovery by programmatically generating candidate features (aggregations, lags, decompositions). They shine for exploratory phases and high-cardinality or relational data where engineered interactions are numerous.

Pros:- Fastly generate large, consistent sets (saves developer time).- Capture non-obvious interactions and temporal aggregations.- Reproducible pipelines and integration with metadata.- Good for prototyping or when domain knowledge is incomplete.

Cons:- Can create huge, noisy feature spaces (overfitting risk).- Many uninterpretable features — harder for debugging and compliance.- May produce unstable features across time (data drift).- Computational and storage costs for feature explosion.

When I’d adopt AutoFE:- Early exploration to surface candidate features quickly.- When dataset is large/complex (many entities, time series) and manual coverage is impractical.- As a supplement to — not replacement for — domain-driven features.

How I validate generated features:- Automated filtering: remove low-variance, high-missing, or collinear features.- Univariate importance (mutual information, correlation with target) as initial screen.- Model-based selection: L1-regularized models, tree-based feature importance with cross-validation.- Stability checks: evaluate performance across time splits (temporal CV) and different cohorts.- Monitor feature distributions and upstream data quality; run permutation importance and SHAP for interpretability.

Guardrails to avoid explosion & instability:- Limit feature templates and depth (FeatureTools primitives / max aggregation depth).- Cardinality thresholds for categorical joins; avoid expanding extremely sparse keys.- Budgeting: cap total generated features and compute time.- Enforce provenance: metadata linking each auto-feature to source and generation rule.- Automated sanity checks: reject features with excessive drift, high missingness, or leakage indicators (lookahead).- Human-in-the-loop review for top candidates before productionizing.- Continuous monitoring in production with alerts for distribution shift, importance changes, and model degradation.

By combining AutoFE for breadth and domain-driven features for critical, interpretable signals — plus rigorous validation and strict guardrails — I balance productivity with robustness and maintainability.

MediumTechnical

43 practiced

Given numeric features and a continuous target, design a pipeline (pseudocode or Python) that computes mutual information between each feature and the target, ranks features by MI, and selects top-k. Explain discretization choices for continuous variables, which scikit-learn estimator you would use (e.g., mutual_info_regression), and how to guard against spurious correlations.

Sample Answer

Approach: compute mutual information (MI) between each numeric feature and a continuous target using scikit-learn's mutual_info_regression (non-parametric, kNN-based). Preprocess by handling missing values, optionally standardizing, and discretize only if you prefer histogram-based MI (I recommend using mutual_info_regression which works on continuous variables directly). Use permutation tests / bootstrap and control for confounders to avoid spurious correlations. Finally rank and select top-k.

python

import numpy as np
import pandas as pd
from sklearn.feature_selection import mutual_info_regression
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import permutation_test_score

def select_topk_by_mi(X: pd.DataFrame, y: pd.Series, k=10, random_state=0):
    # Impute missing values
    X_imputed = pd.DataFrame(SimpleImputer(strategy='median').fit_transform(X), columns=X.columns)
    # Optional scaling (not required for mutual_info_regression but helps numeric stability)
    X_scaled = pd.DataFrame(StandardScaler().fit_transform(X_imputed), columns=X.columns)
    # Estimate MI (continuous features -> continuous target) using kNN estimator
    mi = mutual_info_regression(X_scaled.values, y.values, random_state=random_state)
    mi_series = pd.Series(mi, index=X.columns).sort_values(ascending=False)
    # Guard: permutation test to assess significance of top features
    significant = []
    for feat in mi_series.index[:k]:
        score, perm_scores, pvalue = permutation_test_score(
            lambda X_sub, y_sub: mutual_info_regression(X_sub.reshape(-1,1), y_sub, random_state=random_state)[0],
            X_scaled[[feat]].values, y.values, scoring=None, cv=None, n_permutations=100, random_state=random_state
        )
        if pvalue < 0.05:
            significant.append(feat)
    return mi_series, significant[:k]

Key choices explained:- mutual_info_regression: non-parametric estimator for continuous targets; no explicit binning required.- Discretization: if you must discretize (for other MI estimators), use equal-frequency bins to preserve distribution or use adaptive binning (e.g., KBinsDiscretizer with strategy='quantile'). Beware bin count: too few bins lose info, too many increase variance.- Guard against spurious MI: use permutation tests or bootstrap to get p-values/confidence intervals; compute conditional MI or include known confounders in a regression residualization step (i.e., compute MI with residuals after regressing out confounders); cross-validate MI estimates; correct for multiple testing (Benjamini–Hochberg).- Practical tips: ensure sample size is sufficient for kNN MI; interpret MI magnitudes relatively; combine MI ranking with model-based feature importance and domain knowledge.

MediumSystem Design

28 practiced

Design a monitoring system to detect feature drift and instability for deployed features. Define which metrics you would compute (PSI, KL divergence, KS, mean/std changes), sampling strategies, alert thresholds, and automated mitigation actions (retrain, freeze, or rollback). Explain how you'd evaluate false positives and incorporate seasonality.

Sample Answer

Requirements:- Detect distributional (feature) drift and instability per-feature and for joint behavior; low latency for critical models, batch for non-critical; support seasonality and robust false-positive control; automated mitigation (retrain, freeze, rollback) with human-in-loop control.

High-level design:- Inference-time probe collects feature vectors and model outputs into a staging store (time-windowed, hashed for privacy).- Monitoring service computes per-feature and multivariate stats and raises alerts to orchestration layer that can trigger mitigation pipelines.

Metrics:- Univariate: Population Stability Index (PSI) over binned features, Kolmogorov–Smirnov (KS) for continuous CDF shifts, mean/std changes with z-score.- Multivariate: KL-divergence on joint categorical distributions or embeddings; Mahalanobis distance for covariate shift in embedding space.- Prediction-level: Drift in score distribution, calibration shift (ECE), change in downstream business metrics (CTR, conversion).

Sampling strategies:- Stratified sampling by label, region, user cohort and by time windows (hourly for realtime, daily/weekly for batch).- Reservoir sampling + time-decay weighting to keep memory bounded while emphasizing recent data.

Thresholds & alerting:- Thresholds per metric per feature set using historical variability: e.g., PSI > 0.2 (moderate), >0.3 (severe); KS p-value < 0.01; KL > baseline + 3σ. Use adaptive thresholds computed from rolling baseline and control charts (EWMA) to limit false positives.

Automated mitigation:- Triage rules combine signal strength, affected features, and business impact: - Low-confidence, non-critical drift -> freeze model weights and route to shadow mode for diagnosis. - Moderate drift with degraded business metrics -> trigger automated retrain pipeline using recent data, run validation and canary rollout. - Severe drift or safety-critical failures -> automatic rollback to last known-good model and page on-call.- All automated actions log events and require post-action human review.

Handling seasonality & false positives:- Maintain seasonal baselines (daily/weekly/monthly) and perform seasonal decomposition (STL) to compare residuals rather than raw series.- Use permutation tests and bootstrap to estimate false-positive rates; tune thresholds to target acceptable alert precision (e.g., <10% FP).- Implement cooldown windows, alert aggregation, and minimum sample-size checks to avoid noisy signals.

Evaluation:- Backtest on historical incidents and synthetic drifts (covariate, prior, concept) to measure detection latency, precision/recall, and mitigation safety.- KPIs: time-to-detect, false positive rate, time-to-recover, business impact after mitigation.- Continuous improvement via labeled incidents and feedback loop: incorporate human adjudications to retrain alerting models and thresholds.

Unlock Full Question Bank

Get access to hundreds of Feature Engineering and Selection interview questions and detailed answers.

Join thousands of developers preparing for their dream job.