Feature Engineering & Selection Basics Questions
Understand why features matter and basic techniques: scaling/normalization, handling categorical variables (one-hot encoding, label encoding), creating interaction features, and feature importance. Know that good features are as important as good algorithms. Understand why feature scaling matters for algorithms like KNN or linear models.
MediumTechnical
70 practiced
You find two features with Pearson correlation > 0.95. How do you decide which one to keep? Describe quantitative and qualitative approaches (e.g., VIF, model-based checks, domain knowledge) and the downstream implications for interpretability and model stability.
Sample Answer
When two features have Pearson r > 0.95, treat them as essentially collinear and evaluate both quantitative and qualitative signals before dropping or transforming one.Quantitative approaches- VIF: compute VIF for each feature; if VIF >> 10, multicollinearity is severe. Drop or combine the one that reduces VIF most.- Model-based importance: fit regularized models (Lasso, Elastic Net, tree-based). L1 may zero out one; feature importances in trees can show which contributes more to predictive power.- Partial dependence / SHAP: compare marginal/SHAP contributions — keep the feature with clearer, more stable effect.- Stability checks: bootstrap models with different samples; a feature whose coefficient/sign flips often is less stable and a candidate to remove.- Dimensionality reduction: PCA/PLS to create an orthogonal component if interpretability is less important.Qualitative approaches- Domain knowledge: prefer the feature that is more directly measurable, less noisy, or more actionable for stakeholders.- Data quality & provenance: keep the feature with fewer missing values, lower measurement error, or stronger business meaning.Downstream implications- Interpretability: keeping both harms interpretability — coefficients are unstable and hard to explain. Prefer one-or-combined approach if explanations are needed.- Model stability & generalization: high collinearity inflates variance of coefficients and can hurt generalization; regularization or removal improves stability.- Monitoring & data drift: choose the feature less likely to drift or easier to monitor in production.Recommended workflow1. Check VIF and correlations.2. Use domain knowledge to shortlist.3. Run model-based importance and stability (bootstrap/SHAP).4. Decide: keep the more stable/actionable one, or combine (e.g., average or PCA) if predictive performance requires both. Document the decision for reproducibility.
HardSystem Design
64 practiced
Design an end-to-end production feature engineering process for a company that needs reproducible offline features for training and consistent online features for serving. Describe components: feature definitions, transformations, feature store (online/offline), validation tests, backfills, monitoring, and contracts with engineers. Explain how you ensure consistency and reproducibility.
Sample Answer
Requirements & constraints:- Reproducible offline features for model training (deterministic, versioned).- Low-latency, consistent online features for serving.- Support backfills, pipeline idempotency, schema evolution, and auditability.- Scale to billions of rows, multiple teams, SLAs for freshness/latency.High-level architecture:- Ingestion layer (stream/batch) → Feature transformation layer (batch/stream runners) → Feature Store (offline data lake + online KV store) → Serving layer (online feature API) → Model training/validation → Monitoring & governance.Components & responsibilities:1. Feature definitions- Central repo of feature specs (YAML/SQL/DSL) containing: logical name, owner, description, input sources, transformation SQL/udf, aggregation windows, TTL, dtype, freshness SLA, expected distribution, lineage.- Stored in git with PR-based reviews and semantic versioning.2. Transformations- Two implementations per feature: batch (Spark/DBT/SQL on data lake) that writes materialized Parquet/Delta to offline store and stream (Flink/Beam/Kafka Streams) that emits feature updates to online store.- Use a single canonical transformation expression (SQL/IR) compiled into both batch and stream jobs to avoid drift.3. Feature Store (offline/online)- Offline: partitioned, versioned data lake (Delta Lake/Iceberg) with event-time partitioning and snapshotting. Stores raw feature vectors per entity-time, indexed by entity_id + event_time + feature_version.- Online: low-latency KV store (Redis, DynamoDB) with per-entity keys and feature shards; supports atomic writes from stream processors and TTL based on feature_spec.4. Validation tests- Unit tests of transformation logic (small fixtures).- Integration tests that run compiled transformations on sampled data.- Data quality checks (rowcounts, null rates, cardinality), distribution drift tests, freshness checks. Implement as DAG tasks (Great Expectations / Deequ).- CI gates: PR cannot merge without passing tests.5. Backfills & materialization- Backfill orchestrator (Airflow/Argo) runs batch transformation over historical event time ranges and writes to offline store using idempotent writes (UPSERT with transaction support).- Backfill produces a materialization manifest (feature_version, input_snapshot_hash, run_id).- Online backfill: either stream-replay (recompute and push to online) or on-demand upsert jobs; controlled by throttling and blackout windows.6. Monitoring & alerting- Feature serving metrics: latency, error rates, freshness, missing rate per feature.- Training/production drift: PSI, KL divergence, distribution alarms, population skew.- Data lineage and audit logs: forensics of model inputs; store input snapshot hashes per training job.- Dashboards + automated alerts to owners.7. Contracts with engineers (SLAs & expectations)- Feature contract document required before production: ownership, interface (name, dtype), SLA (freshness, latency), privacy constraints, allowed transformations.- Change policy: breaking changes require deprecation window + migration path; version bump and dual-write for a period.- Ownership responsibilities: ensure upstream data quality, respond to alerts, maintain tests.Ensuring consistency & reproducibility:- Single-source feature spec compiled into both batch and stream codepaths—no separate hand-coded transformations.- Event-time semantics: compute features using event-time, not processing-time; include watermarking for late data.- Versioning: every feature materialization annotated with feature_version + input_snapshot_hash (hash of upstream data commit IDs). Training jobs record exact feature versions and offline snapshot IDs used.- Deterministic transforms: avoid non-deterministic functions (rand(), now()); if used, seed and record seeds.- Idempotent writes and transactional updates in offline store; materialization manifests enable reproducible backfills.- CI/CD: tests for correctness, schema, and performance, plus deployment via automated pipelines.- Validation before serving: nightly run comparing offline recomputed features vs online served values on a sample of keys; alerts on divergence beyond tolerance.Trade-offs & scaling:- Compiling one canonical spec increases engineering upfront but minimizes drift.- Dual storage (offline + online) duplicates data but is necessary for low-latency serving and reproducible training.- Using transactional data lake (Delta/Iceberg) simplifies backfills but requires supporting infra.Example flow (concise):- Data arrives in Kafka + raw events in data lake.- Feature spec compiled into Beam job (stream) and Spark job (batch).- Stream job writes updates to DynamoDB; batch job writes materialized Parquet with feature_version to Delta.- Training job points to specific feature_version + snapshot; model registry records the manifest.- Continuous validation compares a sample of online values to offline recomputation to ensure parity.This design emphasizes single-definition transformations, strict versioning, event-time computation, automated testing, and clear ownership — together ensuring reproducible offline features and consistent online serving.
HardTechnical
49 practiced
Your model's performance has steadily degraded in production. Describe a process to detect feature drift vs label drift, methods to root-cause which feature(s) changed (e.g., PSI, KS test, shift detection), and mitigation strategies including retraining cadence, adaptive features, and alerting. Include practical thresholds and tooling you might use.
Sample Answer
Process overview (detect → diagnose → mitigate):1) Detect degradation & distinguish feature vs label drift- Monitor model-level KPIs: AUC, accuracy, precision/recall, calibration (Brier), business metric. Set alert triggers: relative drop >5 percentage points or >10% relative; Brier increase >10%.- If labels available with lag: compare recent-label distribution vs historical (label drift). If labels delayed/unavailable, use proxy metrics (prediction distribution shift, uplift in low-confidence predictions).- Statistical tests: - Feature drift (input): Population Stability Index (PSI) per feature — thresholds: <0.1 stable, 0.1–0.2 moderate, >0.2 significant; Kolmogorov–Smirnov (KS) test p<0.05 indicates shift for continuous features; Chi-square for categoricals. - Label drift: KL divergence or Chi-square between historical and recent label distributions; KS for continuous targets. - Concept drift (P(y|X) change): monitor conditional metrics (per-segment performance) and calibration drift; use PSI/KS on predicted probabilities.2) Root-cause analysis (which features changed)- Univariate checks: rank features by PSI/KS delta; visualize distributions and missingness.- Multivariate checks: classifier two-sample test — train a domain classifier to distinguish between historical vs recent X; high AUC indicates overall covariate shift; use SHAP on domain classifier to find features that most separate distributions.- Model behavior checks: compare feature importances (SHAP/TreeSHAP) between training and recent inference; compute permutation importance on recent data (if labels) to see changed predictive power.- Missingness/encoding: monitor null rates and new/unseen categories; drift in embedding stats for categorical encodings.3) Mitigation & operational strategy- Short term: - Input validation: block or flag records with invalid ranges or new categories (Great Expectations, Tecton). - Feature transformations: re-normalize using rolling statistics, map rare/new categories to "other". - Fallback rules or simpler safe-model when confidence low.- Medium term: - Retraining strategy: hybrid cadence — scheduled (weekly/monthly depending on data velocity) plus trigger-based retrain when monitored metrics cross thresholds (e.g., PSI>0.2 on top-3 features or AUC drop >5%). - Use rolling-window training (e.g., last N weeks) or incremental learning (online updates) for fast-moving data. - Domain adaptation: re-weight training samples via importance weighting using density ratio estimates.- Long term: - Feature hardening: add features robust to upstream change, add feature provenance & lineage, automated tests for upstream data pipelines. - Ensemble models with diverse feature sets to reduce single-feature brittleness.4) Tools & pipelines- Monitoring: Evidently AI, WhyLabs, Seldon, Monte Carlo for data ops; Prometheus + Grafana for metric dashboards; MLflow for model registry.- Statistical tests & alerts: implement PSI/KS/KL in pipeline (Evidently includes these); domain classifier via scikit-learn.- Automation: CI for data (Great Expectations), retrain jobs via Airflow/Kubernetes; deployment via Seldon/TF Serving with canary rollouts and automated rollback.- Alerting: push alerts to Slack/PagerDuty when thresholds hit; include diagnostic payload (top drifting features, plots, domain classifier SHAP).5) Practical example- If production AUC drops from 0.82 → 0.75 and PSI for feature "device_type" = 0.28 and domain classifier SHAP points to "device_type" and "country", trigger retrain on last 2 weeks, validate candidate model on holdout, deploy via canary; if immediate risk, deploy fallback model and raise incident.This process combines statistical detection, explainability-driven root cause, and a mixed scheduled+trigger retraining cadence to keep models robust while minimizing unnecessary churn.
EasyTechnical
57 practiced
Given the following pandas DataFrame schema:- user_id: int- age: float- country: string- last_login_days_ago: float- purchase_amount: float (target)Write Python code (pandas) to impute missing numeric features with median and missing categorical 'country' with the mode. Show the function signature and an example of applying it to a DataFrame 'df'.
Sample Answer
A straightforward approach: compute medians for numeric columns and mode for the categorical 'country', then fillna in a copy of the DataFrame. Keep the function generic so you can pass column lists.Key points:- Uses median for robustness to outliers.- Uses first mode for categorical; if no non-null values exist, remains NA.Time complexity: O(n * m) where n = rows, m = columns (computing medians/modes single pass). Edge cases: all-NaN column, multiple modes, non-existent columns — function handles these by leaving as NA or ignoring missing column names. Alternative: scikit-learn's SimpleImputer if integrating into a pipeline.
python
import pandas as pd
from typing import List, Dict
def impute_missing(df: pd.DataFrame,
numeric_cols: List[str],
categorical_cols: List[str],
inplace: bool = False) -> pd.DataFrame:
"""
Impute numeric_cols with median and categorical_cols with mode.
Returns the imputed DataFrame (a copy unless inplace=True).
"""
if not inplace:
df = df.copy()
# Compute medians for numeric columns
medians: Dict[str, float] = df[numeric_cols].median(skipna=True).to_dict()
# Compute modes for categorical columns (take first mode; if no mode, leave NaN)
modes: Dict[str, object] = {}
for col in categorical_cols:
if col in df.columns:
mode_series = df[col].mode(dropna=True)
modes[col] = mode_series.iloc[0] if not mode_series.empty else pd.NA
# Fill numeric and categorical
df = df.fillna({**medians, **modes})
return df
# Example usage:
# Assume df has columns: ['user_id','age','country','last_login_days_ago','purchase_amount']
numeric_cols = ['age', 'last_login_days_ago', 'purchase_amount']
categorical_cols = ['country']
imputed_df = impute_missing(df, numeric_cols, categorical_cols)EasyTechnical
70 practiced
Define interaction features and provide three examples where an interaction term is likely to improve predictive performance. Explain how you would test whether an interaction adds value without causing overfitting.
Sample Answer
Interaction features (interaction terms) are new features formed by combining two or more original features so the model can capture effects that depend on the value of both features simultaneously. They model non-additive relationships: the effect of X1 on the target changes depending on X2.Three examples where interactions often help:- Age × ChronicCondition (healthcare): the impact of a condition on hospitalization risk varies by age.- Price × PromotionFlag (retail): a promotion’s uplift depends on base price level.- SessionDuration × DeviceType (digital product): longer sessions on mobile may indicate different intent than on desktop.How to test whether an interaction adds value without causing overfitting:1. Add the interaction to a baseline model (e.g., logistic/linear with regularization).2. Use k-fold cross-validation (or nested CV if tuning hyperparameters) to compare out-of-sample metrics (AUC, RMSE, log-loss). Improvement on validation folds indicates genuine gain.3. Apply regularization (L1/L2, elastic net) so the interaction is penalized if not useful; check if its coefficient survives.4. Perform a likelihood-ratio test or compare information criteria (AIC/BIC) for nested parametric models if assumptions hold.5. Use permutation importance / partial dependence to verify the interaction’s effect size and shape.6. Monitor complexity: if validation improves but test/generalization set performance or calibration worsens, it’s likely overfitting—remove or simplify.Combining cross-validation and regularization gives robust evidence an interaction improves predictive performance without overfitting.
Unlock Full Question Bank
Get access to hundreds of Feature Engineering & Selection Basics interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.