Feature Engineering and Selection Questions

Covers the end to end process of transforming raw data into predictive and stable model inputs and choosing the most useful subset of those inputs. Topics include generating features from domain signals and timestamps, numerical transformations such as scaling binning and logarithmic transforms, categorical encodings including one hot and target encoding, creation of interaction and polynomial features, construction of dense feature vectors for model consumption, handling missing values and outliers, and strategies for class imbalance. Also includes feature selection and dimensionality reduction methods such as filter techniques statistical tests wrapper methods embedded model based selection mutual information analysis and tree based importance measures. Emphasis is placed on avoiding data leakage validating feature stability over time interpreting feature contributions and documenting rationale for feature creation or removal. For senior roles include designing feature engineering best practices mentoring others and considering feature impact on model interpretability and business metrics.

MediumTechnical

27 practiced

You have a high-cardinality categorical feature (product_id with ~100k unique values). Describe at least five distinct encoding strategies you could use, discuss trade-offs in model performance, memory/serving cost, and robustness, and explain how you'd handle unseen values at inference time for each approach.

Sample Answer

1) One-hot / sparse binary- Description: Create a binary column per product_id (sparse representation).- Performance: Good for simple linear models with low interaction complexity; can overfit and not capture similarity.- Memory/serving: Very high (100k dims); requires sparse storage and models supporting sparse inputs.- Robustness/unseen: Map unseen to all-zero or to an explicit “unknown” column.- When to use: Baselines or tree/linear models that accept sparse features and when memory isn’t a bottleneck.

2) Target / mean encoding- Description: Replace product_id by aggregated target statistic (e.g., mean conversion rate), optionally smoothed with global mean and count-based shrinkage.- Performance: Often strong predictive power; risk of leakage/overfitting if not properly regularized.- Memory/serving: Low (store mapping table ~100k entries).- Robustness/unseen: Fall back to global mean or prior; can assign Bayesian prior using few-count smoothing.- When to use: Tabular tasks with leakage controls (folding, CV encoding).

3) Frequency / count encoding- Description: Encode by product frequency or log-frequency.- Performance: Captures popularity signal; lower variance than target encoding, less predictive for nuanced effects.- Memory/serving: Very low (single numeric).- Robustness/unseen: Map unseen to zero or minimum frequency; robust and fast.- When to use: When popularity matters and model simplicity is desired.

4) Embeddings (learned)- Description: Learn dense vector per product via embedding layer (neural nets) or entity embeddings from supervised training.- Performance: High for capturing latent similarity and interactions; needs enough data per id.- Memory/serving: Medium (embedding_dim * 100k floats; e.g., 32-d → ~12.8M floats ≈ 50MB); needs lookup service.- Robustness/unseen: Map unseen to a dedicated “unknown” vector (random/learned or average embedding); can also use attribute-based cold-start initialization.- When to use: Deep models, collaborative filtering, when capturing similarity is important.

5) Hashing / feature hashing- Description: Hash product_id into a fixed number of buckets (e.g., 2^16) producing dense or sparse hashed features.- Performance: Reduces dimensionality, avoids dictionary maintenance; risk of collisions reducing signal.- Memory/serving: Predictable and low; no mapping table required.- Robustness/unseen: Unseen hash to a bucket naturally; performance stable but collisions can degrade accuracy.- When to use: High throughput systems where memory and lookup latency matter.

Extras / combinations:- Hierarchical / attribute-based encoding: Use product category, brand, price bins when available—reduces cardinality and helps cold start.- Hybrid strategies: Use embedding + frequency + target-smoothed features for best-of-both worlds.

Practical safeguards:- For any mapping table (target, embedding), use time-based splits and cross-fold encoding to avoid leakage.- Monitor drift and retrain mappings periodically; implement fallback “unknown” logic and default priors for inference.- Consider serving constraints: if low-latency and memory-limited, prefer hashing or compact embeddings with quantization (e.g., 8-bit).

EasyTechnical

27 practiced

Explain class imbalance and list three feature-level strategies (not model-level class weighting) to mitigate its impact before model training. For each strategy, describe a situation where it helps and where it might introduce problems.

EasyTechnical

23 practiced

Describe the differences between standard scaling (z-score), min-max scaling, and robust scaling for numerical features. For each method, explain when you would prefer it (outliers, model assumptions, interpretability) and name the corresponding sklearn transformer class you would use in Python.

MediumSystem Design

20 practiced

Design a test and CI strategy to ensure that newly implemented engineered features do not leak future information into training or evaluation. Include unit tests for transformation logic, integration tests that simulate time-based splits, and data-contract checks you would run in continuous integration before merging feature code.

Sample Answer

Requirements:- Prevent leakage of future info into training/eval.- Verify transformation correctness, time-aware splits, and schema/contract integrity in CI before merge.- Fast feedback in PRs and fuller pipelines on main.

High-level strategy:1. Local/unit tests for deterministic transformation logic.2. Integration tests that simulate realistic time-series pipelines (time-based train/val/test splits).3. CI gates: data-contract checks, lineage/time-consistency checks, and sample-based statistical drift tests.

Components & tests:

Unit tests (fast, run on PR)- Pure-function tests for each transformer: fixed inputs -> expected outputs, including edge timestamps (e.g., timezone, DST).- Mock clock tests: inject "current_time" param to ensure features that use "now" are deterministic.- Null/NA handling and type assertions.Examples:- assert transform_feature(row_with_t=2024-01-01) == expected_value- test that feature_window_agg(window=7 days) only reads rows with timestamp <= reference_time

Integration tests (run in CI matrix or nightly)- Time-split simulation: create synthetic dataset spanning timeline; run featurization then perform rolling-origin evaluation to verify no future rows used for a given training window.- Full pipeline smoke: featurize, save artifacts, train lightweight model, evaluate on holdout — assert that training timestamps < evaluation timestamps and metrics reasonable.- Backfill/regression test: compare new feature outputs vs. baseline on historical snapshot.

CI data-contract checks (pre-merge)- Schema validation: column names, types, nullability using JSON schema or Great Expectations.- Timestamp monotonicity: timestamps sorted per entity; no future timestamp <= reference.- Lineage/time-window assertion: for every feature that aggregates over window W, assert that produced value at t only uses source rows with timestamp in [t-W, t] (static analysis of code or runtime sampling).- Cardinality & uniqueness checks for keys.- Statistical sanity: simple delta checks vs. recent snapshot (means, quantiles) to catch inadvertent leakage.

Automation & tooling- Use pytest for unit/integration; parametrized tests for time windows.- Use Great Expectations / Deequ for contract checks.- CI workflow: fast PR job (unit tests + schema + mock-clock checks), extended job on merge (integration + rolling-origin evaluation).- Artifacts: store featurization logs, provenance (datasets, code hash, reference_time) for audits.

Trade-offs- Static analysis for window safety is hard; prefer runtime assertions and code patterns (inject reference_time) to make correctness testable.- Synthetic integration tests add maintenance but catch subtle leaks.

Outcome: deterministic transforms, provable time boundaries in tests, CI gates to block merges that could introduce leakage.

EasyTechnical

21 practiced

Explain cyclical encoding for timestamp-derived features (for example, hour-of-day and day-of-week). Show the mathematical transform you would use to encode an 'hour' column into two features and explain why cyclical encoding is preferred over integer encoding for periodic signals.

Sample Answer

Approach: encode periodic timestamp features (hour, day-of-week) as two continuous features using sine and cosine of the angle on the unit circle so the model sees cyclical proximity (e.g., 23:00 close to 0:00).

Mathematical transform:Given hour h (0..23), angle θ = 2π * h / 24.Encode as:x1 = cos(θ) = cos(2π * h / 24)x2 = sin(θ) = sin(2π * h / 24)

Python implementation:

python

import numpy as np
import pandas as pd

def cyclical_encode_hour(df, hour_col='hour'):
    """
    Adds two columns: hour_cos, hour_sin for cyclical encoding of hour in [0,23].
    """
    hours = df[hour_col].astype(float)  # handle numeric or int-like
    theta = 2 * np.pi * hours / 24.0
    df = df.copy()
    df['hour_cos'] = np.cos(theta)
    df['hour_sin'] = np.sin(theta)
    return df

Key points:- Preserves cyclic distance: |23 - 0| should be small; integer encoding fails here.- Makes relationships smooth for models that assume Euclidean distance (linear, tree-based often benefit).- Handles other cycles similarly (day-of-week: divide by 7; month: by 12).

Time/Space complexity: O(n) time, O(1) extra memory per row (two new cols).

Edge cases & considerations:- Missing values: impute before encoding or keep NaNs.- Timezones / DST: convert timestamps to consistent timezone and decide whether to use local or UTC.- If higher-frequency patterns exist, consider adding harmonics (sin/cos of k*θ) or learned embeddings for complex cyclic patterns.

Alternatives: ordinal/integer encoding (bad for cyclic continuity), one-hot (avoids order but increases dimensionality), or learned embeddings for high-cardinality cyclic features.

Unlock Full Question Bank

Get access to hundreds of Feature Engineering and Selection interview questions and detailed answers.

Join thousands of developers preparing for their dream job.