Feature Engineering & Selection Basics Questions

Understand why features matter and basic techniques: scaling/normalization, handling categorical variables (one-hot encoding, label encoding), creating interaction features, and feature importance. Know that good features are as important as good algorithms. Understand why feature scaling matters for algorithms like KNN or linear models.

MediumTechnical

64 practiced

Implement a scikit-learn Pipeline using ColumnTransformer in Python that applies StandardScaler to numeric_cols and OneHotEncoder(handle_unknown='ignore') to categorical_cols, then trains a LogisticRegression with class_weight='balanced'. Signature: def build_and_train(X_train, y_train, numeric_cols, categorical_cols): -> returns fitted_pipeline. Include model persistence considerations.

Sample Answer

To build this training pipeline, we'll use ColumnTransformer to apply StandardScaler to numeric columns and OneHotEncoder(handle_unknown='ignore') to categorical columns, then chain a LogisticRegression with class_weight='balanced' in a scikit-learn Pipeline. We'll include simple input validation and model persistence notes (joblib, versioning, storing preprocessing metadata).

python

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.exceptions import NotFittedError
import joblib

def build_and_train(X_train, y_train, numeric_cols, categorical_cols, random_state=42):
    """
    Builds and fits a Pipeline that preprocesses numeric and categorical features
    and trains a LogisticRegression with class_weight='balanced'.
    Returns the fitted pipeline.
    """
    # Validate columns
    missing = set(numeric_cols + categorical_cols) - set(X_train.columns)
    if missing:
        raise ValueError(f"Columns not in X_train: {missing}")

    # Preprocessing transformers
    numeric_transformer = Pipeline(steps=[
        ('scaler', StandardScaler())
    ])
    categorical_transformer = Pipeline(steps=[
        ('onehot', OneHotEncoder(handle_unknown='ignore', sparse=False))
    ])

    preprocessor = ColumnTransformer(transformers=[
        ('num', numeric_transformer, numeric_cols),
        ('cat', categorical_transformer, categorical_cols)
    ], remainder='drop')  # drop any other columns

    # Full pipeline
    pipeline = Pipeline(steps=[
        ('preprocessor', preprocessor),
        ('clf', LogisticRegression(class_weight='balanced', solver='liblinear', random_state=random_state))
    ])

    # Fit and return
    pipeline.fit(X_train, y_train)
    return pipeline

# Example persistence:
# joblib.dump(pipeline, 'models/logreg_pipeline_v1.joblib')
# loaded = joblib.load('models/logreg_pipeline_v1.joblib')

Key points:- Using ColumnTransformer keeps preprocessing and model together for reproducibility.- OneHotEncoder handle_unknown='ignore' prevents runtime errors on unseen categories.- Persist pipeline with joblib (or cloud artifact store). Also store training feature lists, encoder categories, model version, hyperparams, and software/package versions for reproducibility.- Considerations: scale of data, class imbalance (we use class_weight), and using sparse=True to save memory if many categories. Validate inputs at serving and include unit tests for preprocessing.

EasyTechnical

62 practiced

Write Python code (without using sklearn scalers) to standardize numeric columns in a pandas DataFrame: subtract column mean and divide by column std. Signature: def zscore_scale(df, numeric_cols, ddof=0): -> returns scaled_df and stats_dict. Handle zero-variance columns by setting scaled values to 0 and preserve NaNs. Mention how you would persist the stats_dict for production use.

Sample Answer

Approach: compute per-column mean and std (with given ddof), subtract mean and divide by std; if std == 0 or NaN, set scaled values to 0 while preserving NaNs. Return scaled DataFrame and a stats_dict mapping col -> {"mean":..., "std":...}. Persist stats_dict (JSON/pickle/joblib) for reuse in production.

python

import pandas as pd
import numpy as np
import json

def zscore_scale(df, numeric_cols, ddof=0):
    """
    Standardize numeric columns in df: (x - mean) / std
    Args:
        df: pandas.DataFrame
        numeric_cols: list of column names to scale
        ddof: degrees of freedom for std (0 => population, 1 => sample)
    Returns:
        scaled_df: DataFrame copy with scaled columns
        stats_dict: {col: {"mean": float, "std": float}}
    """
    df = df.copy()
    stats = {}

    for col in numeric_cols:
        series = df[col]
        mean = series.mean(skipna=True)
        std = series.std(ddof=ddof, skipna=True)

        # Handle zero or NaN std: treat as zero-variance
        if pd.isna(std) or std == 0:
            # Keep NaNs; set non-NaN entries to 0
            scaled = series.where(series.isna(), 0.0)
            std_val = 0.0
        else:
            scaled = (series - mean) / std
            std_val = float(std)

        df[col] = scaled
        stats[col] = {"mean": float(mean) if not pd.isna(mean) else None,
                      "std": std_val}

    return df, stats

Key points:- Preserves original NaNs because pandas ops propagate NaN; when std==0 we explicitly set non-NaN to 0.- stats_dict contains floats (or None) suitable for serialization.

Complexity:- Time: O(n * m) where n = rows, m = number of numeric_cols (single pass per column).- Space: O(n * m) for copy plus O(m) for stats.

Edge cases:- Columns with all NaNs => mean/std are NaN; function sets mean=None, std=0 and leaves values NaN.- Mixed dtypes: ensure numeric_cols are numeric; non-numeric will raise on mean/std.

Persisting stats_dict for production:- Serialize stats_dict to JSON if only floats/None: json.dump(stats, open("scaler_stats.json","w"))- For richer types or compatibility use pickle/joblib or store in model registry (MLflow) or object storage (S3). Example:

python

with open("scaler_stats.json","w") as f:
    json.dump(stats, f)
# load:
with open("scaler_stats.json") as f:
    stats = json.load(f)

At serving time, load stats and apply the same transform (use stored ddof if needed).

HardTechnical

69 practiced

Explain strategies to handle Missing Not At Random (MNAR) in a medical dataset where lab tests are ordered selectively for sicker patients. Discuss modeling missingness explicitly, using missingness indicators, joint modeling, multiple imputation approaches that include predictors of missingness, and causal considerations to avoid biased estimates.

Sample Answer

Start by acknowledging MNAR: lab tests ordered selectively for sicker patients means probability of missing depends on unobserved health status — naive methods (complete-case, MCAR assumptions) will bias estimates. Practical strategies:

1. Model missingness explicitly- Build a missingness model P(R=1 | X, Y*, A) where R indicates lab observed, X are covariates, Y* is latent health. Use proxies (vitals, prior diagnoses, orders) as predictors. Fit a logistic model or flexible classifier (XGBoost) for R to quantify selection.

2. Missingness indicators / informative missingness features- Add indicator flags (lab_missing) and interactions to prediction models. In many clinical tasks, "not measured" is informative and improves predictive performance even if it doesn't recover causal effects.

3. Joint / selection models- Use joint modeling (e.g., shared-parameter mixed models, Bayesian hierarchical models) that specify outcome and missingness processes together. Selection models factorize P(Y, R | X) = P(Y | X) P(R | Y, X); estimate via maximum likelihood or Bayesian MCMC, which can partially identify MNAR when structure/priors are credible.

4. Multiple imputation that includes predictors of missingness- Use MI with imputation models that include strong auxiliary variables predictive of both lab value and R (vitals, prior labs, clinician notes). Implement chained equations or Bayesian MI; include the fitted missingness score as a predictor to make MAR more plausible (MAR conditional on observed auxiliaries).

5. Pattern-mixture and sensitivity analysis- Use pattern-mixture models to model distributions conditional on missingness pattern, and conduct targeted sensitivity analyses by varying a sensitivity parameter delta that shifts imputed values for missing labs to reflect worse health among untested patients.

6. Causal considerations to avoid bias- Draw a DAG: show that clinician decision to order (R) is a collider between unobserved severity and other covariates. Avoid conditioning on colliders when estimating causal effects. If estimating treatment effects, consider IPTW where weights are inverse probability of being measured (using R model) combined with outcome modeling (doubly robust/targeted maximum likelihood) — but note that MNAR breaks standard IPW unless unmeasured confounding is addressed.

Practical workflow- 1) Map data-generating process and construct DAGs; 2) engineer rich auxiliary variables from EHR (timestamps, notes, order reasons); 3) fit and evaluate missingness model; 4) choose approach: joint/Bayesian if you can specify plausible priors, MI+auxiliaries plus sensitivity analysis otherwise; 5) report sensitivity bounds and calibration; 6) in production, monitor changes in ordering patterns and re-evaluate.

Key caveat: MNAR is not fully testable from data alone; rely on domain knowledge, auxiliary data, and transparent sensitivity analyses.

EasyTechnical

63 practiced

Explain why feature engineering is often as important as algorithm choice in applied machine learning. Provide a concrete example: you have two binary classification pipelines with identical model architecture but different feature sets — describe scenarios where engineering better features would outperform switching to a more complex model. How would you prioritize feature work vs model tuning when time and compute are limited?

MediumTechnical

58 practiced

Implement a function in Python that computes mutual information (MI) between each feature and a binary target in a dataset containing both continuous and categorical features. Use scikit-learn's mutual_info_classif for numeric features and an appropriate estimator for categorical ones; include a discretization strategy for continuous features and return a ranked list of features by MI.

Sample Answer

Approach: detect feature types, discretize continuous features with KBinsDiscretizer (or optional quantile binning), encode categorical features to integer labels, then compute mutual information using scikit-learn's mutual_info_classif (treating all inputs as discrete after discretization/encoding). Return a DataFrame/ordered list of features sorted by MI.

python

import numpy as np
import pandas as pd
from sklearn.preprocessing import KBinsDiscretizer, OrdinalEncoder
from sklearn.feature_selection import mutual_info_classif

def compute_feature_mi(X: pd.DataFrame, y: pd.Series,
                       n_bins=10, strategy='quantile', random_state=0):
    """
    Compute mutual information between each feature and a binary target.
    Continuous features are discretized using KBinsDiscretizer.
    Categorical features are ordinal-encoded.
    Returns pandas DataFrame with features ranked by MI desc.
    """
    X = X.copy()
    y = y.values.ravel()
    contin_cols = X.select_dtypes(include=[np.number]).columns.tolist()
    cat_cols = X.select_dtypes(exclude=[np.number]).columns.tolist()

    # Discretize continuous features into n_bins (quantile by default)
    if contin_cols:
        kbd = KBinsDiscretizer(n_bins=n_bins, encode='ordinal', strategy=strategy)
        # KBinsDiscretizer requires 2D array
        X_cont = kbd.fit_transform(X[contin_cols])
        X[contin_cols] = X_cont

    # Encode categorical features to ordinal integers
    if cat_cols:
        enc = OrdinalEncoder(handle_unknown='use_encoded_value', unknown_value=-1)
        X_cat = enc.fit_transform(X[cat_cols])
        X[cat_cols] = X_cat

    # After discretization/encoding treat all features as discrete
    mi = mutual_info_classif(X.values, y, discrete_features=True, random_state=random_state)
    res = pd.DataFrame({'feature': X.columns, 'mi': mi})
    res = res.sort_values('mi', ascending=False).reset_index(drop=True)
    return res

Key points:- Discretization: KBinsDiscretizer with strategy='quantile' handles skewed distributions; 'uniform' or 'kmeans' are alternatives.- Encoding: OrdinalEncoder is fast; use one-hot if ordinal encoding may inject false order (but mutual_info_classif on integers treats them as categories when discrete_features=True).Time & space:- Time: O(n_features * n_samples * estimator_cost). mutual_info_classif uses nearest-neighbor estimator with roughly O(n_samples log n_samples) per feature in practice.- Space: O(n_samples * n_features) for transformed arrays.

Edge cases:- Missing values: impute before calling this function.- Low-variance or constant features produce MI ~ 0.- y must be binary (or discrete labels); mutual_info_classif supports multiclass as well.

Alternatives:- Use mutual_info_regression for continuous targets.- Use permutation-based MI (model-agnostic) or conditional MI to control for correlated features.

Unlock Full Question Bank

Get access to hundreds of Feature Engineering & Selection Basics interview questions and detailed answers.

Join thousands of developers preparing for their dream job.