ML Fundamentals: Supervised Learning Algorithms Questions

Deep understanding of linear regression, logistic regression, decision trees, random forests, SVMs, and ensemble methods. Be able to explain: how each algorithm works, advantages/disadvantages, when to use each, regularization techniques (L1/L2), hyperparameter tuning, and how to handle overfitting.

MediumTechnical

55 practiced

You are building a fraud classifier with heavy class imbalance (0.1% fraud). As an Applied Scientist, describe techniques at the data, algorithm, and evaluation levels you would use to train and validate supervised models. Include specific algorithm choices and sampling or cost adjustment strategies.

Sample Answer

**Overview / goal**Detect rare fraud (0.1%) while minimizing false negatives and controlling false positives for business impact. I'll address strategies at the data, algorithm, and evaluation levels.

**Data-level**- Enrich signals: feature engineering (behavioral sequences, time deltas, device fingerprints), aggregation windows, anomaly scores.- Sampling: use stratified undersampling of majority + limited oversampling of fraud (SMOTE-NC for mixed features) only in training folds; keep validation/test distribution untouched.- Synthetic data & augmentation: generate realistic fraud via adversarial augmentation or conditional GAN when labeled fraud is extremely scarce.- Time-awareness: preserve temporal ordering; avoid leakage by splitting on time.

**Algorithm-level**- Cost-sensitive models: XGBoost/LightGBM with scale_pos_weight or custom objective to weight fraud higher.- Probabilistic models: calibrated Logistic Regression with class weights for interpretability.- Ensemble: stacking (weighted) ensembles combining tree models and neural nets; use focal loss or class-balanced loss for deep models.- Anomaly detectors as complements: isolation forest or autoencoder reconstruction error to surface novel fraud.

**Evaluation-level**- Metrics: prioritize precision@k, recall at fixed FPR, AUCPR (preferred over AUC-ROC), cost-weighted expected loss.- Cross-validation: time-series CV (rolling window), ensure fraud examples appear in training only from earlier times.- Calibration & thresholding: use business cost matrix to pick operating threshold; evaluate lift and ROI simulations.- Robustness: test on adversarial / concept-drift scenarios; monitor feature drift and periodic re-training cadence.

**Trade-offs**- Oversampling risks overfitting; cost-weighting balances rare classes without altering distribution.- Ensemble complexity vs. latency—use simpler scored models for real-time, heavier models for batch scoring.

This mix yields a practical, production-ready approach emphasizing realistic validation and business-aligned metrics.

MediumTechnical

61 practiced

Propose a reproducible experiment to compare logistic regression, random forest, and linear SVM on a time-series forecasting regression task (predicting next-day metric). As an Applied Scientist, describe data splitting to avoid leakage, feature engineering considerations, evaluation metrics, and deployment implications for each model class.

Sample Answer

**Experiment goal (brief)** Compare logistic regression (note: logistic is for classification—use linear regression instead) — linear regression, random forest, and linear SVM (SVR) for next-day numeric forecasting on one metric.

**Reproducibility setup** - Fix seeds, record package versions, containerize, snapshot dataset and preprocessing code. - Use a clear experiment config (hyperparams, features, train windows) and CI to rerun.

**Data splitting / leakage avoidance** - Time-based split: train on t0..tN, validate on rolling-window forward-chaining (e.g., expanding window) and test on a held-out future period. - Use nested hyperparameter tuning inside each train fold to avoid peeking. - Ensure no future-derived features (no target lag leakage); compute aggregations using only historical rows per split.

**Feature engineering** - Core: lag features (t-1..t-k), rolling stats (mean,std) computed causally, calendar (day-of-week, holiday), exponential decay features. - For SVM and linear model: scale numeric features (StandardScaler fit on train only). - For random forest: handle categorical via ordinal/target encoding; tuning can handle unscaled inputs. - Feature selection: use train-only permutation importance or Lasso for linear baseline.

**Evaluation metrics & protocol** - Metrics: MAE (robust), RMSE (penalize large errors), and MAPE or SMAPE if relative error matters. - Report mean and 95% CI across CV folds and compare on held-out test. - Statistical test: paired Wilcoxon or bootstrap to assess significance.

**Model-specific deployment considerations** - Linear regression: fast, highly interpretable, easy monitoring; retrain cadence moderate; good baseline. - Random forest: better nonlinearity capture, heavier memory and latency; produce per-tree uncertainty via ensemble variance; feature importance available. - Linear SVM (SVR): handles high-dim sparse features, needs scaling, can be slower at prediction for large support vectors; consider approximate/linearized solvers for production.

**Operational & monitoring** - Monitor drift (feature and residual), latency, and model staleness; automate retrain on drift triggers. - Log inputs, predictions, and evaluation on live data; compare to baseline.

**Outcome & decision rule** Choose model balancing test performance, latency, explainability, and maintenance cost; deploy simpler model if performance delta small.

HardSystem Design

60 practiced

Design an automated pipeline to detect and prevent target leakage during supervised model development. Include static checks, data lineage, feature provenance, and automated unit tests or validators an Applied Scientist would implement to ensure features available at training are also available at inference time without leakage.

Sample Answer

**Clarify requirements & success criteria**- Prevent target leakage end-to-end: detect features that use future/label-derived info, ensure parity between training and inference availability, produce actionable alerts and block deployments.- Constraints: low latency for inference, scalable to many features/datasets, integrate with CI/CD and feature store.

**High-level architecture**- Data ingestion → Feature store + lineage tracker → Static analyzer → Runtime validators → CI/CD gating → Monitoring & drift detection.

**Core components**- Feature provenance & lineage: capture column-level lineage metadata (source table, SQL/transformation DAG, timestamps, last-updated). Use open lineage formats (e.g., OpenLineage).- Static leakage checker: analyze DAGs/SQL for operations that join on future timestamps, label joins, lookahead windows, or use target columns; heuristics + rule engine + ML classifier for patterns.- Feature registry & availability matrix: records for each feature whether it is materialized, online-storeable, latency, required upstream signals.- Unit-test validators (CI): synthetic scenario tests that replay historical time windows and assert that feature values at training time are reproducible using only past data; include time-travel tests and “frozen pipeline” reproducer.- Runtime guardrails: feature access control blocking features marked as non-online or with >allowed latency; shadow inference to compare train vs. online feature distributions.- Monitoring & alerting: data drift, feature missingness, leakage score trends, and automatic rollback trigger.

**Data flow / checks**1. On feature creation: lineage capture → static analyzer runs rules (joins on label, future timestamps) → if flagged, block or require justification.2. In CI: run unit validators (temporal backtests) that ensure forward-only access and that model predictions can be recreated from online features.3. Pre-deployment: gate on all validators passing plus availability matrix confirming online serving hooks.4. Post-deploy: continuous runtime checks for missing features, distribution shift, and periodic re-run of leakage detection with recent lineage.

**Scalability & trade-offs**- Static rules are fast but produce false positives; complement with time-based unit tests for precision.- Full replay tests are compute-heavy; run nightly for full dataset, lightweight smoke tests per PR.- Prioritize features by business impact for expensive checks.

**Why this works (Applied Scientist perspective)**- Combines programmatic static analysis with empirical temporal tests—catch both obvious and subtle leakage.- Lineage + availability matrix bridges training vs inference gap.- CI gates and runtime guards operationalize research constraints into reproducible, deploy-safe pipelines.

HardSystem Design

56 practiced

You must build a supervised learning benchmark for dozens of business datasets to compare logistic regression, random forest, and gradient-boosted trees consistently. As an Applied Scientist, design the benchmarking framework including data preprocessing rules, metric selection, statistical significance testing, and reproducibility practices.

Sample Answer

**Clarify scope & requirements**- Goal: consistent, automated benchmark comparing Logistic Regression (LR), Random Forest (RF), Gradient-Boosted Trees (GBT) across ~dozens of business datasets (classification/regression).- Constraints: varied imbalance, feature types, time-series vs IID, compute limits, explainability needs.

**High-level architecture**- Orchestrator (Airflow/Luigi) runs per-dataset experiment jobs → Training pipeline (scikit-learn/LightGBM) → Evaluation & stats → Registry (MLflow) + artifacts (models, metrics, plots) → Dashboard.

**Data preprocessing rules**- Standard pipeline (scikit-learn Pipelines): - Schema validation; drop/flag columns with >X% missing. - Missing: impute numerics with median, categoricals with new category; record indicator flags. - Categorical: target-aware encoding only inside CV folds (no leakage); use ordinal/one-hot hashing depending cardinality. - Numerical: Robust scaling (median/IQR) for LR; tree models use raw. - Time-series: time-based split; no future leakage. - Feature selection: consistent filter (low-variance, high-correlation) applied inside CV.

**Modeling & hyperparameter search**- Use nested CV: outer k-fold for evaluation, inner CV for hyperparameter tuning (Bayesian/Optuna).- Fixed compute budget per model for fairness.- Same feature pipeline for all models where applicable; model-specific transforms wrapped inside pipeline.

**Metric selection**- Primary metric per problem class: - Binary classification (balanced): ROC-AUC. - Binary (imbalanced): PR-AUC + recall@fixed-fpr. - Multi-class: macro-F1 and per-class ROC-AUC. - Regression: RMSE and MAE.- Secondary: calibration (Brier, reliability plots), inference latency, model size, fairness metrics where relevant.

**Statistical significance**- Per-dataset paired testing comparing model predictions from outer CV folds: - Use paired t-test if normal; else Wilcoxon signed-rank test. - Complement with bootstrap confidence intervals (stratified) for metric differences. - Correct for multiple comparisons across datasets/models: Benjamini-Hochberg FDR.- Aggregate comparison: hierarchical/mixed-effect model treating dataset as random effect to estimate average effect size.- Report effect size (Cohen’s d) + p-values.

**Reproducibility & provenance**- Fix random seeds at pipeline, CV split, and model RNG; record seeds.- Environment: Docker images, pinned packages (requirements.txt/conda), reproducible hardware spec.- Data & code versioning: DVC for data; Git for code; MLflow/Artifact store for models, metrics, and metadata (parameters, git SHA, dataset hash).- Automated CI that runs smoke tests and a small reproducible sample of benchmark.- Extensive logging (structured), and notebooks + README showing how to re-run specific dataset experiments.

**Automation & reporting**- Per-dataset HTML report: metrics, CI plots, calibration, feature importances, failure cases.- Global dashboard: rank models, per-domain performance, statistical significance flags.- Decision rules: e.g., prefer simpler LR when difference is small and calibration/latency matters.

This design ensures fair, statistically sound comparisons, prevents leakage, and makes every result reproducible and auditable.

HardTechnical

51 practiced

You are building a credit risk model subject to regulatory scrutiny. Compare tree ensemble models and logistic regression on interpretability, fairness, stability, and auditability. As an Applied Scientist, propose concrete mitigation strategies to meet regulatory requirements while maintaining predictive performance.

Sample Answer

**Clarify objective & constraints**- Goal: high predictive power for default prediction while satisfying regulators on interpretability, fairness, stability, auditability. Constraints: explainable decisions, reproducible training, controlled feature drift, demonstrable non-discrimination.

**Compare models (concise)**- Logistic regression - Interpretability: inherently transparent (coefficients, odds ratios, monotonicity easy). - Fairness: easier to inspect group effects; can include fairness-aware constraints. - Stability: high — small data changes produce gradual coefficient shifts. - Auditability: simple to document, fast to reproduce, easy model cards and decision logic.- Tree ensembles (e.g., XGBoost, Random Forest) - Interpretability: global/local opaque; needs post-hoc methods (SHAP, treeinterpreter). - Fairness: can amplify biases via interaction terms; harder to attribute cause. - Stability: more sensitive to data sampling and hyperparameters. - Auditability: more complex lineage, requires more artifacting.

**Mitigation strategies (concrete, regulatory-ready)**- Interpretability - Use constrained tree ensembles: monotonic constraints and max_depth to limit complexity. - Provide surrogate GLM or rule list (e.g., decision set) that approximates ensemble (report fidelity). - Produce SHAP global summaries, per-decision SHAP explanations, and simple natural-language decision rationales.- Fairness - Define regulatory metrics (e.g., equalized odds, disparate impact) per protected group. - Preprocess: reweighing / adversarial debiasing; in-training: fairness-regularized objective; postprocess: calibrated equalized odds. - Run counterfactual and subgroup performance audits; threshold per-group or use constrained optimization to meet targets.- Stability & robustness - Feature selection favoring stable, business-meaningful features; remove proxies of protected attributes. - Stability tests: bootstrap model retraining, population-shift simulations, and report parameter/SHAP variance. - Enforce rigorous CI for data schema; automated drift detection and automated retraining policies.- Auditability & governance - End-to-end lineage: data snapshot, feature code, seed, hyperparameters, model binary, evaluation notebooks in version control (DVC/Mlflow). - Produce a regulatory bundle: model card, technical note, validation results, counterfactuals, fairness reports, stress tests. - Explainability validation: demonstrate surrogate model fidelity, and human-review process for edge cases. - Logging: store per-decision inputs, scores, explanations, and appeals workflow for at least retention period required.

**Trade-offs & recommendation**- Start with ensemble for performance but enforce interpretability/constraint layers; maintain a vetted logistic regression baseline as a fallback and for audits. Demonstrate parity in key fairness and stability metrics; document trade-offs quantitatively (AUC vs fairness loss vs explanation fidelity). This combination meets regulatory needs while preserving predictive performance.

Unlock Full Question Bank

Get access to hundreds of ML Fundamentals: Supervised Learning Algorithms interview questions and detailed answers.

Join thousands of developers preparing for their dream job.