ML Fundamentals: Supervised Learning Algorithms Questions

Deep understanding of linear regression, logistic regression, decision trees, random forests, SVMs, and ensemble methods. Be able to explain: how each algorithm works, advantages/disadvantages, when to use each, regularization techniques (L1/L2), hyperparameter tuning, and how to handle overfitting.

MediumTechnical

54 practiced

Design a cross-validation approach suitable for time-series supervised learning where future information must never leak into training. Explain walk-forward validation, expanding-window versus sliding-window strategies, and how to tune hyperparameters without leaking future data.

Sample Answer

Walk-forward validation (a.k.a. rolling-origin) is the standard for time-series: you simulate how models would have been trained in production by repeatedly training on past data and testing on a subsequent holdout period — ensuring no future information leaks into training.

Core patterns:- Expanding-window: start with an initial training window, then at each step add the next train block (train grows), and evaluate on the next fixed-length test window. Good when more data improves the model and the process is relatively stationary.- Sliding-window (fixed-size rolling window): keep the training window length fixed (drop oldest data as you add newest). Use this when the series is non-stationary and older data becomes less relevant.

How to tune hyperparameters without leaking future data:- Use nested time-aware validation. Outer loop: walk-forward splits to estimate generalization. Inner loop: for each outer training set, perform hyperparameter selection using additional inner walk-forward splits that only use data up to that outer training end. Select hyperparameters based solely on inner validation results and then retrain on the full outer training set before testing on the outer test fold.- Apply purging/embargo around test windows when features or labels are derived with temporal overlap (e.g., look-ahead indicators, overlapping target windows) to avoid leakage.- Use time-series-aware CV utilities (e.g., sklearn's TimeSeriesSplit) or custom generators that yield (train_idx, test_idx) respecting temporal order.

Practical tips:- Choose test window length to match business horizon (e.g., weekly, monthly).- Report metrics across all outer folds (mean ± std).- If compute-limited, tune on a recent contiguous walk-forward subset and validate on later periods.- Monitor calibration and concept drift; consider re-tuning periodically.

Example pseudo-process:1) Define initial_train_end, test_horizon, step.2) For t in rollout: - train = data[:t] - inner CV on train using smaller walk-forward splits -> choose params - fit final model on train with chosen params - test on data[t:t+test_horizon]3) Aggregate metrics.

EasyTechnical

48 practiced

Describe common regression metrics: MSE, RMSE, MAE, and R-squared. Explain their sensitivities to outliers and give guidance on when to prefer one metric over another based on business objectives.

EasyTechnical

56 practiced

Describe how decision trees choose splits for classification tasks. Explain impurity measures such as Gini impurity and entropy, how information gain is computed, and typical stopping criteria. Discuss the tendency of deep trees to overfit and simple remedies like pruning and max depth.

EasyTechnical

58 practiced

Define bagging, boosting, and stacking at a high level. For each technique, explain the core idea, typical algorithms that implement it, and when you might prefer one over the others in supervised learning problems.

EasyTechnical

60 practiced

Explain precision, recall, F1 score, specificity, and accuracy for binary classification. Give concrete business scenarios where precision is more important than recall and vice versa. Describe how class imbalance affects metric choice and threshold selection.

Sample Answer

Precision, recall, F1, specificity and accuracy (binary classification) — definitions and intuition:- Precision = TP / (TP + FP). Of predicted positives, how many are correct. High precision means few false alarms.- Recall (sensitivity) = TP / (TP + FN). Of actual positives, how many we catch. High recall means few misses.- F1 score = 2 * (Precision * Recall) / (Precision + Recall). Harmonic mean balancing precision and recall; useful when you want a single metric and classes are imbalanced.- Specificity = TN / (TN + FP). Of actual negatives, how many are correctly identified (true negative rate).- Accuracy = (TP + TN) / (TP + TN + FP + FN). Overall correctness; can be misleading with class imbalance.

Business scenarios:- Precision more important: Email spam filter for an executive inbox — false positives (legitimate emails marked spam) are costly, so prioritize precision even if some spam slips through. Medical diagnostic confirmation test where a positive triggers invasive procedure.- Recall more important: Screening for a contagious disease in a pandemic — missing cases is dangerous, so maximize recall even if you get more false positives. Fraud detection initial flagging step where human review follows.

Class imbalance and metric/threshold choices:- With rare positive class, accuracy can be high by always predicting negative; use precision/recall or AUCPR instead. If costs differ (FP vs FN), choose metric accordingly (precision vs recall).- Threshold selection: move threshold up to increase precision (fewer positives, fewer FPs) and down to increase recall (more positives, fewer FNs). Use precision-recall curve and choose threshold based on business cost (e.g., expected cost = FP_cost * FP_rate + FN_cost * FN_rate). Calibrate probabilities (Platt/isotonic) so threshold decisions map to real risk.

Practical tip: report a set of metrics (precision, recall, F1, specificity, ROC-AUC / PR-AUC) and choose threshold with stakeholder-aligned cost function or by optimizing a metric (e.g., maximize F1 or a weighted utility).

Unlock Full Question Bank

Get access to hundreds of ML Fundamentals: Supervised Learning Algorithms interview questions and detailed answers.

Join thousands of developers preparing for their dream job.