Machine Learning Fundamentals Questions

Core machine learning concepts and terminology for conceptual understanding. Topics include supervised and unsupervised learning, regression and classification problems, training validation and test splits, cross validation, loss functions and optimization at a high level, model evaluation metrics, overfitting and underfitting, regularization concepts, and common basic model families such as linear models decision trees nearest neighbors and simple neural networks. Emphasis is on conceptual explanations and trade offs rather than deep mathematical derivations.

MediumTechnical

150 practiced

Explain nested cross-validation at a practical level and why it is recommended when you need an unbiased performance estimate while performing hyperparameter tuning, particularly for small datasets. Describe the roles of the outer and inner loops and how you would summarize results.

Sample Answer

Nested cross-validation is a practical protocol to get an unbiased estimate of a model’s generalization performance when you also tune hyperparameters — especially important for small datasets where optimistic bias is large.

High-level idea:- You have two loops: an outer CV that estimates final performance, and an inner CV that selects hyperparameters.

Outer loop (model evaluation):- Split data into K_outer folds. For each outer fold: hold out one fold as a true test set (never touched for tuning).- Train and tune only on the remaining K_outer-1 folds.

Inner loop (hyperparameter tuning / model selection):- Inside the training portion from the outer loop, run an inner cross-validation (e.g., K_inner folds or repeated CV) to evaluate hyperparameter candidates (grid, random, Bayesian, etc.).- Pick the hyperparameters that give the best average inner CV metric.

Then:- Retrain the model with the selected hyperparameters on the entire outer training set and evaluate once on the outer test fold. Record that outer test score.

Why this is unbiased:- Hyperparameter tuning leaks information about the test distribution if you tune and evaluate on the same splits; nested CV prevents that by reserving an untouched outer test for final evaluation. This avoids optimistic estimates that occur when hyperparameters overfit to validation folds — a major risk on small datasets.

Practical choices & tips:- Use K_outer = 5 or 10 and K_inner = 3–5; with very small data prefer more repeats or stratification rather than very large K to avoid high variance.- If compute is limited, use fewer outer folds and a smarter inner search (Bayesian/hyperband).- Always keep preprocessing (scaling, imputation, feature selection) inside the inner loop to avoid leakage.

Summarizing results:- Report the distribution of outer test scores: mean ± standard deviation (or median and IQR) and confidence intervals (e.g., 95% bootstrap CI).- Report the selected hyperparameters per outer fold (shows stability), and aggregate feature importances if relevant.- If you need a final model for deployment, retrain using the hyperparameter setting that was most frequently selected (or the one with best average inner performance) on the entire dataset, and note that its expected performance is the outer CV mean (not the inner CV best).

Example summary block:- “Nested CV (5 outer × 3 inner) mean AUC = 0.78 ± 0.04 (95% CI 0.70–0.85). Best hyperparameter sets were stable in 4/5 folds: C=1.0, penalty='l2'.”

This procedure balances honest evaluation with robust hyperparameter selection and is the recommended approach when you need trustworthy performance estimates, particularly on small datasets.

EasyTechnical

83 practiced

What are training, validation, and test splits? Describe a typical split strategy for a dataset of 100k examples and explain how you would modify splits if data is time-series or suffers from class imbalance.

HardTechnical

96 practiced

Compare bagging, boosting, and stacking ensemble methods. For each, explain the core idea, typical performance and robustness characteristics, training and inference cost implications, and scenarios where each is preferred in production (for example, high-latency batch scoring vs low-latency online scoring).

Sample Answer

Core idea- Bagging: train many independent base learners (often same algorithm) on bootstrap samples and average/vote (e.g., Random Forest). Reduces variance by decorrelating models.- Boosting: sequentially train weak learners where each focuses on previous errors (AdaBoost, Gradient Boosting, XGBoost/LightGBM/CatBoost). Reduces bias and builds strong learner.- Stacking: train diverse base models in parallel, then train a meta-learner on their out-of-fold predictions to combine strengths.

Typical performance & robustness- Bagging: strong when base learners are high-variance (trees). Robust to noise/outliers and less prone to overfitting than single model; gains plateau with many estimators.- Boosting: often achieves state-of-the-art accuracy on tabular data, but more sensitive to label noise/outliers and overfitting if not regularized (early stopping, shrinkage).- Stacking: can outperform both if base models are complementary; risk of overfitting the meta-learner if not using proper CV blending.

Training & inference cost- Bagging: training is embarrassingly parallel (scale horizontally). Inference cost = sum of base model costs (can be reduced via model pruning or selecting subset). Memory and compute scale with number of estimators.- Boosting: training is sequential (less parallelizable across boosting rounds), but many libraries optimize and support multi-threading. Inference cost is number of trees × depth — typically cheaper than large bagged ensembles for similar accuracy.- Stacking: highest complexity. Train multiple different models (parallelizable) plus meta-learner. Inference requires running all base models then meta model—latency and memory heavy.

Production scenarios- High-latency batch scoring (daily/weekly): stacking or large bagged ensembles are fine — maximize accuracy; storage and compute acceptable.- Low-latency online scoring (real-time): prefer single compact boosting model (lightweight XGBoost/LightGBM with constrained trees) or a pruned bagged model. Stacking is risky unless you can cache intermediate predictions or distill ensemble into a single model.- Resource-constrained edge: use a single optimized model or distill ensemble into a smaller model (knowledge distillation) or use shallow bagged trees.- Noisy labels / adversarial contexts: bagging is more robust; boosting needs strong regularization.- Interpretability/Compliance: single tree or small boosting model with SHAP explanations preferred over large ensembles; bagging/stacking harder to explain.

Practical tips- Use out-of-fold predictions and proper CV for stacking to avoid leakage.- Regularize boosting (early stopping, learning rate) to avoid overfit.- Consider model distillation, feature hashing, or approximation for low-latency deployments.

HardTechnical

79 practiced

A model shows high variance: training accuracy is much higher than validation accuracy. Provide a prioritized debugging checklist you would follow as an AI engineer to find root causes and resolve the issue. Include data checks, model adjustments, cross-validation strategies, and practical production constraints.

Sample Answer

1) Quick triage (1–2 hours)- Confirm metrics: compare train/val loss, accuracy, calibration, per-class performance; plot learning curves and confusion matrices to verify gap and overfitting pattern.- Reproduce with a fixed seed and single run to rule out randomness.

2) Data integrity & leakage (highest priority)- Verify no target leakage: inspect feature engineering pipelines, timestamps, join keys; run permutation importance and remove suspicious features.- Check train/val split correctness: ensure no overlap of entities, time-based leakage, or duplicated rows. For time series, use forward-chaining.- Label quality: sample-label audit, compute label noise rate; correct or relabel if noisy.

3) Dataset representativeness & scale- Compare feature distributions between train/val/test (KS, PCA, histograms). If mismatch, re-split or collect more representative data.- Address class imbalance with stratified splits, class-weighting, or resampling.

4) Model complexity & regularization- Reduce capacity: smaller network, fewer trees/depth, lower embedding dims.- Add/adjust regularization: L2, dropout, batchnorm; increase weight decay.- Use early stopping on validation metric with patience tuned conservatively.- Apply data augmentation for vision/NLP (mixup, synonym replacement).

5) Training procedure- Check learning rate schedule (too high can overfit quickly), batch size effects, optimizer stability.- Ensure sufficient training data and epochs — sometimes underfitting then overfitting behavior confused.

6) Robust evaluation & cross-validation- Replace single split with K-fold (stratified/time-series aware) to estimate variance; use nested CV for hyperparameter tuning.- Use holdout test only for final evaluation.

7) Ensembling & calibration- Use bagging or model averaging to reduce variance.- Calibrate probabilities (Platt, isotonic) if miscalibration observed.

8) Diagnostics and interpretability- Use SHAP/feature ablation to find unstable features.- Train simple baseline (logistic/regression) to check signal-to-noise.

9) Production constraints & operational checks- Verify latency/memory limits when reducing model size or ensembling.- Monitor drift: deploy canary, monitor input distributions and prediction confidence; set automated alerts and rollback policies.- Build retraining pipeline with validation gating and reproducible preprocessing.

10) Prioritized experiment plan (2–4 weeks)- Short tests: fix leakage/check splits, run K-fold, add regularization.- Medium: data cleaning/augmentation and model-size tuning.- Longer: collect more representative data or remodel features.

Metrics & stopping rules:- Aim to reduce generalization gap by X% or improve validation metric by a meaningful delta (e.g., +2–5% absolute) before deploying changes.- Use statistical tests across CV folds to confirm improvement.

EasyTechnical

88 practiced

Define the entries of a binary classification confusion matrix (TP, FP, TN, FN). Explain with a short numeric example where accuracy is misleading and why metrics like precision, recall, or F1 make more sense in that context.

Unlock Full Question Bank

Get access to hundreds of Machine Learning Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.