Machine Learning Fundamentals Questions

Core machine learning concepts and terminology for conceptual understanding. Topics include supervised and unsupervised learning, regression and classification problems, training validation and test splits, cross validation, loss functions and optimization at a high level, model evaluation metrics, overfitting and underfitting, regularization concepts, and common basic model families such as linear models decision trees nearest neighbors and simple neural networks. Emphasis is on conceptual explanations and trade offs rather than deep mathematical derivations.

MediumTechnical

139 practiced

Compare grid search, random search, and Bayesian optimization for hyperparameter tuning. Discuss their strengths and weaknesses in terms of convergence speed, ability to find important hyperparameters, parallelizability, and handling conditional hyperparameters for expensive models.

Sample Answer

Grid search, random search, and Bayesian optimization each trade off exploration, efficiency, and practicality for hyperparameter tuning—here's a focused comparison across the requested axes.

Convergence speed- Grid search: slow. Exhaustive across a predefined grid; scales exponentially with dims so convergence to good values is poor unless grid is tiny.- Random search: faster than grid in practice because it explores more unique combinations; expected to find near-optimal values faster when only a few hyperparameters matter.- Bayesian optimization (BO): fastest for expensive objective evaluations. Builds a surrogate (e.g., Gaussian Process) and uses acquisition functions to focus evaluations where improvement is likely—fewer expensive runs needed.

Ability to find important hyperparameters- Grid search: inefficient—wastes budget on unimportant dims and can miss good values between grid points.- Random search: strong—by sampling across full ranges it has higher probability to hit important hyperparameters, especially when few are relevant.- BO: best at concentrating search on promising regions and revealing interactions; surrogate models can prioritize important dims but may struggle if dimensionality is high.

Parallelizability- Grid search: trivially parallelizable (independent jobs).- Random search: also trivially parallelizable.- BO: inherently sequential (acquisition depends on prior results). Parallel BO variants exist (batch acquisition, asynchronous methods) but are more complex and less straightforward than grid/random.

Handling conditional (nested) hyperparameters & expensive models- Grid search: awkward—conditional dims blow up grid size; often infeasible for expensive training.- Random search: tolerates conditional spaces naturally (sample only active parameters); still may be wasteful but simpler to implement for costly models.- BO: can handle conditional hyperparameters using tree-structured or conditional-aware surrogates (e.g., Tree-structured Parzen Estimator, SMAC). For very expensive models BO is preferred because it reduces the number of runs, but surrogate quality degrades with high dimensionality or noisy evaluations.

Practical guidance for a data scientist- Use random search (or simple Bayesian wrappers like Hyperopt) for moderate budgets and many dims.- Use BO (e.g., Gaussian Process-based or TPE/SMAC) for very expensive evaluations and low-to-moderate dimensionality, plus when conditional logic is present choose TPE/SMAC variants.- Use grid only for small, low-dimensional, well-understood searches or to produce reproducible baselines.- Combine strategies: start with random to explore broadly, then switch to BO to refine promising regions.

EasyTechnical

83 practiced

What are training, validation, and test splits? Describe a typical split strategy for a dataset of 100k examples and explain how you would modify splits if data is time-series or suffers from class imbalance.

HardTechnical

97 practiced

Propose a practical framework to detect and mitigate unfairness in a binary classification model used for loan approvals. Include selection of fairness metrics, data collection considerations, pre-processing, in-processing and post-processing mitigation techniques, and how to communicate trade-offs with stakeholders.

Sample Answer

Requirements & goals:- Binary loan approval model must minimize unfair disparate treatment and disparate impact across protected groups (e.g., race, gender, age), comply with regulations, and preserve business utility (approval rate, default rate).

Framework overview (detect + mitigate):1. Data collection & exploration- Gather sensitive attributes where legally permitted; if unavailable, collect proxies cautiously and document limitations.- Perform exploratory parity checks: base rates (approval, default) by group, feature distributions, missingness patterns, label bias (e.g., historical denial patterns).- Check sample size per group; if small, quantify uncertainty (confidence intervals, bootstrap).

2. Fairness metrics (selection)- Group fairness: Demographic parity difference, Equal opportunity difference (TPR gap), Equalized odds (TPR & FPR gaps).- Individual fairness: Similar individuals treated similarly (use distance-based checks).- Calibration-by-group: predicted probability matches observed default per group.- Choose metrics tied to business goals: for credit, prioritize equal opportunity (similar default detection) and calibration for risk-priced decisions.

3. Pre-processing techniques- Reweighing or stratified resampling to correct label or representation imbalance.- Feature repair / remove proxies: test impact of dropping or transforming features correlated with sensitive attributes.- Generate synthetic samples (SMOTE variants) only with caution and preserve label integrity.

4. In-processing techniques- Regularized objectives: add constraints or penalties (e.g., adversarial debiasing, fairness-constrained optimization like equalized odds constraints).- Use calibrated probabilistic models (e.g., constrained logistic regression) to maintain interpretability.- Monitor trade-offs via Pareto frontier of accuracy vs fairness during training.

5. Post-processing techniques- Threshold adjustment per group to equalize TPR/FPR or achieve target acceptance rates.- Score transformation methods that maintain calibration when possible.- Use post-hoc explanations to validate decisions (SHAP) and detect residual disparate treatment.

6. Evaluation & monitoring- Split evaluation: validation across demographic slices, and temporal holdouts to detect drift.- Deploy fairness dashboards: metrics, confidence intervals, sample counts, and alerting for threshold breaches.- A/B test mitigations measuring business KPIs (default rate, revenue) and fairness metrics.

7. Governance & communication with stakeholders- Present clear trade-offs: show Pareto curves and example borrower profiles illustrating who gains/loses under each mitigation.- Use business terms: expected change in approval rate, expected change in default/loss, compliance risk reduction.- Recommend pilot timeline, rollback criteria, and monitoring SLAs.- Document assumptions, data gaps, and legal constraints; involve legal/compliance and impacted-community feedback early.

Example: If TPR gap (minority vs majority) = 8%, propose reweighing + constrained training to reduce gap <2% with estimated 0.7% increase in predicted default — show simulations and a 3-month pilot to validate in production.

Key trade-off reasoning:- Some fairness constraints reduce overall accuracy or change risk mix—quantify downstream financial impact.- Prefer interventions that preserve calibration for risk-based pricing; use post-processing when a model must remain unchanged for regulatory reasons.- Iterate: mitigation + monitoring + stakeholder alignment ensures practical, auditable fairness.

HardTechnical

82 practiced

Explain gradient descent in intuitive terms and discuss common convergence issues such as plateaus, oscillations, and saddle points. Compare learning rate schedules and optimizers (SGD, momentum, RMSprop, Adam) conceptually and describe when you would prefer one over another.

Sample Answer

Gradient descent — intuition:Think of minimizing a loss as walking downhill on a landscape. At each step you estimate the slope (gradient) and take a step opposite it. The step size is the learning rate: too small → slow; too large → overshoot.

Common convergence issues:- Plateaus: regions with very small gradients (flat) slow progress. Solutions: increase learning rate temporarily, use adaptive methods (Adam/RMSprop), or add momentum to carry you through.- Oscillations: if the learning rate is too high or curvature differs across directions, updates bounce across narrow valleys. Solutions: reduce learning rate, use momentum to damp oscillations, or use second-moment normalization (RMSprop/Adam).- Saddle points: points with zero gradient but not minima (some directions up, some down). High-dimensional problems have many saddles. Momentum and noise (stochastic mini-batches) help escape saddles; adaptive methods also re-scale updates to move away.

Learning-rate schedules (conceptual):- Fixed: simple but brittle.- Step decay / exponential decay: lower LR over time to refine convergence.- Linear/cyclical schedules: cyclic LR can help escape local traps and improve generalization.- Warm restarts/annealing: combine exploration early with fine-tuning later.

Optimizers — conceptual comparison and when to prefer:- SGD (vanilla mini-batch): simple, low memory, often better generalization for large datasets and convex problems. Prefer when you need reproducibility and simpler regularization tuning.- SGD + Momentum: adds velocity to smooth updates and accelerate across shallow directions; good default for deep nets when gradients are noisy.- RMSprop: divides by root mean square of recent gradients (adaptive per-parameter LR), helps with ill-conditioned problems and faster convergence on nonstationary objectives.- Adam: combines momentum and RMSprop (first and second moments). Fast, robust, and works well out-of-the-box for many architectures and sparse gradients; beware of possible worse generalization and need for weight decay tuning.

Practical rule-of-thumb:Start with Adam for fast prototyping; switch to SGD with momentum (and tuned LR schedule) for final training when aiming for best generalization. Use adaptive optimizers or cyclical LR when facing plateaus or difficult curvature; use small batches/momentum to help escape saddle points.

EasyTechnical

100 practiced

Define overfitting in practical terms. List and explain five actionable techniques you would try to reduce overfitting when training a model on tabular data for a production use case.

Unlock Full Question Bank

Get access to hundreds of Machine Learning Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.