Classification and Regression Fundamentals Questions
Covers the core concepts and distinctions between classification and regression in supervised learning. Classification predicts discrete categories, either binary or multi class, while regression predicts continuous numerical values. Candidates should understand how to format and encode target variables for each task, common algorithms for each family, and the theoretical foundations of representative models such as linear regression and logistic regression. For regression, know least squares estimation, coefficients interpretation, residual analysis, assumptions of the linear model, R squared, and common loss and error measures including mean squared error, root mean squared error, and mean absolute error. For classification, know logistic regression with its sigmoid transformation and probability interpretation, decision trees, k nearest neighbors, and other basic classifiers; understand loss functions such as cross entropy and evaluation metrics including accuracy, precision, recall, F one score, and area under the receiver operating characteristic curve. Also be prepared to discuss model selection, regularization techniques such as L one and L two regularization, handling class imbalance, calibration and probability outputs, feature preprocessing and encoding for targets and inputs, and trade offs when choosing approaches based on problem constraints and data characteristics.
EasyTechnical
32 practiced
Describe why feature scaling matters. For each of these algorithms explain whether scaling is required and why: k-NN, SVM, logistic regression trained with gradient descent, decision trees, and PCA. Given numeric features with heavy outliers and skew, which scaling/transformations would you apply and why?
Sample Answer
Feature scaling matters because many ML algorithms rely on distance metrics or gradient magnitudes; features on larger scales can dominate training, slow or destabilize convergence, and distort model decisions. Below I state whether scaling is required and why.k-NN — Required. k-NN uses distance (Euclidean, etc.); unscaled features with larger ranges dominate neighbors and mislead similarity.SVM — Required (usually). SVM with RBF or linear kernels is sensitive to feature scales: scale affects margin and kernel distances; scaling helps optimization and avoids one feature dominating the decision boundary.Logistic regression (gradient descent) — Required. Feature scales affect gradient magnitudes and therefore convergence speed; scaling leads to better-conditioned optimization and comparable regularization across features.Decision trees — Not required. Trees split on single features using thresholds; scale doesn't change split ordering. Scaling typically provides no benefit for pure tree-based models (but can matter for tree ensembles when combined with methods that use distance).PCA — Required. PCA finds directions of maximum variance; unscaled features with larger variance will dominate principal components. Standardize (zero mean, unit variance) if you want components reflecting correlation structure rather than raw scale.Heavy outliers and skew — recommended transforms:- Outliers: prefer robust scaling (remove median, divide by IQR) rather than standard scaler; or use clipping. RobustScaler reduces outlier influence.- Skew: apply monotonic transforms to reduce skew before scaling: log(x) for positive values, or Yeo-Johnson/Box-Cox (Box-Cox requires positive) to stabilize variance.- If you need bounded ranges (e.g., for neural nets or min-max sensitive algorithms), after robust or log transform apply MinMaxScaler.- For extreme distributions, QuantileTransformer (uniform or normal output) can make features Gaussian-like and reduce outlier impact, but it warps relationships (use with caution).Practical pipeline: handle zeros/negatives, apply log or Yeo-Johnson for skew, then RobustScaler (or StandardScaler if outliers removed), and finally optional MinMaxScaler for algorithms needing bounded inputs.
MediumTechnical
29 practiced
Compare model-agnostic interpretability techniques (LIME, SHAP) and global explanations like Partial Dependence Plots (PDP). For a production classifier, describe how you would use these tools to explain a single prediction and to audit model behavior across cohorts.
Sample Answer
LIME, SHAP, and PDP serve different but complementary roles. LIME and SHAP are model-agnostic local explainers: they explain individual predictions. PDPs are global: they show average effect of a feature on model output (marginalized over data).Comparison (brief):- LIME: fits a local surrogate (usually linear) around one instance. Fast, intuitive, but sensitive to neighborhood sampling and can be unstable.- SHAP: grounded in game theory (Shapley values). Offers consistent, additive attributions and theoretical guarantees. TreeSHAP is exact and fast for tree models; KernelSHAP is model-agnostic but slower.- PDP: plots E[f(x) | x_j = v] vs v. Good for global trends and detecting nonlinearity, but it marginalizes other features and can mislead with correlated features or interactions.How to explain a single prediction in production:1. Use SHAP (TreeSHAP if model is tree-based) to get feature attributions — stable, additive, and comparable across features.2. Complement with LIME as a sanity check: see if LIME’s local linear surrogate agrees with SHAP for top features.3. Present top positive/negative contributors with feature values and counterfactual suggestions (what minimal change flips prediction).4. Capture and log explanations and model inputs for auditability.How to audit model behavior across cohorts:1. Compute SHAP summaries (mean absolute SHAP) per cohort to see which features drive differences.2. Generate PDPs and ICE plots per cohort to detect differing marginal effects or interactions.3. Use cohort-wise feature importance drift detection (monitor distributions of SHAP values over time).4. Watch for pitfalls: correlated features bias PDPs; KernelSHAP is expensive on large cohorts; LIME instability — use multiple runs and aggregate.5. Action: if cohort differences reveal bias, run targeted counterfactual tests, retrain with stratified sampling or add interaction terms, and re-evaluate post-deployment.Key takeaways: use SHAP for principled local attributions and cohort-level aggregation, LIME as a quick local check, and PDP/ICE for global effect visualization — always account for feature correlation, interactions, and computational cost.
EasyTechnical
30 practiced
Given this confusion matrix for a binary classifier evaluated on 1,000 samples: TP=70, FP=30, FN=20, TN=880. Compute accuracy, precision, recall, F1 score, specificity and briefly interpret each metric in the context of a rare positive class. Which metric(s) would you prioritize if the positive class represents fraud?
Sample Answer
Accuracy: (TP+TN)/N = (70+880)/1000 = 950/1000 = 0.95 (95%).Precision: TP/(TP+FP) = 70/(70+30) = 70/100 = 0.70 (70%).Recall (Sensitivity): TP/(TP+FN) = 70/(70+20) = 70/90 ≈ 0.777... (77.8%).F1 score: 2 * (Precision * Recall) / (Precision + Recall) = 2 * 0.70 * 0.7778 / (0.70 + 0.7778) ≈ 0.737 (73.7%).Specificity: TN/(TN+FP) = 880/(880+30) = 880/910 ≈ 0.967 (96.7%).Interpretation (rare positive class):- Accuracy (95%): Seems high but is misleading with a rare positive class—most samples are negatives, so a trivial classifier predicting all negatives would still get high accuracy.- Precision (70%): Of instances predicted fraud, 70% were true frauds — measures the trustworthiness of positive predictions; important to avoid wasting investigation effort on false alarms.- Recall (77.8%): The model detects ~78% of actual frauds — measures how many real frauds are caught; missing frauds (false negatives) can be costly.- F1 (73.7%): Harmonic mean of precision and recall — useful when you want a single balance metric.- Specificity (96.7%): Most legitimate transactions are correctly identified as non-fraud — low false positive rate.Which metrics to prioritize for fraud:- Primarily prioritize recall if the business cost of missed fraud is very high (loss, regulatory risk), but ensure precision doesn't collapse (too many false positives).- Practically, optimize for a good trade-off: maximize recall subject to a minimum acceptable precision (or optimize F-beta with beta>1 if recall is more important).- Use precision-recall curve and PR-AUC (better than ROC-AUC under class imbalance), and consider business costs to pick an operating point.
MediumTechnical
32 practiced
Explain limitations of ROC-AUC when classes are highly imbalanced. Define precision-recall curve and PR-AUC and show conceptually why PR-AUC is often preferred for rare positive classes. What is partial AUC (pAUC) and when would you use it?
Sample Answer
ROC-AUC measures the trade-off between true positive rate (TPR = recall) and false positive rate (FPR) across thresholds. Limitation with highly imbalanced classes: FPR is computed using negatives (TN+FP). When positives are rare, even many false positives produce a very small FPR, so ROC curves can look deceptively good—high AUC even if the classifier yields poor precision (many false alarms). ROC-AUC thus can mask poor positive-prediction quality that matters in rare-event settings.Precision-Recall (PR) curve plots Precision = TP / (TP+FP) versus Recall = TP / (TP+FN) across thresholds. PR-AUC is the area under that curve. Because precision directly penalizes false positives, PR curves emphasize performance on the positive class. Conceptually, for rare positives you care about how many predicted positives are correct (precision) as well as coverage (recall); PR-AUC focuses on that whereas ROC-AUC dilutes the impact of FP by large negative class counts. Thus PR-AUC is often preferred when positives are rare or when the cost of false positives is high.Partial AUC (pAUC) is the AUC computed over a restricted range of interest on the x-axis (e.g., FPR in [0, 0.01] for ROC, or recall in [0.7,1] for PR). Use pAUC when only a specific operating region matters—e.g., clinical screening where you require FPR < 1%, or business rules that require minimum recall. pAUC focuses evaluation and model selection on the practically relevant threshold region rather than averaging over all thresholds.
EasyTechnical
32 practiced
Explain the logistic regression model: show how the linear predictor is transformed to a probability using the sigmoid function, derive the relationship between coefficients and log-odds, and describe how multiclass classification is handled (one-vs-rest vs softmax). What are practical signs that logistic regression is a reasonable first model for a classification task?
Sample Answer
Logistic regression models the probability of a binary outcome y∈{0,1} given features x by first forming a linear predictor z = w^T x + b, then mapping z to [0,1] with the sigmoid σ(z)=1/(1+e^{-z}). So P(y=1|x)=σ(w^T x + b).Odds = P(y=1)/P(y=0) = σ(z)/(1−σ(z)) = e^{z}. Taking logs gives the log-odds (logit):logit(P(y=1|x)) = log(P(y=1)/P(y=0)) = z = w^T x + b.Thus each coefficient w_j is the change in log-odds per unit change in x_j; exp(w_j) is the multiplicative change in odds (odds ratio).Multiclass:- One-vs-Rest (OvR): train K binary logistic models, each distinguishing class k vs all others; predict class with highest predicted probability. Simple and works well when classes are imbalanced or roughly separable pairwise.- Softmax (multinomial logistic): generalizes the linear predictor to K scores z_k = w_k^T x + b_k and uses softmax: P(y=k|x)=exp(z_k)/Σ_j exp(z_j). Softmax models class probabilities jointly and is preferred when classes are mutually exclusive and you want calibrated multiclass probabilities.Practical signs logistic regression is a good first model:- Relationship roughly linear in log-odds (no strong nonlinear interactions).- Dataset size moderate; features not massively high-dimensional relative to samples.- Features are numeric or easily transformed (one-hot categorical).- Need for interpretability and feature importance (coefficients map to odds ratios).- Baseline performance acceptable and fast training/prediction required.Regularization (L2/L1) is commonly used to prevent overfitting, handle multicollinearity, and perform feature selection (L1). Check calibration, residual patterns, and feature interactions to decide if a more complex model is needed.
Unlock Full Question Bank
Get access to hundreds of Classification and Regression Fundamentals interview questions and detailed answers.
