Machine Learning Fundamentals Questions

Core machine learning concepts and terminology for conceptual understanding. Topics include supervised and unsupervised learning, regression and classification problems, training validation and test splits, cross validation, loss functions and optimization at a high level, model evaluation metrics, overfitting and underfitting, regularization concepts, and common basic model families such as linear models decision trees nearest neighbors and simple neural networks. Emphasis is on conceptual explanations and trade offs rather than deep mathematical derivations.

EasyTechnical

99 practiced

Briefly describe k-fold cross-validation and when it's useful. Mention one drawback of cross-validation for large datasets or specific production workflows.

HardSystem Design

68 practiced

You're designing a retraining cadence for a classification model. Describe factors that should influence retraining frequency (data volume, drift detection, cost, SLA), propose a hybrid schedule combining periodic and event-driven retraining, and explain how you'd validate retrained models before swap-in.

MediumTechnical

86 practiced

Explain early stopping in training. How does it act as implicit regularization? Describe how you would implement early stopping in a production training job with noisy validation metrics to avoid premature stopping.

MediumTechnical

128 practiced

Implement batch gradient descent to fit a linear regression model y = Xw + b in Python using numpy. Include an option for L2 regularization and early stopping based on validation loss. Provide runnable pseudo-code or real code and explain the computational complexity per iteration.

Sample Answer

**Approach (brief)** Use batch gradient descent updating weights w and bias b using gradients computed over the full training set. Add L2 penalty lambda for weights. Track validation loss and stop early if it doesn't improve for `patience` epochs.

**Code (runnable numpy)**

python

import numpy as np

def batch_grad_descent(X_train, y_train, X_val, y_val,
                       lr=1e-3, epochs=1000, l2=0.0, tol=1e-6, patience=10, verbose=False):
    n, d = X_train.shape
    # initialize
    w = np.zeros(d)
    b = 0.0
    best_val = np.inf
    best_params = (w.copy(), b)
    wait = 0

    def mse_loss(X, y, w, b, l2):
        preds = X.dot(w) + b
        mse = np.mean((preds - y) ** 2)
        reg = l2 * np.sum(w * w)
        return mse + reg

    for epoch in range(epochs):
        preds = X_train.dot(w) + b
        err = preds - y_train

        # gradients (batch)
        grad_w = (2.0 / n) * (X_train.T.dot(err)) + 2.0 * l2 * w
        grad_b = (2.0 / n) * np.sum(err)

        # parameter update
        w -= lr * grad_w
        b -= lr * grad_b

        # validation check
        val_loss = mse_loss(X_val, y_val, w, b, l2)
        if verbose and epoch % 50 == 0:
            print(f"epoch {epoch}, val_loss {val_loss:.6f}")

        # early stopping
        if val_loss + tol < best_val:
            best_val = val_loss
            best_params = (w.copy(), b)
            wait = 0
        else:
            wait += 1
            if wait >= patience:
                if verbose:
                    print(f"Early stopping at epoch {epoch}, best_val {best_val:.6f}")
                break

    return best_params, best_val

**Key concepts & reasoning**- Use full-batch gradients for stable convergence (suitable for research reproducibility).- L2 regularization adds 2*l2*w to gradient; it penalizes large weights and improves generalization.- Early stopping on validation loss provides implicit regularization and prevents overfitting.

**Complexity per iteration**- Computing preds: X.dot(w) costs O(n * d)- Computing grad_w: X.T.dot(err) costs O(n * d)- Other ops: O(d) or O(n)- Total per-iteration time: O(n * d)- Memory: O(n * d) to store X; parameters O(d)

**Edge cases / tips**- Standardize features for numeric stability.- For very large n, prefer mini-batch SGD or vectorized libraries.- Tune lr, l2, patience with validation.

MediumTechnical

99 practiced

Compare bagging (e.g., random forests) and boosting (e.g., gradient boosting machines) in terms of bias–variance behavior, sensitivity to noisy labels and outliers, interpretability, and typical use cases. Provide intuition about why the two approaches differ in practice.

Sample Answer

**Summary comparison**

- Bagging (e.g., Random Forests): reduces variance by averaging many independent high-variance learners. Tends to keep bias similar to base learner (often low for deep trees). Robust to noisy labels/outliers because averaging dilutes their influence. Interpretability: per-tree explanations exist (feature importance, partial dependence), but ensemble-level behavior is relatively interpretable. Typical use cases: stable baseline for tabular data, cases with many features and nonlinearities, when robustness is required.

- Boosting (e.g., Gradient Boosting Machines): sequentially fits weak learners to residuals, reducing bias by focusing on errors; can also reduce variance but is more prone to overfitting if not regularized. More sensitive to noisy labels and outliers because later learners focus on hard (possibly mislabeled) points. Interpretability: single learned function is more complex; SHAP/feature contributions help but model is harder to reason about than bagged trees. Typical use cases: high-accuracy tabular problems, ranking, when predictive performance is paramount and careful regularization/tuning is possible.

**Intuition for difference**

- Bagging relies on ensemble diversity: bootstrap + random feature splits make trees uncorrelated; averaging cancels variance but does not change systematic biases of individual trees.- Boosting constructs a correlated sequence where each stage corrects residual bias; that targeted correction reduces bias aggressively but amplifies the influence of difficult examples (including noise), explaining sensitivity to outliers.

**Practical notes for research applications**

- Use bagging when dataset has label noise or when you want robust, interpretable baseline. Use boosting for state-of-the-art accuracy; mitigate overfitting via learning rate, early stopping, subsampling, and robust loss functions (e.g., Huber) to reduce sensitivity to outliers.- For theoretical work, view bagging as variance reduction via model averaging; view boosting as functional gradient descent minimizing empirical risk, which explains its bias-reduction behavior and potential to overfit.

Unlock Full Question Bank

Get access to hundreds of Machine Learning Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.