Regularization and Generalization Questions

Covers the principles and practices used to improve model generalization and prevent overfitting. Candidates should understand overfitting and underfitting, how to diagnose them using learning curves and evaluation on validation and test sets, and the bias variance trade off. Know common regularization techniques including L one and L two regularization and elastic net, weight decay, dropout and its variants, batch normalization and layer normalization, early stopping, data augmentation, label smoothing, and ensemble methods such as bagging and boosting. Discuss practical considerations: how to select and tune regularization strength and other hyperparameters using cross validation, how training data size and model capacity affect choices, and how to detect noisy labels and class imbalance and mitigate their effects. Be prepared to explain implementation details in machine learning frameworks, the interaction between optimization and regularization, and production concerns for large models including scaling and monitoring generalization. For senior candidates, demonstrate deeper knowledge of theoretical generalization bounds, regularization strategies for very large models, and trade offs when combining multiple techniques.