Linear Algebra and Numerical Computing Questions

The linear algebra and numerical methods that machine-learning and scientific code rest on. Covers vectors, matrices and their operations, eigenvalues and eigenvectors and their geometry, singular value decomposition and PCA as linear algebra (variance explained, randomized SVD), least squares, the normal equations and rank deficiency, matrix calculus and gradients of quadratic objectives, conditioning and its effect on solvers, feature scaling and gradient descent, ill-conditioned covariance matrices and the fixes for them (shrinkage, truncated pseudo-inverse, dimension reduction), floating-point error and numerically stable algorithms (compensated and pairwise summation), iterative methods such as power iteration, and the mathematics of optimization: convexity, smoothness, strong convexity, convergence rates of gradient descent, preconditioning, saddle points and Hessian tests for stationary points, Lagrange multipliers and KKT conditions. Questions are about deriving, computing and reasoning about precision and convergence. Probability and statistical inference, learning-theory bounds, neural-network training practice, and library API mechanics are covered elsewhere.