Neural Network Fundamentals Questions

Core concepts and building blocks of artificial neural networks including neuron models, network layers and layer types (dense fully connected, convolutional, recurrent, and transformer blocks), activation functions and their properties (for example ReLU, sigmoid, tanh, GELU), forward propagation, loss functions, and the basics of optimization such as gradient descent. Explain backpropagation conceptually and mathematically using the chain rule, describe forward and backward passes, and discuss common practical issues such as vanishing and exploding gradients, weight initialization, regularization techniques, and when specific architectures are appropriate for particular problem domains (for example convolutional networks for vision and sequence models for temporal data).