Understand backpropagation and gradient flow in neural networks
Gradients become too small, preventing effective learning in deeper layers.
Gradients become too large, causing unstable training and divergence.
Gradients are in the optimal range for effective learning.
Residual Connections: Allow gradients to flow directly to earlier layers
Batch Normalization: Normalize inputs to stabilize gradients
Gradient Clipping: Limit gradient magnitude to prevent explosion
Better Initialization: Use Xavier or He initialization
LSTM/GRU: Use gated architectures for sequential data