Deep Learning (DL) Review. Optimization for DL. Initialization. Neural Networks Loss Landscape. Implicit Regularization of Stochastic Gradient Descent (SGD). Sharpness Aware Minimization (SAM). Sharp minima and flat minima. Edge of Stability of Training. Geometric complexity. Normalization. Residual Connections. Double descent. Grokking. Lottery Ticket Hypothesis (LTH). Invariance. Pruning. Scaling and phase transitions. Understanding Transformers. Mode connectivity. Plasticity. Continual Learning.
- Responsable du site: Sarath Chandar Anbil Parthipan