Optimization

Knows that training minimises a loss function by adjusting parameters via gradient descent.

Implements SGD, momentum, Adam from scratch; understands learning rate schedules.

Diagnoses and fixes common training instabilities; chooses optimisers based on problem type; applies gradient clipping and warm-up.

Analyses convergence bounds; applies second-order methods, distributed optimisation, and constrained optimisation.

Develops new optimisation algorithms or convergence proofs for ML settings.