Sharpness-Aware Minimisation (SAM)

hard
Unit 22 · Optimizers0/1 to milestone

SAM seeks flat minima. Each step:

  1. compute the gradient gg at θ\theta
  2. move to the worst nearby point: ϵ^=ρ g∥g∥+10−12\hat\epsilon = \rho\,\frac{g}{\|g\| + 10^{-12}}
  3. compute the gradient gSAMg_{SAM} at θ+ϵ^\theta + \hat\epsilon
  4. update the original weights: θ←θ−η gSAM\theta \leftarrow \theta - \eta\,g_{SAM}
Pro problem

Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.

Start 7-day free trial
solution.py · Python 3.13
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Sharpness-Aware Minimisation (SAM) · TechGradify