Cosine Annealing with Warm Restarts

medium
Unit 15 · Learning Rate & Optimizers0/2 to milestone

SGDR decays the learning rate along a cosine curve, then jumps back up. Cycle ii lasts Ti=T0⋅miT_i = T_0 \cdot m^i steps.

Within a cycle, at position tcurt_{cur}: η=ηmin+12(ηmax−ηmin)(1+cos⁡(π tcur/Ti))\eta = \eta_{min} + \tfrac12(\eta_{max} - \eta_{min})(1 + \cos(\pi\,t_{cur}/T_i)).

Implement sgdr_lr(step, lr_max, lr_min, T0, mult=1) for integer step >= 0.

#learning-rate#schedules2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Cosine Annealing with Warm Restarts · TechGradify