Polynomial Decay with Linear Warm-up

easy
Checkpoint challenge
Solve this to show you know the material and jump straight to Unit 9. Everything before it counts as cleared.

BERT-style fine-tuning warms the learning rate up linearly, then decays it polynomially to an end value.

Implement poly_lr(step, total, warmup, lr_max, lr_end=0.0, power=1.0):

  • step < warmup: lr_max * step / warmup
  • otherwise: lr_end + (lr_max - lr_end) * (1 - (step - warmup) / (total - warmup)) ** power
  • steps past total stay at lr_end
#learning-rate#schedules2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Polynomial Decay with Linear Warm-up · TechGradify