Training in float16 is fast, but small gradients underflow to zero. Mixed precision multiplies the loss by a large scale before backprop and divides the gradients afterwards. If any gradient overflows to inf or nan, the step is skipped and the scale is halved. After a run of good steps, the scale doubles.
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial