Cross-Entropy with Label Smoothing

easy
Unit 5 · Losses0/3 to milestone

Training on hard one-hot labels pushes models to be over-confident. Label smoothing spreads a little probability to the other classes: with smoothing ϵ\epsilon and CC classes, the target for the true class is 1−ϵ+ϵ/C1 - \epsilon + \epsilon/C, and every other class gets ϵ/C\epsilon/C.

Implement smoothed_targets(labels, n_classes, eps) and smoothed_cross_entropy(logits, labels, eps=0.1): the mean over the batch of −∑ctclog⁡pc-\sum_c t_c \log p_c, using a numerically stable log-softmax. Also return the gradient with respect to the logits, which is (softmax - targets) / N: (loss, grad).

#loss-functions#regularization#training2 visible + 2 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Cross-Entropy with Label Smoothing · TechGradify