Cross-Entropy with an Ignore Index

medium
Unit 14 · Losses & CNNs0/3 to milestone

Padded sequence batches use a special target (often -100) for positions that shouldn't count. Implement cross_entropy(logits, targets, ignore_index=-100):

  • logits has shape (N, C), targets shape (N,)
  • the loss is the mean negative log-likelihood over non-ignored positions only (stable log-softmax)
  • if every position is ignored, return 0.0

Also implement cross_entropy_grad(...) with the same arguments, returning the gradient with respect to logits (zero rows for ignored positions).

#losses#sequence-models#batching2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Cross-Entropy with an Ignore Index · TechGradify