A single bad batch can produce a huge gradient that wrecks training. Gradient clipping rescales all gradients together when their combined size is too large.
Implement clip_grad_norm(grads, max_norm):
grads is a list of NumPy arrays (one per parameter, any shapes).max_norm, multiply every gradient by max_norm / global_norm. Otherwise leave them as they are.(clipped_grads, global_norm), where global_norm is the value before clipping. Don't modify the input arrays.