KL Divergence · TechGradify

KL divergence measures how different one probability distribution $P$ is from another $Q$ :

D_{KL}(P \,\|\, Q) = \sum_i p_i \log \frac{p_i}{q_i}

It's everywhere in AI engineering: the KL penalty that keeps RLHF-tuned models close to the base model, distillation losses, and comparing a fine-tuned model's token distribution with the original's.

Implement kl_divergence(p, q) using the natural log.

Rules

Terms where $p_i = 0$ contribute 0, even if $q_i = 0$ .
If some $p_i > 0$ where $q_i = 0$ , the divergence is infinite: return float("inf").
Inputs are probability vectors (non-negative, summing to 1).

Note that KL is not symmetric: $D_{KL}(P\|Q) \ne D_{KL}(Q\|P)$ in general.