Entropy and Information Gain

easy
Unit 5 · Clustering & Decision Trees0/3 to milestone

Decision trees like ID3 and C4.5 choose splits by information gain: how much a split reduces label entropy.

Implement:

  • entropy(labels): Shannon entropy in bits, −∑pklog⁡2pk-\sum p_k \log_2 p_k (0 for an empty or pure set)
  • information_gain(labels, mask): entropy of the parent minus the size-weighted entropy of labels[mask] and labels[~mask]
#decision-trees#information-theory3 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Entropy and Information Gain · TechGradify