Fit Bradley–Terry Ratings from Pairwise Votes

Chatbot Arena ranks models from millions of "A vs B" votes. Online Elo depends on the order of the votes. The Bradley–Terry model instead fits all the votes at once: each model has a strength $p_i > 0$ , and $P(i \text{ beats } j) = \frac{p_i}{p_i + p_j}$ .

Fit it with Hunter's MM algorithm: repeat

p_i \leftarrow \frac{W_i}{\sum_{j \ne i} \frac{n_{ij}}{p_i + p_j}}

where $W_i$ is i's total wins and $n_{ij}$ the number of games between i and j, then rescale so the geometric mean of p is 1. Stop when the largest change in p is below tol.

Implement:

fit_bt(battles, iters=1000, tol=1e-10): battles is a list of (winner, loser) name pairs. Return {name: p}. (Assume every model has at least one win and one loss.)
to_elo(strengths, base=1000): base + 400 * log10(p)
win_prob(strengths, a, b)