BM25 Keyword Search

Vector search misses exact keywords like product codes and names, so production RAG pairs it with BM25, the classic keyword-ranking function behind Elasticsearch and Lucene.

For a query $q$ and document $D$:

where $f(t, D)$ is how often term $t$ appears in $D$, $|D|$ is the document's length in tokens, avgdl is the average document length, $N$ is the number of documents and $n_t$ is how many documents contain $t$.

Implement bm25_scores(query, docs, k1=1.5, b=0.75) returning one score per document, in order.

Tokenising: lowercase, then split on whitespace. Count each distinct query term once, even if the query repeats it. Terms that appear in no document contribute nothing.