RAG Retrieval Metrics: Recall@k, MRR, nDCG

medium
Unit 4 · RAG II0/3 to milestone

Before tuning prompts, check that your retriever finds the right documents. Three standard metrics, for one query with a ranked list retrieved and a set of relevant document ids:

  • Recall@k: the fraction of relevant docs that appear in the top k.
  • Reciprocal rank: 1/rank1/\text{rank} of the first relevant doc (ranks start at 1), or 0 if none appears.
  • nDCG@k with binary relevance: DCG=∑i=1krelilog⁡2(i+1)\text{DCG} = \sum_{i=1}^{k} \frac{rel_i}{\log_2(i + 1)}, divided by the best possible DCG, where all relevant docs come first.

Implement recall_at_k, reciprocal_rank and ndcg_at_k, each taking (retrieved, relevant) (plus k where named). Then implement evaluate(runs, k), where runs is a list of (retrieved, relevant) pairs, returning a dict with the mean "recall@k", "mrr" and "ndcg@k" across queries.

If relevant is empty, recall and nDCG are 0.

#rag#evaluation#retrieval2 visible + 3 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
RAG Retrieval Metrics: Recall@k, MRR, nDCG · TechGradify