QA Evaluation: Exact Match and Token F1

easy
Checkpoint challenge
Solve this to show you know the material and jump straight to Unit 5. Everything before it counts as cleared.

For question answering and RAG, answers rarely match the gold answer character for character. The SQuAD metrics normalise text first:

  1. lowercase,
  2. remove punctuation,
  3. remove the articles a, an and the as whole words,
  4. collapse whitespace.

Then:

  • Exact match: 1.0 if the normalised strings are equal, else 0.0.
  • F1: token-level overlap (counting repeated tokens with a multiset), F1=2PRP+RF1 = \frac{2PR}{P+R}. If either side has no tokens, F1 is 1.0 when both are empty and 0.0 otherwise.

Implement normalize_answer, exact_match and f1_score, then score(prediction, gold_answers), which returns the max EM and the max F1 over all gold answers: {"em": ..., "f1": ...}.

#evaluation#nlp#rag2 visible + 2 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
QA Evaluation: Exact Match and Token F1 · TechGradify