Check That Citations Are Grounded

medium
Unit 17 · RAG II0/3 to milestone

A RAG answer cites sources like [1] and [2]. Does each cited source actually support the sentence? A cheap first check is lexical overlap.

Implement check_citations(answer, sources, threshold=0.5):

  • Split the answer into sentences on ., ! or ? followed by whitespace or the end, and keep non-empty sentences.
  • For each sentence, find citation markers [n] (1-based indices into sources).
  • Support for a cited source = the fraction of the sentence's content words (lowercase alphanumeric words of 4+ letters, with citation markers removed) that appear in that source (also lowercased).
  • Return a list with one dict per sentence: {"sentence": ..., "citations": [n, ...], "supported": bool}, where supported is True if the sentence has at least one citation and every cited source scores at least threshold. A citation number out of range counts as unsupported. A sentence with no content words is supported if it has a valid citation.
#rag#evaluation#hallucination2 visible + 2 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Check That Citations Are Grounded · TechGradify