Faithfulness from a Claim–Source Support Matrix

easy
Checkpoint challenge
Solve this to show you know the material and jump straight to Unit 4. Everything before it counts as cleared.

RAG evaluators (RAGAS, TruLens) split an answer into claims, score how strongly each source passage supports each claim (with an NLI model or an LLM judge), and aggregate.

Given a support matrix S of shape (claims, sources) with values in [0, 1], implement:

  • faithfulness(S, threshold=0.5): the fraction of claims supported by at least one source (max ≥ threshold). An answer with no claims scores 1.0.
  • unsupported_claims(S, threshold=0.5): indices of claims no source supports
  • citation_precision(S, citations, threshold=0.5): citations[i] is the list of sources the answer cited for claim i. Return the fraction of all citations that actually support their claim (1.0 if there are no citations).
  • citation_recall(S, citations, threshold=0.5): the fraction of supported claims that cite at least one supporting source (1.0 if none are supported)
#rag#evaluation#hallucination2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Faithfulness from a Claim–Source Support Matrix · TechGradify