RAG Evaluation & The RAG Triad
Quantify RAG quality using deterministic evaluation metrics: Context Relevance, Groundedness (Faithfulness), and Answer Relevance.
By the end
You will be able to
- Implement the RAG Triad: Context Relevance, Groundedness, and Answer Relevance.
- Use LLM-as-a-judge patterns with deterministic structured scoring rubrics and logit bias.
- Identify and diagnose retrieval failure modes vs synthesis hallucination failure modes.
- Build automated regression testing CI pipelines for enterprise knowledge updates.
The Three Core Pillars of RAG Quality
Measuring end-to-end user satisfaction does not tell you why a RAG pipeline failed. The RAG Triad isolates the three failure domains:
1. Context Relevance: Did the retrieval engine fetch chunks that actually answer the query? 2. Groundedness (Faithfulness): Are all claims in the generated response directly supported by the retrieved context? 3. Answer Relevance: Does the generated response directly address the user inquiry without extraneous drift?
GROUNDEDNESS_JUDGE_PROMPT = """You are a rigorous evaluation judge. Evaluate if every statement in the ANSWER is supported by the CONTEXT.
CONTEXT:
{context}
ANSWER:
{answer}
Respond with strict JSON:
{{
"unsupported_claims": ["list of unsupported statements"],
"groundedness_score": 0.0 to 1.0,
"verdict": "PASS" | "FAIL"
}}"""Practice activity
Implement Automated RAG Triad Regression Test
- Create a test fixture with 10 synthetic query-context-answer triples (5 grounded, 5 hallucinated).
- Run an automated evaluation harness computing Groundedness and Answer Relevance scores.
- Assert that ungrounded answers are flagged with 100% precision.
What to produce
- Evaluation report JSON with per-metric scores and claim attribution logs.
Reflect before continuing
Why is groundedness scoring critical before deploying automated customer-facing RAG systems?
Evidence
Sources and verification
- Evaluating RAG Applications: The RAG TriadTruLens · verified 2026-08-22
Knowledge check
Make it stick.
Choose the strongest answer for each question. Your attempts become part of your account transcript.