# 5. Generation and the faithfulness check

prev: [4. Retrieval and reranking](04-retrieval-and-reranking.md) · [Index](../README.md) · next: [6. Observability and a production checklist](06-observability-and-production-checklist.md)

Source: [`RagQueryService`](../src/main/java/com/ankurm/rag/query/RagQueryService.java). Test:
[`FaithfulnessTest`](../src/test/java/com/ankurm/rag/FaithfulnessTest.java) ([output 09](output/09-faithfulness-check.txt)).

After the model answers, `RagQueryService` asks a second model call, `FactCheckingEvaluator`, whether the answer is supported by the
chunks that were in the prompt. The response reports one of three statuses:

| status | when | fact check runs? |
|---|---|---|
| `answered` | chunks retrieved and the judge said "yes" | yes |
| `ungrounded` | chunks retrieved and the judge said anything other than "yes" (case aside), or the judge call failed | yes |
| `no_context` | nothing retrieved | no |

Behaviours the test pins down:

- **The verdict is matched literally.** A judge reply of `Yes.` counted as *not* grounded; `YES` counted as grounded. The evaluator
  compares against "yes" ignoring case, not ignoring punctuation. A real judge model that likes to add a full stop would fail
  every answer; instruct it to answer with one word, and watch `rag_queries_total{status="ungrounded"}`.
- **A failing judge is not a passing judge.** If the judge call throws (a rate limit, a timeout) the answer is reported `ungrounded` and
  `rag.faithfulness.judge_failures` increments. The answer text is still returned, so the caller decides what to do with an unchecked answer.
- **No chunks, no check.** With nothing to check against, the service skips the judge (zero fact-check calls in the output) and reports `no_context`.
- **The check is only as good as the judge.** The tests script the judge, so they prove the wiring and the four outcomes, not that a real model
  spots an invented claim. Measure that on your own questions before you rely on it, and note that it doubles the model calls per question.

Next: [6. Observability and a production checklist](06-observability-and-production-checklist.md)