# 6. Observability and a production checklist

prev: [5. Generation and the faithfulness check](05-generation-and-the-faithfulness-check.md) · [Index](../README.md)

## What the application exports

`GET /actuator/prometheus` from the end-to-end run ([output 10](output/10-end-to-end.txt)) contained these `rag_` series after two
ingested files, one skipped re-upload and two questions:

```
rag_chunks_ingested_total 5.0
rag_context_chunks_count 2
rag_context_chunks_sum 5.0
rag_context_chunks_max 4.0
rag_ingestion_skipped_total 1.0
rag_queries_total{status="answered"} 2.0
rag_query_duration_seconds_count{status="answered"} 2
```

Counters that exist in the code but were not triggered in that run (so they do not appear): `rag_rerank_failures_total` and
`rag_faithfulness_judge_failures_total`. Micrometer registers a counter when it is first incremented.

The ones worth alerting on: a rising share of `ungrounded` and `no_context` in `rag_queries_total`; any increase in `rag_rerank_failures_total`
(silent degradation, chapter 4); any increase in `rag_faithfulness_judge_failures_total`; and `rag_context_chunks` drifting toward its maximum
(the threshold is accepting everything).

## Checklist

- [ ] Threshold set from questions your corpus cannot answer (chapter 4), not copied from a tutorial.
- [ ] `allowEmptyContext(false)`, and a test that an off-topic question produces `no_context`.
- [ ] Tenant filter built with `FilterExpressionBuilder`, never string concatenation; a test with a hostile tenant id.
- [ ] Filtered queries tested on a table the size of production (HNSW and `WHERE`, chapter 4).
- [ ] The ingestion tracker persisted, or deletion by `source_file` filter (chapter 3).
- [ ] Judge prompt forces a one-word answer; `ungrounded` is shown to the user as such, not hidden.
- [ ] Reranker replies parsed with a fallback and the failure counter alerted (chapter 4). Consider a structured-output call instead of parsing text.
- [ ] `init.sql` (or your migration tool) owns the schema; `initialize-schema` stays `false`.
- [ ] Embedding model and vector dimension changed together, and the table rebuilt when either changes: vectors from two models are not comparable.
- [ ] An evaluation set of real questions with known answers, run on every change to chunking, threshold or models. **This repository has none**; it is the largest gap between this code and a production system.