Add rag module: Spring AI 2.0 RAG with pgvector, chunking, reranking and a faithfulness check

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01B38FGKKam5SCGgwgduVAh3

rag/docs/04-retrieval-and-reranking.md

@@ -0,0 +1,65 @@
+# 4. Retrieval and reranking
+
+prev: [3. Ingestion](03-ingestion.md) · [Index](../README.md) · next: [5. Generation and the faithfulness check](05-generation-and-the-faithfulness-check.md)
+
+Tests: [`RetrievalTest`](../src/test/java/com/ankurm/rag/RetrievalTest.java),
+[`TenantFilterTest`](../src/test/java/com/ankurm/rag/TenantFilterTest.java),
+[`RerankTest`](../src/test/java/com/ankurm/rag/RerankTest.java). Wiring: [`RagConfig`](../src/main/java/com/ankurm/rag/config/RagConfig.java).
+
+**All similarity scores in this chapter come from the hashing embedding model.** They show how the components behave, not what a
+real model would score.
+
+## Top-k, and the threshold that is not there by default
+
+`VectorStoreDocumentRetriever` returns the `topK` nearest chunks. Its default similarity threshold accepts everything, and
+[output 06](output/06-retrieval-threshold-and-prompt.txt) shows what follows. For a question about leave, a threshold of 0.0 returned all four
+chunks (scores 0.59, 0.41, 0.25, 0.20); 0.3 returned two. For an off-topic question ("capital of Mongolia") the default still
+returned all four chunks, each with score 0.0000, and all four went into the prompt: **the "nothing found" safety net never
+fires while the threshold accepts everything.** With 0.3 nothing was retrieved.
+
+Which threshold is right depends entirely on the embedding model and cannot be copied from here. Find yours by asking questions
+you know the corpus cannot answer and looking at the top scores.
+
+## The empty-context path
+
+`ContextualQueryAugmenter` builds the prompt. With `allowEmptyContext(false)` (what `RagConfig` sets) and no chunks, the model
+receives only "The user query is outside your knowledge base. Politely inform the user that you can't answer it."
+With `allowEmptyContext(true)` the model receives the bare question and answers from whatever it was trained on, which for a
+document-QA service is the failure you built RAG to avoid. Both prompts are in output 06.
+`QuestionAnswerAdvisor` has no such switch: [output 11](output/11-question-answer-advisor.txt) shows it sending an empty context block instead.
+
+## Tenant isolation
+
+Put `tenant_id` in every chunk's metadata at ingestion (the controller does) and pass a filter at query time.
+[Output 07](output/07-tenant-filter-and-injection.txt), on the in-memory store and on PostgreSQL alike:
+
+- with no filter, the top three chunks for "How many days of annual leave" included one from the other tenant;
+- `new FilterExpressionBuilder().eq("tenant_id", "acme").build()` returned only that tenant's chunks;
+- a filter *string* built as `"tenant_id == '" + input + "'"` with `input = globex' || tenant_id == 'acme` parsed to
+  `tenant_id == 'globex' || tenant_id == 'acme'` and returned both tenants' chunks;
+- the same hostile text handed to `eq("tenant_id", input)` returned nothing, with a single or a double quote in it, on both stores.
+
+So: never build a filter string from a request value; pass the value to the builder. That is what
+[`RagController`](../src/main/java/com/ankurm/rag/web/RagController.java) does. The two hostile strings tried are the ones in the
+test; this is not a security audit of the filter parser.
+
+**Not tested:** how the filter performs. In pgvector, an approximate (HNSW) index is scanned first and the `WHERE` is applied to what
+it returns, so a selective filter can return fewer than `topK` rows; pgvector 0.8.0 added iterative index scans to address this
+([pgvector README, "Filtering"](https://github.com/pgvector/pgvector#filtering)). This repository ran pgvector **0.6.0**, which predates them,
+and never ran a filtered query against enough rows to see the effect. If one tenant is a small fraction of the table, test it.
+
+## Reranking
+
+Spring AI 2.0.1 has the hook (`DocumentPostProcessor`) and no reranker: `spring-ai-rag` contains no class with "rerank" in its
+name ([output 12](output/12-api-facts.txt)). [`LlmReranker`](../src/main/java/com/ankurm/rag/query/LlmReranker.java) is ours: it asks the
+chat model to rate each candidate 0-10 and keeps the best `topN` ([output 08](output/08-reranking.txt)).
+
+- **Cost:** one model call per candidate. Four candidates made four calls; with `topK` 20 every question costs 20 rating calls before the answer.
+- **Latency:** the calls run on virtual threads. With 200 ms per call standing in for the network, 20 calls took over 4 s one after another
+  and under 1 s through the reranker. That shows the structure works; a real API's rate limits are not modelled.
+- **Failure:** the reply must be a bare integer. `"Score: 8"` failed to parse for every candidate, all scores became 0, and the survivors were the
+  vector-search order. Nothing errors; only the `rag.rerank.failures` counter moves. A reply of `" 9\n"` parsed fine. Watch the counter.
+
+**Not tested:** whether reranking improves which chunks reach the prompt. The scripted rater is word overlap and agrees with the fake embeddings.
+
+Next: [5. Generation and the faithfulness check](05-generation-and-the-faithfulness-check.md)