spring-ai/rag/docs/01-the-shape-of-a-rag-pipeline.md

1. The shape of a RAG pipeline

Index · next: 2. Chunking

A language model only knows what it was trained on and what is in the prompt. Retrieval-augmented generation (RAG) puts the right few paragraphs of your documents into the prompt, so the model can answer from them. Everything else in this repository is a decision about which paragraphs, how they get there, and what to do when the answer is not in them.

Two phases, six stages

 ingest (once per file)                      query (once per question)
 ──────────────────────                      ─────────────────────────
 PDF ─► read pages ─► chunk ─► embed ─► store      question ─► embed ─► search ─► rerank ─► prompt ─► model ─► check
        IngestionService        VectorStore                    └────── RetrievalAugmentationAdvisor ──────┘  RagQueryService

Stage	Class here	Spring AI part it stands on
read	IngestionService	PagePdfDocumentReader
chunk	DocumentTransformer chunker in RagConfig	TokenTextSplitter (or RecursiveChunker, SemanticChunker)
embed + store	VectorStore bean	PgVectorStore, built by the pgvector starter
search	retrievalAdvisor bean	VectorStoreDocumentRetriever
rerank	LlmReranker	the DocumentPostProcessor hook (Spring AI ships no reranker)
prompt	retrievalAdvisor bean	ContextualQueryAugmenter
model + check	RagQueryService	ChatClient, FactCheckingEvaluator

The beans as the application actually wires them are in output 01.

The smallest thing that works

One advisor, no other class of ours:

ChatClient.builder(chatModel)
    .defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore)
        .searchRequest(SearchRequest.builder().topK(2).similarityThreshold(0.3).build())
        .build())
    .build();

That is SimpleAdvisorTest. Output 11 shows the two prompts it produces. For a question the store can answer, the chunks are pasted between two lines of dashes. For a question nothing matches, the prompt still has the same template with an empty block between the dashes, and the model is left to follow the template's last sentence ("if the answer is not in the context, inform the user that you can't answer"). Nothing in code decides that nothing was found. That is the first thing the larger pipeline adds: see chapter 4.

What this repository does not test

The chat model and the embedding model are scripted stand-ins (FakeChatModel, HashingEmbeddingModel). That is deliberate: anyone can run every test with no API key, and what is asserted is what Spring AI sends and how this code handles what comes back. It also means nothing here shows how a real model answers, how a real embedding model scores similarity, whether reranking improves answers, or how good a real judge is at spotting an invented claim. Similarity scores and thresholds in the outputs belong to the hashing model; do not carry them over.

The vector store is real: every test that touches persistence runs against PostgreSQL with pgvector (scripts/pg-up.sh or docker-compose.yml), and the end-to-end test starts the whole application over HTTP.

Next: 2. Chunking