spring-ai/rag/docs/02-chunking.md

2. Chunking

prev: 1. The shape of a RAG pipeline · Index · next: 3. Ingestion

A chunk is the unit that gets embedded, stored, retrieved and pasted into the prompt. Cut too big and the prompt fills with text unrelated to the question; cut too small and an answer is split across two chunks.

This chapter measures what the chunkers do to one fixed document, not which one retrieves best. There is no retrieval-quality benchmark in this repository, so nothing here says which strategy is best for your documents. The fixture is forty numbered sentences, 4,268 characters, 880 cl100k_base tokens.

TokenTextSplitter in Spring AI 2.0.1

Transcript: output 02. Test: ChunkingTest.

• The defaults are large. new TokenTextSplitter() (800-token chunks) turned the 880-token fixture into two chunks of 792 and 88 tokens. A short handbook page becomes one chunk.
• It prefers sentence ends. With withChunkSize(100) the result was ten chunks of 88 tokens, every one ending on a full stop; it did not cut at exactly 100 tokens.
• It has no overlap. Zero of nine neighbouring boundaries repeated any text. A sentence that straddles a cut belongs to one chunk only.
• withMinChunkSizeChars did nothing on this fixture. 350 and 50 produced identical ten-chunk results. Reading the bytecode of the splitter: when a chunk exceeds the token limit it is cut back to its last sentence-ending mark, but only if that mark is more than minChunkSizeChars characters into the chunk. On text with a full stop every ~100 characters the mark is always far enough in, so neither value matters. Do not read this as "the setting is useless", only as "this document does not exercise it".
• The constructor changed. The 1.x five-argument constructor new TokenTextSplitter(512, 128, 5, 10_000, true) does not exist in 2.0.1; use the builder. See output 14 and output 12.

RecursiveChunker (ours)

Transcript: output 03. Source: RecursiveChunker.

Cuts at the coarsest boundary that fits (blank line, line break, sentence end, space, and only then mid-word), and repeats the last overlapChars characters at the start of the next chunk. With (400, 80) the fixture became 14 chunks, longest 395 characters, and all 13 boundaries carried overlap. Sizes are characters, not tokens. A 900-character string with no separators is cut hard at [400, 400, 100].

The overlap starts at a word boundary, so a chunk often opens mid-sentence ("for item 3 and states th..."). That is the price of overlap; whether it helps retrieval was not measured.

SemanticChunker (ours)

Same transcript. Source: SemanticChunker.

Embeds every sentence in one batched call and starts a new chunk where the cosine distance between neighbours exceeds a threshold. On twelve sentences about three topics it produced three chunks, one per topic, and sent 12 texts to the embedding model. That result is from the hashing model, which is good at exactly this (topics share words). With a real embedding model the right threshold is different and must be found on your own text; the value 0.9 has no meaning outside this test. It costs an embedding call per sentence at ingestion time.

Choosing

Start with TokenTextSplitter and a size you set on purpose. Move to the recursive chunker when a fact is being cut in half at the boundary and you can see it in real retrievals. Consider semantic chunking only when documents mix topics without headings and you can afford the extra embedding calls. Whichever you use, change it by replacing the one DocumentTransformer bean; nothing else in the pipeline notices.

prev: 1 · next: 3. Ingestion