Files

T

History

Claude d7e68a14af Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0146opDKBYtADXvkc39zdiyw

2026-10-22 18:04:49 +00:00

output

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

2026-10-22 18:04:49 +00:00

scripts

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

2026-10-22 18:04:49 +00:00

src

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

2026-10-22 18:04:49 +00:00

.gitignore

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

2026-10-22 18:04:49 +00:00

pom.xml

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

2026-10-22 18:04:49 +00:00

README.md

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

2026-10-22 18:04:49 +00:00

README.md

rag-ingestion

Companion code for Spring AI RAG Ingestion Pipeline: PDF, Word and HTML Readers with Chunking Strategies Compared.

PDF, Word and HTML versions of one 12-section runbook go through PagePdfDocumentReader, TikaDocumentReader and JsoupDocumentReader, then through several chunkers, and retrieval hit rate is measured on 37 fixed questions. Also: metadata in the embedding, content-hash ingestion, and what one edit costs to re-embed. Spring Boot 4.1.2, Spring AI 2.0.2, Java 25.

Run scripts/run-all.sh (needs PostgreSQL 16 with pgvector; see the script header). Every number in the article is a line in output/.