asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add rag-ingestion module: PDF, Word and HTML readers and chunking strategies compared by retrieval hit rate; metadata in the embedding, content-hash idempotent ingestion, re-embedding cost of an edit

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0146opDKBYtADXvkc39zdiyw
This commit is contained in:
Claude
2026-10-22 18:04:49 +00:00
parent c13195295d
commit d7e68a14af
35 changed files with 1699 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rag-ingestion/output/08-edit-cost.txt
+9
View File
@@ -0,0 +1,9 @@
# Chunks to embed again after appending one sentence to one paragraph (all 36 paragraphs tried)
splitter chunks mean median max
TokenTextSplitter 128 tokens 19 5.0 2 17
TokenTextSplitter 256 tokens 9 2.5 2 10
SentenceSplitter 128, no overlap 19 2.4 2 8
SentenceSplitter 128, 1 sentence overlap 24 3.1 2 13
one chunk per paragraph 36 1.0 1 1
columns: chunks in the unedited file; mean, median and max chunks re-embedded over the 36 edits