Claude
d7e68a14af
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0146opDKBYtADXvkc39zdiyw
10 lines
619 B
Plaintext
10 lines
619 B
Plaintext
# Chunks to embed again after appending one sentence to one paragraph (all 36 paragraphs tried)
|
|
|
|
splitter chunks mean median max
|
|
TokenTextSplitter 128 tokens 19 5.0 2 17
|
|
TokenTextSplitter 256 tokens 9 2.5 2 10
|
|
SentenceSplitter 128, no overlap 19 2.4 2 8
|
|
SentenceSplitter 128, 1 sentence overlap 24 3.1 2 13
|
|
one chunk per paragraph 36 1.0 1 1
|
|
columns: chunks in the unedited file; mean, median and max chunks re-embedded over the 36 edits
|