Claude
1d4625a1c2
Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01B38FGKKam5SCGgwgduVAh3
32 lines
2.3 KiB
Plaintext
32 lines
2.3 KiB
Plaintext
# RecursiveChunker and SemanticChunker
|
|
|
|
|
|
--- RecursiveChunker(maxChars=400, overlapChars=80) ---
|
|
chunks: 14, longest: 395 characters
|
|
chunk 0: 314 chars, starts "Sentence 1 describes the", ends "very full-time employee."
|
|
chunk 1: 388 chars, starts "for item 3 and states th", ends "very full-time employee."
|
|
chunk 2: 387 chars, starts "for item 6 and states th", ends "very full-time employee."
|
|
chunk 3: 394 chars, starts "for item 9 and states th", ends "very full-time employee."
|
|
chunk 4: 394 chars, starts "for item 12 and states t", ends "very full-time employee."
|
|
chunk 5: 395 chars, starts "for item 15 and states t", ends "very full-time employee."
|
|
chunk 6: 395 chars, starts "for item 18 and states t", ends "very full-time employee."
|
|
chunk 7: 394 chars, starts "for item 21 and states t", ends "very full-time employee."
|
|
chunk 8: 395 chars, starts "for item 24 and states t", ends "very full-time employee."
|
|
chunk 9: 394 chars, starts "for item 27 and states t", ends "very full-time employee."
|
|
chunk 10: 395 chars, starts "for item 30 and states t", ends "very full-time employee."
|
|
chunk 11: 395 chars, starts "for item 33 and states t", ends "very full-time employee."
|
|
chunk 12: 394 chars, starts "for item 36 and states t", ends "very full-time employee."
|
|
chunk 13: 180 chars, starts "for item 39 and states t", ends "very full-time employee."
|
|
metadata of chunk 2: {chunk_index=2, chunk_total=14, page_number=1, source_file=long.txt}
|
|
boundaries where the next chunk opens with words the previous one ended with: 13 of 13
|
|
|
|
--- one 900-character word-salad with no separators falls back to a hard cut ---
|
|
chunks: [400, 400, 100]
|
|
|
|
--- SemanticChunker(distance 0.9) on three topics, four sentences each ---
|
|
chunks: 3
|
|
[0] Annual leave is twenty days per year. Unused annual leave carries over until March. Leave requests go through the HR portal. Annual leave accrues monthly.
|
|
[1] Expenses need a receipt above fifty euros. Expense claims must be filed within thirty days. Receipts for expenses are uploaded as photos. Approved expenses are paid with salary.
|
|
[2] Remote work is allowed two days per week. Managers agree the remote work days. Remote work needs a quiet workspace. Remote work days are recorded in the calendar.
|
|
texts sent to the embedding model: 12 (12 sentences, one batched call)
|