# One runbook, three formats, TokenTextSplitter(128), 37 questions

reader and configuration                             chunks  meanToks  hit@1  hit@3  hit@5   cut  top3Toks
PDF  PagePdfDocumentReader, each page split alone        36     106.6  24/37  32/37  33/37     1        333
PDF  pages joined first, footer still there              34     112.9  26/37  33/37  34/37     1        338
PDF  pages joined, pageBottomMargin=60                   34     110.5  27/37  33/37  34/37     1        330
PDF  pages joined, margin 60, whitespace collapsed       19     117.7  27/37  36/37  37/37     0        348
DOCX TikaDocumentReader                                  19     114.1  31/37  35/37  36/37     0        335
HTML JsoupDocumentReader, default selector (body)        19     114.8  30/37  34/37  36/37     0        342
HTML JsoupDocumentReader, selector=article               19     113.8  29/37  34/37  36/37     0        342

--- tokens the splitter had to cut, by reader ---
PDF pages as read     : 30358 characters, 3933 tokens
PDF whitespace collapsed: 10655 characters, 2251 tokens
DOCX                  : 10657 characters, 2172 tokens