Add rerank module: bi-encoder retrieval plus an ONNX Runtime cross-encoder reranker behind a Spring AI DocumentPostProcessor; hit@k and MRR by pool size and question kind, latency by candidates, batch size, threads and int8, and five reproduced traps. Local MiniLM models, no database or API key.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Cfo2K4u4rF18QsPXPFnytf
2026-10-24 18:00:05 +00:00
parent 2ccf1d8d19
commit a429aa40f6
34 changed files with 1617 additions and 0 deletions
@@ -0,0 +1,10 @@
+# Time to rerank N candidates (median and slowest of 12 calls, after 3 warm-up calls)
+
+machine: 2 vCPU, JDK 25.0.5
+fp32 cross-encoder, max length 256 tokens; the first column of each cell is the median in ms, the second the slowest
+
+candidates   batch 1        batch 8        batch 32       batch 32, 1 thread
+10           125 / 188      121 / 142      122 / 138      148 / 208     
+20           205 / 226      195 / 220      220 / 257      322 / 411     
+50           546 / 661      512 / 578      489 / 549      800 / 913     
+94           897 / 1062     838 / 966      962 / 1075     1635 / 1928