Add rerank module: bi-encoder retrieval plus an ONNX Runtime cross-encoder reranker behind a Spring AI DocumentPostProcessor; hit@k and MRR by pool size and question kind, latency by candidates, batch size, threads and int8, and five reproduced traps. Local MiniLM models, no database or API key.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Cfo2K4u4rF18QsPXPFnytf
2026-10-24 18:00:05 +00:00
parent 2ccf1d8d19
commit a429aa40f6
34 changed files with 1617 additions and 0 deletions
@@ -0,0 +1,17 @@
+# Quality and added latency for each setting (60 questions; latency is the median of 12 calls)
+
+machine: 2 vCPU, JDK 25.0.5
+first stage (embed the question, compare with 94 vectors): median 3.8 ms
+
+setting                      hit@1    hit@3    hit@5    MRR@10   rerank ms (median)
+no reranker                  41/60    51/60    55/60    0.782    0
+fp32, top 5                  48/60    55/60    55/60    0.863    70
+int8, top 5                  48/60    55/60    55/60    0.860    68
+fp32, top 10                 49/60    54/60    56/60    0.868    116
+int8, top 10                 49/60    54/60    56/60    0.868    106
+fp32, top 20                 49/60    55/60    56/60    0.870    199
+int8, top 20                 49/60    55/60    56/60    0.870    197
+fp32, top 50                 49/60    56/60    56/60    0.869    461
+int8, top 50                 49/60    56/60    56/60    0.869    503
+
+int8 and fp32 put the same passage first for 58 of 60 questions (top 50)