Add hybrid-search module: vector, tsvector and BM25 fused with Reciprocal Rank Fusion in Java and SQL over one PostgreSQL table; quality by query type, held-out k check, plans and latency at 50k rows

Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0146opDKBYtADXvkc39zdiyw
2026-10-22 18:04:49 +00:00
parent d7e68a14af
commit 2ccf1d8d19
26 changed files with 1508 additions and 0 deletions
@@ -0,0 +1,34 @@
+# Reciprocal Rank Fusion on two short rankings
+
+vector ranking : [A, B, C, D]
+keyword ranking: [C, A, E, B]
+
+--- k = 1 ---
+A  score 0.83333   (vector rank 1, keyword rank 2)
+C  score 0.75000   (vector rank 3, keyword rank 1)
+B  score 0.53333   (vector rank 2, keyword rank 4)
+E  score 0.25000   (vector rank 0, keyword rank 3)
+D  score 0.20000   (vector rank 4, keyword rank 0)
+
+--- k = 60 ---
+A  score 0.03252   (vector rank 1, keyword rank 2)
+C  score 0.03227   (vector rank 3, keyword rank 1)
+B  score 0.03175   (vector rank 2, keyword rank 4)
+E  score 0.01587   (vector rank 0, keyword rank 3)
+D  score 0.01563   (vector rank 4, keyword rank 0)
+
+--- k = 1000 ---
+A  score 0.00200   (vector rank 1, keyword rank 2)
+C  score 0.00200   (vector rank 3, keyword rank 1)
+B  score 0.00199   (vector rank 2, keyword rank 4)
+E  score 0.00100   (vector rank 0, keyword rank 3)
+D  score 0.00100   (vector rank 4, keyword rank 0)
+
+--- the arithmetic for A at k = 60 ---
+rank 1 in the vector list: 1/(60+1) = 0.01639
+rank 2 in the keyword list: 1/(60+2) = 0.01613
+sum = 0.03252
+
+--- an item in one list only ---
+E is rank 3 in the keyword list and absent from the vector list: 1/(60+3) = 0.01587
+D is rank 4 in the vector list and absent from the keyword list: 1/(60+4) = 0.01563