Claude
a429aa40f6
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Cfo2K4u4rF18QsPXPFnytf
18 lines
973 B
Plaintext
18 lines
973 B
Plaintext
# Quality and added latency for each setting (60 questions; latency is the median of 12 calls)
|
|
|
|
machine: 2 vCPU, JDK 25.0.5
|
|
first stage (embed the question, compare with 94 vectors): median 3.8 ms
|
|
|
|
setting hit@1 hit@3 hit@5 MRR@10 rerank ms (median)
|
|
no reranker 41/60 51/60 55/60 0.782 0
|
|
fp32, top 5 48/60 55/60 55/60 0.863 70
|
|
int8, top 5 48/60 55/60 55/60 0.860 68
|
|
fp32, top 10 49/60 54/60 56/60 0.868 116
|
|
int8, top 10 49/60 54/60 56/60 0.868 106
|
|
fp32, top 20 49/60 55/60 56/60 0.870 199
|
|
int8, top 20 49/60 55/60 56/60 0.870 197
|
|
fp32, top 50 49/60 56/60 56/60 0.869 461
|
|
int8, top 50 49/60 56/60 56/60 0.869 503
|
|
|
|
int8 and fp32 put the same passage first for 58 of 60 questions (top 50)
|