asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add semantic-cache module: CallAdvisor over a Redis Stack vector index with a real in-process embedding model; threshold sweep (hit rate against wrong answers), 600-request replay with an illustrative cost model, tenant filter, expiry, look-alike and embedding-model-change traps; optional real local model

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
asmhatre
2026-10-11 09:46:19 +00:00
co-authored by Claude Sonnet 5.5
parent cdee85d3f4
commit d51ef56468
24 changed files with 1383 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
semantic-cache/output/01-similarity-scores.txt
+35
View File
@@ -0,0 +1,35 @@
# What a similarity score looks like (all-MiniLM-L6-v2, cosine)
Plain cosine similarity (NOT the Spring AI score, see the end) of each later phrasing with the cached phrasing of the SAME question (90 pairs):
min 0.337 p10 0.537 median 0.729 p90 0.869 max 0.916
lowest: 0.337 Where is my package right now? -> How can I track my order?
Best score each of those phrasings gets against a cached question of a DIFFERENT intent (look-alikes like reset password / reset router):
min 0.209 p10 0.311 median 0.446 p90 0.563 max 0.633
highest: 0.633 Can you help me reset the password for my login? -> How do I reset two-factor authentication on my account?
Best score of 20 off-topic questions against any cached question:
min 0.096 p10 0.133 median 0.180 p90 0.225 max 0.289
Does Redis report the same number? "I forgot my password, how can I get back into my account?"
cosine computed in Java : 0.838726
Document.getScore() from Redis : 0.919363 (matched stored question: "How do I reset my account password?")
(1 + cosine) / 2 : 0.919363 <- the score is this, not the cosine
same question as the cached one -> score 1.000000
The closest pairs of DIFFERENT cached questions (Spring AI score; 435 pairs in all). Above the threshold, one answers for the other:
0.852 "How do I reset my account password?" / "How do I reset two-factor authentication on my account?"
0.824 "How do I cancel an order I just placed?" / "How do I cancel a return I already requested?"
0.796 "How do I cancel an order I just placed?" / "How do I cancel my monthly subscription?"
0.791 "How can I track my order?" / "How can I track the return I sent back?"
0.789 "How can I track my order?" / "Can I pick up my order at a store?"
0.779 "How long is the warranty on your laptops?" / "How do I make a warranty claim?"
0.775 "How long does a refund take to arrive?" / "How can I track the return I sent back?"
0.767 "How do I update the firmware on my headphones?" / "How do I pair my headphones over Bluetooth?"
Spring AI's similarityThreshold is applied to that score. Threshold -> the cosine it really means (cosine = 2 x score - 1):
similarityThreshold 0.70 = cosine 0.40
similarityThreshold 0.80 = cosine 0.60
similarityThreshold 0.85 = cosine 0.70
similarityThreshold 0.90 = cosine 0.80
similarityThreshold 0.95 = cosine 0.90