asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add semantic-cache module: CallAdvisor over a Redis Stack vector index with a real in-process embedding model; threshold sweep (hit rate against wrong answers), 600-request replay with an illustrative cost model, tenant filter, expiry, look-alike and embedding-model-change traps; optional real local model

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
asmhatre
2026-10-11 09:46:19 +00:00
co-authored by Claude Sonnet 5.5
parent cdee85d3f4
commit d51ef56468
24 changed files with 1383 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
semantic-cache/output/10-real-model-latency.txt
+16
View File
@@ -0,0 +1,16 @@
# A real local model (TinyLlama 1.1B, Q4_0, 60 tokens) behind the advisor, 2 vCPUs
first ask 2100 ms model "How do I reset my account password?"
first ask 1953 ms model "How do I reset my router to factory settings?"
first ask 5 ms CACHE "How do I reset two-factor authentication on my account?" <- served the answer to "How do I reset my account password?" (score 0.852), a DIFFERENT question
first ask 1902 ms model "How do I cancel an order I just placed?"
first ask 1920 ms model "How do I cancel my monthly subscription?"
first ask 10 ms CACHE "How do I cancel a return I already requested?" <- served the answer to "How do I cancel an order I just placed?" (score 0.824), a DIFFERENT question
reworded 17 ms CACHE "I forgot my password, how can I get back into my account?" <- How do I reset my account password?
reworded 9 ms CACHE "What is the way to restore my router to its factory defaults?" <- How do I reset my router to factory settings?
reworded 1941 ms model "I lost my phone, how can I reset my 2FA?"
reworded 9 ms CACHE "I made an order by mistake, can I cancel it?" <- How do I cancel an order I just placed?
reworded 6 ms CACHE "I want to stop my recurring subscription payments" <- How do I cancel my monthly subscription?
reworded 2013 ms model "I changed my mind about sending the item back, can I withdraw the return?"
median model call 1920 ms; median cache hit 9 ms (4 of 6 rewordings hit); first-time questions wrongly answered from the cache: 2 of 6