asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add semantic-cache module: CallAdvisor over a Redis Stack vector index with a real in-process embedding model; threshold sweep (hit rate against wrong answers), 600-request replay with an illustrative cost model, tenant filter, expiry, look-alike and embedding-model-change traps; optional real local model

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
asmhatre
2026-10-11 09:46:19 +00:00
co-authored by Claude Sonnet 5.5
parent cdee85d3f4
commit d51ef56468
24 changed files with 1383 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
semantic-cache/output/09-hit-latency.txt
+5
View File
@@ -0,0 +1,5 @@
# Where the milliseconds go on a lookup (110 questions, 30 entries cached, 2 vCPUs)
embed the question (MiniLM in-process) : median 2.4 ms, p95 3.6 ms
Redis KNN search with the tenant filter : median 0.3 ms, p95 2.5 ms (lookup time minus one embedding)
Timings drift run to run; the shape (embedding dominates, Redis is small) does not.