asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add evaluation module: RelevancyEvaluator and FactCheckingEvaluator, golden dataset with a pass-rate gate, deterministic CI judge and simulated judge noise

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
This commit is contained in:
Claude
2026-10-09 06:15:41 +00:00
parent 80db4eb8ac
commit 65f580c5bc
37 changed files with 1353 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
evaluation/output/06-judge-limits.txt
+8
View File
@@ -0,0 +1,8 @@
# RuleBasedJudge(0.6) on hand-labelled answers to: Can I get a refund on a monthly plan?
answer type correct? judge says verdict
correct, same words true pass right
WRONG: negation dropped false pass WRONG
WRONG: roles swapped false pass WRONG
correct, paraphrased true fail WRONG
correct, but a refusal true fail WRONG