asmhatre/spring-ai
1
0
Fork 0
Code Activity

Add evaluation module: RelevancyEvaluator and FactCheckingEvaluator, golden dataset with a pass-rate gate, deterministic CI judge and simulated judge noise

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
This commit is contained in:
Claude
2026-10-09 06:15:41 +00:00
parent 80db4eb8ac
commit 65f580c5bc
37 changed files with 1353 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
evaluation/output/02-verdict-parsing.txt
+13
View File
@@ -0,0 +1,13 @@
# Which judge replies the built-in evaluators accept
judge reply relevancy fact-chk wrapped
'yes' true true true
'YES' true true true
' yes\n' true true true
'Yes.' false false true
'yes!' false false true
'Yes, the response is in line with the context.' false false true
'**Yes**' false false true
'no' false false false
'No.' false false false
'' false false false