Add evaluation module: RelevancyEvaluator and FactCheckingEvaluator, golden dataset with a pass-rate gate, deterministic CI judge and simulated judge noise

Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
2026-10-09 06:15:41 +00:00
parent 80db4eb8ac
commit 65f580c5bc
37 changed files with 1353 additions and 0 deletions
@@ -0,0 +1,4 @@
+# LiveJudgeTest as recorded by surefire when OPENAI_API_KEY is not set
+
+testcase: "realJudgeAgreesWithTheKnownGoodAndKnownBadBuilds"
+skipped : "Environment variable [OPENAI_API_KEY] does not exist"