Claude
65f580c5bc
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
38 lines
1.9 KiB
Plaintext
38 lines
1.9 KiB
Plaintext
# Golden dataset: 12 cases, deterministic judge, pass-rate gate at 0.90
|
|
|
|
== healthy build ==
|
|
case relevant grounded has facts passed
|
|
refund-annual true true true true
|
|
refund-monthly true true true true
|
|
storage-pro true true true true
|
|
support-hours true true true true
|
|
sso-plan true true true true
|
|
data-region true true true true
|
|
api-rate-limit true true true true
|
|
backup-retention true true true true
|
|
macos-agent true true true true
|
|
reset-link true true true true
|
|
extra-seat true true true true
|
|
trial-card true true true true
|
|
pass rate: 1.00 failing: []
|
|
|
|
== regressed build ==
|
|
case relevant grounded has facts passed
|
|
refund-annual false false false false
|
|
refund-monthly true true true true
|
|
storage-pro true true true true
|
|
support-hours true true true true
|
|
sso-plan false false false false
|
|
data-region true true true true
|
|
api-rate-limit true true true true
|
|
backup-retention true true true true
|
|
macos-agent true true true true
|
|
reset-link true true true true
|
|
extra-seat true true false false
|
|
trial-card true true true true
|
|
pass rate: 0.75 failing: [refund-annual, sso-plan, extra-seat]
|
|
|
|
== the gate ==
|
|
healthy build : requirePassRate(0.90) returned normally
|
|
regressed build : java.lang.AssertionError: pass rate 0.75 is below the 0.90 threshold; failing cases: [refund-annual, sso-plan, extra-seat]
|