Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
16 lines
975 B
Plaintext
16 lines
975 B
Plaintext
# The simulated provider: same input, same outcome
|
|
|
|
Two separate provider instances, same seed, tickets 1..20,000, one call each: identical outcome lists = true
|
|
|
|
Outcome of the first call per ticket (the plan: 1.5% 429, 0.5% 503, 0.3% never valid, 1.2% invalid on the first reply only):
|
|
429 304 1.52%
|
|
503 93 0.47%
|
|
bad JSON: category not in schema 76 0.38%
|
|
bad JSON: missing field 77 0.39%
|
|
bad JSON: not JSON 137 0.69%
|
|
valid JSON 17,361 86.81%
|
|
valid JSON (in a markdown fence, stripped) 1,952 9.76%
|
|
|
|
Quota check: quota 100 calls/second, 300 calls fired back to back -> 201 rejected with 429 (quota rejections plus any injected 429)
|
|
Every number above is a property of the TEST PLAN, not a measurement of any provider.
|