Claude
85a3359186
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
14 lines
1.0 KiB
Plaintext
14 lines
1.0 KiB
Plaintext
# Eight questions, scored three ways (the model is a script, not a language model)
|
|
|
|
question string execution what happened
|
|
How many customers are in Germany? no match same result
|
|
What is the total revenue of paid orders? no match same result
|
|
Which 3 products sold the most units? no match same result
|
|
Which customers have never ordered? no match same result
|
|
How many orders are there per status? no no DIFFERENT result: got [paid, 750], expected [paid, 250]
|
|
What is the average order value? no no DIFFERENT result: got [15.5000000000000000], expected [78.0150000000000000]
|
|
What is the revenue by country? no no refused by the guard: table sales is not allowed
|
|
How many orders were placed in March 2025? no match same result
|
|
|
|
string match: 0/8 execution match: 5/8 refused before running: 1 ran and returned a wrong answer: 2
|