Claude
65f580c5bc
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
43 lines
1.3 KiB
Plaintext
43 lines
1.3 KiB
Plaintext
# What RelevancyEvaluator and FactCheckingEvaluator send to the judge
|
|
|
|
--- RelevancyEvaluator: prompt sent to the judge (1 message(s)) ---
|
|
Your task is to evaluate if the response for the query
|
|
is in line with the context information provided.
|
|
|
|
You have two options to answer. Either YES or NO.
|
|
|
|
Answer YES, if the response for the query
|
|
is in line with context information otherwise NO.
|
|
|
|
Query:
|
|
How long do I have to request a refund on an annual plan?
|
|
|
|
Response:
|
|
You can request a refund on an annual plan within 30 days of purchase.
|
|
|
|
Context:
|
|
Annual plans can be refunded within 30 days of purchase.
|
|
Monthly plans are not refundable.
|
|
|
|
Answer:
|
|
|
|
--- RelevancyEvaluator: judge said "yes" ---
|
|
pass=true score=1.0 feedback='' metadata={}
|
|
|
|
--- FactCheckingEvaluator: prompt sent to the judge ---
|
|
Evaluate whether or not the following claim is supported by the provided document.
|
|
Respond with "yes" if the claim is supported, or "no" if it is not.
|
|
|
|
Document:
|
|
Annual plans can be refunded within 30 days of purchase.
|
|
Monthly plans are not refundable.
|
|
|
|
Claim:
|
|
You can request a refund on an annual plan within 30 days of purchase.
|
|
|
|
--- FactCheckingEvaluator: judge said "yes" ---
|
|
pass=true score=0.0 feedback='' metadata={}
|
|
|
|
--- RelevancyEvaluator: judge said "no" ---
|
|
pass=false score=0.0 feedback=''
|