Claude
1d4625a1c2
Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01B38FGKKam5SCGgwgduVAh3
62 lines
2.6 KiB
Plaintext
62 lines
2.6 KiB
Plaintext
# Retrieval: threshold, prompt shape and the empty-context path
|
|
|
|
the store holds the 4 pages of one handbook as 4 chunks; topK = 20
|
|
|
|
--- question: "How many days of annual leave do employees get?" ---
|
|
similarityThreshold 0.0 (the default): 4 chunk(s)
|
|
score 0.5891 page 2 "4.1 Annual Leave Entitlement. Full-time empl..."
|
|
score 0.4077 page 3 "4.3 Requesting Leave. All leave requests mus..."
|
|
score 0.2535 page 1 "Acme Employee Handbook 2026 3.5 Probationar..."
|
|
score 0.2023 page 4 "6.1 Expenses. Receipts are required for ever..."
|
|
similarityThreshold 0.3: 2 chunk(s)
|
|
score 0.5891 page 2 "4.1 Annual Leave Entitlement. Full-time empl..."
|
|
score 0.4077 page 3 "4.3 Requesting Leave. All leave requests mus..."
|
|
|
|
--- question: "What is the capital of Mongolia?" ---
|
|
similarityThreshold 0.0 (the default): 4 chunk(s)
|
|
score 0.0000 page 1 "Acme Employee Handbook 2026 3.5 Probationar..."
|
|
score 0.0000 page 2 "4.1 Annual Leave Entitlement. Full-time empl..."
|
|
score 0.0000 page 3 "4.3 Requesting Leave. All leave requests mus..."
|
|
score 0.0000 page 4 "6.1 Expenses. Receipts are required for ever..."
|
|
similarityThreshold 0.3: 0 chunk(s)
|
|
|
|
--- the prompt the model receives (threshold 0.3, question about leave) ---
|
|
Context information is below.
|
|
|
|
---------------------
|
|
4.1 Annual Leave Entitlement. Full-time employees are entitled to 20 working days
|
|
of annual leave per calendar year. Part-time employees receive leave pro rata.
|
|
|
|
4.2 Leave Carryover. Unused annual leave may be carried over for a maximum of 5 days
|
|
into the next calendar year and must be used by 31 March.
|
|
4.3 Requesting Leave. All leave requests must be submitted through the HR portal
|
|
at least two weeks in advance for absences longer than three days.
|
|
|
|
4.7 Sick Leave. Sick leave is separate from annual leave and is not deducted from it.
|
|
A medical certificate is required after three consecutive days.
|
|
---------------------
|
|
|
|
Given the context information and no prior knowledge, answer the query.
|
|
|
|
Follow these rules:
|
|
|
|
1. If the answer is not in the context, just say that you don't know.
|
|
2. Avoid statements like "Based on the context..." or "The provided information...".
|
|
|
|
Query: How many days of annual leave do employees get?
|
|
|
|
Answer:
|
|
|
|
|
|
--- nothing retrieved, allowEmptyContext(false): the prompt the model receives ---
|
|
The user query is outside your knowledge base.
|
|
Politely inform the user that you can't answer it.
|
|
|
|
|
|
--- nothing retrieved, allowEmptyContext(true): the prompt the model receives ---
|
|
What is the capital of Mongolia?
|
|
|
|
--- off-topic question, default threshold 0.0, allowEmptyContext(false) ---
|
|
chunks placed in the prompt: 4 of 4
|
|
the empty-context safety net fired: false
|