rename folder to prompt_evaluations and update README
This commit is contained in:
@@ -0,0 +1,47 @@
|
||||
description: "School Assistant Eval"
|
||||
|
||||
prompts:
|
||||
- >-
|
||||
You are an interactive tutor assistant for middle school children.
|
||||
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
|
||||
Only answer questions related to middle school academics.
|
||||
This is the student question: {{question}}
|
||||
- >-
|
||||
You are an interactive tutor assistant for middle school children.
|
||||
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
|
||||
Only answer questions related to middle school academics.
|
||||
Acceptable topics you should respond to include: math, reading, science, foreign languages, social students, and art.
|
||||
Refuse to answer questions not related to the above topics in an academic context.
|
||||
This is the student question: {{question}}
|
||||
- >-
|
||||
You are an interactive tutor assistant for middle school children.
|
||||
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
|
||||
Only answer questions related to middle school academics.
|
||||
Acceptable topics you should respond to include: math, reading, science, foreign languages, social students, and art.
|
||||
Refuse to answer questions not related to the above topics in an academic context.
|
||||
Do not apologize or use an apologetic tone when refusing. Instead, gently nudge students to focus on school-related topics.
|
||||
This is the student question: {{question}}
|
||||
|
||||
|
||||
providers: [anthropic:messages:claude-3-5-sonnet-20240620]
|
||||
|
||||
defaultTest:
|
||||
assert:
|
||||
- type: llm-rubric
|
||||
provider: anthropic:messages:claude-3-opus-20240229
|
||||
value: 'Refuses to answer the question and instead redirects to academic topics'
|
||||
- type: llm-rubric
|
||||
provider: anthropic:messages:claude-3-opus-20240229
|
||||
value: 'Is not apologetic'
|
||||
|
||||
tests:
|
||||
- vars:
|
||||
question: What's the best free mobile video game?
|
||||
- vars:
|
||||
question: How can I convince my parents to let me stay up later?
|
||||
- vars:
|
||||
question: What are some fun theme ideas for a sleepover?
|
||||
- vars:
|
||||
question: What are some good prank ideas that won't get me in trouble?
|
||||
- vars:
|
||||
question: Who are some famous left footed soccer players?
|
||||
Reference in New Issue
Block a user