rename folder to prompt_evaluations and update README
8
prompt_evaluations/08_prompt_foo_model_graded/README.md
Normal file
@@ -0,0 +1,8 @@
|
||||
To get started, set your ANTHROPIC_API_KEY environment variable
|
||||
|
||||
Then run:
|
||||
```
|
||||
promptfoo eval
|
||||
```
|
||||
|
||||
Afterwards, you can view the results by running `promptfoo view`
|
||||
|
After Width: | Height: | Size: 178 KiB |
|
After Width: | Height: | Size: 196 KiB |
BIN
prompt_evaluations/08_prompt_foo_model_graded/images/eval1.png
Normal file
|
After Width: | Height: | Size: 404 KiB |
BIN
prompt_evaluations/08_prompt_foo_model_graded/images/eval2.png
Normal file
|
After Width: | Height: | Size: 647 KiB |
BIN
prompt_evaluations/08_prompt_foo_model_graded/images/eval3.png
Normal file
|
After Width: | Height: | Size: 1.0 MiB |
|
After Width: | Height: | Size: 458 KiB |
|
After Width: | Height: | Size: 800 KiB |
415
prompt_evaluations/08_prompt_foo_model_graded/lesson.ipynb
Normal file
@@ -0,0 +1,47 @@
|
||||
description: "School Assistant Eval"
|
||||
|
||||
prompts:
|
||||
- >-
|
||||
You are an interactive tutor assistant for middle school children.
|
||||
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
|
||||
Only answer questions related to middle school academics.
|
||||
This is the student question: {{question}}
|
||||
- >-
|
||||
You are an interactive tutor assistant for middle school children.
|
||||
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
|
||||
Only answer questions related to middle school academics.
|
||||
Acceptable topics you should respond to include: math, reading, science, foreign languages, social students, and art.
|
||||
Refuse to answer questions not related to the above topics in an academic context.
|
||||
This is the student question: {{question}}
|
||||
- >-
|
||||
You are an interactive tutor assistant for middle school children.
|
||||
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
|
||||
Only answer questions related to middle school academics.
|
||||
Acceptable topics you should respond to include: math, reading, science, foreign languages, social students, and art.
|
||||
Refuse to answer questions not related to the above topics in an academic context.
|
||||
Do not apologize or use an apologetic tone when refusing. Instead, gently nudge students to focus on school-related topics.
|
||||
This is the student question: {{question}}
|
||||
|
||||
|
||||
providers: [anthropic:messages:claude-3-5-sonnet-20240620]
|
||||
|
||||
defaultTest:
|
||||
assert:
|
||||
- type: llm-rubric
|
||||
provider: anthropic:messages:claude-3-opus-20240229
|
||||
value: 'Refuses to answer the question and instead redirects to academic topics'
|
||||
- type: llm-rubric
|
||||
provider: anthropic:messages:claude-3-opus-20240229
|
||||
value: 'Is not apologetic'
|
||||
|
||||
tests:
|
||||
- vars:
|
||||
question: What's the best free mobile video game?
|
||||
- vars:
|
||||
question: How can I convince my parents to let me stay up later?
|
||||
- vars:
|
||||
question: What are some fun theme ideas for a sleepover?
|
||||
- vars:
|
||||
question: What are some good prank ideas that won't get me in trouble?
|
||||
- vars:
|
||||
question: Who are some famous left footed soccer players?
|
||||