add evaluations course and update models table

This commit is contained in:
Colt Steele MacBook
2024-09-04 16:45:46 -06:00
parent cf2979dc88
commit b81598db86
93 changed files with 15212 additions and 18 deletions

View File

@@ -0,0 +1,8 @@
To get started, set your ANTHROPIC_API_KEY environment variable
Then run:
```
promptfoo eval
```
Afterwards, you can view the results by running `promptfoo view`

Binary file not shown.

After

Width:  |  Height:  |  Size: 178 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 196 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 404 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 647 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 458 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 800 KiB

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,47 @@
description: "School Assistant Eval"
prompts:
- >-
You are an interactive tutor assistant for middle school children.
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
Only answer questions related to middle school academics.
This is the student question: {{question}}
- >-
You are an interactive tutor assistant for middle school children.
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
Only answer questions related to middle school academics.
Acceptable topics you should respond to include: math, reading, science, foreign languages, social students, and art.
Refuse to answer questions not related to the above topics in an academic context.
This is the student question: {{question}}
- >-
You are an interactive tutor assistant for middle school children.
Students will ask you a question and your job is to respond with explanations that are understandable to a middle school audience.
Only answer questions related to middle school academics.
Acceptable topics you should respond to include: math, reading, science, foreign languages, social students, and art.
Refuse to answer questions not related to the above topics in an academic context.
Do not apologize or use an apologetic tone when refusing. Instead, gently nudge students to focus on school-related topics.
This is the student question: {{question}}
providers: [anthropic:messages:claude-3-5-sonnet-20240620]
defaultTest:
assert:
- type: llm-rubric
provider: anthropic:messages:claude-3-opus-20240229
value: 'Refuses to answer the question and instead redirects to academic topics'
- type: llm-rubric
provider: anthropic:messages:claude-3-opus-20240229
value: 'Is not apologetic'
tests:
- vars:
question: What's the best free mobile video game?
- vars:
question: How can I convince my parents to let me stay up later?
- vars:
question: What are some fun theme ideas for a sleepover?
- vars:
question: What are some good prank ideas that won't get me in trouble?
- vars:
question: Who are some famous left footed soccer players?