Compare commits
2 Commits
bedrock-co
...
vertex
| Author | SHA1 | Date | |
|---|---|---|---|
|
c6ed07bac7 | ||
|
|
891ba16451 |
@@ -1,8 +1,5 @@
|
||||
# Anthropic courses
|
||||
|
||||
Welcome to Anthropic's educational courses. This repository currently contains four courses. We suggest completing the courses in the following order:
|
||||
Welcome to Anthropic's educational courses specifically for Google Vertex
|
||||
|
||||
1. [Anthropic API fundamentals course](./anthropic_api_fundamentals/README.md) - teaches the essentials of working with the Claude SDK: getting an API key, working with model parameters, writing multimodal prompts, streaming responses, etc.
|
||||
2. [Prompt engineering interactive tutorial](./prompt_engineering_interactive_tutorial/README.md) - a comprehensive step-by-step guide to key prompting techniques
|
||||
3. [Real world prompting course](./real_world_prompting/README.md) - learn how to incorporate prompting techniques into complex, real world prompts
|
||||
4. [Tool use course](./tool_use/README.md) - teaches everything you need to know to implement tool use successfully in your workflows with Claude.
|
||||
1. [Real world prompting course](./real_world_prompting/README.md) - learn how to incorporate prompting techniques into complex, real world prompts
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1,9 +0,0 @@
|
||||
# Anthropic API fundamentals course
|
||||
A series of notebook tutorials that cover the essentials of working with Claude models and the Anthropic SDK including:
|
||||
|
||||
* [Getting an API key and making simple requests](./01_getting_started.ipynb)
|
||||
* [Working with the messages format](./02_messages_format.ipynb)
|
||||
* [Comparing capabilities and performance of the Claude model family](./03_models.ipynb)
|
||||
* [Understanding model parameters](./04_parameters.ipynb)
|
||||
* [Working with streaming responses](./05_Streaming.ipynb)
|
||||
* [Vision prompting](./06_vision.ipynb)
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 195 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 220 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 801 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 356 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 105 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 170 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 204 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 107 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 293 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 695 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.3 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.5 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 142 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 246 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.5 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 372 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 364 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 228 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 843 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.2 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.5 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.5 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 126 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 116 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 224 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.5 MiB |
@@ -1,59 +0,0 @@
|
||||
# Contributing Guidelines
|
||||
|
||||
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
|
||||
documentation, we greatly value feedback and contributions from our community.
|
||||
|
||||
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
|
||||
information to effectively respond to your bug report or contribution.
|
||||
|
||||
|
||||
## Reporting Bugs/Feature Requests
|
||||
|
||||
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
|
||||
|
||||
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
|
||||
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
|
||||
|
||||
* A reproducible test case or series of steps
|
||||
* The version of our code being used
|
||||
* Any modifications you've made relevant to the bug
|
||||
* Anything unusual about your environment or deployment
|
||||
|
||||
|
||||
## Contributing via Pull Requests
|
||||
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
|
||||
|
||||
1. You are working against the latest source on the *main* branch.
|
||||
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
|
||||
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
|
||||
|
||||
To send us a pull request, please:
|
||||
|
||||
1. Fork the repository.
|
||||
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
|
||||
3. Ensure local tests pass.
|
||||
4. Commit to your fork using clear commit messages.
|
||||
5. Send us a pull request, answering any default questions in the pull request interface.
|
||||
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
|
||||
|
||||
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
|
||||
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
|
||||
|
||||
|
||||
## Finding contributions to work on
|
||||
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
|
||||
|
||||
|
||||
## Code of Conduct
|
||||
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
|
||||
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
|
||||
opensource-codeofconduct@amazon.com with any additional questions or comments.
|
||||
|
||||
|
||||
## Security issue notifications
|
||||
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
|
||||
|
||||
|
||||
## Licensing
|
||||
|
||||
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
|
||||
@@ -1,17 +0,0 @@
|
||||
MIT No Attribution
|
||||
|
||||
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
this software and associated documentation files (the "Software"), to deal in
|
||||
the Software without restriction, including without limitation the rights to
|
||||
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
||||
the Software, and to permit persons to whom the Software is furnished to do so.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
||||
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
||||
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
# Welcome to Anthropic's Prompt Engineering Interactive Tutorial - Bedrock Edition
|
||||
|
||||
## Course introduction and goals
|
||||
|
||||
This course is intended to provide you with a comprehensive step-by-step understanding of how to engineer optimal prompts within Claude, using Bedrock.
|
||||
|
||||
**After completing this course, you will be able to**:
|
||||
- Master the basic structure of a good prompt
|
||||
- Recognize common failure modes and learn the '80/20' techniques to address them
|
||||
- Understand Claude's strengths and weaknesses
|
||||
- Build strong prompts from scratch for common use cases
|
||||
|
||||
## Course structure and content
|
||||
|
||||
This course is structured to allow you many chances to practice writing and troubleshooting prompts yourself. The course is broken up into **9 chapters with accompanying exercises**, as well as an appendix of even more advanced methods. It is intended for you to **work through the course in chapter order**.
|
||||
|
||||
**Each lesson has an "Example Playground" area** at the bottom where you are free to experiment with the examples in the lesson and see for yourself how changing prompts can change Claude's responses. There is also an [answer key](https://docs.google.com/spreadsheets/d/1jIxjzUWG-6xBVIa2ay6yDpLyeuOh_hR_ZB75a47KX_E/edit?usp=sharing). While this answer key is structured for 1P API requests, the solutions are the same.
|
||||
|
||||
Note: This tutorial uses our smallest, fastest, and cheapest model, Claude 3 Haiku. Anthropic has [two other models](https://docs.anthropic.com/claude/docs/models-overview), Claude 3 Sonnet and Claude 3 Opus, which are more intelligent than Haiku, with Opus being the most intelligent.
|
||||
|
||||
When you are ready to begin, go to `01_Basic Prompt Structure` to proceed.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
Each chapter consists of a lesson and a set of exercises.
|
||||
|
||||
### Beginner
|
||||
- **Chapter 1:** Basic Prompt Structure
|
||||
|
||||
- **Chapter 2:** Being Clear and Direct
|
||||
|
||||
- **Chapter 3:** Assigning Roles
|
||||
|
||||
### Intermediate
|
||||
- **Chapter 4:** Separating Data from Instructions
|
||||
|
||||
- **Chapter 5:** Formatting Output & Speaking for Claude
|
||||
|
||||
- **Chapter 6:** Precognition (Thinking Step by Step)
|
||||
|
||||
- **Chapter 7:** Using Examples
|
||||
|
||||
### Advanced
|
||||
- **Chapter 8:** Avoiding Hallucinations
|
||||
|
||||
- **Chapter 9:** Building Complex Prompts (Industry Use Cases)
|
||||
- Complex Prompts from Scratch - Chatbot
|
||||
- Complex Prompts for Legal Services
|
||||
- **Exercise:** Complex Prompts for Financial Services
|
||||
- **Exercise:** Complex Prompts for Coding
|
||||
- Congratulations & Next Steps
|
||||
|
||||
- **Appendix:** Beyond Standard Prompting
|
||||
- Chaining Prompts
|
||||
- Tool Use
|
||||
- Empriical Performance Evaluations
|
||||
- Search & Retrieval
|
||||
@@ -1,185 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Tutorial How-To\n",
|
||||
"\n",
|
||||
"This tutorial requires this initial notebook to be run first so that the requirements and environment variables are stored for all notebooks in the workshop"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## How to get started\n",
|
||||
"\n",
|
||||
"1. Clone this repository to your local machine.\n",
|
||||
"\n",
|
||||
"2. Install the required dependencies by running the following command:\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Note: you may need to restart the kernel to use updated packages.\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%pip install -qU pip\n",
|
||||
"%pip install -qr ../requirements.txt"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"3. Restart the kernel after installing dependencies"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# restart kernel\n",
|
||||
"from IPython.core.display import HTML\n",
|
||||
"HTML(\"<script>Jupyter.notebook.kernel.restart()</script>\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Usage Notes & Tips 💡\n",
|
||||
"\n",
|
||||
"- This course uses Claude 3 Haiku with temperature 0. We will talk more about temperature later in the course. For now, it's enough to understand that these settings yield more deterministic results. All prompt engineering techniques in this course also apply to previous generation legacy Claude models such as Claude 2 and Claude Instant 1.2.\n",
|
||||
"\n",
|
||||
"- You can use `Shift + Enter` to execute the cell and move to the next one.\n",
|
||||
"\n",
|
||||
"- When you reach the bottom of a tutorial page, navigate to the next numbered file in the folder, or to the next numbered folder if you're finished with the content within that chapter file.\n",
|
||||
"\n",
|
||||
"### The Anthropic SDK & the Messages API\n",
|
||||
"We will be using the [Anthropic python SDK](https://docs.anthropic.com/claude/reference/claude-on-amazon-bedrock) and the [Messages API](https://docs.anthropic.com/claude/reference/messages_post) throughout this tutorial.\n",
|
||||
"\n",
|
||||
"Below is an example of what running a prompt will look like in this tutorial."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"First, we set and store the model name and region."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import boto3\n",
|
||||
"session = boto3.Session() # create a boto3 session to dynamically get and set the region name\n",
|
||||
"AWS_REGION = session.region_name\n",
|
||||
"print(\"AWS Region:\", AWS_REGION)\n",
|
||||
"MODEL_NAME = \"anthropic.claude-3-haiku-20240307-v1:0\"\n",
|
||||
"\n",
|
||||
"%store MODEL_NAME\n",
|
||||
"%store AWS_REGION"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then, we create `get_completion`, which is a helper function that sends a prompt to Claude and returns Claude's generated response. Run that cell now."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now we will write out an example prompt for Claude and print Claude's output by running our `get_completion` helper function. Running the cell below will print out a response from Claude beneath it.\n",
|
||||
"\n",
|
||||
"Feel free to play around with the prompt string to elicit different responses from Claude."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"prompt = \"Hello, Claude!\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"print(get_completion(prompt))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `MODEL_NAME` and `AWS_REGION` variables defined earlier will be used throughout the tutorial. Just make sure to run the cells for each tutorial page from top to bottom."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "py310",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,493 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 1: Basic Prompt Structure\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Anthropic offers two APIs, the legacy [Text Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-text-completion.html) and the current [Messages API](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html). For this tutorial, we will be exclusively using the Messages API.\n",
|
||||
"\n",
|
||||
"At minimum, a call to Claude using the Messages API requires the following parameters:\n",
|
||||
"- `model`: the [API model name](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns) of the model that you intend to call\n",
|
||||
"\n",
|
||||
"- `max_tokens`: the maximum number of tokens to generate before stopping. Note that Claude may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. Furthermore, this is a *hard* stop, meaning that it may cause Claude to stop generating mid-word or mid-sentence.\n",
|
||||
"\n",
|
||||
"- `messages`: an array of input messages. Our models are trained to operate on alternating `user` and `assistant` conversational turns. When creating a new `Message`, you specify the prior conversational turns with the messages parameter, and the model then generates the next `Message` in the conversation.\n",
|
||||
" - Each input message must be an object with a `role` and `content`. You can specify a single `user`-role message, or you can include multiple `user` and `assistant` messages (they must alternate, if so). The first message must always use the user `role`.\n",
|
||||
"\n",
|
||||
"There are also optional parameters, such as:\n",
|
||||
"- `system`: the system prompt - more on this below.\n",
|
||||
" \n",
|
||||
"- `temperature`: the degree of variability in Claude's response. For these lessons and exercises, we have set `temperature` to 0.\n",
|
||||
"\n",
|
||||
"For a complete list of all API parameters, visit our [API documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-claude.html)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Let's take a look at how Claude responds to some correctly-formatted prompts. For each of the following cells, run the cell (`shift+enter`), and Claude's response will appear below the block."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hi Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Can you tell me the color of the ocean?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"What year was Celine Dion born in?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's take a look at some prompts that do not include the correct Messages API formatting. For these malformatted prompts, the Messages API returns an error.\n",
|
||||
"\n",
|
||||
"First, we have an example of a Messages API call that lacks `role` and `content` fields in the `messages` array."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> ⚠️ **Warning:** Due to the incorrect formatting of the messages parameter in the prompt, the following cell will return an error. This is expected behavior."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"Hi Claude, how are you?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's a prompt that fails to alternate between the `user` and `assistant` roles."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> ⚠️ **Warning:** Due to the lack of alternation between `user` and `assistant` roles, Claude will return an error message. This is expected behavior."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": \"What year was Celine Dion born in?\"},\n",
|
||||
" {\"role\": \"user\", \"content\": \"Also, can you tell me some other facts about her?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`user` and `assistant` messages **MUST alternate**, and messages **MUST start with a `user` turn**. You can have multiple `user` & `assistant` pairs in a prompt (as if simulating a multi-turn conversation). You can also put words into a terminal `assistant` message for Claude to continue from where you left off (more on that in later chapters).\n",
|
||||
"\n",
|
||||
"#### System Prompts\n",
|
||||
"\n",
|
||||
"You can also use **system prompts**. A system prompt is a way to **provide context, instructions, and guidelines to Claude** before presenting it with a question or task in the \"User\" turn. \n",
|
||||
"\n",
|
||||
"Structurally, system prompts exist separately from the list of `user` & `assistant` messages, and thus belong in a separate `system` parameter (take a look at the structure of the `get_completion` helper function in the [Setup](#setup) section of the notebook). \n",
|
||||
"\n",
|
||||
"Within this tutorial, wherever we might utilize a system prompt, we have provided you a `system` field in your completions function. Should you not want to use a system prompt, simply set the `SYSTEM_PROMPT` variable to an empty string."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### System Prompt Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Why is the sky blue?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why use a system prompt? A **well-written system prompt can improve Claude's performance** in a variety of ways, such as increasing Claude's ability to follow rules and instructions. For more information, visit our documentation on [how to use system prompts](https://docs.anthropic.com/claude/docs/how-to-use-system-prompts) with Claude.\n",
|
||||
"\n",
|
||||
"Now we'll dive into some exercises. If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 1.1 - Counting to Three](#exercise-11---counting-to-three)\n",
|
||||
"- [Exercise 1.2 - System Prompt](#exercise-12---system-prompt)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 1.1 - Counting to Three\n",
|
||||
"Using proper `user` / `assistant` formatting, edit the `PROMPT` below to get Claude to **count to three.** The output will also indicate whether your solution is correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" pattern = re.compile(r'^(?=.*1)(?=.*2)(?=.*3).*$', re.DOTALL)\n",
|
||||
" return bool(pattern.match(text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_1_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 1.2 - System Prompt\n",
|
||||
"\n",
|
||||
"Modify the `SYSTEM_PROMPT` to make Claude respond like it's a 3 year old child."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - this is the only field you should change\n",
|
||||
"SYSTEM_PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"How big is the sky?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(r\"giggles\", text) or re.search(r\"soo\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_1_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hi Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Can you tell me the color of the ocean?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"What year was Celine Dion born in?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"Hi Claude, how are you?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": \"What year was Celine Dion born in?\"},\n",
|
||||
" {\"role\": \"user\", \"content\": \"Also, can you tell me some other facts about her?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Why is the sky blue?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,399 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 2: Being Clear and Direct\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Claude responds best to clear and direct instructions.**\n",
|
||||
"\n",
|
||||
"Think of Claude like any other human that is new to the job. **Claude has no context** on what to do aside from what you literally tell it. Just as when you instruct a human for the first time on a task, the more you explain exactly what you want in a straightforward manner to Claude, the better and more accurate Claude's response will be.\"\t\t\t\t\n",
|
||||
"\t\t\t\t\n",
|
||||
"When in doubt, follow the **Golden Rule of Clear Prompting**:\n",
|
||||
"- Show your prompt to a colleague or friend and have them follow the instructions themselves to see if they can produce the result you want. If they're confused, Claude's confused.\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Let's take a task like writing poetry. (Ignore any syllable mismatch - LLMs aren't great at counting syllables yet.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This haiku is nice enough, but users may want Claude to go directly into the poem without the \"Here is a haiku\" preamble.\n",
|
||||
"\n",
|
||||
"How do we achieve that? We **ask for it**!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots. Skip the preamble; go straight into the poem.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's another example. Let's ask Claude who's the best basketball player of all time. You can see below that while Claude lists a few names, **it doesn't respond with a definitive \"best\"**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Can we get Claude to make up its mind and decide on a best player? Yes! Just ask!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time? Yes, there are differing opinions, but if you absolutely had to pick one player, who would it be?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 2.1 - Spanish](#exercise-21---spanish)\n",
|
||||
"- [Exercise 2.2 - One Player Only](#exercise-22---one-player-only)\n",
|
||||
"- [Exercise 2.3 - Write a Story](#exercise-23---write-a-story)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.1 - Spanish\n",
|
||||
"Modify the `SYSTEM_PROMPT` to make Claude output its answer in Spanish."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - this is the only field you should chnage\n",
|
||||
"SYSTEM_PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hello Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return \"hola\" in text.lower()\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_2_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.2 - One Player Only\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` so that Claude doesn't equivocate at all and responds with **ONLY** the name of one specific player, with **no other words or punctuation**. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return text == \"Michael Jordan\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_2_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.3 - Write a Story\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` so that Claude responds with as long a response as you can muster. If your answer is **over 800 words**, Claude's response will be graded as correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" trimmed = text.strip()\n",
|
||||
" words = len(trimmed.split())\n",
|
||||
" return words >= 800\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_2_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots. Skip the preamble; go straight into the poem.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time? Yes, there are differing opinions, but if you absolutely had to pick one player, who would it be?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,331 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 3: Assigning Roles (Role Prompting)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Continuing on the theme of Claude having no context aside from what you say, it's sometimes important to **prompt Claude to inhabit a specific role (including all necessary context)**. This is also known as role prompting. The more detail to the role context, the better.\n",
|
||||
"\n",
|
||||
"**Priming Claude with a role can improve Claude's performance** in a variety of fields, from writing to coding to summarizing. It's like how humans can sometimes be helped when told to \"think like a ______\". Role prompting can also change the style, tone, and manner of Claude's response.\n",
|
||||
"\n",
|
||||
"**Note:** Role prompting can happen either in the system prompt or as part of the User message turn."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In the example below, we see that without role prompting, Claude provides a **straightforward and non-stylized answer** when asked to give a single sentence perspective on skateboarding.\n",
|
||||
"\n",
|
||||
"However, when we prime Claude to inhabit the role of a cat, Claude's perspective changes, and thus **Claude's response tone, style, content adapts to the new role**. \n",
|
||||
"\n",
|
||||
"**Note:** A bonus technique you can use is to **provide Claude context on its intended audience**. Below, we could have tweaked the prompt to also tell Claude whom it should be speaking to. \"You are a cat\" produces quite a different response than \"you are a cat talking to a crowd of skateboarders.\n",
|
||||
"\n",
|
||||
"Here is the prompt without role prompting in the system prompt:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here is the same user question, except with role prompting."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a cat.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can use role prompting as a way to get Claude to emulate certain styles in writing, speak in a certain voice, or guide the complexity of its answers. **Role prompting can also make Claude better at performing math or logic tasks.**\n",
|
||||
"\n",
|
||||
"For example, in the example below, there is a definitive correct answer, which is yes. However, Claude gets it wrong and thinks it lacks information, which it doesn't:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, what if we **prime Claude to act as a logic bot**? How will that change Claude's answer? \n",
|
||||
"\n",
|
||||
"It turns out that with this new role assignment, Claude gets it right. (Although notably not for all the right reasons)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a logic bot designed to answer complex logic problems.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Note:** What you'll learn throughout this course is that there are **many prompt engineering techniques you can use to derive similar results**. Which techniques you use is up to you and your preference! We encourage you to **experiment to find your own prompt engineering style**.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 3.1 - Math Correction](#exercise-31---math-correction)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 3.1 - Math Correction\n",
|
||||
"In some instances, **Claude may struggle with mathematics**, even simple mathematics. Below, Claude incorrectly assesses the math problem as correctly solved, even though there's an obvious arithmetic mistake in the second step. Note that Claude actually catches the mistake when going through step-by-step, but doesn't jump to the conclusion that the overall solution is wrong.\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` and / or the `SYSTEM_PROMPT` to make Claude grade the solution as `incorrectly` solved, rather than correctly solved. \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - if you don't want to use a system prompt, you can leave this variable set to an empty string\n",
|
||||
"SYSTEM_PROMPT = \"\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this equation solved correctly below?\n",
|
||||
"\n",
|
||||
"2x - 3 = 9\n",
|
||||
"2x = 6\n",
|
||||
"x = 3\"\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" if \"incorrect\" in text or \"not correct\" in text.lower():\n",
|
||||
" return True\n",
|
||||
" else:\n",
|
||||
" return False\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_3_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a cat.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a logic bot designed to answer complex logic problems.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,558 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 4: Separating Data and Instructions\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Oftentimes, we don't want to write full prompts, but instead want **prompt templates that can be modified later with additional input data before submitting to Claude**. This might come in handy if you want Claude to do the same thing every time, but the data that Claude uses for its task might be different each time. \n",
|
||||
"\n",
|
||||
"Luckily, we can do this pretty easily by **separating the fixed skeleton of the prompt from variable user input, then substituting the user input into the prompt** before sending the full prompt to Claude. \n",
|
||||
"\n",
|
||||
"Below, we'll walk step by step through how to write a substitutable prompt template, as well as how to substitute in user input."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In this first example, we're asking Claude to act as an animal noise generator. Notice that the full prompt submitted to Claude is just the `PROMPT_TEMPLATE` substituted with the input (in this case, \"Cow\"). Notice that the word \"Cow\" replaces the `ANIMAL` placeholder via an f-string when we print out the full prompt.\n",
|
||||
"\n",
|
||||
"**Note:** You don't have to call your placeholder variable anything in particular in practice. We called it `ANIMAL` in this example, but just as easily, we could have called it `CREATURE` or `A` (although it's generally good to have your variable names be specific and relevant so that your prompt template is easy to understand even without the substitution, just for user parseability). Just make sure that whatever you name your variable is what you use for the prompt template f-string."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cow\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why would we want to separate and substitute inputs like this? Well, **prompt templates simplify repetitive tasks**. Let's say you build a prompt structure that invites third party users to submit content to the prompt (in this case the animal whose sound they want to generate). These third party users don't have to write or even see the full prompt. All they have to do is fill in variables.\n",
|
||||
"\n",
|
||||
"We do this substitution here using variables and f-strings, but you can also do it with the format() method.\n",
|
||||
"\n",
|
||||
"**Note:** Prompt templates can have as many variables as desired!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When introducing substitution variables like this, it is very important to **make sure Claude knows where variables start and end** (vs. instructions or task descriptions). Let's look at an example where there is no separation between the instructions and the substitution variable.\n",
|
||||
"\n",
|
||||
"To our human eyes, it is very clear where the variable begins and ends in the prompt template below. However, in the fully substituted prompt, that delineation becomes unclear."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here, **Claude thinks \"Yo Claude\" is part of the email it's supposed to rewrite**! You can tell because it begins its rewrite with \"Dear Claude\". To the human eye, it's clear, particularly in the prompt template where the email begins and ends, but it becomes much less clear in the prompt after substitution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"How do we solve this? **Wrap the input in XML tags**! We did this below, and as you can see, there's no more \"Dear Claude\" in the output.\n",
|
||||
"\n",
|
||||
"[XML tags](https://docs.anthropic.com/claude/docs/use-xml-tags) are angle-bracket tags like `<tag></tag>`. They come in pairs and consist of an opening tag, such as `<tag>`, and a closing tag marked by a `/`, such as `</tag>`. XML tags are used to wrap around content, like this: `<tag>content</tag>`.\n",
|
||||
"\n",
|
||||
"**Note:** While Claude can recognize and work with a wide range of separators and delimeters, we recommend that you **use specifically XML tags as separators** for Claude, as Claude was trained specifically to recognize XML tags as a prompt organizing mechanism. Outside of function calling, **there are no special sauce XML tags that Claude has been trained on that you should use to maximally boost your performance**. We have purposefully made Claude very malleable and customizable this way."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's see another example of how XML tags can help us. \n",
|
||||
"\n",
|
||||
"In the following prompt, **Claude incorrectly interprets what part of the prompt is the instruction vs. the input**. It incorrectly considers `Each is about an animal, like rabbits` to be part of the list due to the formatting, when the user (the one filling out the `SENTENCES` variable) presumably did not want that."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"{SENTENCES}\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To fix this, we just need to **surround the user input sentences in XML tags**. This shows Claude where the input data begins and ends despite the misleading hyphen before `Each is about an animal, like rabbits.`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"<sentences>\n",
|
||||
"{SENTENCES}\n",
|
||||
"</sentences>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Note:** In the incorrect version of the \"Each is about an animal\" prompt, we had to include the hyphen to get Claude to respond incorrectly in the way we wanted to for this example. This is an important lesson about prompting: **small details matter**! It's always worth it to **scrub your prompts for typos and grammatical errors**. Claude is sensitive to patterns (in its early years, before finetuning, it was a raw text-prediction tool), and it's more likely to make mistakes when you make mistakes, smarter when you sound smart, sillier when you sound silly, and so on.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 4.1 - Haiku Topic](#exercise-41---haiku-topic)\n",
|
||||
"- [Exercise 4.2 - Dog Question with Typos](#exercise-42---dog-question-with-typos)\n",
|
||||
"- [Exercise 4.3 - Dog Question Part 2](#exercise-42---dog-question-part-2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.1 - Haiku Topic\n",
|
||||
"Modify the `PROMPT` so that it's a template that will take in a variable called `TOPIC` and output a haiku about the topic. This exercise is just meant to test your understanding of the variable templating structure with f-strings."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"TOPIC = \"Pigs\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"pigs\", text.lower()) and re.search(\"haiku\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_4_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.2 - Dog Question with Typos\n",
|
||||
"Fix the `PROMPT` by adding XML tags so that Claude produces the right answer. \n",
|
||||
"\n",
|
||||
"Try not to change anything else about the prompt. The messy and mistake-ridden writing is intentional, so you can see how Claude reacts to such mistakes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"QUESTION = \"ar cn brown?\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"brown\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_4_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.3 - Dog Question Part 2\n",
|
||||
"Fix the `PROMPT` **WITHOUT** adding XML tags. Instead, remove only one or two words from the prompt.\n",
|
||||
"\n",
|
||||
"Just as with the above exercises, try not to change anything else about the prompt. This will show you what kind of language Claude can parse and understand."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"QUESTION = \"ar cn brown?\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"brown\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_4_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cow\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"{SENTENCES}\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"<sentences>\n",
|
||||
"{SENTENCES}\n",
|
||||
"</sentences>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,528 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 5: Formatting Output and Speaking for Claude\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Claude can format its output in a wide variety of ways**. You just need to ask for it to do so!\n",
|
||||
"\n",
|
||||
"One of these ways is by using XML tags to separate out the response from any other superfluous text. You've already learned that you can use XML tags to make your prompt clearer and more parseable to Claude. It turns out, you can also ask Claude to **use XML tags to make its output clearer and more easily understandable** to humans."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Remember the 'poem preamble problem' we solved in Chapter 2 by asking Claude to skip the preamble entirely? It turns out we can also achieve a similar outcome by **telling Claude to put the poem in XML tags**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Rabbit\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why is this something we'd want to do? Well, having the output in **XML tags allows the end user to reliably get the poem and only the poem by writing a short program to extract the content between XML tags**.\n",
|
||||
"\n",
|
||||
"An extension of this technique is to **put the first XML tag in the `assistant` turn. When you put text in the `assistant` turn, you're basically telling Claude that Claude has already said something, and that it should continue from that point onward. This technique is called \"speaking for Claude\" or \"prefilling Claude's response.\"\n",
|
||||
"\n",
|
||||
"Below, we've done this with the first `<haiku>` XML tag. Notice how Claude continues directly from where we left off."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Claude also excels at using other output formatting styles, notably `JSON`. If you want to enforce JSON output (not deterministically, but close to it), you can also prefill Claude's response with the opening bracket, `{`}."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"{\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Below is an example of **multiple input variables in the same prompt AND output formatting specification, all done using XML tags**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ADJECTIVE = \"olde english\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response (now as an f-string with a variable)\n",
|
||||
"PREFILL = f\"<{ADJECTIVE}_email>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Bonus lesson\n",
|
||||
"\n",
|
||||
"If you are calling Claude through the API, you can pass the closing XML tag to the `stop_sequences` parameter to get Claude to stop sampling once it emits your desired tag. This can save money and time-to-last-token by eliminating Claude's concluding remarks after it's already given you the answer you care about.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 5.1 - Steph Curry GOAT](#exercise-51---steph-curry-goat)\n",
|
||||
"- [Exercise 5.2 - Two Haikus](#exercise-52---two-haikus)\n",
|
||||
"- [Exercise 5.3 - Two Haikus, Two Animals](#exercise-53---two-haikus-two-animals)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.1 - Steph Curry GOAT\n",
|
||||
"Forced to make a choice, Claude designates Michael Jordan as the best basketball player of all time. Can we get Claude to pick someone else?\n",
|
||||
"\n",
|
||||
"Change the `PREFILL` variable to **compell Claude to make a detailed argument that the best basketball player of all time is Stephen Curry**. Try not to change anything except `PREFILL` as that is the focus of this exercise."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Who is the best basketball player of all time? Please choose one specific player.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"Warrior\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_5_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.2 - Two Haikus\n",
|
||||
"Modify the `PROMPT` below using XML tags so that Claude writes two haikus about the animal instead of just one. It should be clear where one poem ends and the other begins."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"cats\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(\n",
|
||||
" (re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
|
||||
" and (text.count(\"\\n\") + 1) > 5\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_5_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.3 - Two Haikus, Two Animals\n",
|
||||
"Modify the `PROMPT` below so that **Claude produces two haikus about two different animals**. Use `{ANIMAL1}` as a stand-in for the first substitution, and `{ANIMAL2}` as a stand-in for the second substitution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"ANIMAL1 = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ANIMAL2 = \"Dog\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL1}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"tail\", text.lower()) and re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_5_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Rabbit\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"{\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ADJECTIVE = \"olde english\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response (now as an f-string with a variable)\n",
|
||||
"PREFILL = f\"<{ADJECTIVE}_email>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,508 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 6: Precognition (Thinking Step by Step)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"If someone woke you up and immediately started asking you several complicated questions that you had to respond to right away, how would you do? Probably not as good as if you were given time to **think through your answer first**. \n",
|
||||
"\n",
|
||||
"Guess what? Claude is the same way.\n",
|
||||
"\n",
|
||||
"**Giving Claude time to think step by step sometimes makes Claude more accurate**, particularly for complex tasks. However, **thinking only counts when it's out loud**. You cannot ask Claude to think but output only the answer - in this case, no thinking has actually occurred."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In the prompt below, it's clear to a human reader that the second sentence belies the first. But **Claude takes the word \"unrelated\" too literally**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this movie review sentiment positive or negative?\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since the year 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To improve Claude's response, let's **allow Claude to think things out first before answering**. We do that by literally spelling out the steps that Claude should take in order to process and think through its task. Along with a dash of role prompting, this empowers Claude to understand the review more deeply."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a savvy reader of movie reviews.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment positive or negative? First, write the best arguments for each side in <positive-argument> and <negative-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Claude is sometimes sensitive to ordering**. This example is on the frontier of Claude's ability to understand nuanced text, and when we swap the order of the arguments from the previous example so that negative is first and positive is second, this changes Claude's overall assessment to positive.\n",
|
||||
"\n",
|
||||
"In most situations (but not all, confusingly enough), **Claude is more likely to choose the second of two options**, possibly because in its training data from the web, second options were more likely to be correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment negative or positive? First write the best arguments for each side in <negative-argument> and <positive-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. Unrelatedly, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Letting Claude think can shift Claude's answer from incorrect to correct**. It's that simple in many cases where Claude makes mistakes!\n",
|
||||
"\n",
|
||||
"Let's go through an example where Claude's answer is incorrect to see how asking Claude to think can fix that."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's fix this by asking Claude to think step by step, this time in `<brainstorm>` tags."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 6.1 - Classifying Emails](#exercise-61---classifying-emails)\n",
|
||||
"- [Exercise 6.2 - Email Classification Formatting](#exercise-62---email-classification-formatting)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 6.1 - Classifying Emails\n",
|
||||
"In this exercise, we'll be instructing Claude to sort emails into the following categories:\t\t\t\t\t\t\t\t\t\t\n",
|
||||
"- (A) Pre-sale question\n",
|
||||
"- (B) Broken or defective item\n",
|
||||
"- (C) Billing question\n",
|
||||
"- (D) Other (please explain)\n",
|
||||
"\n",
|
||||
"For the first part of the exercise, change the `PROMPT` to **make Claude output the correct classification and ONLY the classification**. Your answer needs to **include the letter (A - D) of the correct choice, with the parentheses, as well as the name of the category**.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list to know which category that email should be classified under."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response, if any\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Dictionary of string values for each category to be used for regex grading\n",
|
||||
"REGEX_CATEGORIES = {\n",
|
||||
" \"A\": \"A\\) P\",\n",
|
||||
" \"B\": \"B\\) B\",\n",
|
||||
" \"C\": \"C\\) B\",\n",
|
||||
" \"D\": \"D\\) O\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_6_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Still stuck? Run the cell below for an example solution.\t\t\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_6_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 6.2 - Email Classification Formatting\n",
|
||||
"In this exercise, we're going to refine the output of the above prompt to yield an answer formatted exactly how we want it. \n",
|
||||
"\n",
|
||||
"Use your favorite output formatting technique to make Claude wrap JUST the letter of the correct classification in `<answer></answer>` tags. For instance, the answer to the first email should contain the exact string `<answer>B</answer>`.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list if you forget which letter category is correct for each email."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response, if any\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Dictionary of string values for each category to be used for regex grading\n",
|
||||
"REGEX_CATEGORIES = {\n",
|
||||
" \"A\": \"<answer>A</answer>\",\n",
|
||||
" \"B\": \"<answer>B</answer>\",\n",
|
||||
" \"C\": \"<answer>C</answer>\",\n",
|
||||
" \"D\": \"<answer>D</answer>\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_6_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this movie review sentiment positive or negative?\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since the year 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a savvy reader of movie reviews.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment positive or negative? First, write the best arguments for each side in <positive-argument> and <negative-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment negative or positive? First write the best arguments for each side in <negative-argument> and <positive-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. Unrelatedly, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,397 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 7: Using Examples (Few-Shot Prompting)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Giving Claude examples of how you want it to behave (or how you want it not to behave) is extremely effective** for:\n",
|
||||
"- Getting the right answer\n",
|
||||
"- Getting the answer in the right format\n",
|
||||
"\n",
|
||||
"This sort of prompting is also called \"**few shot prompting**\". You might also encounter the phrase \"zero-shot\" or \"n-shot\" or \"one-shot\". The number of \"shots\" refers to how many examples are used within the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Pretend you're a developer trying to build a \"parent bot\" that responds to questions from kids. **Claude's default response is quite formal and robotic**. This is going to break a child's heart."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Will Santa bring me presents on Christmas?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You could take the time to describe your desired tone, but it's much easier just to **give Claude a few examples of ideal responses**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Please complete the conversation by writing the next line, speaking as \"A\".\n",
|
||||
"Q: Is the tooth fairy real?\n",
|
||||
"A: Of course, sweetie. Wrap up your tooth and put it under your pillow tonight. There might be something waiting for you in the morning.\n",
|
||||
"Q: Will Santa bring me presents on Christmas?\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In the following formatting example, we could walk Claude step by step through a set of formatting instructions on how to extract names and professions and then format them exactly the way we want, or we could just **provide Claude with some correctly-formatted examples and Claude can extrapolate from there**. Note the `<individuals>` in the `assistant` turn to start Claude off on the right foot."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Silvermist Hollow, a charming village, was home to an extraordinary group of individuals.\n",
|
||||
"Among them was Dr. Liam Patel, a neurosurgeon who revolutionized surgical techniques at the regional medical center.\n",
|
||||
"Olivia Chen was an innovative architect who transformed the village's landscape with her sustainable and breathtaking designs.\n",
|
||||
"The local theater was graced by the enchanting symphonies of Ethan Kovacs, a professionally-trained musician and composer.\n",
|
||||
"Isabella Torres, a self-taught chef with a passion for locally sourced ingredients, created a culinary sensation with her farm-to-table restaurant, which became a must-visit destination for food lovers.\n",
|
||||
"These remarkable individuals, each with their distinct talents, contributed to the vibrant tapestry of life in Silvermist Hollow.\n",
|
||||
"<individuals>\n",
|
||||
"1. Dr. Liam Patel [NEUROSURGEON]\n",
|
||||
"2. Olivia Chen [ARCHITECT]\n",
|
||||
"3. Ethan Kovacs [MISICIAN AND COMPOSER]\n",
|
||||
"4. Isabella Torres [CHEF]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"At the heart of the town, Chef Oliver Hamilton has transformed the culinary scene with his farm-to-table restaurant, Green Plate. Oliver's dedication to sourcing local, organic ingredients has earned the establishment rave reviews from food critics and locals alike.\n",
|
||||
"Just down the street, you'll find the Riverside Grove Library, where head librarian Elizabeth Chen has worked diligently to create a welcoming and inclusive space for all. Her efforts to expand the library's offerings and establish reading programs for children have had a significant impact on the town's literacy rates.\n",
|
||||
"As you stroll through the charming town square, you'll be captivated by the beautiful murals adorning the walls. These masterpieces are the work of renowned artist, Isabella Torres, whose talent for capturing the essence of Riverside Grove has brought the town to life.\n",
|
||||
"Riverside Grove's athletic achievements are also worth noting, thanks to former Olympic swimmer-turned-coach, Marcus Jenkins. Marcus has used his experience and passion to train the town's youth, leading the Riverside Grove Swim Team to several regional championships.\n",
|
||||
"<individuals>\n",
|
||||
"1. Oliver Hamilton [CHEF]\n",
|
||||
"2. Elizabeth Chen [LIBRARIAN]\n",
|
||||
"3. Isabella Torres [ARTIST]\n",
|
||||
"4. Marcus Jenkins [COACH]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"Oak Valley, a charming small town, is home to a remarkable trio of individuals whose skills and dedication have left a lasting impact on the community.\n",
|
||||
"At the town's bustling farmer's market, you'll find Laura Simmons, a passionate organic farmer known for her delicious and sustainably grown produce. Her dedication to promoting healthy eating has inspired the town to embrace a more eco-conscious lifestyle.\n",
|
||||
"In Oak Valley's community center, Kevin Alvarez, a skilled dance instructor, has brought the joy of movement to people of all ages. His inclusive dance classes have fostered a sense of unity and self-expression among residents, enriching the local arts scene.\n",
|
||||
"Lastly, Rachel O'Connor, a tireless volunteer, dedicates her time to various charitable initiatives. Her commitment to improving the lives of others has been instrumental in creating a strong sense of community within Oak Valley.\n",
|
||||
"Through their unique talents and unwavering dedication, Laura, Kevin, and Rachel have woven themselves into the fabric of Oak Valley, helping to create a vibrant and thriving small town.\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<individuals>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 7.1 - Email Formatting via Examples](#exercise-71---email-formatting-via-examples)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 7.1 - Email Formatting via Examples\n",
|
||||
"We're going to redo Exercise 6.2, but this time, we're going to edit the `PROMPT` to use \"few-shot\" examples of emails + proper classification (and formatting) to get Claude to output the correct answer. We want the *last* letter of Claude's output to be the letter of the category.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list if you forget which letter category is correct for each email.\n",
|
||||
"\n",
|
||||
"Remember that these are the categories for the emails:\t\t\t\t\t\t\t\t\t\t\n",
|
||||
"- (A) Pre-sale question\n",
|
||||
"- (B) Broken or defective item\n",
|
||||
"- (C) Billing question\n",
|
||||
"- (D) Other (please explain)\t\t\t\t\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(ans, response[-1])) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_7_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Still stuck? Run the cell below for an example solution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_7_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Will Santa bring me presents on Christmas?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Please complete the conversation by writing the next line, speaking as \"A\".\n",
|
||||
"Q: Is the tooth fairy real?\n",
|
||||
"A: Of course, sweetie. Wrap up your tooth and put it under your pillow tonight. There might be something waiting for you in the morning.\n",
|
||||
"Q: Will Santa bring me presents on Christmas?\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Silvermist Hollow, a charming village, was home to an extraordinary group of individuals.\n",
|
||||
"Among them was Dr. Liam Patel, a neurosurgeon who revolutionized surgical techniques at the regional medical center.\n",
|
||||
"Olivia Chen was an innovative architect who transformed the village's landscape with her sustainable and breathtaking designs.\n",
|
||||
"The local theater was graced by the enchanting symphonies of Ethan Kovacs, a professionally-trained musician and composer.\n",
|
||||
"Isabella Torres, a self-taught chef with a passion for locally sourced ingredients, created a culinary sensation with her farm-to-table restaurant, which became a must-visit destination for food lovers.\n",
|
||||
"These remarkable individuals, each with their distinct talents, contributed to the vibrant tapestry of life in Silvermist Hollow.\n",
|
||||
"<individuals>\n",
|
||||
"1. Dr. Liam Patel [NEUROSURGEON]\n",
|
||||
"2. Olivia Chen [ARCHITECT]\n",
|
||||
"3. Ethan Kovacs [MISICIAN AND COMPOSER]\n",
|
||||
"4. Isabella Torres [CHEF]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"At the heart of the town, Chef Oliver Hamilton has transformed the culinary scene with his farm-to-table restaurant, Green Plate. Oliver's dedication to sourcing local, organic ingredients has earned the establishment rave reviews from food critics and locals alike.\n",
|
||||
"Just down the street, you'll find the Riverside Grove Library, where head librarian Elizabeth Chen has worked diligently to create a welcoming and inclusive space for all. Her efforts to expand the library's offerings and establish reading programs for children have had a significant impact on the town's literacy rates.\n",
|
||||
"As you stroll through the charming town square, you'll be captivated by the beautiful murals adorning the walls. These masterpieces are the work of renowned artist, Isabella Torres, whose talent for capturing the essence of Riverside Grove has brought the town to life.\n",
|
||||
"Riverside Grove's athletic achievements are also worth noting, thanks to former Olympic swimmer-turned-coach, Marcus Jenkins. Marcus has used his experience and passion to train the town's youth, leading the Riverside Grove Swim Team to several regional championships.\n",
|
||||
"<individuals>\n",
|
||||
"1. Oliver Hamilton [CHEF]\n",
|
||||
"2. Elizabeth Chen [LIBRARIAN]\n",
|
||||
"3. Isabella Torres [ARTIST]\n",
|
||||
"4. Marcus Jenkins [COACH]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"Oak Valley, a charming small town, is home to a remarkable trio of individuals whose skills and dedication have left a lasting impact on the community.\n",
|
||||
"At the town's bustling farmer's market, you'll find Laura Simmons, a passionate organic farmer known for her delicious and sustainably grown produce. Her dedication to promoting healthy eating has inspired the town to embrace a more eco-conscious lifestyle.\n",
|
||||
"In Oak Valley's community center, Kevin Alvarez, a skilled dance instructor, has brought the joy of movement to people of all ages. His inclusive dance classes have fostered a sense of unity and self-expression among residents, enriching the local arts scene.\n",
|
||||
"Lastly, Rachel O'Connor, a tireless volunteer, dedicates her time to various charitable initiatives. Her commitment to improving the lives of others has been instrumental in creating a strong sense of community within Oak Valley.\n",
|
||||
"Through their unique talents and unwavering dedication, Laura, Kevin, and Rachel have woven themselves into the fabric of Oak Valley, helping to create a vibrant and thriving small town.\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<individuals>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,707 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 8: Avoiding Hallucinations\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" system=system\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Some bad news: **Claude sometimes \"hallucinates\" and makes claims that are untrue or unjustified**. The good news: there are techniques you can use to minimize hallucinations.\n",
|
||||
"\t\t\t\t\n",
|
||||
"Below, we'll go over a few of these techniques, namely:\n",
|
||||
"- Giving Claude the option to say it doesn't know the answer to a question\n",
|
||||
"- Asking Claude to find evidence before answering\n",
|
||||
"\n",
|
||||
"However, **there are many methods to avoid hallucinations**, including many of the techniques you've already learned in this course. If Claude hallucinates, experiment with multiple techniques to get Claude to increase its accuracy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Here is a question about general factual knowledge in answer to which **Claude hallucinates several large hippos because it's trying to be as helpful as possible**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"A solution we can try here is to \"**give Claude an out**\" — tell Claude that it's OK for it to decline to answer, or to only answer if it actually knows the answer with certainty."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time? Only answer if you know the answer with certainty.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In the prompt below, we give Claude a long document containing some \"distractor information\" that is almost but not quite relevant to the user's question. **Without prompting help, Claude falls for the distractor information** and gives an incorrect \"hallucinated\" answer as to the size of Matterport's subscriber base as of May 31, 2020.\n",
|
||||
"\n",
|
||||
"**Note:** As you'll learn later in the next chapter, **it's best practice to have the question at the bottom *after* any text or document**, but we put it at the top here to make the prompt easier to read. Feel free to double click on the prompt cell to get the full prompt text (it's very long!)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then write a brief numerical answer inside <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"How do we fix this? Well, a great way to reduce hallucinations on long documents is to **make Claude gather evidence first.** \n",
|
||||
"\n",
|
||||
"In this case, we **tell Claude to first extract relevant quotes, then base its answer on those quotes**. Telling Claude to do so here makes it correctly notice that the quote does not answer the question."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then, in <scratchpad> tags, pull the most relevant quote from the document and consider whether it answers the user's question or whether it lacks sufficient detail. Then write a brief numerical answer in <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Bonus lesson\n",
|
||||
"\n",
|
||||
"Sometimes, Claude's hallucinations can be solved by lowering the `temperature` of Claude's responses. Temperature is a measurement of answer creativity between 0 and 1, with 1 being more unpredictable and less standardized, and 0 being the most consistent. \n",
|
||||
"\n",
|
||||
"Asking Claude something at temperature 0 will generally yield an almost-deterministic answer set across repeated trials (although complete determinism is not guaranteed). Asking Claude something at temperature 1 (or gradations in between) will yield more variable answers. Learn more about temperature and other parameters [here](https://docs.anthropic.com/claude/reference/messages_post).\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 8.1 - Beyoncé Hallucination](#exercise-81---beyoncé-hallucination)\n",
|
||||
"- [Exercise 8.2 - Prospectus Hallucination](#exercise-82---prospectus-hallucination)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 8.1 - Beyoncé Hallucination\n",
|
||||
"Modify the `PROMPT` to fix Claude's hallucination issue by giving Claude an out. (Renaissance is Beyoncé's seventh studio album, not her eigthth.)\n",
|
||||
"\n",
|
||||
"We suggest you run the cell first to see what Claude hallucinates before trying to fix it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In what year did star performer Beyoncé release her eighth studio album?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" contains = bool(\n",
|
||||
" re.search(\"Unfortunately\", text) or\n",
|
||||
" re.search(\"I do not\", text) or\n",
|
||||
" re.search(\"I don't\", text)\n",
|
||||
" )\n",
|
||||
" does_not_contain = not bool(re.search(\"2022\", text))\n",
|
||||
" return contains and does_not_contain\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_8_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 8.1 - Prospectus Hallucination\n",
|
||||
"Modify the `PROMPT` to fix Claude's hallucination issue by asking for citations. The correct answer is that subscribers went up 49x."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"From December 2018 to December 2022, by what amount did Matterport's subscribers grow?\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"49-fold\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_8_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time? Only answer if you know the answer with certainty.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then write a brief numerical answer inside <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then, in <scratchpad> tags, pull the most relevant quote from the document and consider whether it answers the user's question or whether it lacks sufficient detail. Then write a brief numerical answer in <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,706 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.1: Chaining Prompts\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(messages, system_prompt=''):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=messages,\n",
|
||||
" system=system_prompt\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"The saying goes, \"Writing is rewriting.\" It turns out, **Claude can often improve the accuracy of its response when asked to do so**!\n",
|
||||
"\n",
|
||||
"There are many ways to prompt Claude to \"think again\". The ways that feel natural to ask a human to double check their work will also generally work for Claude. (Check out our [prompt chaining documentation](https://docs.anthropic.com/claude/docs/chain-prompts) for further examples of when and how to use prompt chaining.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In this example, we ask Claude to come up with ten words... but one or more of them isn't a real word."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Asking Claude to make its answer more accurate** fixes the error! \n",
|
||||
"\n",
|
||||
"Below, we've pulled down Claude's incorrect response from above and added another turn to the conversation asking Claude to fix its previous answer."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"But is Claude revising its answer just because we told it to? What if we start off with a correct answer already? Will Claude lose its confidence? Here, we've placed a correct response in the place of `first_response` and asked it to double check again."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You may notice that if you generate a respnse from the above block a few times, Claude leaves the words as is most of the time, but still occasionally changes the words even though they're all already correct. What can we do to mitigate this? Per Chapter 8, we can give Claude an out! Let's try this one more time."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words. If all the words are real words, return the original list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Try generating responses from the above code a few times to see that Claude is much better at sticking to its guns now.\n",
|
||||
"\n",
|
||||
"You can also use prompt chaining to **ask Claude to make its responses better**. Below, we asked Claude to first write a story, and then improve the story it wrote. Your personal tastes may vary, but many might agree that Claude's second version is better.\n",
|
||||
"\n",
|
||||
"First, let's generate Claude's first version of the story."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Write a three-sentence short story about a girl who likes to run.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's have Claude improve on its first draft."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Make the story better.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This form of substitution is very powerful. We've been using substitution placeholders to pass in lists, words, Claude's former responses, and so on. You can also **use substitution to do what we call \"function calling,\" which is asking Claude to perform some function, and then taking the results of that function and asking Claude to do even more afterward with the results**. It works like any other substitution. More on this in the next appendix.\n",
|
||||
"\n",
|
||||
"Below is one more example of taking the results of one call to Claude and plugging it into another, longer call. Let's start with the first prompt (which includes prefilling Claude's response this time)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"\"\"Find all names from the below text:\n",
|
||||
"\n",
|
||||
"\"Hey, Jesse. It's me, Erin. I'm calling about the party that Joey is throwing tomorrow. Keisha said she would come and I think Mel will be there too.\"\"\"\n",
|
||||
"\n",
|
||||
"prefill = \"<names>\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's pass this list of names into another prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Alphabetize the list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill + \"\\n\" + first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that you've learned about prompt chaining, head over to Appendix 10.2 to learn how to implement function calling using prompt chaining."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words. If all the words are real words, return the original list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Write a three-sentence short story about a girl who likes to run.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Make the story better.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"\"\"Find all names from the below text:\n",
|
||||
"\n",
|
||||
"\"Hey, Jesse. It's me, Erin. I'm calling about the party that Joey is throwing tomorrow. Keisha said she would come and I think Mel will be there too.\"\"\"\n",
|
||||
"\n",
|
||||
"prefill = \"<names>\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Alphabetize the list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill + \"\\n\" + first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,800 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.2: Tool Use\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install anthropic --quiet\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"# Override the MODEL_NAME variable in the IPython store to use Sonnet instead of the Haiku model\n",
|
||||
"MODEL_NAME='anthropic.claude-3-sonnet-20240229-v1:0'\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)\n",
|
||||
"\n",
|
||||
"# Rewrittten to call Claude 3 Sonnet, which is generally better at tool use, and include stop_sequences\n",
|
||||
"def get_completion(messages, system_prompt=\"\", prefill=\"\",stop_sequences=None):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=messages,\n",
|
||||
" system=system_prompt,\n",
|
||||
" stop_sequences=stop_sequences\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"While it might seem conceptually complex at first, tool use, a.k.a. function calling, is actually quite simple! You already know all the skills necessary to implement tool use, which is really just a combination of substitution and prompt chaining.\n",
|
||||
"\n",
|
||||
"In previous substitution exercises, we substituted text into prompts. With tool use, we substitute tool or function results into prompts. Claude can't literally call or access tools and functions. Instead, we have Claude:\n",
|
||||
"1. Output the tool name and arguments it wants to call\n",
|
||||
"2. Halt any further response generation while the tool is called\n",
|
||||
"3. Then we reprompt with the appended tool results"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Function calling is useful because it expands Claude's capabilities and enables Claude to handle much more complex, multi-step tasks.\n",
|
||||
"Some examples of functions you can give Claude:\n",
|
||||
"- Calculator\n",
|
||||
"- Word counter\n",
|
||||
"- SQL database querying and data retrieval\n",
|
||||
"- Weather API"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can get Claude to do tool use by combining these two elements:\n",
|
||||
"\n",
|
||||
"1. A system prompt, in which we give Claude an explanation of the concept of tool use as well as a detailed descriptive list of the tools it has access to\n",
|
||||
"2. The control logic with which to orchestrate and execute Claude's tool use requests"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Tool use roadmap\n",
|
||||
"\n",
|
||||
"*This lesson teaches our current tool use format. However, we will be updating and improving tool use functionality in the near future, including:*\n",
|
||||
"* *A more streamlined format for function definitions and calls*\n",
|
||||
"* *More robust error handilgj and edge case coverage*\n",
|
||||
"* *Tighter integration with the rest of our API*\n",
|
||||
"* *Better reliability and performance, especially for more complex tool use tasks*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"To enable tool use in Claude, we start with the system prompt. In this special tool use system prompt, wet tell Claude:\n",
|
||||
"* The basic premise of tool use and what it entails\n",
|
||||
"* How Claude can call and use the tools it's been given\n",
|
||||
"* A detailed list of tools it has access to in this specific scenario \n",
|
||||
"\n",
|
||||
"Here's the first part of the system prompt, explaining tool use to Claude. This part of the system prompt is generalizable across all instances of prompting Claude for tool use. The tool calling structure we're giving Claude (`<function_calls> [...] </function_calls>`) is a structure Claude has been specifically trained to use, so we recommend that you stick with this."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_general_explanation = \"\"\"You have access to a set of functions you can use to answer the user's question. This includes access to a\n",
|
||||
"sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external\n",
|
||||
"resources, except by invoking the below functions.\n",
|
||||
"\n",
|
||||
"You can invoke one or more functions by writing a \"<function_calls>\" block like the following as part of your\n",
|
||||
"reply to the user:\n",
|
||||
"<function_calls>\n",
|
||||
"<invoke name=\"$FUNCTION_NAME\">\n",
|
||||
"<antml:parameter name=\"$PARAMETER_NAME\">$PARAMETER_VALUE</parameter>\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"<nvoke name=\"$FUNCTION_NAME2\">\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"</function_calls>\n",
|
||||
"\n",
|
||||
"String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that\n",
|
||||
"spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular\n",
|
||||
"expressions.\n",
|
||||
"\n",
|
||||
"The output and/or any errors will appear in a subsequent \"<function_results>\" block, and remain there as part of\n",
|
||||
"your reply to the user.\n",
|
||||
"You may then continue composing the rest of your reply to the user, respond to any errors, or make further function\n",
|
||||
"calls as appropriate.\n",
|
||||
"If a \"<function_results>\" does NOT appear after your function calls, then they are likely malformatted and not\n",
|
||||
"recognized as a call.\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's the second part of the system prompt, which defines the exact tools Claude has access to in this specific situation. In this example, we will be giving Claude a calculator tool, which takes three parameters: two operands and an operator. \n",
|
||||
"\n",
|
||||
"Then we combine the two parts of the system prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools = \"\"\"Here are the functions available in JSONSchema format:\n",
|
||||
"<tools>\n",
|
||||
"<tool_description>\n",
|
||||
"<tool_name>calculator</tool_name>\n",
|
||||
"<description>\n",
|
||||
"Calculator function for doing basic arithmetic.\n",
|
||||
"Supports addition, subtraction, multiplication\n",
|
||||
"</description>\n",
|
||||
"<parameters>\n",
|
||||
"<parameter>\n",
|
||||
"<name>first_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>First operand (before the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>second_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>Second operand (after the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>operator</name>\n",
|
||||
"<type>str</type>\n",
|
||||
"<description>The operation to perform. Must be either +, -, *, or /</description>\n",
|
||||
"</parameter>\n",
|
||||
"</parameters>\n",
|
||||
"</tool_description>\n",
|
||||
"</tools>\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now we can give Claude a question that requires use of the `calculator` tool. We will use `<function_calls\\>` in `stop_sequences` to detect if and when Claude calls the function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Multiply 1,984,135 by 9,343,116\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, we can extract out the parameters from Claude's function call and actually run the function on Claude's behalf.\n",
|
||||
"\n",
|
||||
"First we'll define the function's code."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def do_pairwise_arithmetic(num1, num2, operation):\n",
|
||||
" if operation == '+':\n",
|
||||
" return num1 + num2\n",
|
||||
" elif operation == \"-\":\n",
|
||||
" return num1 - num2\n",
|
||||
" elif operation == \"*\":\n",
|
||||
" return num1 * num2\n",
|
||||
" elif operation == \"/\":\n",
|
||||
" return num1 / num2\n",
|
||||
" else:\n",
|
||||
" return \"Error: Operation not supported.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then we'll extract the parameters from Claude's function call response. If all the parameters exist, we run the calculator tool."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def find_parameter(message, parameter_name):\n",
|
||||
" parameter_start_string = f\"name=\\\"{parameter_name}\\\">\"\n",
|
||||
" start = message.index(parameter_start_string)\n",
|
||||
" if start == -1:\n",
|
||||
" return None\n",
|
||||
" if start > 0:\n",
|
||||
" start = start + len(parameter_start_string)\n",
|
||||
" end = start\n",
|
||||
" while message[end] != \"<\":\n",
|
||||
" end += 1\n",
|
||||
" return message[start:end]\n",
|
||||
"\n",
|
||||
"first_operand = find_parameter(function_calling_response, \"first_operand\")\n",
|
||||
"second_operand = find_parameter(function_calling_response, \"second_operand\")\n",
|
||||
"operator = find_parameter(function_calling_response, \"operator\")\n",
|
||||
"\n",
|
||||
"if first_operand and second_operand and operator:\n",
|
||||
" result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)\n",
|
||||
" print(\"---------------- RESULT ----------------\")\n",
|
||||
" print(f\"{result:,}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that we have a result, we have to properly format that result so that when we pass it back to Claude, Claude understands what tool that result is in relation to. There is a set format for this that Claude has been trained to recognize:\n",
|
||||
"```\n",
|
||||
"<function_results>\n",
|
||||
"<result>\n",
|
||||
"<tool_name>{TOOL_NAME}</tool_name>\n",
|
||||
"<stdout>\n",
|
||||
"{TOOL_RESULT}\n",
|
||||
"</stdout>\n",
|
||||
"</result>\n",
|
||||
"</function_results>\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Run the cell below to format the above tool result into this structure."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_successful_function_run_injection_prompt(invoke_results):\n",
|
||||
" constructed_prompt = (\n",
|
||||
" \"<function_results>\\n\"\n",
|
||||
" + '\\n'.join(\n",
|
||||
" f\"<result>\\n<tool_name>{res['tool_name']}</tool_name>\\n<stdout>\\n{res['tool_result']}\\n</stdout>\\n</result>\"\n",
|
||||
" for res in invoke_results\n",
|
||||
" ) + \"\\n</function_results>\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return constructed_prompt\n",
|
||||
"\n",
|
||||
"formatted_results = [{\n",
|
||||
" 'tool_name': 'do_pairwise_arithmetic',\n",
|
||||
" 'tool_result': result\n",
|
||||
"}]\n",
|
||||
"function_results = construct_successful_function_run_injection_prompt(formatted_results)\n",
|
||||
"print(function_results)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now all we have to do is send this result back to Claude by appending the result to the same message chain as before, and we're good!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"full_first_response = function_calling_response + \"</function_calls>\"\n",
|
||||
"\n",
|
||||
"# Construct the full conversation\n",
|
||||
"messages = [multiplication_message,\n",
|
||||
"{\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": full_first_response\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": function_results\n",
|
||||
"}]\n",
|
||||
" \n",
|
||||
"# Print Claude's response\n",
|
||||
"final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(\"------------- FINAL RESULT -------------\")\n",
|
||||
"print(final_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Congratulations on running an entire tool use chain end to end!\n",
|
||||
"\n",
|
||||
"Now what if we give Claude a question that doesn't that doesn't require using the given tool at all?"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"non_multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Tell me the capital of France.\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Success! As you can see, Claude knew not to call the function when it wasn't needed.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 10.2.1 - SQL](#exercise-1021---SQL)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 10.2.1 - SQL\n",
|
||||
"In this exercise, you'll be writing a tool use prompt for querying and writing to the world's smallest \"database\". Here's the initialized database, which is really just a dictionary."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"db = {\n",
|
||||
" \"users\": [\n",
|
||||
" {\"id\": 1, \"name\": \"Alice\", \"email\": \"alice@example.com\"},\n",
|
||||
" {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@example.com\"},\n",
|
||||
" {\"id\": 3, \"name\": \"Charlie\", \"email\": \"charlie@example.com\"}\n",
|
||||
" ],\n",
|
||||
" \"products\": [\n",
|
||||
" {\"id\": 1, \"name\": \"Widget\", \"price\": 9.99},\n",
|
||||
" {\"id\": 2, \"name\": \"Gadget\", \"price\": 14.99},\n",
|
||||
" {\"id\": 3, \"name\": \"Doohickey\", \"price\": 19.99}\n",
|
||||
" ]\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"And here is the code for the functions that write to and from the database."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_user(user_id):\n",
|
||||
" for user in db[\"users\"]:\n",
|
||||
" if user[\"id\"] == user_id:\n",
|
||||
" return user\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def get_product(product_id):\n",
|
||||
" for product in db[\"products\"]:\n",
|
||||
" if product[\"id\"] == product_id:\n",
|
||||
" return product\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def add_user(name, email):\n",
|
||||
" user_id = len(db[\"users\"]) + 1\n",
|
||||
" user = {\"id\": user_id, \"name\": name, \"email\": email}\n",
|
||||
" db[\"users\"].append(user)\n",
|
||||
" return user\n",
|
||||
"\n",
|
||||
"def add_product(name, price):\n",
|
||||
" product_id = len(db[\"products\"]) + 1\n",
|
||||
" product = {\"id\": product_id, \"name\": name, \"price\": price}\n",
|
||||
" db[\"products\"].append(product)\n",
|
||||
" return product"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To solve the exercise, start by defining a system prompt like `system_prompt_tools_specific_tools` above. Make sure to include the name and description of each tool, along with the name and type and description of each parameter for each function. We've given you some starting scaffolding below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools_sql = \"\"\"\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools_sql"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When you're ready, you can try out your tool definition system prompt on the examples below. Just run the below cell!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"examples = [\n",
|
||||
" \"Add a user to the database named Deborah.\",\n",
|
||||
" \"Add a product to the database named Thingo\",\n",
|
||||
" \"Tell me the name of User 2\",\n",
|
||||
" \"Tell me the name of Product 3\"\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"for example in examples:\n",
|
||||
" message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": example\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" # Get & print Claude's response\n",
|
||||
" function_calling_response = get_completion([message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
" print(example, \"\\n----------\\n\\n\", function_calling_response, \"\\n*********\\n*********\\n*********\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you did it right, the function calling messages should call the `add_user`, `add_product`, `get_user`, and `get_product` functions correctly.\n",
|
||||
"\n",
|
||||
"For extra credit, add some code cells and write parameter-parsing code. Then call the functions with the parameters Claude gives you to see the state of the \"database\" after the call."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want to see a possible solution, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_10_2_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"Congratulations on learning tool use and function calling! Head over to the last appendix section if you would like to learn more about search & RAG."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_general_explanation = \"\"\"You have access to a set of functions you can use to answer the user's question. This includes access to a\n",
|
||||
"sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external\n",
|
||||
"resources, except by invoking the below functions.\n",
|
||||
"\n",
|
||||
"You can invoke one or more functions by writing a \"<function_calls>\" block like the following as part of your\n",
|
||||
"reply to the user:\n",
|
||||
"<function_calls>\n",
|
||||
"<invoke name=\"$FUNCTION_NAME\">\n",
|
||||
"<antml:parameter name=\"$PARAMETER_NAME\">$PARAMETER_VALUE</parameter>\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"<nvoke name=\"$FUNCTION_NAME2\">\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"</function_calls>\n",
|
||||
"\n",
|
||||
"String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that\n",
|
||||
"spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular\n",
|
||||
"expressions.\n",
|
||||
"\n",
|
||||
"The output and/or any errors will appear in a subsequent \"<function_results>\" block, and remain there as part of\n",
|
||||
"your reply to the user.\n",
|
||||
"You may then continue composing the rest of your reply to the user, respond to any errors, or make further function\n",
|
||||
"calls as appropriate.\n",
|
||||
"If a \"<function_results>\" does NOT appear after your function calls, then they are likely malformatted and not\n",
|
||||
"recognized as a call.\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools = \"\"\"Here are the functions available in JSONSchema format:\n",
|
||||
"<tools>\n",
|
||||
"<tool_description>\n",
|
||||
"<tool_name>calculator</tool_name>\n",
|
||||
"<description>\n",
|
||||
"Calculator function for doing basic arithmetic.\n",
|
||||
"Supports addition, subtraction, multiplication\n",
|
||||
"</description>\n",
|
||||
"<parameters>\n",
|
||||
"<parameter>\n",
|
||||
"<name>first_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>First operand (before the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>second_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>Second operand (after the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>operator</name>\n",
|
||||
"<type>str</type>\n",
|
||||
"<description>The operation to perform. Must be either +, -, *, or /</description>\n",
|
||||
"</parameter>\n",
|
||||
"</parameters>\n",
|
||||
"</tool_description>\n",
|
||||
"</tools>\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Multiply 1,984,135 by 9,343,116\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def do_pairwise_arithmetic(num1, num2, operation):\n",
|
||||
" if operation == '+':\n",
|
||||
" return num1 + num2\n",
|
||||
" elif operation == \"-\":\n",
|
||||
" return num1 - num2\n",
|
||||
" elif operation == \"*\":\n",
|
||||
" return num1 * num2\n",
|
||||
" elif operation == \"/\":\n",
|
||||
" return num1 / num2\n",
|
||||
" else:\n",
|
||||
" return \"Error: Operation not supported.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def find_parameter(message, parameter_name):\n",
|
||||
" parameter_start_string = f\"name=\\\"{parameter_name}\\\">\"\n",
|
||||
" start = message.index(parameter_start_string)\n",
|
||||
" if start == -1:\n",
|
||||
" return None\n",
|
||||
" if start > 0:\n",
|
||||
" start = start + len(parameter_start_string)\n",
|
||||
" end = start\n",
|
||||
" while message[end] != \"<\":\n",
|
||||
" end += 1\n",
|
||||
" return message[start:end]\n",
|
||||
"\n",
|
||||
"first_operand = find_parameter(function_calling_response, \"first_operand\")\n",
|
||||
"second_operand = find_parameter(function_calling_response, \"second_operand\")\n",
|
||||
"operator = find_parameter(function_calling_response, \"operator\")\n",
|
||||
"\n",
|
||||
"if first_operand and second_operand and operator:\n",
|
||||
" result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)\n",
|
||||
" print(\"---------------- RESULT ----------------\")\n",
|
||||
" print(f\"{result:,}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_successful_function_run_injection_prompt(invoke_results):\n",
|
||||
" constructed_prompt = (\n",
|
||||
" \"<function_results>\\n\"\n",
|
||||
" + '\\n'.join(\n",
|
||||
" f\"<result>\\n<tool_name>{res['tool_name']}</tool_name>\\n<stdout>\\n{res['tool_result']}\\n</stdout>\\n</result>\"\n",
|
||||
" for res in invoke_results\n",
|
||||
" ) + \"\\n</function_results>\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return constructed_prompt\n",
|
||||
"\n",
|
||||
"formatted_results = [{\n",
|
||||
" 'tool_name': 'do_pairwise_arithmetic',\n",
|
||||
" 'tool_result': result\n",
|
||||
"}]\n",
|
||||
"function_results = construct_successful_function_run_injection_prompt(formatted_results)\n",
|
||||
"print(function_results)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"full_first_response = function_calling_response + \"</function_calls>\"\n",
|
||||
"\n",
|
||||
"# Construct the full conversation\n",
|
||||
"messages = [multiplication_message,\n",
|
||||
"{\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": full_first_response\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": function_results\n",
|
||||
"}]\n",
|
||||
" \n",
|
||||
"# Print Claude's response\n",
|
||||
"final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(\"------------- FINAL RESULT -------------\")\n",
|
||||
"print(final_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"non_multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Tell me the capital of France.\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,353 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Evaluating AI Models: Code, Human, and Model-Based Grading\n",
|
||||
"\n",
|
||||
"In this notebook, we'll delve into a trio of widely-used techniques for assessing the effectiveness of AI models, like Claude v3:\n",
|
||||
"\n",
|
||||
"1. Code-based grading\n",
|
||||
"2. Human grading\n",
|
||||
"3. Model-based grading\n",
|
||||
"\n",
|
||||
"We'll illustrate each approach through examples and examine their respective advantages and limitations, when gauging AI performance."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Code-Based Grading Example: Sentiment Analysis\n",
|
||||
"\n",
|
||||
"In this example, we'll evaluate Claude's ability to classify the sentiment of movie reviews as positive or negative. We can use code to check if the model's output matches the expected sentiment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store the model name and AWS region for later use\n",
|
||||
"MODEL_NAME = \"anthropic.claude-3-haiku-20240307-v1:0\"\n",
|
||||
"AWS_REGION = \"us-west-2\"\n",
|
||||
"\n",
|
||||
"%store MODEL_NAME\n",
|
||||
"%store AWS_REGION"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Install the Anthropic package\n",
|
||||
"%pip install anthropic --quiet"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import the AnthropicBedrock class and create a client instance\n",
|
||||
"from anthropic import AnthropicBedrock\n",
|
||||
"\n",
|
||||
"client = AnthropicBedrock(aws_region=AWS_REGION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for sentiment analysis\n",
|
||||
"def build_input_prompt(review):\n",
|
||||
" user_content = f\"\"\"Classify the sentiment of the following movie review as either 'positive' or 'negative' provide only one of those two choices:\n",
|
||||
" <review>{review}</review>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"review\": \"This movie was amazing! The acting was superb and the plot kept me engaged from start to finish.\",\n",
|
||||
" \"golden_answer\": \"positive\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"review\": \"I was thoroughly disappointed by this film. The pacing was slow and the characters were one-dimensional.\",\n",
|
||||
" \"golden_answer\": \"negative\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Function to get completions from the model\n",
|
||||
"def get_completion(messages):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=messages\n",
|
||||
" )\n",
|
||||
" return message.content[0].text\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"review\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Print the outputs and golden answers\n",
|
||||
"for output, question in zip(outputs, eval):\n",
|
||||
" print(f\"Review: {question['review']}\\nGolden Answer: {question['golden_answer']}\\nOutput: {output}\\n\")\n",
|
||||
"\n",
|
||||
"# Function to grade the completions\n",
|
||||
"def grade_completion(output, golden_answer):\n",
|
||||
" return output.lower() == golden_answer.lower()\n",
|
||||
"\n",
|
||||
"# Grade the completions and print the accuracy\n",
|
||||
"grades = [grade_completion(output, item[\"golden_answer\"]) for output, item in zip(outputs, eval)]\n",
|
||||
"print(f\"Accuracy: {sum(grades) / len(grades) * 100}%\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Human Grading Example: Essay Scoring\n",
|
||||
"\n",
|
||||
"Some tasks, like scoring essays, are difficult to evaluate with code alone. In this case, we can provide guidelines for human graders to assess the model's output."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for essay generation\n",
|
||||
"def build_input_prompt(topic):\n",
|
||||
" user_content = f\"\"\"Write a short essay discussing the following topic:\n",
|
||||
" <topic>{topic}</topic>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"topic\": \"The importance of education in personal development and societal progress\",\n",
|
||||
" \"golden_answer\": \"A high-scoring essay should have a clear thesis, well-structured paragraphs, and persuasive examples discussing how education contributes to individual growth and broader societal advancement.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"topic\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Print the outputs and golden answers\n",
|
||||
"for output, item in zip(outputs, eval):\n",
|
||||
" print(f\"Topic: {item['topic']}\\n\\nGrading Rubric:\\n {item['golden_answer']}\\n\\nModel Output:\\n{output}\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Model-Based Grading Examples\n",
|
||||
"\n",
|
||||
"We can use Claude to grade its own outputs by providing the model's response and a grading rubric. This allows us to automate the evaluation of tasks that would typically require human judgment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example 1: Summarization\n",
|
||||
"\n",
|
||||
"In this example, we'll use Claude to assess the quality of a summary it generated. This can be useful when you need to evaluate the model's ability to capture key information from a longer text concisely and accurately. By providing a rubric that outlines the essential points that should be covered, we can automate the grading process and quickly assess the model's performance on summarization tasks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for summarization\n",
|
||||
"def build_input_prompt(text):\n",
|
||||
" user_content = f\"\"\"Please summarize the main points of the following text:\n",
|
||||
" <text>{text}</text>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Function to build the grader prompt for assessing summary quality\n",
|
||||
"def build_grader_prompt(output, rubric):\n",
|
||||
" user_content = f\"\"\"Assess the quality of the following summary based on this rubric:\n",
|
||||
" <rubric>{rubric}</rubric>\n",
|
||||
" <summary>{output}</summary>\n",
|
||||
" Provide a score from 1-5, where 1 is poor and 5 is excellent.\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"text\": \"The Magna Carta, signed in 1215, was a pivotal document in English history. It limited the powers of the monarchy and established the principle that everyone, including the king, was subject to the law. This laid the foundation for constitutional governance and the rule of law in England and influenced legal systems worldwide.\",\n",
|
||||
" \"golden_answer\": \"A high-quality summary should concisely capture the key points: 1) The Magna Carta's significance in English history, 2) Its role in limiting monarchical power, 3) Establishing the principle of rule of law, and 4) Its influence on legal systems around the world.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"text\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Grade the completions\n",
|
||||
"grades = [get_completion(build_grader_prompt(output, item[\"golden_answer\"])) for output, item in zip(outputs, eval)]\n",
|
||||
"\n",
|
||||
"# Print the summary quality score\n",
|
||||
"print(f\"Summary quality score: {grades[0]}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example 2: Fact-Checking\n",
|
||||
"\n",
|
||||
"In this example, we'll use Claude to fact-check a claim and then assess the accuracy of its fact-checking. This can be useful when you need to evaluate the model's ability to distinguish between accurate and inaccurate information. By providing a rubric that outlines the key points that should be covered in a correct fact-check, we can automate the grading process and quickly assess the model's performance on fact-checking tasks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for fact-checking\n",
|
||||
"def build_input_prompt(claim):\n",
|
||||
" user_content = f\"\"\"Determine if the following claim is true or false:\n",
|
||||
" <claim>{claim}</claim>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Function to build the grader prompt for assessing fact-check accuracy\n",
|
||||
"def build_grader_prompt(output, rubric):\n",
|
||||
" user_content = f\"\"\"Based on the following rubric, assess whether the fact-check is correct:\n",
|
||||
" <rubric>{rubric}</rubric>\n",
|
||||
" <fact-check>{output}</fact-check>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"claim\": \"The Great Wall of China is visible from space.\",\n",
|
||||
" \"golden_answer\": \"A correct fact-check should state that this claim is false. While the Great Wall is an impressive structure, it is not visible from space with the naked eye.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"claim\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"grades = []\n",
|
||||
"for output, item in zip(outputs, eval):\n",
|
||||
" # Print the claim, fact-check, and rubric\n",
|
||||
" print(f\"Claim: {item['claim']}\\n\")\n",
|
||||
" print(f\"Fact-check: {output}]\\n\")\n",
|
||||
" print(f\"Rubric: {item['golden_answer']}\\n\")\n",
|
||||
" \n",
|
||||
" # Grade the fact-check\n",
|
||||
" grader_prompt = build_grader_prompt(output, item[\"golden_answer\"])\n",
|
||||
" grade = get_completion(grader_prompt)\n",
|
||||
" grades.append(\"correct\" in grade.lower())\n",
|
||||
"\n",
|
||||
"# Print the fact-checking accuracy\n",
|
||||
"accuracy = sum(grades) / len(grades)\n",
|
||||
"print(f\"Fact-checking accuracy: {accuracy * 100}%\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example 3: Tone Analysis\n",
|
||||
"\n",
|
||||
"In this example, we'll use Claude to analyze the tone of a given text and then assess the accuracy of its analysis. This can be useful when you need to evaluate the model's ability to identify and interpret the emotional content and attitudes expressed in a piece of text. By providing a rubric that outlines the key aspects of tone that should be identified, we can automate the grading process and quickly assess the model's performance on tone analysis tasks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for tone analysis\n",
|
||||
"def build_input_prompt(text):\n",
|
||||
" user_content = f\"\"\"Analyze the tone of the following text:\n",
|
||||
" <text>{text}</text>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Function to build the grader prompt for assessing tone analysis accuracy\n",
|
||||
"def build_grader_prompt(output, rubric):\n",
|
||||
" user_content = f\"\"\"Assess the accuracy of the following tone analysis based on this rubric:\n",
|
||||
" <rubric>{rubric}</rubric>\n",
|
||||
" <tone-analysis>{output}</tone-analysis>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"text\": \"I can't believe they canceled the event at the last minute. This is completely unacceptable and unprofessional!\",\n",
|
||||
" \"golden_answer\": \"The tone analysis should identify the text as expressing frustration, anger, and disappointment. Key words like 'can't believe', 'last minute', 'unacceptable', and 'unprofessional' indicate strong negative emotions.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"text\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Grade the completions\n",
|
||||
"grades = [get_completion(build_grader_prompt(output, item[\"golden_answer\"])) for output, item in zip(outputs, eval)]\n",
|
||||
"\n",
|
||||
"# Print the tone analysis quality\n",
|
||||
"print(f\"Tone analysis quality: {grades[0]}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"These examples demonstrate how code-based, human, and model-based grading can be used to evaluate AI models like Claude on various tasks. The choice of evaluation method depends on the nature of the task and the resources available. Model-based grading offers a promising approach for automating the assessment of complex tasks that would otherwise require human judgment."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "conda_pytorch_p310",
|
||||
"language": "python",
|
||||
"name": "conda_pytorch_p310"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.3: Search & Retrieval\n",
|
||||
"\n",
|
||||
"Did you know you can use Claude to **search through Wikipedia for you**? Claude can find and retrieve articles, at which point you can also use Claude to summarize and synthesize them, write novel content from what it found, and much more. And not just Wikipedia! You can also search over your own docs, whether stored as plain text or embedded in a vector datastore.\n",
|
||||
"\n",
|
||||
"See our [RAG cookbook examples](https://github.com/anthropics/anthropic-cookbook/blob/main/third_party/Wikipedia/wikipedia-search-cookbook.ipynb) to learn how to supplement Claude's knowledge and improve the accuracy and relevance of Claude's responses with data retrieved from vector databases, Wikipedia, the internet, and more. There, you can also learn about how to use certain [embeddings](https://docs.anthropic.com/claude/docs/embeddings) and vector database tools.\n",
|
||||
"\n",
|
||||
"If you are interested in learning about advanced RAG architectures using Claude, check out our [Claude 3 technical presentation slides on RAG architectures](https://docs.google.com/presentation/d/1zxkSI7lLUBrZycA-_znwqu8DDyVhHLkQGScvzaZrUns/edit#slide=id.g2c736259dac_63_782)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,195 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Tutorial How-To\n",
|
||||
"\n",
|
||||
"This tutorial requires this initial notebook to be run first so that the requirements and environment variables are stored for all notebooks in the workshop."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## How to get started\n",
|
||||
"\n",
|
||||
"1. Clone this repository to your local machine.\n",
|
||||
"\n",
|
||||
"2. Install the required dependencies by running the following command:\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> ⚠️ **Please ignore error messages related to pip's dependency resolver.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install -qU pip\n",
|
||||
"%pip install -qr ../requirements.txt"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"3. Restart the kernel after installing dependencies"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# restart kernel\n",
|
||||
"from IPython.core.display import HTML\n",
|
||||
"HTML(\"<script>Jupyter.notebook.kernel.restart()</script>\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"4. Run the notebook cells in order, following the instructions provided."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Usage Notes & Tips 💡\n",
|
||||
"\n",
|
||||
"- This course uses Claude 3 Haiku with temperature 0. We will talk more about temperature later in the course. For now, it's enough to understand that these settings yield more deterministic results. All prompt engineering techniques in this course also apply to previous generation legacy Claude models such as Claude 2 and Claude Instant 1.2.\n",
|
||||
"\n",
|
||||
"- You can use `Shift + Enter` to execute the cell and move to the next one.\n",
|
||||
"\n",
|
||||
"- When you reach the bottom of a tutorial page, navigate to the next numbered file in the folder, or to the next numbered folder if you're finished with the content within that chapter file.\n",
|
||||
"\n",
|
||||
"### The Anthropic SDK & the Messages API\n",
|
||||
"We will be using the [Anthropic python SDK](https://docs.anthropic.com/claude/reference/client-sdks) and the [Messages API](https://docs.anthropic.com/claude/reference/messages_post) throughout this tutorial. \n",
|
||||
"\n",
|
||||
"Below is an example of what running a prompt will look like in this tutorial. First, we create `get_completion`, which is a helper function that sends a prompt to Claude and returns Claude's generated response. Run that cell now."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"First, we set and store the model name and region."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import boto3\n",
|
||||
"session = boto3.Session() # create a boto3 session to dynamically get and set the region name\n",
|
||||
"AWS_REGION = session.region_name\n",
|
||||
"print(\"AWS Region:\", AWS_REGION)\n",
|
||||
"MODEL_NAME = \"anthropic.claude-3-haiku-20240307-v1:0\"\n",
|
||||
"\n",
|
||||
"%store MODEL_NAME\n",
|
||||
"%store AWS_REGION"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then, we create `get_completion`, which is a helper function that sends a prompt to Claude and returns Claude's generated response. Run that cell now."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"bedrock = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": ''\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = bedrock.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now we will write out an example prompt for Claude and print Claude's output by running our `get_completion` helper function. Running the cell below will print out a response from Claude beneath it.\n",
|
||||
"\n",
|
||||
"Feel free to play around with the prompt string to elicit different responses from Claude."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"prompt = \"Hello, Claude!\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"print(get_completion(prompt))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `MODEL_NAME` and `AWS_REGION` variables defined earlier will be used throughout the tutorial. Just make sure to run the cells for each tutorial page from top to bottom."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "py310",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,503 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 1: Basic Prompt Structure\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt,system=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Anthropic offers two APIs, the legacy [Text Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-text-completion.html) and the current [Messages API](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html). For this tutorial, we will be exclusively using the Messages API.\n",
|
||||
"\n",
|
||||
"At minimum, a call to Claude using the Messages API requires the following parameters:\n",
|
||||
"- `model`: the [API model name](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns) of the model that you intend to call\n",
|
||||
"\n",
|
||||
"- `max_tokens`: the maximum number of tokens to generate before stopping. Note that Claude may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. Furthermore, this is a *hard* stop, meaning that it may cause Claude to stop generating mid-word or mid-sentence.\n",
|
||||
"\n",
|
||||
"- `messages`: an array of input messages. Our models are trained to operate on alternating `user` and `assistant` conversational turns. When creating a new `Message`, you specify the prior conversational turns with the messages parameter, and the model then generates the next `Message` in the conversation.\n",
|
||||
" - Each input message must be an object with a `role` and `content`. You can specify a single `user`-role message, or you can include multiple `user` and `assistant` messages (they must alternate, if so). The first message must always use the user `role`.\n",
|
||||
"\n",
|
||||
"There are also optional parameters, such as:\n",
|
||||
"- `system`: the system prompt - more on this below.\n",
|
||||
" \n",
|
||||
"- `temperature`: the degree of variability in Claude's response. For these lessons and exercises, we have set `temperature` to 0.\n",
|
||||
"\n",
|
||||
"For a complete list of all API parameters, visit our [API documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-claude.html)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Let's take a look at how Claude responds to some correctly-formatted prompts. For each of the following cells, run the cell (`shift+enter`), and Claude's response will appear below the block."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hi Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Can you tell me the color of the ocean?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"What year was Celine Dion born in?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's take a look at some prompts that do not include the correct Messages API formatting. For these malformatted prompts, the Messages API returns an error.\n",
|
||||
"\n",
|
||||
"First, we have an example of a Messages API call that lacks `role` and `content` fields in the `messages` array."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> ⚠️ **Warning:** Due to the incorrect formatting of the messages parameter in the prompt, the following cell will return an error. This is expected behavior."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"Hi Claude, how are you?\"}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": ''\n",
|
||||
" }\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's a prompt that fails to alternate between the `user` and `assistant` roles."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"> ⚠️ **Warning:** Due to the lack of alternation between `user` and `assistant` roles, Claude will return an error message. This is expected behavior."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [\n",
|
||||
" {\"role\": \"user\", \"content\": \"What year was Celine Dion born in?\"},\n",
|
||||
" {\"role\": \"user\", \"content\": \"Also, can you tell me some other facts about her?\"}\n",
|
||||
" ],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": ''\n",
|
||||
" }\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`user` and `assistant` messages **MUST alternate**, and messages **MUST start with a `user` turn**. You can have multiple `user` & `assistant` pairs in a prompt (as if simulating a multi-turn conversation). You can also put words into a terminal `assistant` message for Claude to continue from where you left off (more on that in later chapters).\n",
|
||||
"\n",
|
||||
"#### System Prompts\n",
|
||||
"\n",
|
||||
"You can also use **system prompts**. A system prompt is a way to **provide context, instructions, and guidelines to Claude** before presenting it with a question or task in the \"User\" turn. \n",
|
||||
"\n",
|
||||
"Structurally, system prompts exist separately from the list of `user` & `assistant` messages, and thus belong in a separate `system` parameter (take a look at the structure of the `get_completion` helper function in the [Setup](#setup) section of the notebook). \n",
|
||||
"\n",
|
||||
"Within this tutorial, wherever we might utilize a system prompt, we have provided you a `system` field in your completions function. Should you not want to use a system prompt, simply set the `SYSTEM_PROMPT` variable to an empty string."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### System Prompt Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Why is the sky blue?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why use a system prompt? A **well-written system prompt can improve Claude's performance** in a variety of ways, such as increasing Claude's ability to follow rules and instructions. For more information, visit our documentation on [how to use system prompts](https://docs.anthropic.com/claude/docs/how-to-use-system-prompts) with Claude.\n",
|
||||
"\n",
|
||||
"Now we'll dive into some exercises. If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 1.1 - Counting to Three](#exercise-11---counting-to-three)\n",
|
||||
"- [Exercise 1.2 - System Prompt](#exercise-12---system-prompt)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 1.1 - Counting to Three\n",
|
||||
"Using proper `user` / `assistant` formatting, edit the `PROMPT` below to get Claude to **count to three.** The output will also indicate whether your solution is correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" pattern = re.compile(r'^(?=.*1)(?=.*2)(?=.*3).*$', re.DOTALL)\n",
|
||||
" return bool(pattern.match(text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_1_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 1.2 - System Prompt\n",
|
||||
"\n",
|
||||
"Modify the `SYSTEM_PROMPT` to make Claude respond like it's a 3 year old child."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - this is the only field you should change\n",
|
||||
"SYSTEM_PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"How big is the sky?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(r\"giggles\", text) or re.search(r\"soo\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_1_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hi Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Can you tell me the color of the ocean?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"What year was Celine Dion born in?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"Hi Claude, how are you?\"}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": ''\n",
|
||||
" }\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [\n",
|
||||
" {\"role\": \"user\", \"content\": \"What year was Celine Dion born in?\"},\n",
|
||||
" {\"role\": \"user\", \"content\": \"Also, can you tell me some other facts about her?\"}\n",
|
||||
" ],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": ''\n",
|
||||
" }\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Why is the sky blue?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,403 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 2: Being Clear and Direct\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt,system=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Claude responds best to clear and direct instructions.**\n",
|
||||
"\n",
|
||||
"Think of Claude like any other human that is new to the job. **Claude has no context** on what to do aside from what you literally tell it. Just as when you instruct a human for the first time on a task, the more you explain exactly what you want in a straightforward manner to Claude, the better and more accurate Claude's response will be.\"\t\t\t\t\n",
|
||||
"\t\t\t\t\n",
|
||||
"When in doubt, follow the **Golden Rule of Clear Prompting**:\n",
|
||||
"- Show your prompt to a colleague or friend and have them follow the instructions themselves to see if they can produce the result you want. If they're confused, Claude's confused.\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Let's take a task like writing poetry. (Ignore any syllable mismatch - LLMs aren't great at counting syllables yet.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This haiku is nice enough, but users may want Claude to go directly into the poem without the \"Here is a haiku\" preamble.\n",
|
||||
"\n",
|
||||
"How do we achieve that? We **ask for it**!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots. Skip the preamble; go straight into the poem.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's another example. Let's ask Claude who's the best basketball player of all time. You can see below that while Claude lists a few names, **it doesn't respond with a definitive \"best\"**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Can we get Claude to make up its mind and decide on a best player? Yes! Just ask!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time? Yes, there are differing opinions, but if you absolutely had to pick one player, who would it be?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 2.1 - Spanish](#exercise-21---spanish)\n",
|
||||
"- [Exercise 2.2 - One Player Only](#exercise-22---one-player-only)\n",
|
||||
"- [Exercise 2.3 - Write a Story](#exercise-23---write-a-story)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.1 - Spanish\n",
|
||||
"Modify the `SYSTEM_PROMPT` to make Claude output its answer in Spanish."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - this is the only field you should chnage\n",
|
||||
"SYSTEM_PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hello Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return \"hola\" in text.lower()\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_2_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.2 - One Player Only\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` so that Claude doesn't equivocate at all and responds with **ONLY** the name of one specific player, with **no other words or punctuation**. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return text == \"Michael Jordan\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_2_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.3 - Write a Story\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` so that Claude responds with as long a response as you can muster. If your answer is **over 800 words**, Claude's response will be graded as correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" trimmed = text.strip()\n",
|
||||
" words = len(trimmed.split())\n",
|
||||
" return words >= 800\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_2_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots. Skip the preamble; go straight into the poem.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time? Yes, there are differing opinions, but if you absolutely had to pick one player, who would it be?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,335 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 3: Assigning Roles (Role Prompting)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt,system=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Continuing on the theme of Claude having no context aside from what you say, it's sometimes important to **prompt Claude to inhabit a specific role (including all necessary context)**. This is also known as role prompting. The more detail to the role context, the better.\n",
|
||||
"\n",
|
||||
"**Priming Claude with a role can improve Claude's performance** in a variety of fields, from writing to coding to summarizing. It's like how humans can sometimes be helped when told to \"think like a ______\". Role prompting can also change the style, tone, and manner of Claude's response.\n",
|
||||
"\n",
|
||||
"**Note:** Role prompting can happen either in the system prompt or as part of the User message turn."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In the example below, we see that without role prompting, Claude provides a **straightforward and non-stylized answer** when asked to give a single sentence perspective on skateboarding.\n",
|
||||
"\n",
|
||||
"However, when we prime Claude to inhabit the role of a cat, Claude's perspective changes, and thus **Claude's response tone, style, content adapts to the new role**. \n",
|
||||
"\n",
|
||||
"**Note:** A bonus technique you can use is to **provide Claude context on its intended audience**. Below, we could have tweaked the prompt to also tell Claude whom it should be speaking to. \"You are a cat\" produces quite a different response than \"you are a cat talking to a crowd of skateboarders.\n",
|
||||
"\n",
|
||||
"Here is the prompt without role prompting in the system prompt:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here is the same user question, except with role prompting."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a cat.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can use role prompting as a way to get Claude to emulate certain styles in writing, speak in a certain voice, or guide the complexity of its answers. **Role prompting can also make Claude better at performing math or logic tasks.**\n",
|
||||
"\n",
|
||||
"For example, in the example below, there is a definitive correct answer, which is yes. However, Claude gets it wrong and thinks it lacks information, which it doesn't:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, what if we **prime Claude to act as a logic bot**? How will that change Claude's answer? \n",
|
||||
"\n",
|
||||
"It turns out that with this new role assignment, Claude gets it right. (Although notably not for all the right reasons)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a logic bot designed to answer complex logic problems.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Note:** What you'll learn throughout this course is that there are **many prompt engineering techniques you can use to derive similar results**. Which techniques you use is up to you and your preference! We encourage you to **experiment to find your own prompt engineering style**.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 3.1 - Math Correction](#exercise-31---math-correction)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 3.1 - Math Correction\n",
|
||||
"In some instances, **Claude may struggle with mathematics**, even simple mathematics. Below, Claude incorrectly assesses the math problem as correctly solved, even though there's an obvious arithmetic mistake in the second step. Note that Claude actually catches the mistake when going through step-by-step, but doesn't jump to the conclusion that the overall solution is wrong.\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` and / or the `SYSTEM_PROMPT` to make Claude grade the solution as `incorrectly` solved, rather than correctly solved. \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - if you don't want to use a system prompt, you can leave this variable set to an empty string\n",
|
||||
"SYSTEM_PROMPT = \"\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this equation solved correctly below?\n",
|
||||
"\n",
|
||||
"2x - 3 = 9\n",
|
||||
"2x = 6\n",
|
||||
"x = 3\"\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" if \"incorrect\" in text or \"not correct\" in text.lower():\n",
|
||||
" return True\n",
|
||||
" else:\n",
|
||||
" return False\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_3_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a cat.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a logic bot designed to answer complex logic problems.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,562 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 4: Separating Data and Instructions\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt,system=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Oftentimes, we don't want to write full prompts, but instead want **prompt templates that can be modified later with additional input data before submitting to Claude**. This might come in handy if you want Claude to do the same thing every time, but the data that Claude uses for its task might be different each time. \n",
|
||||
"\n",
|
||||
"Luckily, we can do this pretty easily by **separating the fixed skeleton of the prompt from variable user input, then substituting the user input into the prompt** before sending the full prompt to Claude. \n",
|
||||
"\n",
|
||||
"Below, we'll walk step by step through how to write a substitutable prompt template, as well as how to substitute in user input."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In this first example, we're asking Claude to act as an animal noise generator. Notice that the full prompt submitted to Claude is just the `PROMPT_TEMPLATE` substituted with the input (in this case, \"Cow\"). Notice that the word \"Cow\" replaces the `ANIMAL` placeholder via an f-string when we print out the full prompt.\n",
|
||||
"\n",
|
||||
"**Note:** You don't have to call your placeholder variable anything in particular in practice. We called it `ANIMAL` in this example, but just as easily, we could have called it `CREATURE` or `A` (although it's generally good to have your variable names be specific and relevant so that your prompt template is easy to understand even without the substitution, just for user parseability). Just make sure that whatever you name your variable is what you use for the prompt template f-string."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cow\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why would we want to separate and substitute inputs like this? Well, **prompt templates simplify repetitive tasks**. Let's say you build a prompt structure that invites third party users to submit content to the prompt (in this case the animal whose sound they want to generate). These third party users don't have to write or even see the full prompt. All they have to do is fill in variables.\n",
|
||||
"\n",
|
||||
"We do this substitution here using variables and f-strings, but you can also do it with the format() method.\n",
|
||||
"\n",
|
||||
"**Note:** Prompt templates can have as many variables as desired!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When introducing substitution variables like this, it is very important to **make sure Claude knows where variables start and end** (vs. instructions or task descriptions). Let's look at an example where there is no separation between the instructions and the substitution variable.\n",
|
||||
"\n",
|
||||
"To our human eyes, it is very clear where the variable begins and ends in the prompt template below. However, in the fully substituted prompt, that delineation becomes unclear."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here, **Claude thinks \"Yo Claude\" is part of the email it's supposed to rewrite**! You can tell because it begins its rewrite with \"Dear Claude\". To the human eye, it's clear, particularly in the prompt template where the email begins and ends, but it becomes much less clear in the prompt after substitution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"How do we solve this? **Wrap the input in XML tags**! We did this below, and as you can see, there's no more \"Dear Claude\" in the output.\n",
|
||||
"\n",
|
||||
"[XML tags](https://docs.anthropic.com/claude/docs/use-xml-tags) are angle-bracket tags like `<tag></tag>`. They come in pairs and consist of an opening tag, such as `<tag>`, and a closing tag marked by a `/`, such as `</tag>`. XML tags are used to wrap around content, like this: `<tag>content</tag>`.\n",
|
||||
"\n",
|
||||
"**Note:** While Claude can recognize and work with a wide range of separators and delimeters, we recommend that you **use specifically XML tags as separators** for Claude, as Claude was trained specifically to recognize XML tags as a prompt organizing mechanism. Outside of function calling, **there are no special sauce XML tags that Claude has been trained on that you should use to maximally boost your performance**. We have purposefully made Claude very malleable and customizable this way."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's see another example of how XML tags can help us. \n",
|
||||
"\n",
|
||||
"In the following prompt, **Claude incorrectly interprets what part of the prompt is the instruction vs. the input**. It incorrectly considers `Each is about an animal, like rabbits` to be part of the list due to the formatting, when the user (the one filling out the `SENTENCES` variable) presumably did not want that."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"{SENTENCES}\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To fix this, we just need to **surround the user input sentences in XML tags**. This shows Claude where the input data begins and ends despite the misleading hyphen before `Each is about an animal, like rabbits.`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"<sentences>\n",
|
||||
"{SENTENCES}\n",
|
||||
"</sentences>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Note:** In the incorrect version of the \"Each is about an animal\" prompt, we had to include the hyphen to get Claude to respond incorrectly in the way we wanted to for this example. This is an important lesson about prompting: **small details matter**! It's always worth it to **scrub your prompts for typos and grammatical errors**. Claude is sensitive to patterns (in its early years, before finetuning, it was a raw text-prediction tool), and it's more likely to make mistakes when you make mistakes, smarter when you sound smart, sillier when you sound silly, and so on.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 4.1 - Haiku Topic](#exercise-41---haiku-topic)\n",
|
||||
"- [Exercise 4.2 - Dog Question with Typos](#exercise-42---dog-question-with-typos)\n",
|
||||
"- [Exercise 4.3 - Dog Question Part 2](#exercise-42---dog-question-part-2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.1 - Haiku Topic\n",
|
||||
"Modify the `PROMPT` so that it's a template that will take in a variable called `TOPIC` and output a haiku about the topic. This exercise is just meant to test your understanding of the variable templating structure with f-strings."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"TOPIC = \"Pigs\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"pigs\", text.lower()) and re.search(\"haiku\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_4_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.2 - Dog Question with Typos\n",
|
||||
"Fix the `PROMPT` by adding XML tags so that Claude produces the right answer. \n",
|
||||
"\n",
|
||||
"Try not to change anything else about the prompt. The messy and mistake-ridden writing is intentional, so you can see how Claude reacts to such mistakes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"QUESTION = \"ar cn brown?\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"brown\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_4_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.3 - Dog Question Part 2\n",
|
||||
"Fix the `PROMPT` **WITHOUT** adding XML tags. Instead, remove only one or two words from the prompt.\n",
|
||||
"\n",
|
||||
"Just as with the above exercises, try not to change anything else about the prompt. This will show you what kind of language Claude can parse and understand."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"QUESTION = \"ar cn brown?\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"brown\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_4_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cow\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"{SENTENCES}\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"<sentences>\n",
|
||||
"{SENTENCES}\n",
|
||||
"</sentences>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,528 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 5: Formatting Output and Speaking for Claude\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\":[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Claude can format its output in a wide variety of ways**. You just need to ask for it to do so!\n",
|
||||
"\n",
|
||||
"One of these ways is by using XML tags to separate out the response from any other superfluous text. You've already learned that you can use XML tags to make your prompt clearer and more parseable to Claude. It turns out, you can also ask Claude to **use XML tags to make its output clearer and more easily understandable** to humans."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Remember the 'poem preamble problem' we solved in Chapter 2 by asking Claude to skip the preamble entirely? It turns out we can also achieve a similar outcome by **telling Claude to put the poem in XML tags**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Rabbit\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why is this something we'd want to do? Well, having the output in **XML tags allows the end user to reliably get the poem and only the poem by writing a short program to extract the content between XML tags**.\n",
|
||||
"\n",
|
||||
"An extension of this technique is to **put the first XML tag in the `assistant` turn. When you put text in the `assistant` turn, you're basically telling Claude that Claude has already said something, and that it should continue from that point onward. This technique is called \"speaking for Claude\" or \"prefilling Claude's response.\"\n",
|
||||
"\n",
|
||||
"Below, we've done this with the first `<haiku>` XML tag. Notice how Claude continues directly from where we left off."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Claude also excels at using other output formatting styles, notably `JSON`. If you want to enforce JSON output (not deterministically, but close to it), you can also prefill Claude's response with the opening bracket, `{`}."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"{\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Below is an example of **multiple input variables in the same prompt AND output formatting specification, all done using XML tags**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ADJECTIVE = \"olde english\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response (now as an f-string with a variable)\n",
|
||||
"PREFILL = f\"<{ADJECTIVE}_email>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Bonus lesson\n",
|
||||
"\n",
|
||||
"If you are calling Claude through the API, you can pass the closing XML tag to the `stop_sequences` parameter to get Claude to stop sampling once it emits your desired tag. This can save money and time-to-last-token by eliminating Claude's concluding remarks after it's already given you the answer you care about.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 5.1 - Steph Curry GOAT](#exercise-51---steph-curry-goat)\n",
|
||||
"- [Exercise 5.2 - Two Haikus](#exercise-52---two-haikus)\n",
|
||||
"- [Exercise 5.3 - Two Haikus, Two Animals](#exercise-53---two-haikus-two-animals)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.1 - Steph Curry GOAT\n",
|
||||
"Forced to make a choice, Claude designates Michael Jordan as the best basketball player of all time. Can we get Claude to pick someone else?\n",
|
||||
"\n",
|
||||
"Change the `PREFILL` variable to **compell Claude to make a detailed argument that the best basketball player of all time is Stephen Curry**. Try not to change anything except `PREFILL` as that is the focus of this exercise."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Who is the best basketball player of all time? Please choose one specific player.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"Warrior\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_5_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.2 - Two Haikus\n",
|
||||
"Modify the `PROMPT` below using XML tags so that Claude writes two haikus about the animal instead of just one. It should be clear where one poem ends and the other begins."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"cats\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(\n",
|
||||
" (re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
|
||||
" and (text.count(\"\\n\") + 1) > 5\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_5_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.3 - Two Haikus, Two Animals\n",
|
||||
"Modify the `PROMPT` below so that **Claude produces two haikus about two different animals**. Use `{ANIMAL1}` as a stand-in for the first substitution, and `{ANIMAL2}` as a stand-in for the second substitution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"ANIMAL1 = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ANIMAL2 = \"Dog\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL1}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"tail\", text.lower()) and re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_5_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Rabbit\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"{\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ADJECTIVE = \"olde english\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response (now as an f-string with a variable)\n",
|
||||
"PREFILL = f\"<{ADJECTIVE}_email>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,514 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 6: Precognition (Thinking Step by Step)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\":[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"If someone woke you up and immediately started asking you several complicated questions that you had to respond to right away, how would you do? Probably not as good as if you were given time to **think through your answer first**. \n",
|
||||
"\n",
|
||||
"Guess what? Claude is the same way.\n",
|
||||
"\n",
|
||||
"**Giving Claude time to think step by step sometimes makes Claude more accurate**, particularly for complex tasks. However, **thinking only counts when it's out loud**. You cannot ask Claude to think but output only the answer - in this case, no thinking has actually occurred."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In the prompt below, it's clear to a human reader that the second sentence belies the first. But **Claude takes the word \"unrelated\" too literally**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this movie review sentiment positive or negative?\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since the year 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To improve Claude's response, let's **allow Claude to think things out first before answering**. We do that by literally spelling out the steps that Claude should take in order to process and think through its task. Along with a dash of role prompting, this empowers Claude to understand the review more deeply."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a savvy reader of movie reviews.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment positive or negative? First, write the best arguments for each side in <positive-argument> and <negative-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Claude is sometimes sensitive to ordering**. This example is on the frontier of Claude's ability to understand nuanced text, and when we swap the order of the arguments from the previous example so that negative is first and positive is second, this changes Claude's overall assessment to positive.\n",
|
||||
"\n",
|
||||
"In most situations (but not all, confusingly enough), **Claude is more likely to choose the second of two options**, possibly because in its training data from the web, second options were more likely to be correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment negative or positive? First write the best arguments for each side in <negative-argument> and <positive-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. Unrelatedly, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Letting Claude think can shift Claude's answer from incorrect to correct**. It's that simple in many cases where Claude makes mistakes!\n",
|
||||
"\n",
|
||||
"Let's go through an example where Claude's answer is incorrect to see how asking Claude to think can fix that."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's fix this by asking Claude to think step by step, this time in `<brainstorm>` tags."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 6.1 - Classifying Emails](#exercise-61---classifying-emails)\n",
|
||||
"- [Exercise 6.2 - Email Classification Formatting](#exercise-62---email-classification-formatting)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 6.1 - Classifying Emails\n",
|
||||
"In this exercise, we'll be instructing Claude to sort emails into the following categories:\t\t\t\t\t\t\t\t\t\t\n",
|
||||
"- (A) Pre-sale question\n",
|
||||
"- (B) Broken or defective item\n",
|
||||
"- (C) Billing question\n",
|
||||
"- (D) Other (please explain)\n",
|
||||
"\n",
|
||||
"For the first part of the exercise, change the `PROMPT` to **make Claude output the correct classification and ONLY the classification**. Your answer needs to **include the letter (A - D) of the correct choice, with the parentheses, as well as the name of the category**.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list to know which category that email should be classified under."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response, if any\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Dictionary of string values for each category to be used for regex grading\n",
|
||||
"REGEX_CATEGORIES = {\n",
|
||||
" \"A\": \"A\\) P\",\n",
|
||||
" \"B\": \"B\\) B\",\n",
|
||||
" \"C\": \"C\\) B\",\n",
|
||||
" \"D\": \"D\\) O\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_6_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Still stuck? Run the cell below for an example solution.\t\t\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_6_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 6.2 - Email Classification Formatting\n",
|
||||
"In this exercise, we're going to refine the output of the above prompt to yield an answer formatted exactly how we want it. \n",
|
||||
"\n",
|
||||
"Use your favorite output formatting technique to make Claude wrap JUST the letter of the correct classification in `<answer></answer>` tags. For instance, the answer to the first email should contain the exact string `<answer>B</answer>`.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list if you forget which letter category is correct for each email."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response, if any\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Dictionary of string values for each category to be used for regex grading\n",
|
||||
"REGEX_CATEGORIES = {\n",
|
||||
" \"A\": \"<answer>A</answer>\",\n",
|
||||
" \"B\": \"<answer>B</answer>\",\n",
|
||||
" \"C\": \"<answer>C</answer>\",\n",
|
||||
" \"D\": \"<answer>D</answer>\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_6_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this movie review sentiment positive or negative?\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since the year 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a savvy reader of movie reviews.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment positive or negative? First, write the best arguments for each side in <positive-argument> and <negative-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment negative or positive? First write the best arguments for each side in <negative-argument> and <positive-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. Unrelatedly, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,403 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 7: Using Examples (Few-Shot Prompting)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\":[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Giving Claude examples of how you want it to behave (or how you want it not to behave) is extremely effective** for:\n",
|
||||
"- Getting the right answer\n",
|
||||
"- Getting the answer in the right format\n",
|
||||
"\n",
|
||||
"This sort of prompting is also called \"**few shot prompting**\". You might also encounter the phrase \"zero-shot\" or \"n-shot\" or \"one-shot\". The number of \"shots\" refers to how many examples are used within the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Pretend you're a developer trying to build a \"parent bot\" that responds to questions from kids. **Claude's default response is quite formal and robotic**. This is going to break a child's heart."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Will Santa bring me presents on Christmas?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You could take the time to describe your desired tone, but it's much easier just to **give Claude a few examples of ideal responses**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Please complete the conversation by writing the next line, speaking as \"A\".\n",
|
||||
"Q: Is the tooth fairy real?\n",
|
||||
"A: Of course, sweetie. Wrap up your tooth and put it under your pillow tonight. There might be something waiting for you in the morning.\n",
|
||||
"Q: Will Santa bring me presents on Christmas?\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In the following formatting example, we could walk Claude step by step through a set of formatting instructions on how to extract names and professions and then format them exactly the way we want, or we could just **provide Claude with some correctly-formatted examples and Claude can extrapolate from there**. Note the `<individuals>` in the `assistant` turn to start Claude off on the right foot."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Silvermist Hollow, a charming village, was home to an extraordinary group of individuals.\n",
|
||||
"Among them was Dr. Liam Patel, a neurosurgeon who revolutionized surgical techniques at the regional medical center.\n",
|
||||
"Olivia Chen was an innovative architect who transformed the village's landscape with her sustainable and breathtaking designs.\n",
|
||||
"The local theater was graced by the enchanting symphonies of Ethan Kovacs, a professionally-trained musician and composer.\n",
|
||||
"Isabella Torres, a self-taught chef with a passion for locally sourced ingredients, created a culinary sensation with her farm-to-table restaurant, which became a must-visit destination for food lovers.\n",
|
||||
"These remarkable individuals, each with their distinct talents, contributed to the vibrant tapestry of life in Silvermist Hollow.\n",
|
||||
"<individuals>\n",
|
||||
"1. Dr. Liam Patel [NEUROSURGEON]\n",
|
||||
"2. Olivia Chen [ARCHITECT]\n",
|
||||
"3. Ethan Kovacs [MISICIAN AND COMPOSER]\n",
|
||||
"4. Isabella Torres [CHEF]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"At the heart of the town, Chef Oliver Hamilton has transformed the culinary scene with his farm-to-table restaurant, Green Plate. Oliver's dedication to sourcing local, organic ingredients has earned the establishment rave reviews from food critics and locals alike.\n",
|
||||
"Just down the street, you'll find the Riverside Grove Library, where head librarian Elizabeth Chen has worked diligently to create a welcoming and inclusive space for all. Her efforts to expand the library's offerings and establish reading programs for children have had a significant impact on the town's literacy rates.\n",
|
||||
"As you stroll through the charming town square, you'll be captivated by the beautiful murals adorning the walls. These masterpieces are the work of renowned artist, Isabella Torres, whose talent for capturing the essence of Riverside Grove has brought the town to life.\n",
|
||||
"Riverside Grove's athletic achievements are also worth noting, thanks to former Olympic swimmer-turned-coach, Marcus Jenkins. Marcus has used his experience and passion to train the town's youth, leading the Riverside Grove Swim Team to several regional championships.\n",
|
||||
"<individuals>\n",
|
||||
"1. Oliver Hamilton [CHEF]\n",
|
||||
"2. Elizabeth Chen [LIBRARIAN]\n",
|
||||
"3. Isabella Torres [ARTIST]\n",
|
||||
"4. Marcus Jenkins [COACH]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"Oak Valley, a charming small town, is home to a remarkable trio of individuals whose skills and dedication have left a lasting impact on the community.\n",
|
||||
"At the town's bustling farmer's market, you'll find Laura Simmons, a passionate organic farmer known for her delicious and sustainably grown produce. Her dedication to promoting healthy eating has inspired the town to embrace a more eco-conscious lifestyle.\n",
|
||||
"In Oak Valley's community center, Kevin Alvarez, a skilled dance instructor, has brought the joy of movement to people of all ages. His inclusive dance classes have fostered a sense of unity and self-expression among residents, enriching the local arts scene.\n",
|
||||
"Lastly, Rachel O'Connor, a tireless volunteer, dedicates her time to various charitable initiatives. Her commitment to improving the lives of others has been instrumental in creating a strong sense of community within Oak Valley.\n",
|
||||
"Through their unique talents and unwavering dedication, Laura, Kevin, and Rachel have woven themselves into the fabric of Oak Valley, helping to create a vibrant and thriving small town.\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<individuals>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 7.1 - Email Formatting via Examples](#exercise-71---email-formatting-via-examples)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 7.1 - Email Formatting via Examples\n",
|
||||
"We're going to redo Exercise 6.2, but this time, we're going to edit the `PROMPT` to use \"few-shot\" examples of emails + proper classification (and formatting) to get Claude to output the correct answer. We want the *last* letter of Claude's output to be the letter of the category.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list if you forget which letter category is correct for each email.\n",
|
||||
"\n",
|
||||
"Remember that these are the categories for the emails:\t\t\t\t\t\t\t\t\t\t\n",
|
||||
"- (A) Pre-sale question\n",
|
||||
"- (B) Broken or defective item\n",
|
||||
"- (C) Billing question\n",
|
||||
"- (D) Other (please explain)\t\t\t\t\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(ans, response[-1])) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_7_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Still stuck? Run the cell below for an example solution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_7_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Will Santa bring me presents on Christmas?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Please complete the conversation by writing the next line, speaking as \"A\".\n",
|
||||
"Q: Is the tooth fairy real?\n",
|
||||
"A: Of course, sweetie. Wrap up your tooth and put it under your pillow tonight. There might be something waiting for you in the morning.\n",
|
||||
"Q: Will Santa bring me presents on Christmas?\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Silvermist Hollow, a charming village, was home to an extraordinary group of individuals.\n",
|
||||
"Among them was Dr. Liam Patel, a neurosurgeon who revolutionized surgical techniques at the regional medical center.\n",
|
||||
"Olivia Chen was an innovative architect who transformed the village's landscape with her sustainable and breathtaking designs.\n",
|
||||
"The local theater was graced by the enchanting symphonies of Ethan Kovacs, a professionally-trained musician and composer.\n",
|
||||
"Isabella Torres, a self-taught chef with a passion for locally sourced ingredients, created a culinary sensation with her farm-to-table restaurant, which became a must-visit destination for food lovers.\n",
|
||||
"These remarkable individuals, each with their distinct talents, contributed to the vibrant tapestry of life in Silvermist Hollow.\n",
|
||||
"<individuals>\n",
|
||||
"1. Dr. Liam Patel [NEUROSURGEON]\n",
|
||||
"2. Olivia Chen [ARCHITECT]\n",
|
||||
"3. Ethan Kovacs [MISICIAN AND COMPOSER]\n",
|
||||
"4. Isabella Torres [CHEF]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"At the heart of the town, Chef Oliver Hamilton has transformed the culinary scene with his farm-to-table restaurant, Green Plate. Oliver's dedication to sourcing local, organic ingredients has earned the establishment rave reviews from food critics and locals alike.\n",
|
||||
"Just down the street, you'll find the Riverside Grove Library, where head librarian Elizabeth Chen has worked diligently to create a welcoming and inclusive space for all. Her efforts to expand the library's offerings and establish reading programs for children have had a significant impact on the town's literacy rates.\n",
|
||||
"As you stroll through the charming town square, you'll be captivated by the beautiful murals adorning the walls. These masterpieces are the work of renowned artist, Isabella Torres, whose talent for capturing the essence of Riverside Grove has brought the town to life.\n",
|
||||
"Riverside Grove's athletic achievements are also worth noting, thanks to former Olympic swimmer-turned-coach, Marcus Jenkins. Marcus has used his experience and passion to train the town's youth, leading the Riverside Grove Swim Team to several regional championships.\n",
|
||||
"<individuals>\n",
|
||||
"1. Oliver Hamilton [CHEF]\n",
|
||||
"2. Elizabeth Chen [LIBRARIAN]\n",
|
||||
"3. Isabella Torres [ARTIST]\n",
|
||||
"4. Marcus Jenkins [COACH]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"Oak Valley, a charming small town, is home to a remarkable trio of individuals whose skills and dedication have left a lasting impact on the community.\n",
|
||||
"At the town's bustling farmer's market, you'll find Laura Simmons, a passionate organic farmer known for her delicious and sustainably grown produce. Her dedication to promoting healthy eating has inspired the town to embrace a more eco-conscious lifestyle.\n",
|
||||
"In Oak Valley's community center, Kevin Alvarez, a skilled dance instructor, has brought the joy of movement to people of all ages. His inclusive dance classes have fostered a sense of unity and self-expression among residents, enriching the local arts scene.\n",
|
||||
"Lastly, Rachel O'Connor, a tireless volunteer, dedicates her time to various charitable initiatives. Her commitment to improving the lives of others has been instrumental in creating a strong sense of community within Oak Valley.\n",
|
||||
"Through their unique talents and unwavering dedication, Laura, Kevin, and Rachel have woven themselves into the fabric of Oak Valley, helping to create a vibrant and thriving small town.\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<individuals>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,713 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 8: Avoiding Hallucinations\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, system='', prefill=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\":[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ],\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Some bad news: **Claude sometimes \"hallucinates\" and makes claims that are untrue or unjustified**. The good news: there are techniques you can use to minimize hallucinations.\n",
|
||||
"\t\t\t\t\n",
|
||||
"Below, we'll go over a few of these techniques, namely:\n",
|
||||
"- Giving Claude the option to say it doesn't know the answer to a question\n",
|
||||
"- Asking Claude to find evidence before answering\n",
|
||||
"\n",
|
||||
"However, **there are many methods to avoid hallucinations**, including many of the techniques you've already learned in this course. If Claude hallucinates, experiment with multiple techniques to get Claude to increase its accuracy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Here is a question about general factual knowledge in answer to which **Claude hallucinates several large hippos because it's trying to be as helpful as possible**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"A solution we can try here is to \"**give Claude an out**\" — tell Claude that it's OK for it to decline to answer, or to only answer if it actually knows the answer with certainty."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time? Only answer if you know the answer with certainty.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In the prompt below, we give Claude a long document containing some \"distractor information\" that is almost but not quite relevant to the user's question. **Without prompting help, Claude falls for the distractor information** and gives an incorrect \"hallucinated\" answer as to the size of Matterport's subscriber base as of May 31, 2020.\n",
|
||||
"\n",
|
||||
"**Note:** As you'll learn later in the next chapter, **it's best practice to have the question at the bottom *after* any text or document**, but we put it at the top here to make the prompt easier to read. Feel free to double click on the prompt cell to get the full prompt text (it's very long!)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then write a brief numerical answer inside <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"How do we fix this? Well, a great way to reduce hallucinations on long documents is to **make Claude gather evidence first.** \n",
|
||||
"\n",
|
||||
"In this case, we **tell Claude to first extract relevant quotes, then base its answer on those quotes**. Telling Claude to do so here makes it correctly notice that the quote does not answer the question."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then, in <scratchpad> tags, pull the most relevant quote from the document and consider whether it answers the user's question or whether it lacks sufficient detail. Then write a brief numerical answer in <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Bonus lesson\n",
|
||||
"\n",
|
||||
"Sometimes, Claude's hallucinations can be solved by lowering the `temperature` of Claude's responses. Temperature is a measurement of answer creativity between 0 and 1, with 1 being more unpredictable and less standardized, and 0 being the most consistent. \n",
|
||||
"\n",
|
||||
"Asking Claude something at temperature 0 will generally yield an almost-deterministic answer set across repeated trials (although complete determinism is not guaranteed). Asking Claude something at temperature 1 (or gradations in between) will yield more variable answers. Learn more about temperature and other parameters [here](https://docs.anthropic.com/claude/reference/messages_post).\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 8.1 - Beyoncé Hallucination](#exercise-81---beyoncé-hallucination)\n",
|
||||
"- [Exercise 8.2 - Prospectus Hallucination](#exercise-82---prospectus-hallucination)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 8.1 - Beyoncé Hallucination\n",
|
||||
"Modify the `PROMPT` to fix Claude's hallucination issue by giving Claude an out. (Renaissance is Beyoncé's seventh studio album, not her eigthth.)\n",
|
||||
"\n",
|
||||
"We suggest you run the cell first to see what Claude hallucinates before trying to fix it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In what year did star performer Beyoncé release her eighth studio album?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" contains = bool(\n",
|
||||
" re.search(\"Unfortunately\", text) or\n",
|
||||
" re.search(\"I do not\", text) or\n",
|
||||
" re.search(\"I don't\", text)\n",
|
||||
" )\n",
|
||||
" does_not_contain = not bool(re.search(\"2022\", text))\n",
|
||||
" return contains and does_not_contain\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_8_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 8.1 - Prospectus Hallucination\n",
|
||||
"Modify the `PROMPT` to fix Claude's hallucination issue by asking for citations. The correct answer is that subscribers went up 49x."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"From December 2018 to December 2022, by what amount did Matterport's subscribers grow?\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"49-fold\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_8_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time? Only answer if you know the answer with certainty.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then write a brief numerical answer inside <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then, in <scratchpad> tags, pull the most relevant quote from the document and consider whether it answers the user's question or whether it lacks sufficient detail. Then write a brief numerical answer in <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,712 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.1: Chaining Prompts\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Retrieve the MODEL_NAME variable from the IPython store\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(messages, system_prompt=''):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": messages,\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"system\": system_prompt\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"The saying goes, \"Writing is rewriting.\" It turns out, **Claude can often improve the accuracy of its response when asked to do so**!\n",
|
||||
"\n",
|
||||
"There are many ways to prompt Claude to \"think again\". The ways that feel natural to ask a human to double check their work will also generally work for Claude. (Check out our [prompt chaining documentation](https://docs.anthropic.com/claude/docs/chain-prompts) for further examples of when and how to use prompt chaining.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In this example, we ask Claude to come up with ten words... but one or more of them isn't a real word."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Asking Claude to make its answer more accurate** fixes the error! \n",
|
||||
"\n",
|
||||
"Below, we've pulled down Claude's incorrect response from above and added another turn to the conversation asking Claude to fix its previous answer."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"But is Claude revising its answer just because we told it to? What if we start off with a correct answer already? Will Claude lose its confidence? Here, we've placed a correct response in the place of `first_response` and asked it to double check again."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You may notice that if you generate a respnse from the above block a few times, Claude leaves the words as is most of the time, but still occasionally changes the words even though they're all already correct. What can we do to mitigate this? Per Chapter 8, we can give Claude an out! Let's try this one more time."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words. If all the words are real words, return the original list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Try generating responses from the above code a few times to see that Claude is much better at sticking to its guns now.\n",
|
||||
"\n",
|
||||
"You can also use prompt chaining to **ask Claude to make its responses better**. Below, we asked Claude to first write a story, and then improve the story it wrote. Your personal tastes may vary, but many might agree that Claude's second version is better.\n",
|
||||
"\n",
|
||||
"First, let's generate Claude's first version of the story."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Write a three-sentence short story about a girl who likes to run.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's have Claude improve on its first draft."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Make the story better.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This form of substitution is very powerful. We've been using substitution placeholders to pass in lists, words, Claude's former responses, and so on. You can also **use substitution to do what we call \"function calling,\" which is asking Claude to perform some function, and then taking the results of that function and asking Claude to do even more afterward with the results**. It works like any other substitution. More on this in the next appendix.\n",
|
||||
"\n",
|
||||
"Below is one more example of taking the results of one call to Claude and plugging it into another, longer call. Let's start with the first prompt (which includes prefilling Claude's response this time)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"\"\"Find all names from the below text:\n",
|
||||
"\n",
|
||||
"\"Hey, Jesse. It's me, Erin. I'm calling about the party that Joey is throwing tomorrow. Keisha said she would come and I think Mel will be there too.\"\"\"\n",
|
||||
"\n",
|
||||
"prefill = \"<names>\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's pass this list of names into another prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Alphabetize the list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill + \"\\n\" + first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that you've learned about prompt chaining, head over to Appendix 10.2 to learn how to implement function calling using prompt chaining."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words. If all the words are real words, return the original list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Write a three-sentence short story about a girl who likes to run.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Make the story better.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"\"\"Find all names from the below text:\n",
|
||||
"\n",
|
||||
"\"Hey, Jesse. It's me, Erin. I'm calling about the party that Joey is throwing tomorrow. Keisha said she would come and I think Mel will be there too.\"\"\"\n",
|
||||
"\n",
|
||||
"prefill = \"<names>\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Alphabetize the list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill + \"\\n\" + first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,791 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.2: Tool Use\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Rewrittten to call Claude 3 Sonnet, which is generally better at tool use, and include stop_sequences\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Import the hints module from the utils package\n",
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"module_path = \"..\"\n",
|
||||
"sys.path.append(os.path.abspath(module_path))\n",
|
||||
"from utils import hints\n",
|
||||
"\n",
|
||||
"# Override the MODEL_NAME variable in the IPython store to use Sonnet instead of the Haiku model\n",
|
||||
"MODEL_NAME='anthropic.claude-3-sonnet-20240229-v1:0'\n",
|
||||
"%store -r AWS_REGION\n",
|
||||
"\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(messages, system_prompt=\"\", prefill=\"\", stop_sequences=None):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"temperature\": 0.0,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" \"messages\":messages,\n",
|
||||
" \"system\": system_prompt,\n",
|
||||
" \"stop_sequences\": stop_sequences\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"While it might seem conceptually complex at first, tool use, a.k.a. function calling, is actually quite simple! You already know all the skills necessary to implement tool use, which is really just a combination of substitution and prompt chaining.\n",
|
||||
"\n",
|
||||
"In previous substitution exercises, we substituted text into prompts. With tool use, we substitute tool or function results into prompts. Claude can't literally call or access tools and functions. Instead, we have Claude:\n",
|
||||
"1. Output the tool name and arguments it wants to call\n",
|
||||
"2. Halt any further response generation while the tool is called\n",
|
||||
"3. Then we reprompt with the appended tool results"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Function calling is useful because it expands Claude's capabilities and enables Claude to handle much more complex, multi-step tasks.\n",
|
||||
"Some examples of functions you can give Claude:\n",
|
||||
"- Calculator\n",
|
||||
"- Word counter\n",
|
||||
"- SQL database querying and data retrieval\n",
|
||||
"- Weather API"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can get Claude to do tool use by combining these two elements:\n",
|
||||
"\n",
|
||||
"1. A system prompt, in which we give Claude an explanation of the concept of tool use as well as a detailed descriptive list of the tools it has access to\n",
|
||||
"2. The control logic with which to orchestrate and execute Claude's tool use requests"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Tool use roadmap\n",
|
||||
"\n",
|
||||
"*This lesson teaches our current tool use format. However, we will be updating and improving tool use functionality in the near future, including:*\n",
|
||||
"* *A more streamlined format for function definitions and calls*\n",
|
||||
"* *More robust error handilgj and edge case coverage*\n",
|
||||
"* *Tighter integration with the rest of our API*\n",
|
||||
"* *Better reliability and performance, especially for more complex tool use tasks*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"To enable tool use in Claude, we start with the system prompt. In this special tool use system prompt, wet tell Claude:\n",
|
||||
"* The basic premise of tool use and what it entails\n",
|
||||
"* How Claude can call and use the tools it's been given\n",
|
||||
"* A detailed list of tools it has access to in this specific scenario \n",
|
||||
"\n",
|
||||
"Here's the first part of the system prompt, explaining tool use to Claude. This part of the system prompt is generalizable across all instances of prompting Claude for tool use. The tool calling structure we're giving Claude (`<function_calls> [...] </function_calls>`) is a structure Claude has been specifically trained to use, so we recommend that you stick with this."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_general_explanation = \"\"\"You have access to a set of functions you can use to answer the user's question. This includes access to a\n",
|
||||
"sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external\n",
|
||||
"resources, except by invoking the below functions.\n",
|
||||
"\n",
|
||||
"You can invoke one or more functions by writing a \"<function_calls>\" block like the following as part of your\n",
|
||||
"reply to the user:\n",
|
||||
"<function_calls>\n",
|
||||
"<invoke name=\"$FUNCTION_NAME\">\n",
|
||||
"<antml:parameter name=\"$PARAMETER_NAME\">$PARAMETER_VALUE</parameter>\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"<nvoke name=\"$FUNCTION_NAME2\">\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"</function_calls>\n",
|
||||
"\n",
|
||||
"String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that\n",
|
||||
"spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular\n",
|
||||
"expressions.\n",
|
||||
"\n",
|
||||
"The output and/or any errors will appear in a subsequent \"<function_results>\" block, and remain there as part of\n",
|
||||
"your reply to the user.\n",
|
||||
"You may then continue composing the rest of your reply to the user, respond to any errors, or make further function\n",
|
||||
"calls as appropriate.\n",
|
||||
"If a \"<function_results>\" does NOT appear after your function calls, then they are likely malformatted and not\n",
|
||||
"recognized as a call.\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's the second part of the system prompt, which defines the exact tools Claude has access to in this specific situation. In this example, we will be giving Claude a calculator tool, which takes three parameters: two operands and an operator. \n",
|
||||
"\n",
|
||||
"Then we combine the two parts of the system prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools = \"\"\"Here are the functions available in JSONSchema format:\n",
|
||||
"<tools>\n",
|
||||
"<tool_description>\n",
|
||||
"<tool_name>calculator</tool_name>\n",
|
||||
"<description>\n",
|
||||
"Calculator function for doing basic arithmetic.\n",
|
||||
"Supports addition, subtraction, multiplication\n",
|
||||
"</description>\n",
|
||||
"<parameters>\n",
|
||||
"<parameter>\n",
|
||||
"<name>first_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>First operand (before the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>second_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>Second operand (after the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>operator</name>\n",
|
||||
"<type>str</type>\n",
|
||||
"<description>The operation to perform. Must be either +, -, *, or /</description>\n",
|
||||
"</parameter>\n",
|
||||
"</parameters>\n",
|
||||
"</tool_description>\n",
|
||||
"</tools>\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now we can give Claude a question that requires use of the `calculator` tool. We will use `<function_calls\\>` in `stop_sequences` to detect if and when Claude calls the function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Multiply 1,984,135 by 9,343,116\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, we can extract out the parameters from Claude's function call and actually run the function on Claude's behalf.\n",
|
||||
"\n",
|
||||
"First we'll define the function's code."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def do_pairwise_arithmetic(num1, num2, operation):\n",
|
||||
" if operation == '+':\n",
|
||||
" return num1 + num2\n",
|
||||
" elif operation == \"-\":\n",
|
||||
" return num1 - num2\n",
|
||||
" elif operation == \"*\":\n",
|
||||
" return num1 * num2\n",
|
||||
" elif operation == \"/\":\n",
|
||||
" return num1 / num2\n",
|
||||
" else:\n",
|
||||
" return \"Error: Operation not supported.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then we'll extract the parameters from Claude's function call response. If all the parameters exist, we run the calculator tool."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def find_parameter(message, parameter_name):\n",
|
||||
" parameter_start_string = f\"name=\\\"{parameter_name}\\\">\"\n",
|
||||
" start = message.index(parameter_start_string)\n",
|
||||
" if start == -1:\n",
|
||||
" return None\n",
|
||||
" if start > 0:\n",
|
||||
" start = start + len(parameter_start_string)\n",
|
||||
" end = start\n",
|
||||
" while message[end] != \"<\":\n",
|
||||
" end += 1\n",
|
||||
" return message[start:end]\n",
|
||||
"\n",
|
||||
"first_operand = find_parameter(function_calling_response, \"first_operand\")\n",
|
||||
"second_operand = find_parameter(function_calling_response, \"second_operand\")\n",
|
||||
"operator = find_parameter(function_calling_response, \"operator\")\n",
|
||||
"\n",
|
||||
"if first_operand and second_operand and operator:\n",
|
||||
" result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)\n",
|
||||
" print(\"---------------- RESULT ----------------\")\n",
|
||||
" print(f\"{result:,}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that we have a result, we have to properly format that result so that when we pass it back to Claude, Claude understands what tool that result is in relation to. There is a set format for this that Claude has been trained to recognize:\n",
|
||||
"```\n",
|
||||
"<function_results>\n",
|
||||
"<result>\n",
|
||||
"<tool_name>{TOOL_NAME}</tool_name>\n",
|
||||
"<stdout>\n",
|
||||
"{TOOL_RESULT}\n",
|
||||
"</stdout>\n",
|
||||
"</result>\n",
|
||||
"</function_results>\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Run the cell below to format the above tool result into this structure."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_successful_function_run_injection_prompt(invoke_results):\n",
|
||||
" constructed_prompt = (\n",
|
||||
" \"<function_results>\\n\"\n",
|
||||
" + '\\n'.join(\n",
|
||||
" f\"<result>\\n<tool_name>{res['tool_name']}</tool_name>\\n<stdout>\\n{res['tool_result']}\\n</stdout>\\n</result>\"\n",
|
||||
" for res in invoke_results\n",
|
||||
" ) + \"\\n</function_results>\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return constructed_prompt\n",
|
||||
"\n",
|
||||
"formatted_results = [{\n",
|
||||
" 'tool_name': 'do_pairwise_arithmetic',\n",
|
||||
" 'tool_result': result\n",
|
||||
"}]\n",
|
||||
"function_results = construct_successful_function_run_injection_prompt(formatted_results)\n",
|
||||
"print(function_results)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now all we have to do is send this result back to Claude by appending the result to the same message chain as before, and we're good!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"full_first_response = function_calling_response + \"</function_calls>\"\n",
|
||||
"\n",
|
||||
"# Construct the full conversation\n",
|
||||
"messages = [multiplication_message,\n",
|
||||
"{\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": full_first_response\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": function_results\n",
|
||||
"}]\n",
|
||||
" \n",
|
||||
"# Print Claude's response\n",
|
||||
"final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(\"------------- FINAL RESULT -------------\")\n",
|
||||
"print(final_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Congratulations on running an entire tool use chain end to end!\n",
|
||||
"\n",
|
||||
"Now what if we give Claude a question that doesn't that doesn't require using the given tool at all?"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"non_multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Tell me the capital of France.\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Success! As you can see, Claude knew not to call the function when it wasn't needed.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 10.2.1 - SQL](#exercise-1021---SQL)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 10.2.1 - SQL\n",
|
||||
"In this exercise, you'll be writing a tool use prompt for querying and writing to the world's smallest \"database\". Here's the initialized database, which is really just a dictionary."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"db = {\n",
|
||||
" \"users\": [\n",
|
||||
" {\"id\": 1, \"name\": \"Alice\", \"email\": \"alice@example.com\"},\n",
|
||||
" {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@example.com\"},\n",
|
||||
" {\"id\": 3, \"name\": \"Charlie\", \"email\": \"charlie@example.com\"}\n",
|
||||
" ],\n",
|
||||
" \"products\": [\n",
|
||||
" {\"id\": 1, \"name\": \"Widget\", \"price\": 9.99},\n",
|
||||
" {\"id\": 2, \"name\": \"Gadget\", \"price\": 14.99},\n",
|
||||
" {\"id\": 3, \"name\": \"Doohickey\", \"price\": 19.99}\n",
|
||||
" ]\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"And here is the code for the functions that write to and from the database."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_user(user_id):\n",
|
||||
" for user in db[\"users\"]:\n",
|
||||
" if user[\"id\"] == user_id:\n",
|
||||
" return user\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def get_product(product_id):\n",
|
||||
" for product in db[\"products\"]:\n",
|
||||
" if product[\"id\"] == product_id:\n",
|
||||
" return product\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def add_user(name, email):\n",
|
||||
" user_id = len(db[\"users\"]) + 1\n",
|
||||
" user = {\"id\": user_id, \"name\": name, \"email\": email}\n",
|
||||
" db[\"users\"].append(user)\n",
|
||||
" return user\n",
|
||||
"\n",
|
||||
"def add_product(name, price):\n",
|
||||
" product_id = len(db[\"products\"]) + 1\n",
|
||||
" product = {\"id\": product_id, \"name\": name, \"price\": price}\n",
|
||||
" db[\"products\"].append(product)\n",
|
||||
" return product"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To solve the exercise, start by defining a system prompt like `system_prompt_tools_specific_tools` above. Make sure to include the name and description of each tool, along with the name and type and description of each parameter for each function. We've given you some starting scaffolding below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools_sql = \"\"\"\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools_sql"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When you're ready, you can try out your tool definition system prompt on the examples below. Just run the below cell!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"examples = [\n",
|
||||
" \"Add a user to the database named Deborah.\",\n",
|
||||
" \"Add a product to the database named Thingo\",\n",
|
||||
" \"Tell me the name of User 2\",\n",
|
||||
" \"Tell me the name of Product 3\"\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"for example in examples:\n",
|
||||
" message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": example\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" # Get & print Claude's response\n",
|
||||
" function_calling_response = get_completion([message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
" print(example, \"\\n----------\\n\\n\", function_calling_response, \"\\n*********\\n*********\\n*********\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you did it right, the function calling messages should call the `add_user`, `add_product`, `get_user`, and `get_product` functions correctly.\n",
|
||||
"\n",
|
||||
"For extra credit, add some code cells and write parameter-parsing code. Then call the functions with the parameters Claude gives you to see the state of the \"database\" after the call."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want to see a possible solution, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(hints.exercise_10_2_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"Congratulations on learning tool use and function calling! Head over to the last appendix section if you would like to learn more about search & RAG."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_general_explanation = \"\"\"You have access to a set of functions you can use to answer the user's question. This includes access to a\n",
|
||||
"sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external\n",
|
||||
"resources, except by invoking the below functions.\n",
|
||||
"\n",
|
||||
"You can invoke one or more functions by writing a \"<function_calls>\" block like the following as part of your\n",
|
||||
"reply to the user:\n",
|
||||
"<function_calls>\n",
|
||||
"<invoke name=\"$FUNCTION_NAME\">\n",
|
||||
"<antml:parameter name=\"$PARAMETER_NAME\">$PARAMETER_VALUE</parameter>\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"<nvoke name=\"$FUNCTION_NAME2\">\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"</function_calls>\n",
|
||||
"\n",
|
||||
"String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that\n",
|
||||
"spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular\n",
|
||||
"expressions.\n",
|
||||
"\n",
|
||||
"The output and/or any errors will appear in a subsequent \"<function_results>\" block, and remain there as part of\n",
|
||||
"your reply to the user.\n",
|
||||
"You may then continue composing the rest of your reply to the user, respond to any errors, or make further function\n",
|
||||
"calls as appropriate.\n",
|
||||
"If a \"<function_results>\" does NOT appear after your function calls, then they are likely malformatted and not\n",
|
||||
"recognized as a call.\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools = \"\"\"Here are the functions available in JSONSchema format:\n",
|
||||
"<tools>\n",
|
||||
"<tool_description>\n",
|
||||
"<tool_name>calculator</tool_name>\n",
|
||||
"<description>\n",
|
||||
"Calculator function for doing basic arithmetic.\n",
|
||||
"Supports addition, subtraction, multiplication\n",
|
||||
"</description>\n",
|
||||
"<parameters>\n",
|
||||
"<parameter>\n",
|
||||
"<name>first_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>First operand (before the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>second_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>Second operand (after the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>operator</name>\n",
|
||||
"<type>str</type>\n",
|
||||
"<description>The operation to perform. Must be either +, -, *, or /</description>\n",
|
||||
"</parameter>\n",
|
||||
"</parameters>\n",
|
||||
"</tool_description>\n",
|
||||
"</tools>\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Multiply 1,984,135 by 9,343,116\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def do_pairwise_arithmetic(num1, num2, operation):\n",
|
||||
" if operation == '+':\n",
|
||||
" return num1 + num2\n",
|
||||
" elif operation == \"-\":\n",
|
||||
" return num1 - num2\n",
|
||||
" elif operation == \"*\":\n",
|
||||
" return num1 * num2\n",
|
||||
" elif operation == \"/\":\n",
|
||||
" return num1 / num2\n",
|
||||
" else:\n",
|
||||
" return \"Error: Operation not supported.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def find_parameter(message, parameter_name):\n",
|
||||
" parameter_start_string = f\"name=\\\"{parameter_name}\\\">\"\n",
|
||||
" start = message.index(parameter_start_string)\n",
|
||||
" if start == -1:\n",
|
||||
" return None\n",
|
||||
" if start > 0:\n",
|
||||
" start = start + len(parameter_start_string)\n",
|
||||
" end = start\n",
|
||||
" while message[end] != \"<\":\n",
|
||||
" end += 1\n",
|
||||
" return message[start:end]\n",
|
||||
"\n",
|
||||
"first_operand = find_parameter(function_calling_response, \"first_operand\")\n",
|
||||
"second_operand = find_parameter(function_calling_response, \"second_operand\")\n",
|
||||
"operator = find_parameter(function_calling_response, \"operator\")\n",
|
||||
"\n",
|
||||
"if first_operand and second_operand and operator:\n",
|
||||
" result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)\n",
|
||||
" print(\"---------------- RESULT ----------------\")\n",
|
||||
" print(f\"{result:,}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_successful_function_run_injection_prompt(invoke_results):\n",
|
||||
" constructed_prompt = (\n",
|
||||
" \"<function_results>\\n\"\n",
|
||||
" + '\\n'.join(\n",
|
||||
" f\"<result>\\n<tool_name>{res['tool_name']}</tool_name>\\n<stdout>\\n{res['tool_result']}\\n</stdout>\\n</result>\"\n",
|
||||
" for res in invoke_results\n",
|
||||
" ) + \"\\n</function_results>\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return constructed_prompt\n",
|
||||
"\n",
|
||||
"formatted_results = [{\n",
|
||||
" 'tool_name': 'do_pairwise_arithmetic',\n",
|
||||
" 'tool_result': result\n",
|
||||
"}]\n",
|
||||
"function_results = construct_successful_function_run_injection_prompt(formatted_results)\n",
|
||||
"print(function_results)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"full_first_response = function_calling_response + \"</function_calls>\"\n",
|
||||
"\n",
|
||||
"# Construct the full conversation\n",
|
||||
"messages = [multiplication_message,\n",
|
||||
"{\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": full_first_response\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": function_results\n",
|
||||
"}]\n",
|
||||
" \n",
|
||||
"# Print Claude's response\n",
|
||||
"final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(\"------------- FINAL RESULT -------------\")\n",
|
||||
"print(final_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"non_multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Tell me the capital of France.\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,342 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Evaluating AI Models: Code, Human, and Model-Based Grading\n",
|
||||
"\n",
|
||||
"In this notebook, we'll delve into a trio of widely-used techniques for assessing the effectiveness of AI models, like Claude v3:\n",
|
||||
"\n",
|
||||
"1. Code-based grading\n",
|
||||
"2. Human grading\n",
|
||||
"3. Model-based grading\n",
|
||||
"\n",
|
||||
"We'll illustrate each approach through examples and examine their respective advantages and limitations, when gauging AI performance."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Code-Based Grading Example: Sentiment Analysis\n",
|
||||
"\n",
|
||||
"In this example, we'll evaluate Claude's ability to classify the sentiment of movie reviews as positive or negative. We can use code to check if the model's output matches the expected sentiment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"\n",
|
||||
"# Import boto3 and json\n",
|
||||
"import boto3\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# Store the model name and AWS region for later use\n",
|
||||
"MODEL_NAME = \"anthropic.claude-3-haiku-20240307-v1:0\"\n",
|
||||
"AWS_REGION = \"us-west-2\"\n",
|
||||
"\n",
|
||||
"%store MODEL_NAME\n",
|
||||
"%store AWS_REGION"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for sentiment analysis\n",
|
||||
"def build_input_prompt(review):\n",
|
||||
" user_content = f\"\"\"Classify the sentiment of the following movie review as either 'positive' or 'negative' provide only one of those two choices:\n",
|
||||
" <review>{review}</review>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"review\": \"This movie was amazing! The acting was superb and the plot kept me engaged from start to finish.\",\n",
|
||||
" \"golden_answer\": \"positive\"\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"review\": \"I was thoroughly disappointed by this film. The pacing was slow and the characters were one-dimensional.\",\n",
|
||||
" \"golden_answer\": \"negative\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Function to get completions from the model\n",
|
||||
"client = boto3.client('bedrock-runtime',region_name=AWS_REGION)\n",
|
||||
"\n",
|
||||
"def get_completion(messages):\n",
|
||||
" body = json.dumps(\n",
|
||||
" {\n",
|
||||
" \"anthropic_version\": '',\n",
|
||||
" \"max_tokens\": 2000,\n",
|
||||
" \"messages\": messages,\n",
|
||||
" \"temperature\": 0.5,\n",
|
||||
" \"top_p\": 1,\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" response = client.invoke_model(body=body, modelId=MODEL_NAME)\n",
|
||||
" response_body = json.loads(response.get('body').read())\n",
|
||||
"\n",
|
||||
" return response_body.get('content')[0].get('text')\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"review\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Print the outputs and golden answers\n",
|
||||
"for output, question in zip(outputs, eval):\n",
|
||||
" print(f\"Review: {question['review']}\\nGolden Answer: {question['golden_answer']}\\nOutput: {output}\\n\")\n",
|
||||
"\n",
|
||||
"# Function to grade the completions\n",
|
||||
"def grade_completion(output, golden_answer):\n",
|
||||
" return output.lower() == golden_answer.lower()\n",
|
||||
"\n",
|
||||
"# Grade the completions and print the accuracy\n",
|
||||
"grades = [grade_completion(output, item[\"golden_answer\"]) for output, item in zip(outputs, eval)]\n",
|
||||
"print(f\"Accuracy: {sum(grades) / len(grades) * 100}%\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Human Grading Example: Essay Scoring\n",
|
||||
"\n",
|
||||
"Some tasks, like scoring essays, are difficult to evaluate with code alone. In this case, we can provide guidelines for human graders to assess the model's output."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for essay generation\n",
|
||||
"def build_input_prompt(topic):\n",
|
||||
" user_content = f\"\"\"Write a short essay discussing the following topic:\n",
|
||||
" <topic>{topic}</topic>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"topic\": \"The importance of education in personal development and societal progress\",\n",
|
||||
" \"golden_answer\": \"A high-scoring essay should have a clear thesis, well-structured paragraphs, and persuasive examples discussing how education contributes to individual growth and broader societal advancement.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"topic\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Print the outputs and golden answers\n",
|
||||
"for output, item in zip(outputs, eval):\n",
|
||||
" print(f\"Topic: {item['topic']}\\n\\nGrading Rubric:\\n {item['golden_answer']}\\n\\nModel Output:\\n{output}\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Model-Based Grading Examples\n",
|
||||
"\n",
|
||||
"We can use Claude to grade its own outputs by providing the model's response and a grading rubric. This allows us to automate the evaluation of tasks that would typically require human judgment."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example 1: Summarization\n",
|
||||
"\n",
|
||||
"In this example, we'll use Claude to assess the quality of a summary it generated. This can be useful when you need to evaluate the model's ability to capture key information from a longer text concisely and accurately. By providing a rubric that outlines the essential points that should be covered, we can automate the grading process and quickly assess the model's performance on summarization tasks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for summarization\n",
|
||||
"def build_input_prompt(text):\n",
|
||||
" user_content = f\"\"\"Please summarize the main points of the following text:\n",
|
||||
" <text>{text}</text>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Function to build the grader prompt for assessing summary quality\n",
|
||||
"def build_grader_prompt(output, rubric):\n",
|
||||
" user_content = f\"\"\"Assess the quality of the following summary based on this rubric:\n",
|
||||
" <rubric>{rubric}</rubric>\n",
|
||||
" <summary>{output}</summary>\n",
|
||||
" Provide a score from 1-5, where 1 is poor and 5 is excellent.\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"text\": \"The Magna Carta, signed in 1215, was a pivotal document in English history. It limited the powers of the monarchy and established the principle that everyone, including the king, was subject to the law. This laid the foundation for constitutional governance and the rule of law in England and influenced legal systems worldwide.\",\n",
|
||||
" \"golden_answer\": \"A high-quality summary should concisely capture the key points: 1) The Magna Carta's significance in English history, 2) Its role in limiting monarchical power, 3) Establishing the principle of rule of law, and 4) Its influence on legal systems around the world.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"text\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Grade the completions\n",
|
||||
"grades = [get_completion(build_grader_prompt(output, item[\"golden_answer\"])) for output, item in zip(outputs, eval)]\n",
|
||||
"\n",
|
||||
"# Print the summary quality score\n",
|
||||
"print(f\"Summary quality score: {grades[0]}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example 2: Fact-Checking\n",
|
||||
"\n",
|
||||
"In this example, we'll use Claude to fact-check a claim and then assess the accuracy of its fact-checking. This can be useful when you need to evaluate the model's ability to distinguish between accurate and inaccurate information. By providing a rubric that outlines the key points that should be covered in a correct fact-check, we can automate the grading process and quickly assess the model's performance on fact-checking tasks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for fact-checking\n",
|
||||
"def build_input_prompt(claim):\n",
|
||||
" user_content = f\"\"\"Determine if the following claim is true or false:\n",
|
||||
" <claim>{claim}</claim>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Function to build the grader prompt for assessing fact-check accuracy\n",
|
||||
"def build_grader_prompt(output, rubric):\n",
|
||||
" user_content = f\"\"\"Based on the following rubric, assess whether the fact-check is correct:\n",
|
||||
" <rubric>{rubric}</rubric>\n",
|
||||
" <fact-check>{output}</fact-check>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"claim\": \"The Great Wall of China is visible from space.\",\n",
|
||||
" \"golden_answer\": \"A correct fact-check should state that this claim is false. While the Great Wall is an impressive structure, it is not visible from space with the naked eye.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"claim\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"grades = []\n",
|
||||
"for output, item in zip(outputs, eval):\n",
|
||||
" # Print the claim, fact-check, and rubric\n",
|
||||
" print(f\"Claim: {item['claim']}\\n\")\n",
|
||||
" print(f\"Fact-check: {output}]\\n\")\n",
|
||||
" print(f\"Rubric: {item['golden_answer']}\\n\")\n",
|
||||
" \n",
|
||||
" # Grade the fact-check\n",
|
||||
" grader_prompt = build_grader_prompt(output, item[\"golden_answer\"])\n",
|
||||
" grade = get_completion(grader_prompt)\n",
|
||||
" grades.append(\"correct\" in grade.lower())\n",
|
||||
"\n",
|
||||
"# Print the fact-checking accuracy\n",
|
||||
"accuracy = sum(grades) / len(grades)\n",
|
||||
"print(f\"Fact-checking accuracy: {accuracy * 100}%\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Example 3: Tone Analysis\n",
|
||||
"\n",
|
||||
"In this example, we'll use Claude to analyze the tone of a given text and then assess the accuracy of its analysis. This can be useful when you need to evaluate the model's ability to identify and interpret the emotional content and attitudes expressed in a piece of text. By providing a rubric that outlines the key aspects of tone that should be identified, we can automate the grading process and quickly assess the model's performance on tone analysis tasks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Function to build the input prompt for tone analysis\n",
|
||||
"def build_input_prompt(text):\n",
|
||||
" user_content = f\"\"\"Analyze the tone of the following text:\n",
|
||||
" <text>{text}</text>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Function to build the grader prompt for assessing tone analysis accuracy\n",
|
||||
"def build_grader_prompt(output, rubric):\n",
|
||||
" user_content = f\"\"\"Assess the accuracy of the following tone analysis based on this rubric:\n",
|
||||
" <rubric>{rubric}</rubric>\n",
|
||||
" <tone-analysis>{output}</tone-analysis>\"\"\"\n",
|
||||
" return [{\"role\": \"user\", \"content\": user_content}]\n",
|
||||
"\n",
|
||||
"# Define the evaluation data\n",
|
||||
"eval = [\n",
|
||||
" {\n",
|
||||
" \"text\": \"I can't believe they canceled the event at the last minute. This is completely unacceptable and unprofessional!\",\n",
|
||||
" \"golden_answer\": \"The tone analysis should identify the text as expressing frustration, anger, and disappointment. Key words like 'can't believe', 'last minute', 'unacceptable', and 'unprofessional' indicate strong negative emotions.\"\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Get completions for each input\n",
|
||||
"outputs = [get_completion(build_input_prompt(item[\"text\"])) for item in eval]\n",
|
||||
"\n",
|
||||
"# Grade the completions\n",
|
||||
"grades = [get_completion(build_grader_prompt(output, item[\"golden_answer\"])) for output, item in zip(outputs, eval)]\n",
|
||||
"\n",
|
||||
"# Print the tone analysis quality\n",
|
||||
"print(f\"Tone analysis quality: {grades[0]}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"These examples demonstrate how code-based, human, and model-based grading can be used to evaluate AI models like Claude on various tasks. The choice of evaluation method depends on the nature of the task and the resources available. Model-based grading offers a promising approach for automating the assessment of complex tasks that would otherwise require human judgment."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "conda_pytorch_p310",
|
||||
"language": "python",
|
||||
"name": "conda_pytorch_p310"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.13"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.4: Search & Retrieval\n",
|
||||
"\n",
|
||||
"Did you know you can use Claude to **search through Wikipedia for you**? Claude can find and retrieve articles, at which point you can also use Claude to summarize and synthesize them, write novel content from what it found, and much more. And not just Wikipedia! You can also search over your own docs, whether stored as plain text or embedded in a vector datastore.\n",
|
||||
"\n",
|
||||
"See our [RAG cookbook examples](https://github.com/anthropics/anthropic-cookbook/blob/main/third_party/Wikipedia/wikipedia-search-cookbook.ipynb) to learn how to supplement Claude's knowledge and improve the accuracy and relevance of Claude's responses with data retrieved from vector databases, Wikipedia, the internet, and more. There, you can also learn about how to use certain [embeddings](https://docs.anthropic.com/claude/docs/embeddings) and vector database tools.\n",
|
||||
"\n",
|
||||
"If you are interested in learning about advanced RAG architectures using Claude, check out our [Claude 3 technical presentation slides on RAG architectures](https://docs.google.com/presentation/d/1zxkSI7lLUBrZycA-_znwqu8DDyVhHLkQGScvzaZrUns/edit#slide=id.g2c736259dac_63_782)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,67 +0,0 @@
|
||||
AWSTemplateFormatVersion: '2010-09-09'
|
||||
Description: 'CloudFormation template to create a Jupyter notebook in SageMaker with an execution role and Anthropic Prompt Eng. Repo'
|
||||
|
||||
Parameters:
|
||||
NotebookName:
|
||||
Type: String
|
||||
Default: 'PromptEngWithAnthropicNotebook'
|
||||
DefaultRepoUrl:
|
||||
Type: String
|
||||
Default: 'https://github.com/aws-samples/prompt-engineering-with-anthropic-claude-v-3.git'
|
||||
|
||||
Resources:
|
||||
SageMakerExecutionRole:
|
||||
Type: AWS::IAM::Role
|
||||
Properties:
|
||||
AssumeRolePolicyDocument:
|
||||
Version: '2012-10-17'
|
||||
Statement:
|
||||
- Effect: Allow
|
||||
Principal:
|
||||
Service:
|
||||
- sagemaker.amazonaws.com
|
||||
Action:
|
||||
- sts:AssumeRole
|
||||
ManagedPolicyArns:
|
||||
- arn:aws:iam::aws:policy/AmazonSageMakerFullAccess
|
||||
- arn:aws:iam::aws:policy/AmazonBedrockFullAccess
|
||||
|
||||
KmsKey:
|
||||
Type: AWS::KMS::Key
|
||||
Properties:
|
||||
Description: 'KMS key for SageMaker notebook'
|
||||
KeyPolicy:
|
||||
Version: '2012-10-17'
|
||||
Statement:
|
||||
- Effect: Allow
|
||||
Principal:
|
||||
AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:root'
|
||||
Action: 'kms:*'
|
||||
Resource: '*'
|
||||
EnableKeyRotation: true
|
||||
|
||||
KmsKeyAlias:
|
||||
Type: AWS::KMS::Alias
|
||||
Properties:
|
||||
AliasName: !Sub 'alias/${NotebookName}-kms-key'
|
||||
TargetKeyId: !Ref KmsKey
|
||||
|
||||
SageMakerNotebookInstance:
|
||||
Type: AWS::SageMaker::NotebookInstance
|
||||
Properties:
|
||||
InstanceType: ml.t3.large
|
||||
NotebookInstanceName: !Ref NotebookName
|
||||
RoleArn: !GetAtt SageMakerExecutionRole.Arn
|
||||
DefaultCodeRepository: !Ref DefaultRepoUrl
|
||||
KmsKeyId: !GetAtt KmsKey.Arn
|
||||
|
||||
Outputs:
|
||||
NotebookInstanceName:
|
||||
Description: The name of the created SageMaker Notebook Instance
|
||||
Value: !Ref SageMakerNotebookInstance
|
||||
ExecutionRoleArn:
|
||||
Description: The ARN of the created SageMaker Execution Role
|
||||
Value: !GetAtt SageMakerExecutionRole.Arn
|
||||
KmsKeyArn:
|
||||
Description: The ARN of the created KMS Key for the notebook
|
||||
Value: !GetAtt KmsKey.Arn
|
||||
@@ -1,5 +0,0 @@
|
||||
awscli==1.32.74
|
||||
boto3==1.34.74
|
||||
botocore==1.34.74
|
||||
anthropic==0.21.3
|
||||
pickleshare==0.7.5
|
||||
@@ -1,246 +0,0 @@
|
||||
exercise_1_1_hint = """The grading function in this exercise is looking for an answer that contains the exact Arabic numerals "1", "2", and "3".
|
||||
You can often get Claude to do what you want simply by asking."""
|
||||
|
||||
exercise_1_2_hint = """The grading function in this exercise is looking for answers that contain "soo" or "giggles".
|
||||
There are many ways to solve this, just by asking!"""
|
||||
|
||||
exercise_2_1_hint ="""The grading function in this exercise is looking for any answer that includes the word "hola".
|
||||
Ask Claude to reply in Spanish like you would when speaking with a human. It's that simple!"""
|
||||
|
||||
exercise_2_2_hint = """The grading function in this exercise is looking for EXACTLY "Michael Jordan".
|
||||
How would you ask another human to do this? Reply with no other words? Reply with only the name and nothing else? There are several ways to approach this answer."""
|
||||
|
||||
exercise_2_3_hint = """The grading function in this cell is looking for a response that is equal to or greater than 800 words.
|
||||
Because LLMs aren't great at counting words yet, you may have to overshoot your target."""
|
||||
|
||||
exercise_3_1_hint = """The grading function in this exercise is looking for an answer that includes the words "incorrect" or "not correct".
|
||||
Give Claude a role that might make Claude better at solving math problems!"""
|
||||
|
||||
exercise_4_1_hint = """The grading function in this exercise is looking for a solution that includes the words "haiku" and "pig".
|
||||
Don't forget to include the exact phrase "{TOPIC}" wherever you want the topic to be substituted in. Changing the "TOPIC" variable value should make Claude write a haiku about a different topic."""
|
||||
|
||||
exercise_4_2_hint = """The grading function in this exercise is looking for a response that includes the word "brown".
|
||||
If you surround "{QUESTION}" in XML tags, how does that change Claude's response?"""
|
||||
|
||||
exercise_4_3_hint = """The grading function in this exercise is looking for a response that includes the word "brown".
|
||||
Try removing one word or section of characters at a time, starting with the parts that make the least sense. Doing this one word at a time will also help you see just how much Claude can or can't parse and understand."""
|
||||
|
||||
exercise_5_1_hint = """The grading function for this exercise is looking for a response that includes the word "Warrior".
|
||||
Write more words in Claude's voice to steer Claude to act the way you want it to. For instance, instead of "Stephen Curry is the best because," you could write "Stephen Curry is the best and here are three reasons why. 1:"""
|
||||
|
||||
exercise_5_2_hint = """The grading function looks for a response of over 5 lines in length that includes the words "cat" and "<haiku>".
|
||||
Start simple. Currently, the prompt asks Claude for one haiku. You can change that and ask for two (or even more). Then if you run into formatting issues, change your prompt to fix that after you've already gotten Claude to write more than one haiku."""
|
||||
|
||||
exercise_5_3_hint = """The grading function in this exercise is looking for a response that contains the words "tail", "cat", and "<haiku>".
|
||||
It's helpful to break this exercise down to several steps.
|
||||
1. Modify the initial prompt template so that Claude writes two poems.
|
||||
2. Give Claude indicators as to what the poems will be about, but instead of writing in the subjects directly (e.g., dog, cat, etc.), replace those subjects with the keywords "{ANIMAL1}" and "{ANIMAL2}".
|
||||
3. Run the prompt and make sure that the full prompt with variable substitutions has all the words correctly substituted. If not, check to make sure your {bracket} tags are spelled correctly and formatted correctly with single moustache brackets."""
|
||||
|
||||
exercise_6_1_hint = """The grading function in this exercise is looking for the correct categorization letter + the closing parentheses and the first letter of the name of the category, such as "C) B" or "B) B" etc.
|
||||
Let's take this exercise step by step:
|
||||
1. How will Claude know what categories you want to use? Tell it! Include the four categories you want directly in the prompt. Be sure to include the parenthetical letters as well for easy classification. Feel free to use XML tags to organize your prompt and make clear to Claude where the categories begin and end.
|
||||
2. Try to cut down on superfluous text so that Claude immediately answers with the classification and ONLY the classification. There are several ways to do this, from speaking for Claude (providing anything from the beginning of the sentence to a single open parenthesis so that Claude knows you want the parenthetical letter as the first part of the answer) to telling Claude that you want the classification and only the classification, skipping the preamble.
|
||||
Refer to Chapters 2 and 5 if you want a refresher on these techniques.
|
||||
3. Claude may still be incorrectly categorizing or not including the names of the categories when it answers. Fix this by telling Claude to include the full category name in its answer.)
|
||||
4. Be sure that you still have {email} somewhere in your prompt template so that we can properly substitute in emails for Claude to evaluate."""
|
||||
|
||||
exercise_6_1_solution = """
|
||||
USER TURN
|
||||
Please classify this email into the following categories: {email}
|
||||
|
||||
Do not include any extra words except the category.
|
||||
|
||||
<categories>
|
||||
(A) Pre-sale question
|
||||
(B) Broken or defective item
|
||||
(C) Billing question
|
||||
(D) Other (please explain)
|
||||
</categories>
|
||||
|
||||
ASSISTANT TURN
|
||||
(
|
||||
"""
|
||||
|
||||
exercise_6_2_hint = """The grading function in this exercise is looking for only the correct letter wrapped in <answer> tags, such as "<answer>B</answer>". The correct categorization letters are the same as in the above exercise.
|
||||
Sometimes the simplest way to go about this is to give Claude an example of how you want its output to look. Just don't forget to wrap your example in <example></example> tags! And don't forget that if you prefill Claude's response with anything, Claude won't actually output that as part of its response."""
|
||||
|
||||
exercise_7_1_hint = """You're going to have to write some example emails and classify them for Claude (with the exact formatting you want). There are multiple ways to do this. Here are some guidelines below.
|
||||
1. Try to have at least two example emails. Claude doesn't need an example for all categories, and the examples don't have to be long. It's more helpful to have examples for whatever you think the trickier categories are (which you were asked to think about at the bottom of Chapter 6 Exercise 1). XML tags will help you separate out your examples from the rest of your prompt, although it's unnecessary.
|
||||
2. Make sure your example answer formatting is exactly the format you want Claude to use, so Claude can emulate the format as well. This format should make it so that Claude's answer ends in the letter of the category. Wherever you put the {email} placeholder, make sure that it's formatted exactly like your example emails.
|
||||
3. Make sure you still have the categories listed within the prompt itself, otherwise Claude won't know what categories to reference, as well as {email} as a placeholder for substitution."""
|
||||
|
||||
exercise_7_1_solution = """
|
||||
USER TURN
|
||||
Please classify emails into the following categories, and do not include explanations:
|
||||
<categories>
|
||||
(A) Pre-sale question
|
||||
(B) Broken or defective item
|
||||
(C) Billing question
|
||||
(D) Other (please explain)
|
||||
</categories>
|
||||
|
||||
Here are a few examples of correct answer formatting:
|
||||
<examples>
|
||||
Q: How much does it cost to buy a Mixmaster4000?
|
||||
A: The correct category is: A
|
||||
|
||||
Q: My Mixmaster won't turn on.
|
||||
A: The correct category is: B
|
||||
|
||||
Q: Please remove me from your mailing list.
|
||||
A: The correct category is: D
|
||||
</examples>
|
||||
|
||||
Here is the email for you to categorize: {email}
|
||||
|
||||
ASSISTANT TURN
|
||||
The correct category is:
|
||||
"""
|
||||
exercise_8_1_hint = """The grading function in this exercise is looking for a response that contains the phrase "I do not", "I don't", or "Unfortunately".
|
||||
What should Claude do if it doesn't know the answer?"""
|
||||
|
||||
exercise_8_2_hint = """The grading function in this exercise is looking for a response that contains the phrase "49-fold".
|
||||
Make Claude show its work and thought process first by extracting relevant quotes and seeing whether or not the quotes provide sufficient evidence. Refer back to the Chapter 8 Lesson if you want a refresher."""
|
||||
|
||||
exercise_9_1_solution = """
|
||||
You are a master tax acountant. Your task is to answer user questions using any provided reference documentation.
|
||||
|
||||
Here is the material you should use to answer the user's question:
|
||||
<docs>
|
||||
{TAX_CODE}
|
||||
</docs>
|
||||
|
||||
Here is an example of how to respond:
|
||||
<example>
|
||||
<question>
|
||||
What defines a "qualified" employee?
|
||||
</question>
|
||||
<answer>
|
||||
<quotes>For purposes of this subsection—
|
||||
(A)In general
|
||||
The term "qualified employee" means any individual who—
|
||||
(i)is not an excluded employee, and
|
||||
(ii)agrees in the election made under this subsection to meet such requirements as are determined by the Secretary to be necessary to ensure that the withholding requirements of the corporation under chapter 24 with respect to the qualified stock are met.</quotes>
|
||||
|
||||
<answer>According to the provided documentation, a "qualified employee" is defined as an individual who:
|
||||
|
||||
1. Is not an "excluded employee" as defined in the documentation.
|
||||
2. Agrees to meet the requirements determined by the Secretary to ensure the corporation's withholding requirements under Chapter 24 are met with respect to the qualified stock.</answer>
|
||||
</example>
|
||||
|
||||
First, gather quotes in <quotes></quotes> tags that are relevant to answering the user's question. If there are no quotes, write "no relevant quotes found".
|
||||
|
||||
Then insert two paragraph breaks before answering the user question within <answer></answer> tags. Only answer the user's question if you are confident that the quotes in <quotes></quotes> tags support your answer. If not, tell the user that you unfortunately do not have enough information to answer the user's question.
|
||||
|
||||
Here is the user question: {QUESTION}
|
||||
"""
|
||||
|
||||
exercise_9_2_solution = """
|
||||
You are Codebot, a helpful AI assistant who finds issues with code and suggests possible improvements.
|
||||
|
||||
Act as a Socratic tutor who helps the user learn.
|
||||
|
||||
You will be given some code from a user. Please do the following:
|
||||
1. Identify any issues in the code. Put each issue inside separate <issues> tags.
|
||||
2. Invite the user to write a revised version of the code to fix the issue.
|
||||
|
||||
Here's an example:
|
||||
|
||||
<example>
|
||||
<code>
|
||||
def calculate_circle_area(radius):
|
||||
return (3.14 * radius) ** 2
|
||||
</code>
|
||||
<issues>
|
||||
<issue>
|
||||
3.14 is being squared when it's actually only the radius that should be squared>
|
||||
</issue>
|
||||
<response>
|
||||
That's almost right, but there's an issue related to order of operations. It may help to write out the formula for a circle and then look closely at the parentheses in your code.
|
||||
</response>
|
||||
</example>
|
||||
|
||||
Here is the code you are to analyze:
|
||||
|
||||
<code>
|
||||
{CODE}
|
||||
</code>
|
||||
|
||||
Find the relevant issues and write the Socratic tutor-style response. Do not give the user too much help! Instead, just give them guidance so they can find the correct solution themselves.
|
||||
|
||||
Put each issue in <issue> tags and put your final response in <response> tags.
|
||||
"""
|
||||
|
||||
exercise_10_2_1_solution = """system_prompt = system_prompt_tools_general_explanation + \"""Here are the functions available in JSONSchema format:
|
||||
|
||||
<tools>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>get_user</tool_name>
|
||||
<description>
|
||||
Retrieves a user from the database by their user ID.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>user_id</name>
|
||||
<type>int</type>
|
||||
<description>The ID of the user to retrieve.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>get_product</tool_name>
|
||||
<description>
|
||||
Retrieves a product from the database by its product ID.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>product_id</name>
|
||||
<type>int</type>
|
||||
<description>The ID of the product to retrieve.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>add_user</tool_name>
|
||||
<description>
|
||||
Adds a new user to the database.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>name</name>
|
||||
<type>str</type>
|
||||
<description>The name of the user.</description>
|
||||
</parameter>
|
||||
<parameter>
|
||||
<name>email</name>
|
||||
<type>str</type>
|
||||
<description>The email address of the user.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>add_product</tool_name>
|
||||
<description>
|
||||
Adds a new product to the database.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>name</name>
|
||||
<type>str</type>
|
||||
<description>The name of the product.</description>
|
||||
</parameter>
|
||||
<parameter>
|
||||
<name>price</name>
|
||||
<type>float</type>
|
||||
<description>The price of the product.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
</tools>
|
||||
"""
|
||||
@@ -1,154 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Tutorial How-To\n",
|
||||
"\n",
|
||||
"This tutorial **requires an API key** for interaction. If you don't have an API key, you can sign up for one via the [Anthropic Console](https://console.anthropic.com/) or view our [static tutorial answer key](https://docs.google.com/spreadsheets/u/0/d/1jIxjzUWG-6xBVIa2ay6yDpLyeuOh_hR_ZB75a47KX_E/edit) instead."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## How to get started\n",
|
||||
"\n",
|
||||
"1. Clone this repository to your local machine.\n",
|
||||
"\n",
|
||||
"2. Install the required dependencies by running the following command:\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"3. Set up your API key and model name. Replace `\"your_api_key_here\"` with your actual Anthropic API key."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"API_KEY = \"your_api_key_here\"\n",
|
||||
"MODEL_NAME = \"claude-3-haiku-20240307\"\n",
|
||||
"\n",
|
||||
"# Stores the API_KEY & MODEL_NAME variables for use across notebooks within the IPython store\n",
|
||||
"%store API_KEY\n",
|
||||
"%store MODEL_NAME"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"4. Run the notebook cells in order, following the instructions provided."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Usage Notes & Tips 💡\n",
|
||||
"\n",
|
||||
"- This course uses Claude 3 Haiku with temperature 0. We will talk more about temperature later in the course. For now, it's enough to understand that these settings yield more deterministic results. All prompt engineering techniques in this course also apply to previous generation legacy Claude models such as Claude 2 and Claude Instant 1.2.\n",
|
||||
"\n",
|
||||
"- You can use `Shift + Enter` to execute the cell and move to the next one.\n",
|
||||
"\n",
|
||||
"- When you reach the bottom of a tutorial page, navigate to the next numbered file in the folder, or to the next numbered folder if you're finished with the content within that chapter file.\n",
|
||||
"\n",
|
||||
"### The Anthropic SDK & the Messages API\n",
|
||||
"We will be using the [Anthropic python SDK](https://docs.anthropic.com/claude/reference/client-sdks) and the [Messages API](https://docs.anthropic.com/claude/reference/messages_post) throughout this tutorial. \n",
|
||||
"\n",
|
||||
"Below is an example of what running a prompt will look like in this tutorial. First, we create `get_completion`, which is a helper function that sends a prompt to Claude and returns Claude's generated response. Run that cell now."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now we will write out an example prompt for Claude and print Claude's output by running our `get_completion` helper function. Running the cell below will print out a response from Claude beneath it.\n",
|
||||
"\n",
|
||||
"Feel free to play around with the prompt string to elicit different responses from Claude."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"prompt = \"Hello, Claude!\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"print(get_completion(prompt))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `API_KEY` and `MODEL_NAME` variables defined earlier will be used throughout the tutorial. Just make sure to run the cells for each tutorial page from top to bottom."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "py310",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.14"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,459 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 1: Basic Prompt Structure\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Anthropic offers two APIs, the legacy [Text Completions API](https://docs.anthropic.com/claude/reference/complete_post) and the current [Messages API](https://docs.anthropic.com/claude/reference/messages_post). For this tutorial, we will be exclusively using the Messages API.\n",
|
||||
"\n",
|
||||
"At minimum, a call to Claude using the Messages API requires the following parameters:\n",
|
||||
"- `model`: the [API model name](https://docs.anthropic.com/claude/docs/models-overview#model-recommendations) of the model that you intend to call\n",
|
||||
"\n",
|
||||
"- `max_tokens`: the maximum number of tokens to generate before stopping. Note that Claude may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. Furthermore, this is a *hard* stop, meaning that it may cause Claude to stop generating mid-word or mid-sentence.\n",
|
||||
"\n",
|
||||
"- `messages`: an array of input messages. Our models are trained to operate on alternating `user` and `assistant` conversational turns. When creating a new `Message`, you specify the prior conversational turns with the messages parameter, and the model then generates the next `Message` in the conversation.\n",
|
||||
" - Each input message must be an object with a `role` and `content`. You can specify a single `user`-role message, or you can include multiple `user` and `assistant` messages (they must alternate, if so). The first message must always use the user `role`.\n",
|
||||
"\n",
|
||||
"There are also optional parameters, such as:\n",
|
||||
"- `system`: the system prompt - more on this below.\n",
|
||||
" \n",
|
||||
"- `temperature`: the degree of variability in Claude's response. For these lessons and exercises, we have set `temperature` to 0.\n",
|
||||
"\n",
|
||||
"For a complete list of all API parameters, visit our [API documentation](https://docs.anthropic.com/claude/reference/messages_post)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Let's take a look at how Claude responds to some correctly-formatted prompts. For each of the following cells, run the cell (`shift+enter`), and Claude's response will appear below the block."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hi Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Can you tell me the color of the ocean?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"What year was Celine Dion born in?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's take a look at some prompts that do not include the correct Messages API formatting. For these malformatted prompts, the Messages API returns an error.\n",
|
||||
"\n",
|
||||
"First, we have an example of a Messages API call that lacks `role` and `content` fields in the `messages` array."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"Hi Claude, how are you?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's a prompt that fails to alternate between the `user` and `assistant` roles."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": \"What year was Celine Dion born in?\"},\n",
|
||||
" {\"role\": \"user\", \"content\": \"Also, can you tell me some other facts about her?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`user` and `assistant` messages **MUST alternate**, and messages **MUST start with a `user` turn**. You can have multiple `user` & `assistant` pairs in a prompt (as if simulating a multi-turn conversation). You can also put words into a terminal `assistant` message for Claude to continue from where you left off (more on that in later chapters).\n",
|
||||
"\n",
|
||||
"#### System Prompts\n",
|
||||
"\n",
|
||||
"You can also use **system prompts**. A system prompt is a way to **provide context, instructions, and guidelines to Claude** before presenting it with a question or task in the \"User\" turn. \n",
|
||||
"\n",
|
||||
"Structurally, system prompts exist separately from the list of `user` & `assistant` messages, and thus belong in a separate `system` parameter (take a look at the structure of the `get_completion` helper function in the [Setup](#setup) section of the notebook). \n",
|
||||
"\n",
|
||||
"Within this tutorial, wherever we might utilize a system prompt, we have provided you a `system` field in your completions function. Should you not want to use a system prompt, simply set the `SYSTEM_PROMPT` variable to an empty string."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### System Prompt Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Why is the sky blue?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why use a system prompt? A **well-written system prompt can improve Claude's performance** in a variety of ways, such as increasing Claude's ability to follow rules and instructions. For more information, visit our documentation on [how to use system prompts](https://docs.anthropic.com/claude/docs/how-to-use-system-prompts) with Claude.\n",
|
||||
"\n",
|
||||
"Now we'll dive into some exercises. If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 1.1 - Counting to Three](#exercise-11---counting-to-three)\n",
|
||||
"- [Exercise 1.2 - System Prompt](#exercise-12---system-prompt)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 1.1 - Counting to Three\n",
|
||||
"Using proper `user` / `assistant` formatting, edit the `PROMPT` below to get Claude to **count to three.** The output will also indicate whether your solution is correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" pattern = re.compile(r'^(?=.*1)(?=.*2)(?=.*3).*$', re.DOTALL)\n",
|
||||
" return bool(pattern.match(text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_1_1_hint; print(exercise_1_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 1.2 - System Prompt\n",
|
||||
"\n",
|
||||
"Modify the `SYSTEM_PROMPT` to make Claude respond like it's a 3 year old child."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - this is the only field you should change\n",
|
||||
"SYSTEM_PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"How big is the sky?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(r\"giggles\", text) or re.search(r\"soo\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_1_2_hint; print(exercise_1_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hi Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Can you tell me the color of the ocean?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"What year was Celine Dion born in?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"Hi Claude, how are you?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Get Claude's response\n",
|
||||
"response = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": \"What year was Celine Dion born in?\"},\n",
|
||||
" {\"role\": \"user\", \"content\": \"Also, can you tell me some other facts about her?\"}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(response[0].text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Why is the sky blue?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,394 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 2: Being Clear and Direct\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"# Note that we changed max_tokens to 4K just for this lesson to allow for longer completions in the exercises\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=4000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Claude responds best to clear and direct instructions.**\n",
|
||||
"\n",
|
||||
"Think of Claude like any other human that is new to the job. **Claude has no context** on what to do aside from what you literally tell it. Just as when you instruct a human for the first time on a task, the more you explain exactly what you want in a straightforward manner to Claude, the better and more accurate Claude's response will be.\"\t\t\t\t\n",
|
||||
"\t\t\t\t\n",
|
||||
"When in doubt, follow the **Golden Rule of Clear Prompting**:\n",
|
||||
"- Show your prompt to a colleague or friend and have them follow the instructions themselves to see if they can produce the result you want. If they're confused, Claude's confused.\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Let's take a task like writing poetry. (Ignore any syllable mismatch - LLMs aren't great at counting syllables yet.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This haiku is nice enough, but users may want Claude to go directly into the poem without the \"Here is a haiku\" preamble.\n",
|
||||
"\n",
|
||||
"How do we achieve that? We **ask for it**!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots. Skip the preamble; go straight into the poem.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's another example. Let's ask Claude who's the best basketball player of all time. You can see below that while Claude lists a few names, **it doesn't respond with a definitive \"best\"**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Can we get Claude to make up its mind and decide on a best player? Yes! Just ask!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time? Yes, there are differing opinions, but if you absolutely had to pick one player, who would it be?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 2.1 - Spanish](#exercise-21---spanish)\n",
|
||||
"- [Exercise 2.2 - One Player Only](#exercise-22---one-player-only)\n",
|
||||
"- [Exercise 2.3 - Write a Story](#exercise-23---write-a-story)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.1 - Spanish\n",
|
||||
"Modify the `SYSTEM_PROMPT` to make Claude output its answer in Spanish."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - this is the only field you should chnage\n",
|
||||
"SYSTEM_PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Hello Claude, how are you?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return \"hola\" in text.lower()\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_2_1_hint; print(exercise_2_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.2 - One Player Only\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` so that Claude doesn't equivocate at all and responds with **ONLY** the name of one specific player, with **no other words or punctuation**. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return text == \"Michael Jordan\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_2_2_hint; print(exercise_2_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 2.3 - Write a Story\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` so that Claude responds with as long a response as you can muster. If your answer is **over 800 words**, Claude's response will be graded as correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt - this is the only field you should change\n",
|
||||
"PROMPT = \"[Replace this text]\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" trimmed = text.strip()\n",
|
||||
" words = len(trimmed.split())\n",
|
||||
" return words >= 800\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_2_3_hint; print(exercise_2_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Write a haiku about robots. Skip the preamble; go straight into the poem.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the best basketball player of all time? Yes, there are differing opinions, but if you absolutely had to pick one player, who would it be?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,325 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 3: Assigning Roles (Role Prompting)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Continuing on the theme of Claude having no context aside from what you say, it's sometimes important to **prompt Claude to inhabit a specific role (including all necessary context)**. This is also known as role prompting. The more detail to the role context, the better.\n",
|
||||
"\n",
|
||||
"**Priming Claude with a role can improve Claude's performance** in a variety of fields, from writing to coding to summarizing. It's like how humans can sometimes be helped when told to \"think like a ______\". Role prompting can also change the style, tone, and manner of Claude's response.\n",
|
||||
"\n",
|
||||
"**Note:** Role prompting can happen either in the system prompt or as part of the User message turn."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In the example below, we see that without role prompting, Claude provides a **straightforward and non-stylized answer** when asked to give a single sentence perspective on skateboarding.\n",
|
||||
"\n",
|
||||
"However, when we prime Claude to inhabit the role of a cat, Claude's perspective changes, and thus **Claude's response tone, style, content adapts to the new role**. \n",
|
||||
"\n",
|
||||
"**Note:** A bonus technique you can use is to **provide Claude context on its intended audience**. Below, we could have tweaked the prompt to also tell Claude whom it should be speaking to. \"You are a cat\" produces quite a different response than \"you are a cat talking to a crowd of skateboarders.\n",
|
||||
"\n",
|
||||
"Here is the prompt without role prompting in the system prompt:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here is the same user question, except with role prompting."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a cat.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can use role prompting as a way to get Claude to emulate certain styles in writing, speak in a certain voice, or guide the complexity of its answers. **Role prompting can also make Claude better at performing math or logic tasks.**\n",
|
||||
"\n",
|
||||
"For example, in the example below, there is a definitive correct answer, which is yes. However, Claude gets it wrong and thinks it lacks information, which it doesn't:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, what if we **prime Claude to act as a logic bot**? How will that change Claude's answer? \n",
|
||||
"\n",
|
||||
"It turns out that with this new role assignment, Claude gets it right. (Although notably not for all the right reasons)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a logic bot designed to answer complex logic problems.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Note:** What you'll learn throughout this course is that there are **many prompt engineering techniques you can use to derive similar results**. Which techniques you use is up to you and your preference! We encourage you to **experiment to find your own prompt engineering style**.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 3.1 - Math Correction](#exercise-31---math-correction)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 3.1 - Math Correction\n",
|
||||
"In some instances, **Claude may struggle with mathematics**, even simple mathematics. Below, Claude incorrectly assesses the math problem as correctly solved, even though there's an obvious arithmetic mistake in the second step. Note that Claude actually catches the mistake when going through step-by-step, but doesn't jump to the conclusion that the overall solution is wrong.\n",
|
||||
"\n",
|
||||
"Modify the `PROMPT` and / or the `SYSTEM_PROMPT` to make Claude grade the solution as `incorrectly` solved, rather than correctly solved. \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt - if you don't want to use a system prompt, you can leave this variable set to an empty string\n",
|
||||
"SYSTEM_PROMPT = \"\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this equation solved correctly below?\n",
|
||||
"\n",
|
||||
"2x - 3 = 9\n",
|
||||
"2x = 6\n",
|
||||
"x = 3\"\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" if \"incorrect\" in text or \"not correct\" in text.lower():\n",
|
||||
" return True\n",
|
||||
" else:\n",
|
||||
" return False\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n--------------------------- GRADING ---------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_3_1_hint; print(exercise_3_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a cat.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In one sentence, what do you think about skateboarding?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a logic bot designed to answer complex logic problems.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Jack is looking at Anne. Anne is looking at George. Jack is married, George is not, and we don’t know if Anne is married. Is a married person looking at an unmarried person?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,552 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 4: Separating Data and Instructions\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Oftentimes, we don't want to write full prompts, but instead want **prompt templates that can be modified later with additional input data before submitting to Claude**. This might come in handy if you want Claude to do the same thing every time, but the data that Claude uses for its task might be different each time. \n",
|
||||
"\n",
|
||||
"Luckily, we can do this pretty easily by **separating the fixed skeleton of the prompt from variable user input, then substituting the user input into the prompt** before sending the full prompt to Claude. \n",
|
||||
"\n",
|
||||
"Below, we'll walk step by step through how to write a substitutable prompt template, as well as how to substitute in user input."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In this first example, we're asking Claude to act as an animal noise generator. Notice that the full prompt submitted to Claude is just the `PROMPT_TEMPLATE` substituted with the input (in this case, \"Cow\"). Notice that the word \"Cow\" replaces the `ANIMAL` placeholder via an f-string when we print out the full prompt.\n",
|
||||
"\n",
|
||||
"**Note:** You don't have to call your placeholder variable anything in particular in practice. We called it `ANIMAL` in this example, but just as easily, we could have called it `CREATURE` or `A` (although it's generally good to have your variable names be specific and relevant so that your prompt template is easy to understand even without the substitution, just for user parseability). Just make sure that whatever you name your variable is what you use for the prompt template f-string."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cow\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why would we want to separate and substitute inputs like this? Well, **prompt templates simplify repetitive tasks**. Let's say you build a prompt structure that invites third party users to submit content to the prompt (in this case the animal whose sound they want to generate). These third party users don't have to write or even see the full prompt. All they have to do is fill in variables.\n",
|
||||
"\n",
|
||||
"We do this substitution here using variables and f-strings, but you can also do it with the format() method.\n",
|
||||
"\n",
|
||||
"**Note:** Prompt templates can have as many variables as desired!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When introducing substitution variables like this, it is very important to **make sure Claude knows where variables start and end** (vs. instructions or task descriptions). Let's look at an example where there is no separation between the instructions and the substitution variable.\n",
|
||||
"\n",
|
||||
"To our human eyes, it is very clear where the variable begins and ends in the prompt template below. However, in the fully substituted prompt, that delineation becomes unclear."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here, **Claude thinks \"Yo Claude\" is part of the email it's supposed to rewrite**! You can tell because it begins its rewrite with \"Dear Claude\". To the human eye, it's clear, particularly in the prompt template where the email begins and ends, but it becomes much less clear in the prompt after substitution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"How do we solve this? **Wrap the input in XML tags**! We did this below, and as you can see, there's no more \"Dear Claude\" in the output.\n",
|
||||
"\n",
|
||||
"[XML tags](https://docs.anthropic.com/claude/docs/use-xml-tags) are angle-bracket tags like `<tag></tag>`. They come in pairs and consist of an opening tag, such as `<tag>`, and a closing tag marked by a `/`, such as `</tag>`. XML tags are used to wrap around content, like this: `<tag>content</tag>`.\n",
|
||||
"\n",
|
||||
"**Note:** While Claude can recognize and work with a wide range of separators and delimeters, we recommend that you **use specifically XML tags as separators** for Claude, as Claude was trained specifically to recognize XML tags as a prompt organizing mechanism. Outside of function calling, **there are no special sauce XML tags that Claude has been trained on that you should use to maximally boost your performance**. We have purposefully made Claude very malleable and customizable this way."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's see another example of how XML tags can help us. \n",
|
||||
"\n",
|
||||
"In the following prompt, **Claude incorrectly interprets what part of the prompt is the instruction vs. the input**. It incorrectly considers `Each is about an animal, like rabbits` to be part of the list due to the formatting, when the user (the one filling out the `SENTENCES` variable) presumably did not want that."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"{SENTENCES}\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To fix this, we just need to **surround the user input sentences in XML tags**. This shows Claude where the input data begins and ends despite the misleading hyphen before `Each is about an animal, like rabbits.`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"<sentences>\n",
|
||||
"{SENTENCES}\n",
|
||||
"</sentences>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Note:** In the incorrect version of the \"Each is about an animal\" prompt, we had to include the hyphen to get Claude to respond incorrectly in the way we wanted to for this example. This is an important lesson about prompting: **small details matter**! It's always worth it to **scrub your prompts for typos and grammatical errors**. Claude is sensitive to patterns (in its early years, before finetuning, it was a raw text-prediction tool), and it's more likely to make mistakes when you make mistakes, smarter when you sound smart, sillier when you sound silly, and so on.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 4.1 - Haiku Topic](#exercise-41---haiku-topic)\n",
|
||||
"- [Exercise 4.2 - Dog Question with Typos](#exercise-42---dog-question-with-typos)\n",
|
||||
"- [Exercise 4.3 - Dog Question Part 2](#exercise-42---dog-question-part-2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.1 - Haiku Topic\n",
|
||||
"Modify the `PROMPT` so that it's a template that will take in a variable called `TOPIC` and output a haiku about the topic. This exercise is just meant to test your understanding of the variable templating structure with f-strings."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"TOPIC = \"Pigs\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"pigs\", text.lower()) and re.search(\"haiku\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_4_1_hint; print(exercise_4_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.2 - Dog Question with Typos\n",
|
||||
"Fix the `PROMPT` by adding XML tags so that Claude produces the right answer. \n",
|
||||
"\n",
|
||||
"Try not to change anything else about the prompt. The messy and mistake-ridden writing is intentional, so you can see how Claude reacts to such mistakes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"QUESTION = \"ar cn brown?\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"brown\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_4_2_hint; print(exercise_4_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 4.3 - Dog Question Part 2\n",
|
||||
"Fix the `PROMPT` **WITHOUT** adding XML tags. Instead, remove only one or two words from the prompt.\n",
|
||||
"\n",
|
||||
"Just as with the above exercises, try not to change anything else about the prompt. This will show you what kind of language Claude can parse and understand."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"QUESTION = \"ar cn brown?\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hia its me i have a q about dogs jkaerjv {QUESTION} jklmvca tx it help me muhch much atx fst fst answer short short tx\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"brown\", text.lower()))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_4_3_hint; print(exercise_4_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cow\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"I will tell you the name of an animal. Please respond with the noise that animal makes. {ANIMAL}\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. {EMAIL} <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"EMAIL = \"Show up at 6am tomorrow because I'm the CEO and I say so.\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Yo Claude. <email>{EMAIL}</email> <----- Make this email more polite but don't change anything else about it.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\"Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"{SENTENCES}\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"SENTENCES = \"\"\"- I like how cows sound\n",
|
||||
"- This sentence is about spiders\n",
|
||||
"- This sentence may appear to be about dogs but it's actually about pigs\"\"\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"\"\" Below is a list of sentences. Tell me the second item on the list.\n",
|
||||
"\n",
|
||||
"- Each is about an animal, like rabbits.\n",
|
||||
"<sentences>\n",
|
||||
"{SENTENCES}\n",
|
||||
"</sentences>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,517 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 5: Formatting Output and Speaking for Claude\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"# New argument added for prefill text, with a default value of an empty string\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\", prefill=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Claude can format its output in a wide variety of ways**. You just need to ask for it to do so!\n",
|
||||
"\n",
|
||||
"One of these ways is by using XML tags to separate out the response from any other superfluous text. You've already learned that you can use XML tags to make your prompt clearer and more parseable to Claude. It turns out, you can also ask Claude to **use XML tags to make its output clearer and more easily understandable** to humans."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Remember the 'poem preamble problem' we solved in Chapter 2 by asking Claude to skip the preamble entirely? It turns out we can also achieve a similar outcome by **telling Claude to put the poem in XML tags**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Rabbit\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Why is this something we'd want to do? Well, having the output in **XML tags allows the end user to reliably get the poem and only the poem by writing a short program to extract the content between XML tags**.\n",
|
||||
"\n",
|
||||
"An extension of this technique is to **put the first XML tag in the `assistant` turn. When you put text in the `assistant` turn, you're basically telling Claude that Claude has already said something, and that it should continue from that point onward. This technique is called \"speaking for Claude\" or \"prefilling Claude's response.\"\n",
|
||||
"\n",
|
||||
"Below, we've done this with the first `<haiku>` XML tag. Notice how Claude continues directly from where we left off."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Claude also excels at using other output formatting styles, notably `JSON`. If you want to enforce JSON output (not deterministically, but close to it), you can also prefill Claude's response with the opening bracket, `{`}."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"{\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Below is an example of **multiple input variables in the same prompt AND output formatting specification, all done using XML tags**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ADJECTIVE = \"olde english\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response (now as an f-string with a variable)\n",
|
||||
"PREFILL = f\"<{ADJECTIVE}_email>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Bonus lesson\n",
|
||||
"\n",
|
||||
"If you are calling Claude through the API, you can pass the closing XML tag to the `stop_sequences` parameter to get Claude to stop sampling once it emits your desired tag. This can save money and time-to-last-token by eliminating Claude's concluding remarks after it's already given you the answer you care about.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 5.1 - Steph Curry GOAT](#exercise-51---steph-curry-goat)\n",
|
||||
"- [Exercise 5.2 - Two Haikus](#exercise-52---two-haikus)\n",
|
||||
"- [Exercise 5.3 - Two Haikus, Two Animals](#exercise-53---two-haikus-two-animals)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.1 - Steph Curry GOAT\n",
|
||||
"Forced to make a choice, Claude designates Michael Jordan as the best basketball player of all time. Can we get Claude to pick someone else?\n",
|
||||
"\n",
|
||||
"Change the `PREFILL` variable to **compell Claude to make a detailed argument that the best basketball player of all time is Stephen Curry**. Try not to change anything except `PREFILL` as that is the focus of this exercise."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Who is the best basketball player of all time? Please choose one specific player.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"Warrior\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_5_1_hint; print(exercise_5_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.2 - Two Haikus\n",
|
||||
"Modify the `PROMPT` below using XML tags so that Claude writes two haikus about the animal instead of just one. It should be clear where one poem ends and the other begins."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"cats\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(\n",
|
||||
" (re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
|
||||
" and (text.count(\"\\n\") + 1) > 5\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_5_2_hint; print(exercise_5_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 5.3 - Two Haikus, Two Animals\n",
|
||||
"Modify the `PROMPT` below so that **Claude produces two haikus about two different animals**. Use `{ANIMAL1}` as a stand-in for the first substitution, and `{ANIMAL2}` as a stand-in for the second substitution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"ANIMAL1 = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ANIMAL2 = \"Dog\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL1}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"tail\", text.lower()) and re.search(\"cat\", text.lower()) and re.search(\"<haiku>\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_5_3_hint; print(exercise_5_3_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Rabbit\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Put it in <haiku> tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<haiku>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Variable content\n",
|
||||
"ANIMAL = \"Cat\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Please write a haiku about {ANIMAL}. Use JSON format with the keys as \\\"first_line\\\", \\\"second_line\\\", and \\\"third_line\\\".\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"{\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First input variable\n",
|
||||
"EMAIL = \"Hi Zack, just pinging you for a quick update on that prompt you were supposed to write.\"\n",
|
||||
"\n",
|
||||
"# Second input variable\n",
|
||||
"ADJECTIVE = \"olde english\"\n",
|
||||
"\n",
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = f\"Hey Claude. Here is an email: <email>{EMAIL}</email>. Make this email more {ADJECTIVE}. Write the new version in <{ADJECTIVE}_email> XML tags.\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response (now as an f-string with a variable)\n",
|
||||
"PREFILL = f\"<{ADJECTIVE}_email>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,496 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 6: Precognition (Thinking Step by Step)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\", prefill=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"If someone woke you up and immediately started asking you several complicated questions that you had to respond to right away, how would you do? Probably not as good as if you were given time to **think through your answer first**. \n",
|
||||
"\n",
|
||||
"Guess what? Claude is the same way.\n",
|
||||
"\n",
|
||||
"**Giving Claude time to think step by step sometimes makes Claude more accurate**, particularly for complex tasks. However, **thinking only counts when it's out loud**. You cannot ask Claude to think but output only the answer - in this case, no thinking has actually occurred."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In the prompt below, it's clear to a human reader that the second sentence belies the first. But **Claude takes the word \"unrelated\" too literally**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this movie review sentiment positive or negative?\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since the year 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To improve Claude's response, let's **allow Claude to think things out first before answering**. We do that by literally spelling out the steps that Claude should take in order to process and think through its task. Along with a dash of role prompting, this empowers Claude to understand the review more deeply."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a savvy reader of movie reviews.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment positive or negative? First, write the best arguments for each side in <positive-argument> and <negative-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Claude is sometimes sensitive to ordering**. This example is on the frontier of Claude's ability to understand nuanced text, and when we swap the order of the arguments from the previous example so that negative is first and positive is second, this changes Claude's overall assessment to positive.\n",
|
||||
"\n",
|
||||
"In most situations (but not all, confusingly enough), **Claude is more likely to choose the second of two options**, possibly because in its training data from the web, second options were more likely to be correct."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment negative or positive? First write the best arguments for each side in <negative-argument> and <positive-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. Unrelatedly, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Letting Claude think can shift Claude's answer from incorrect to correct**. It's that simple in many cases where Claude makes mistakes!\n",
|
||||
"\n",
|
||||
"Let's go through an example where Claude's answer is incorrect to see how asking Claude to think can fix that."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's fix this by asking Claude to think step by step, this time in `<brainstorm>` tags."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 6.1 - Classifying Emails](#exercise-61---classifying-emails)\n",
|
||||
"- [Exercise 6.2 - Email Classification Formatting](#exercise-62---email-classification-formatting)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 6.1 - Classifying Emails\n",
|
||||
"In this exercise, we'll be instructing Claude to sort emails into the following categories:\t\t\t\t\t\t\t\t\t\t\n",
|
||||
"- (A) Pre-sale question\n",
|
||||
"- (B) Broken or defective item\n",
|
||||
"- (C) Billing question\n",
|
||||
"- (D) Other (please explain)\n",
|
||||
"\n",
|
||||
"For the first part of the exercise, change the `PROMPT` to **make Claude output the correct classification and ONLY the classification**. Your answer needs to **include the letter (A - D) of the correct choice, with the parentheses, as well as the name of the category**.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list to know which category that email should be classified under."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response, if any\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Dictionary of string values for each category to be used for regex grading\n",
|
||||
"REGEX_CATEGORIES = {\n",
|
||||
" \"A\": \"A\\) P\",\n",
|
||||
" \"B\": \"B\\) B\",\n",
|
||||
" \"C\": \"C\\) B\",\n",
|
||||
" \"D\": \"D\\) O\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_6_1_hint; print(exercise_6_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Still stuck? Run the cell below for an example solution.\t\t\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_6_1_solution; print(exercise_6_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 6.2 - Email Classification Formatting\n",
|
||||
"In this exercise, we're going to refine the output of the above prompt to yield an answer formatted exactly how we want it. \n",
|
||||
"\n",
|
||||
"Use your favorite output formatting technique to make Claude wrap JUST the letter of the correct classification in `<answer></answer>` tags. For instance, the answer to the first email should contain the exact string `<answer>B</answer>`.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list if you forget which letter category is correct for each email."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response, if any\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Dictionary of string values for each category to be used for regex grading\n",
|
||||
"REGEX_CATEGORIES = {\n",
|
||||
" \"A\": \"<answer>A</answer>\",\n",
|
||||
" \"B\": \"<answer>B</answer>\",\n",
|
||||
" \"C\": \"<answer>C</answer>\",\n",
|
||||
" \"D\": \"<answer>D</answer>\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_6_2_hint; print(exercise_6_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this movie review sentiment positive or negative?\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since the year 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# System prompt\n",
|
||||
"SYSTEM_PROMPT = \"You are a savvy reader of movie reviews.\"\n",
|
||||
"\n",
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment positive or negative? First, write the best arguments for each side in <positive-argument> and <negative-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. In totally unrelated news, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT, SYSTEM_PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Is this review sentiment negative or positive? First write the best arguments for each side in <negative-argument> and <positive-argument> XML tags, then answer.\n",
|
||||
"\n",
|
||||
"This movie blew my mind with its freshness and originality. Unrelatedly, I have been living under a rock since 1900.\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,391 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 7: Using Examples (Few-Shot Prompting)\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\", prefill=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"**Giving Claude examples of how you want it to behave (or how you want it not to behave) is extremely effective** for:\n",
|
||||
"- Getting the right answer\n",
|
||||
"- Getting the answer in the right format\n",
|
||||
"\n",
|
||||
"This sort of prompting is also called \"**few shot prompting**\". You might also encounter the phrase \"zero-shot\" or \"n-shot\" or \"one-shot\". The number of \"shots\" refers to how many examples are used within the prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Pretend you're a developer trying to build a \"parent bot\" that responds to questions from kids. **Claude's default response is quite formal and robotic**. This is going to break a child's heart."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Will Santa bring me presents on Christmas?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You could take the time to describe your desired tone, but it's much easier just to **give Claude a few examples of ideal responses**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Please complete the conversation by writing the next line, speaking as \"A\".\n",
|
||||
"Q: Is the tooth fairy real?\n",
|
||||
"A: Of course, sweetie. Wrap up your tooth and put it under your pillow tonight. There might be something waiting for you in the morning.\n",
|
||||
"Q: Will Santa bring me presents on Christmas?\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In the following formatting example, we could walk Claude step by step through a set of formatting instructions on how to extract names and professions and then format them exactly the way we want, or we could just **provide Claude with some correctly-formatted examples and Claude can extrapolate from there**. Note the `<individuals>` in the `assistant` turn to start Claude off on the right foot."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Silvermist Hollow, a charming village, was home to an extraordinary group of individuals.\n",
|
||||
"Among them was Dr. Liam Patel, a neurosurgeon who revolutionized surgical techniques at the regional medical center.\n",
|
||||
"Olivia Chen was an innovative architect who transformed the village's landscape with her sustainable and breathtaking designs.\n",
|
||||
"The local theater was graced by the enchanting symphonies of Ethan Kovacs, a professionally-trained musician and composer.\n",
|
||||
"Isabella Torres, a self-taught chef with a passion for locally sourced ingredients, created a culinary sensation with her farm-to-table restaurant, which became a must-visit destination for food lovers.\n",
|
||||
"These remarkable individuals, each with their distinct talents, contributed to the vibrant tapestry of life in Silvermist Hollow.\n",
|
||||
"<individuals>\n",
|
||||
"1. Dr. Liam Patel [NEUROSURGEON]\n",
|
||||
"2. Olivia Chen [ARCHITECT]\n",
|
||||
"3. Ethan Kovacs [MISICIAN AND COMPOSER]\n",
|
||||
"4. Isabella Torres [CHEF]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"At the heart of the town, Chef Oliver Hamilton has transformed the culinary scene with his farm-to-table restaurant, Green Plate. Oliver's dedication to sourcing local, organic ingredients has earned the establishment rave reviews from food critics and locals alike.\n",
|
||||
"Just down the street, you'll find the Riverside Grove Library, where head librarian Elizabeth Chen has worked diligently to create a welcoming and inclusive space for all. Her efforts to expand the library's offerings and establish reading programs for children have had a significant impact on the town's literacy rates.\n",
|
||||
"As you stroll through the charming town square, you'll be captivated by the beautiful murals adorning the walls. These masterpieces are the work of renowned artist, Isabella Torres, whose talent for capturing the essence of Riverside Grove has brought the town to life.\n",
|
||||
"Riverside Grove's athletic achievements are also worth noting, thanks to former Olympic swimmer-turned-coach, Marcus Jenkins. Marcus has used his experience and passion to train the town's youth, leading the Riverside Grove Swim Team to several regional championships.\n",
|
||||
"<individuals>\n",
|
||||
"1. Oliver Hamilton [CHEF]\n",
|
||||
"2. Elizabeth Chen [LIBRARIAN]\n",
|
||||
"3. Isabella Torres [ARTIST]\n",
|
||||
"4. Marcus Jenkins [COACH]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"Oak Valley, a charming small town, is home to a remarkable trio of individuals whose skills and dedication have left a lasting impact on the community.\n",
|
||||
"At the town's bustling farmer's market, you'll find Laura Simmons, a passionate organic farmer known for her delicious and sustainably grown produce. Her dedication to promoting healthy eating has inspired the town to embrace a more eco-conscious lifestyle.\n",
|
||||
"In Oak Valley's community center, Kevin Alvarez, a skilled dance instructor, has brought the joy of movement to people of all ages. His inclusive dance classes have fostered a sense of unity and self-expression among residents, enriching the local arts scene.\n",
|
||||
"Lastly, Rachel O'Connor, a tireless volunteer, dedicates her time to various charitable initiatives. Her commitment to improving the lives of others has been instrumental in creating a strong sense of community within Oak Valley.\n",
|
||||
"Through their unique talents and unwavering dedication, Laura, Kevin, and Rachel have woven themselves into the fabric of Oak Valley, helping to create a vibrant and thriving small town.\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<individuals>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 7.1 - Email Formatting via Examples](#exercise-71---email-formatting-via-examples)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 7.1 - Email Formatting via Examples\n",
|
||||
"We're going to redo Exercise 6.2, but this time, we're going to edit the `PROMPT` to use \"few-shot\" examples of emails + proper classification (and formatting) to get Claude to output the correct answer. We want the *last* letter of Claude's output to be the letter of the category.\n",
|
||||
"\n",
|
||||
"Refer to the comments beside each email in the `EMAILS` list if you forget which letter category is correct for each email.\n",
|
||||
"\n",
|
||||
"Remember that these are the categories for the emails:\t\t\t\t\t\t\t\t\t\t\n",
|
||||
"- (A) Pre-sale question\n",
|
||||
"- (B) Broken or defective item\n",
|
||||
"- (C) Billing question\n",
|
||||
"- (D) Other (please explain)\t\t\t\t\t\t\t\t"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"\"\n",
|
||||
"\n",
|
||||
"# Variable content stored as a list\n",
|
||||
"EMAILS = [\n",
|
||||
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
|
||||
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
|
||||
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
|
||||
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
|
||||
"ANSWERS = [\n",
|
||||
" [\"B\"],\n",
|
||||
" [\"A\",\"D\"],\n",
|
||||
" [\"C\"],\n",
|
||||
" [\"D\"]\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Iterate through list of emails\n",
|
||||
"for i,email in enumerate(EMAILS):\n",
|
||||
" \n",
|
||||
" # Substitute the email text into the email placeholder variable\n",
|
||||
" formatted_prompt = PROMPT.format(email=email)\n",
|
||||
" \n",
|
||||
" # Get Claude's response\n",
|
||||
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
|
||||
"\n",
|
||||
" # Grade Claude's response\n",
|
||||
" grade = any([bool(re.search(ans, response[-1])) for ans in ANSWERS[i]])\n",
|
||||
" \n",
|
||||
" # Print Claude's response\n",
|
||||
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
" print(\"USER TURN\")\n",
|
||||
" print(formatted_prompt)\n",
|
||||
" print(\"\\nASSISTANT TURN\")\n",
|
||||
" print(PREFILL)\n",
|
||||
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
" print(response)\n",
|
||||
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_7_1_hint; print(exercise_7_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Still stuck? Run the cell below for an example solution."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_7_1_solution; print(exercise_7_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Will Santa bring me presents on Christmas?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"Please complete the conversation by writing the next line, speaking as \"A\".\n",
|
||||
"Q: Is the tooth fairy real?\n",
|
||||
"A: Of course, sweetie. Wrap up your tooth and put it under your pillow tonight. There might be something waiting for you in the morning.\n",
|
||||
"Q: Will Santa bring me presents on Christmas?\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt template with a placeholder for the variable content\n",
|
||||
"PROMPT = \"\"\"Silvermist Hollow, a charming village, was home to an extraordinary group of individuals.\n",
|
||||
"Among them was Dr. Liam Patel, a neurosurgeon who revolutionized surgical techniques at the regional medical center.\n",
|
||||
"Olivia Chen was an innovative architect who transformed the village's landscape with her sustainable and breathtaking designs.\n",
|
||||
"The local theater was graced by the enchanting symphonies of Ethan Kovacs, a professionally-trained musician and composer.\n",
|
||||
"Isabella Torres, a self-taught chef with a passion for locally sourced ingredients, created a culinary sensation with her farm-to-table restaurant, which became a must-visit destination for food lovers.\n",
|
||||
"These remarkable individuals, each with their distinct talents, contributed to the vibrant tapestry of life in Silvermist Hollow.\n",
|
||||
"<individuals>\n",
|
||||
"1. Dr. Liam Patel [NEUROSURGEON]\n",
|
||||
"2. Olivia Chen [ARCHITECT]\n",
|
||||
"3. Ethan Kovacs [MISICIAN AND COMPOSER]\n",
|
||||
"4. Isabella Torres [CHEF]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"At the heart of the town, Chef Oliver Hamilton has transformed the culinary scene with his farm-to-table restaurant, Green Plate. Oliver's dedication to sourcing local, organic ingredients has earned the establishment rave reviews from food critics and locals alike.\n",
|
||||
"Just down the street, you'll find the Riverside Grove Library, where head librarian Elizabeth Chen has worked diligently to create a welcoming and inclusive space for all. Her efforts to expand the library's offerings and establish reading programs for children have had a significant impact on the town's literacy rates.\n",
|
||||
"As you stroll through the charming town square, you'll be captivated by the beautiful murals adorning the walls. These masterpieces are the work of renowned artist, Isabella Torres, whose talent for capturing the essence of Riverside Grove has brought the town to life.\n",
|
||||
"Riverside Grove's athletic achievements are also worth noting, thanks to former Olympic swimmer-turned-coach, Marcus Jenkins. Marcus has used his experience and passion to train the town's youth, leading the Riverside Grove Swim Team to several regional championships.\n",
|
||||
"<individuals>\n",
|
||||
"1. Oliver Hamilton [CHEF]\n",
|
||||
"2. Elizabeth Chen [LIBRARIAN]\n",
|
||||
"3. Isabella Torres [ARTIST]\n",
|
||||
"4. Marcus Jenkins [COACH]\n",
|
||||
"</individuals>\n",
|
||||
"\n",
|
||||
"Oak Valley, a charming small town, is home to a remarkable trio of individuals whose skills and dedication have left a lasting impact on the community.\n",
|
||||
"At the town's bustling farmer's market, you'll find Laura Simmons, a passionate organic farmer known for her delicious and sustainably grown produce. Her dedication to promoting healthy eating has inspired the town to embrace a more eco-conscious lifestyle.\n",
|
||||
"In Oak Valley's community center, Kevin Alvarez, a skilled dance instructor, has brought the joy of movement to people of all ages. His inclusive dance classes have fostered a sense of unity and self-expression among residents, enriching the local arts scene.\n",
|
||||
"Lastly, Rachel O'Connor, a tireless volunteer, dedicates her time to various charitable initiatives. Her commitment to improving the lives of others has been instrumental in creating a strong sense of community within Oak Valley.\n",
|
||||
"Through their unique talents and unwavering dedication, Laura, Kevin, and Rachel have woven themselves into the fabric of Oak Valley, helping to create a vibrant and thriving small town.\"\"\"\n",
|
||||
"\n",
|
||||
"# Prefill for Claude's response\n",
|
||||
"PREFILL = \"<individuals>\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
|
||||
"print(\"USER TURN:\")\n",
|
||||
"print(PROMPT)\n",
|
||||
"print(\"\\nASSISTANT TURN:\")\n",
|
||||
"print(PREFILL)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(PROMPT, prefill=PREFILL))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,701 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Chapter 8: Avoiding Hallucinations\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"def get_completion(prompt: str, system_prompt=\"\", prefill=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": prompt},\n",
|
||||
" {\"role\": \"assistant\", \"content\": prefill}\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"Some bad news: **Claude sometimes \"hallucinates\" and makes claims that are untrue or unjustified**. The good news: there are techniques you can use to minimize hallucinations.\n",
|
||||
"\t\t\t\t\n",
|
||||
"Below, we'll go over a few of these techniques, namely:\n",
|
||||
"- Giving Claude the option to say it doesn't know the answer to a question\n",
|
||||
"- Asking Claude to find evidence before answering\n",
|
||||
"\n",
|
||||
"However, **there are many methods to avoid hallucinations**, including many of the techniques you've already learned in this course. If Claude hallucinates, experiment with multiple techniques to get Claude to increase its accuracy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"Here is a question about general factual knowledge in answer to which **Claude hallucinates several large hippos because it's trying to be as helpful as possible**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"A solution we can try here is to \"**give Claude an out**\" — tell Claude that it's OK for it to decline to answer, or to only answer if it actually knows the answer with certainty."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time? Only answer if you know the answer with certainty.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In the prompt below, we give Claude a long document containing some \"distractor information\" that is almost but not quite relevant to the user's question. **Without prompting help, Claude falls for the distractor information** and gives an incorrect \"hallucinated\" answer as to the size of Matterport's subscriber base as of May 31, 2020.\n",
|
||||
"\n",
|
||||
"**Note:** As you'll learn later in the next chapter, **it's best practice to have the question at the bottom *after* any text or document**, but we put it at the top here to make the prompt easier to read. Feel free to double click on the prompt cell to get the full prompt text (it's very long!)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then write a brief numerical answer inside <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"How do we fix this? Well, a great way to reduce hallucinations on long documents is to **make Claude gather evidence first.** \n",
|
||||
"\n",
|
||||
"In this case, we **tell Claude to first extract relevant quotes, then base its answer on those quotes**. Telling Claude to do so here makes it correctly notice that the quote does not answer the question."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then, in <scratchpad> tags, pull the most relevant quote from the document and consider whether it answers the user's question or whether it lacks sufficient detail. Then write a brief numerical answer in <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Bonus lesson\n",
|
||||
"\n",
|
||||
"Sometimes, Claude's hallucinations can be solved by lowering the `temperature` of Claude's responses. Temperature is a measurement of answer creativity between 0 and 1, with 1 being more unpredictable and less standardized, and 0 being the most consistent. \n",
|
||||
"\n",
|
||||
"Asking Claude something at temperature 0 will generally yield an almost-deterministic answer set across repeated trials (although complete determinism is not guaranteed). Asking Claude something at temperature 1 (or gradations in between) will yield more variable answers. Learn more about temperature and other parameters [here](https://docs.anthropic.com/claude/reference/messages_post).\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 8.1 - Beyoncé Hallucination](#exercise-81---beyoncé-hallucination)\n",
|
||||
"- [Exercise 8.2 - Prospectus Hallucination](#exercise-82---prospectus-hallucination)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 8.1 - Beyoncé Hallucination\n",
|
||||
"Modify the `PROMPT` to fix Claude's hallucination issue by giving Claude an out. (Renaissance is Beyoncé's seventh studio album, not her eigthth.)\n",
|
||||
"\n",
|
||||
"We suggest you run the cell first to see what Claude hallucinates before trying to fix it."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"In what year did star performer Beyoncé release her eighth studio album?\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" contains = bool(\n",
|
||||
" re.search(\"Unfortunately\", text) or\n",
|
||||
" re.search(\"I do not\", text) or\n",
|
||||
" re.search(\"I don't\", text)\n",
|
||||
" )\n",
|
||||
" does_not_contain = not bool(re.search(\"2022\", text))\n",
|
||||
" return contains and does_not_contain\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_8_1_hint; print(exercise_8_1_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 8.1 - Prospectus Hallucination\n",
|
||||
"Modify the `PROMPT` to fix Claude's hallucination issue by asking for citations. The correct answer is that subscribers went up 49x."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"From December 2018 to December 2022, by what amount did Matterport's subscribers grow?\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"response = get_completion(PROMPT)\n",
|
||||
"\n",
|
||||
"# Function to grade exercise correctness\n",
|
||||
"def grade_exercise(text):\n",
|
||||
" return bool(re.search(\"49-fold\", text))\n",
|
||||
"\n",
|
||||
"# Print Claude's response and the corresponding grade\n",
|
||||
"print(response)\n",
|
||||
"print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
|
||||
"print(\"This exercise has been correctly solved:\", grade_exercise(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want a hint, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_8_2_hint; print(exercise_8_2_hint)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time?\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"Who is the heaviest hippo of all time? Only answer if you know the answer with certainty.\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then write a brief numerical answer inside <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Prompt\n",
|
||||
"PROMPT = \"\"\"<question>What was Matterport's subscriber base on the precise date of May 31, 2020?</question>\n",
|
||||
"Please read the below document. Then, in <scratchpad> tags, pull the most relevant quote from the document and consider whether it answers the user's question or whether it lacks sufficient detail. Then write a brief numerical answer in <answer> tags.\n",
|
||||
"\n",
|
||||
"<document>\n",
|
||||
"Matterport SEC filing 10-K 2023\n",
|
||||
"Item 1. Business\n",
|
||||
"Our Company\n",
|
||||
"Matterport is leading the digitization and datafication of the built world. We believe the digital transformation of the built world will fundamentally change the way people interact with buildings and the physical spaces around them.\n",
|
||||
"Since its founding in 2011, Matterport’s pioneering technology has set the standard for digitizing, accessing and managing buildings, spaces and places online. Our platform’s innovative software, spatial data-driven data science, and 3D capture technology have broken down the barriers that have kept the largest asset class in the world, buildings and physical spaces, offline and underutilized for many years. We believe the digitization and datafication of the built world will continue to unlock significant operational efficiencies and property values, and that Matterport is the platform to lead this enormous global transformation.\n",
|
||||
"The world is rapidly moving from offline to online. Digital transformation has made a powerful and lasting impact across every business and industry today. According to International Data Corporation, or IDC, over $6.8 trillion of direct investments will be made on digital transformation from 2020 to 2023, the global digital transformation spending is forecasted to reach $3.4 trillion in 2026 with a five-year compound annual growth rate (“CAGR”) of 16.3%, and digital twin investments are expected to have a five-year CAGR of 35.2%. With this secular shift, there is also growing demand for the built world to transition from physical to digital. Nevertheless, the vast majority of buildings and spaces remain offline and undigitized. The global building stock, estimated by Savills to be $327 trillion in total property value as of 2021, remains largely offline today, and we estimate that less than 0.1% is penetrated by digital transformation.\n",
|
||||
"Matterport was among the first to recognize the increasing need for digitization of the built world and the power of spatial data, the unique details underlying buildings and spaces, in facilitating the understanding of buildings and spaces. In the past, technology advanced physical road maps to the data-rich, digital maps and location services we all rely on today. Matterport now digitizes buildings, creating a data-rich environment to vastly increase our understanding and the full potential of each and every space we capture. Just as we can instantly, at the touch of a button, learn the fastest route from one city to another or locate the nearest coffee shops, Matterport’s spatial data for buildings unlocks a rich set of insights and learnings about properties and spaces worldwide. In addition, just as the geo-spatial mapping platforms of today have opened their mapping data to industry to create new business models such as ridesharing, e-commerce, food delivery marketplaces, and even short-term rental and home sharing, open access to Matterport’s structured spatial data is enabling new opportunities and business models for hospitality, facilities management, insurance, construction, real estate and retail, among others.\n",
|
||||
"We believe the total addressable market opportunity for digitizing the built world is over $240 billion, and could be as high as $1 trillion as the market matures at scale. This is based on our analysis, modeling and understanding of the global building stock of over 4 billion properties and 20 billion spaces in the world today. With the help of artificial intelligence (“AI”), machine learning (“ML”) and deep learning (“DL”) technologies, we believe that, with the additional monetization opportunities from powerful spatial data-driven property insights and analytics, the total addressable market for the digitization and datafication of the built world will reach more than $1 trillion.\n",
|
||||
"\n",
|
||||
"Our spatial data platform and capture of digital twins deliver value across a diverse set of industries and use cases. Large retailers can manage thousands of store locations remotely, real estate agencies can provide virtual open houses for hundreds of properties and thousands of visitors at the same time, property developers can monitor the entirety of the construction process with greater detail and speed, and insurance companies can more precisely document and evaluate claims and underwriting assessments with efficiency and precision. Matterport delivers the critical digital experience, tools and information that matter to our subscribers about properties of virtually any size, shape, and location worldwide.\n",
|
||||
"For nearly a decade, we have been growing our spatial data platform and expanding our capabilities in order to create the most detailed, accurate, and data-rich digital twins available. Moreover, our 3D reconstruction process is fully automated, allowing our solution to scale with equal precision to millions of buildings and spaces of any type, shape, and size in the world. The universal applicability of our service provides Matterport significant scale and reach across diverse verticals and any geography. As of December 31, 2022, our subscriber base had grown approximately 39% to over 701,000 subscribers from 503,000 subscribers as of December 31, 2021, with our digital twins reaching more than 170 countries. We have digitized more than 28 billion square feet of space across multiple industries, representing significant scale and growth over the rest of the market.\n",
|
||||
"\n",
|
||||
"As we continue to transform buildings into data worldwide, we are extending our spatial data platform to further transform property planning, development, management and intelligence for our subscribers across industries to become the de facto building and business intelligence engine for the built world. We believe the demand for spatial data and resulting insights for enterprises, businesses and institutions across industries, including real estate, architecture, engineering and construction (“AEC”), retail, insurance and government, will continue to grow rapidly.\n",
|
||||
"We believe digitization and datafication represent a tremendous greenfield opportunity for growth across this massive category and asset class. From the early stages of design and development to marketing, operations, insurance and building repair and maintenance, our platform’s software and technology provide subscribers critical tools and insights to drive cost savings, increase revenues and optimally manage their buildings and spaces. We believe that hundreds of billions of dollars in unrealized utilization and operating efficiencies in the built world can be unlocked through the power of our spatial data platform. Our platform and data solutions have universal applicability across industries and building categories, giving Matterport a significant advantage as we can address the entirety of this large market opportunity and increase the value of what we believe to be the largest asset class in the world.\n",
|
||||
"With a demonstrated track record of delivering value to our subscribers, our offerings include software subscription, data licensing, services and product hardware. As of December 31, 2022, our subscriber base included over 24% of Fortune 1000 companies, with less than 10% of our total revenue generated from our top 10 subscribers. We expect more than 80% of our revenue to come from our software subscription and data license solutions by 2025. Our innovative 3D capture products, the Pro2 and Pro3 Cameras, have played an integral part in shaping the 3D building and property visualization ecosystem. The Pro2 and Pro3 Cameras have driven adoption of our solutions and have generated the unique high-quality and scaled data set that has enabled Cortex, our proprietary AI software engine, to become the pioneering engine for digital twin creation. With this data advantage initially spurred by the Pro2 Camera, we have developed a capture device agnostic platform that scales and can generate new building and property insights for our subscribers across industries and geographies.\n",
|
||||
"We have recently experienced rapid growth. Our subscribers have grown approximately 49-fold from December 31, 2018 to December 31, 2022. Our revenue increased by approximately 22% to $136.1 million for the year ended December 31, 2022, from approximately $111.2 million for the year ended December 31, 2021. Our gross profit decreased by $8.1 million or 14%, to $51.8 million for the year ended December 31, 2022, from $60.0 million for the year ended December 31, 2021, primarily attributable to certain disruptive and incremental costs due to the global supply chain constraints in fiscal year 2022. Our ability to retain and grow the subscription revenue generated by our existing subscribers is an important measure of the health of our business and our future growth prospects. We track our performance in this area by measuring our net dollar expansion rate from the same set of customers across comparable periods. Our net dollar expansion rate of 103% for the three months ended December 31, 2022 demonstrates the stickiness and growth potential of our platform.\n",
|
||||
"Our Industry and Market Opportunity\n",
|
||||
"Today, the vast majority of buildings and spaces remain undigitized. We estimate our current serviceable addressable market includes approximately 1.3 billion spaces worldwide, primarily from the real estate and travel and hospitality sectors. With approximately 9.2 million spaces under management as of December 31, 2022, we are continuing to penetrate the global building stock and expand our footprint across various end markets, including residential and commercial real estate, facilities management, retail, AEC, insurance and repair, and travel and hospitality. We estimate our total addressable market to be more than 4 billion buildings and 20 billion spaces globally, yielding a more than $240 billion market opportunity. We believe that as Matterport’s unique spatial data library and property data services continue to grow, this opportunity could increase to more than $1 trillion based on the size of the building stock and the untapped value creation available to buildings worldwide. The constraints created by the COVID-19 pandemic have only reinforced and accelerated the importance of our scaled 3D capture solution that we have developed for diverse industries and markets over the past decade.\n",
|
||||
"\n",
|
||||
"Our Spatial Data Platform\n",
|
||||
"Overview\n",
|
||||
"Our technology platform uses spatial data collected from a wide variety of digital capture devices to transform physical buildings and spaces into dimensionally accurate, photorealistic digital twins that provide our subscribers access to previously unavailable building information and insights.\n",
|
||||
"As a first mover in this massive market for nearly a decade, we have developed and scaled our industry-leading 3D reconstruction technology powered by Cortex, our proprietary AI-driven software engine that uses machine learning to recreate a photorealistic, 3D virtual representation of an entire building structure, including contents, equipment and furnishings. The finished product is a detailed and dynamic replication of the physical space that can be explored, analyzed and customized from a web browser on any device, including smartphones. The power to manage even large-scale commercial buildings is in the palm of each subscriber’s hands, made possible by our advanced technology and breakthrough innovations across our entire spatial data technology stack.\n",
|
||||
"Key elements of our spatial data platform include:\n",
|
||||
"•Bringing offline buildings online. Traditionally, our customers needed to conduct in-person site visits to understand and assess their buildings and spaces. While photographs and floor plans can be helpful, these forms of two-dimensional (“2D”) representation have limited information and tend to be static and rigid, and thus lack the interactive element critical to a holistic understanding of each building and space. With the AI-powered capabilities of Cortex, our proprietary AI software, representation of physical objects is no longer confined to static 2D images and physical visits can be eliminated. Cortex helps to move the buildings and spaces from offline to online and makes them accessible to our customers in real-time and on demand from anywhere. After subscribers scan their buildings, our visualization algorithms accurately infer spatial positions and depths from flat, 2D imagery captured through the scans and transform them into high- fidelity and precise digital twin models. This creates a fully automated image processing pipeline to ensure that each digital twin is of professional grade image quality.\n",
|
||||
"•Driven by spatial data. We are a data-driven company. Each incremental capture of a space grows the richness and depth of our spatial data library. Spatial data represents the unique and idiosyncratic details that underlie and compose the buildings and spaces in the human- made environment. Cortex uses the breadth of the billions of data points we have accumulated over the years to improve the 3D accuracy of our digital twins. We help our subscribers pinpoint the height, location and other characteristics of objects in their digital twin. Our sophisticated algorithms also deliver significant commercial value to our subscribers by generating data-based insights that allow them to confidently make assessments and decisions about their properties. For instance, property developers can assess the amount of natural heat and daylight coming from specific windows, retailers can ensure each store layout is up to the same level of code and brand requirements, and factories can insure machinery layouts meet specifications and location guidelines. With approximately 9.2 million spaces under management as of December 31, 2022, our spatial data library is the clearinghouse for information about the built world.\n",
|
||||
"•Powered by AI and ML. Artificial intelligence and machine learning technologies effectively utilize spatial data to create a robust virtual experience that is dynamic, realistic, interactive, informative and permits multiple viewing angles. AI and ML also make costly cameras unnecessary for everyday scans—subscribers can now scan their spaces by simply tapping a button on their smartphones. As a result, Matterport is a device agnostic platform, helping us more rapidly scale and drive towards our mission of digitizing and indexing the built world.\n",
|
||||
"Our value proposition to subscribers is designed to serve the entirety of the digital building lifecycle, from design and build to maintenance and operations, promotion, sale, lease, insure, repair, restore, secure and finance. As a result, we believe we are uniquely positioned to grow our revenue with our subscribers as we help them to discover opportunities to drive short- and long-term return on investment by taking their buildings and spaces from offline to online across their portfolios of properties.\n",
|
||||
"Ubiquitous Capture\n",
|
||||
"Matterport has become the standard for 3D space capture. Our technology platform empowers subscribers worldwide to quickly, easily and accurately digitize, customize and manage interactive and dimensionally accurate digital twins of their buildings and spaces.\n",
|
||||
"The Matterport platform is designed to work with a wide range of LiDAR, spherical, 3D and 360 cameras, as well as smartphones, to suit the capture needs of all of our subscribers. This provides the flexibility to capture a space of any size, scale, and complexity, at anytime and anywhere.\n",
|
||||
"•Matterport Pro3 is our newest 3D camera that scans properties faster than earlier versions to help accelerate project completion. Pro3 provides the highest accuracy scans of both indoor and outdoor spaces and is designed for speed, fidelity, versatility and accuracy. Capturing 3D data up to 100 meters away at less than 20 seconds per sweep, Pro3’s ultra-fast, high-precision LiDAR sensor can run for hours and takes millions of measurements in any conditions.\n",
|
||||
"•Matterport Pro2 is our proprietary 3D camera that has been used to capture millions of spaces around the world with a high degree of fidelity, precision, speed and simplicity. Capable of capturing buildings more than 500,000 square feet in size, it has become the camera of choice for many residential, commercial, industrial and large-scale properties.\n",
|
||||
"•360 Cameras. Matterport supports a selection of 360 cameras available in the market. These affordable, pocket sized devices deliver precision captures with high fidelity and are appropriate for capturing smaller homes, condos, short-term rentals, apartments, and more. The spherical lens image capture technology of these devices gives Cortex robust, detailed image data to transform panoramas into our industry-leading digital twins.\n",
|
||||
"•LEICA BLK360. Through our partnership with Leica, our 3D reconstruction technology and our AI powered software engine, Cortex, transform this powerful LiDAR camera into an ultra-precise capture device for creating Matterport digital twins. It is the solution of choice for AEC professionals when exacting precision is required.\n",
|
||||
"•Smartphone Capture. Our capture apps are commercially available for both iOS and Android. Matterport’s smartphone capture solution has democratized 3D capture, making it easy and accessible for anyone to digitize buildings and spaces with a recent iPhone device since the initial introduction of Matterport for iPhone in May 2020. In April 2021, we announced the official release of the Android Capture app, giving Android users the ability to quickly and easily capture buildings and spaces in immersive 3D. In February 2022, we launched Matterport Axis, a motorized mount that holds a smartphone and can be used with the Matterport Capture app to capture 3D digital twins of any physical space with increased speed, precision, and consistency.\n",
|
||||
"Cortex and 3D Reconstruction (the Matterport Digital Twin)\n",
|
||||
"With a spatial data library, as of December 31, 2022, of approximately 9.2 million spaces under management, representing approximately 28 billion captured square feet of space, we use our advanced ML and DL technologies to algorithmically transform the spatial data we capture into an accurate 3D digital reproduction of any physical space. This intelligent, automated 3D reconstruction is made possible by Cortex, our AI-powered software engine that includes a deep learning neural network that uses our spatial data library to understand how a building or space is divided into floors and rooms, where the doorways and openings are located, and what types of rooms are present, such that those forms are compiled and aligned with dimensional accuracy into a dynamic, photorealistic digital twin. Other components of Cortex include AI-powered computer vision technologies to identify and classify the contents inside a building or space, and object recognition technologies to identify and segment everything from furnishings and equipment to doors, windows, light fixtures, fire suppression sprinklers and fire escapes. Our highly scalable artificial intelligence platform enables our subscribers to tap into powerful, enhanced building data and insights at the click of a button.\n",
|
||||
"\n",
|
||||
"The Science Behind the Matterport Digital Twin: Cortex AI Highlights\n",
|
||||
"Matterport Runs on Cortex\n",
|
||||
"Cortex is our AI-powered software engine that includes a precision deep learning neural network to create digital twins of any building or space. Developed using our proprietary spatial data captured with our Pro2 and Pro3 cameras, Cortex delivers a high degree of precision and accuracy while enabling 3D capture using everyday devices.\n",
|
||||
"Generic neural networks struggle with 3D reconstruction of the real world. Matterport-optimized networks deliver more accurate and robust results. More than just raw training data, Matterport’s datasets allow us to develop new neural network architectures and evaluate them against user behavior and real-world data in millions of situations.\n",
|
||||
"•Deep learning: Connecting and optimizing the detailed neural network data architecture of each space is key to creating robust, highly accurate 3D digital twins. Cortex evaluates and optimizes each 3D model against Matterport’s rich spatial data aggregated from millions of buildings and spaces and the human annotations of those data provided by tens of thousands of subscribers worldwide. Cortex’s evaluative abilities and its data-driven optimization of 3D reconstruction yield consistent, high-precision results across a wide array of building configurations, spaces and environments.\n",
|
||||
"•Dynamic 3D reconstruction: Creating precise 3D spatial data at scale from 2D visuals and static images requires a combination of photorealistic, detailed data from multiple viewpoints and millions of spaces that train and optimize Cortex’s neural network and learning capabilities for improved 3D reconstruction of any space. Cortex’s capabilities combined with real-time spatial alignment algorithms in our 3D capture technology create an intuitive “preview” of any work in progress, allowing subscribers to work with their content interactively and in real-time.\n",
|
||||
"•Computer vision: Cortex enables a suite of powerful features to enhance the value of digital twins. These include automatic measurements for rooms or objects in a room, automatic 2D-from-3D high-definition photo gallery creation, auto face blurring for privacy protection, custom videos, walkthroughs, auto room labeling and object recognition.\n",
|
||||
"•Advanced image processing: Matterport’s computational photography algorithms create a fully automated image processing pipeline to help ensure that each digital twin is of professional grade image quality. Our patented technology makes 3D capture as simple as pressing a single button. Matterport’s software and technology manage the remaining steps, including white balance and camera-specific color correction, high dynamic range tone mapping, de-noising, haze removal, sharpening, saturation and other adjustments to improve image quality.\n",
|
||||
"Spatial Data and AI-Powered Insights\n",
|
||||
"Every Matterport digital twin contains extensive information about a building, room or physical space. The data uses our AI-powered Cortex engine. In addition to the Matterport digital twin itself, our spatial data consists of precision building geometry and structural detail, building contents, fixtures and condition, along with high-definition imagery and photorealistic detail from many vantage points in a space. Cortex employs a technique we call deep spatial indexing. Deep spatial indexing uses artificial intelligence, computer vision and deep learning to identify and convey important details about each space, its structure and its contents with precision and fidelity. We have created a robust spatial data standard that enables Matterport subscribers to harness an interoperable digital system of record for any building.\n",
|
||||
"In addition to creating a highly interactive digital experience for subscribers through the construction of digital twins, we ask ourselves two questions for every subscriber: (1) what is important about their building or physical space and (2) what learnings and insights can we deliver for this space? Our AI-powered Cortex engine helps us answer these questions using our spatial data library to provide aggregated property trends and operational and valuation insights. Moreover, as the Matterport platform ecosystem continues to expand, our subscribers, partners and other third-party developers can bring their own tools to further the breadth and depth of insights they can harvest from our rich spatial data layer.\n",
|
||||
"Extensible Platform Ecosystem\n",
|
||||
"Matterport offers the largest and most accurate library of spatial data in the world, with, as of December 31, 2022, approximately 9.2 million spaces under management and approximately 28 billion captured square feet. The versatility of our spatial data platform and extensive enterprise software development kit and application programming interfaces (“APIs”) has allowed us to develop a robust global ecosystem of channels and partners that extend the Matterport value proposition by geography and vertical market. We intend to continue to deploy a broad set of workflow integrations with our partners and their subscribers to promote an integrated Matterport solution across our target markets. We are also developing a third-party software marketplace to extend the power of our spatial data platform with easy-to-deploy and easy-to-access Matterport software add-ons. The marketplace enables developers to build new applications and spatial data mining tools, enhance the Matterport 3D experience, and create new productivity and property management tools that supplement our core offerings. These value-added capabilities created by third-party developers enable a scalable new revenue stream, with Matterport sharing the subscription and services revenue from each add-on that is deployed to subscribers through the online marketplace. The network effects of our platform ecosystem contributes to the growth of our business, and we believe that it will continue to bolster future growth by enhancing subscriber stickiness and user engagement.\n",
|
||||
"Examples of Matterport add-ons and extensions include:\n",
|
||||
"•Add-ons: Encircle (easy-to-use field documentation tools for faster claims processing); WP Matterport Shortcode (free Wordpress plugin that allows Matterport to be embedded quickly and easily with a Matterport shortcode), WP3D Models (WordPress + Matterport integration plugin); Rela (all-in-one marketing solution for listings); CAPTUR3D (all-in-one Content Management System that extends value to Matterport digital twins); Private Model Emded (feature that allows enterprises to privately share digital twins with a large group of employees on the corporate network without requiring additional user licenses); Views (new workgroup collaboration framework to enable groups and large organizations to create separate, permissions-based workflows to manage different tasks with different teams); and Guided Tours and Tags (tool to elevate the visitor experience by creating directed virtual tours of any commercial or residential space tailored to the interests of their visitors). We unveiled our private beta integration with Amazon Web Services (AWS) IoT TwinMaker to enable enterprise customers to seamlessly connect IoT data into visually immersive and dimensionally accurate Matterport digital twin.\n",
|
||||
"•Services: Matterport ADA Compliant Digital Twin (solution to provide American Disability Act compliant digital twins) and Enterprise Cloud Software Platform (reimagined cloud software platform for the enterprise that creates, publishes, and manages digital twins of buildings and spaces of any size of shape, indoors or outdoors).\n",
|
||||
"Our Competitive Strengths\n",
|
||||
"We believe that we have a number of competitive strengths that will enable our market leadership to grow. Our competitive strengths include:\n",
|
||||
"•Breadth and depth of the Matterport platform. Our core strength is our all-in-one spatial data platform with broad reach across diverse verticals and geographies such as capture to processing to industries without customization. With the ability to integrate seamlessly with various enterprise systems, our platform delivers value across the property lifecycle for diverse end markets, including real estate, AEC, travel and hospitality, repair and insurance, and industrial and facilities. As of December 31, 2022, our global reach extended to subscribers in more than 170 countries, including over 24% of Fortune 1000 companies.\n",
|
||||
"•Market leadership and first-mover advantage. Matterport defined the category of digitizing and datafying the built world almost a decade ago, and we have become the global leader in the category. As of December 31, 2022, we had over 701,000 subscribers on our platform and approximately 9.2 million spaces under management. Our leadership is primarily driven by the fact that we were the first mover in digital twin creation. As a result of our first mover advantage, we have amassed a deep and rich library of spatial data that continues to compound and enhance our leadership position.\n",
|
||||
"•Significant network effect. With each new capture and piece of data added to our platform, the richness of our dataset and the depth of insights from our spaces under management grow. In addition, the combination of our ability to turn data into insights with incremental data from new data captures by our subscribers enables Matterport to develop features for subscribers to our platform. We were a first mover in building a spatial data library for the built world, and our leadership in gathering and deriving insights from data continues to compound and the relevance of those insights attracts more new subscribers.\n",
|
||||
"•Massive spatial data library as the raw material for valuable property insights. The scale of our spatial data library is a significant advantage in deriving insights for our subscribers. Our spatial data library serves as vital ground truth for Cortex, enabling Matterport to create powerful 3D digital twins using a wide range of camera technology, including low-cost digital and smartphone cameras. As of December 31, 2022, our data came from approximately 9.2 million spaces under management and approximately 28 billion captured square feet. As a result, we have taken property insights and analytics to new levels, benefiting subscribers across various industries. For example, facilities managers significantly reduce the time needed to create building layouts, leading to a significant decrease in the cost of site surveying and as-built modeling. AEC subscribers use the analytics of each as-built space to streamline documentation and collaborate with ease.\n",
|
||||
"•Global reach and scale. We are focused on continuing to expand our AI-powered spatial data platform worldwide. We have a significant presence in North America, Europe and Asia, with leadership teams and a go-to-market infrastructure in each of these regions. We have offices in London, Singapore and several across the United States, and we are accelerating our international expansion. As of December 31, 2022, we had over 701,000 subscribers in more than 170 countries. We believe that the geography-agnostic nature of our spatial data platform is a significant advantage as we continue to grow internationally.\n",
|
||||
"•Broad patent portfolio supporting 10 years of R&D and innovation. As of December 31, 2022, we had 54 issued and 37 pending patent applications. Our success is based on almost 10 years of focus on innovation. Innovation has been at the center of Matterport, and we will continue to prioritize our investments in R&D to further our market leading position.\n",
|
||||
"•Superior capture technology. Matterport’s capture technology platform is a software framework that enables support for a wide variety of capture devices required to create a Matterport digital twin of a building or space.\n",
|
||||
"This includes support for LiDAR cameras, 360 cameras, smartphones, Matterport Axis and the Matterport Pro2 and Pro3 cameras. The Pro2 camera was foundational to our spatial data advantage, and we have expanded that advantage with an array of Matterport-enabled third-party capture devices. In August 2022, we launched and began shipment of our Pro3 Camera along with major updates to our industry-leading digital twin cloud platform. The Matterport Pro3 Camera is an advanced 3D capture device, which includes faster boot time, swappable batteries, and a lighter design. The Pro3 camera can perform both indoors and outdoors and is designed for speed, fidelity, versatility and accuracy. Along with our Pro2 Camera, we expect that future sales of our Pro3 Camera will continue to drive increased adoption of our solutions. Matterport is democratizing the 3D capture experience, making high-fidelity and high-accuracy 3D digital twins readily available for any building type and any subscriber need in the property life cycle. While there are other 3D capture solution providers, very few can produce true, dimensionally accurate 3D results, and fewer still can automatically create a final product in photorealistic 3D, and at global scale. This expansive capture technology offering would not be possible without our rich spatial data library available to train the AI-powered Cortex engine to automatically generate accurate digital twins from photos captured with a smartphone or 360 camera.\n",
|
||||
"</document>\"\"\"\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(get_completion(PROMPT))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,708 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.1: Chaining Prompts\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY & MODEL_NAME variables from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"%store -r MODEL_NAME\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"# Has been rewritten to take in a messages list of arbitrary length\n",
|
||||
"def get_completion(messages, system_prompt=\"\"):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=MODEL_NAME,\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=messages\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"The saying goes, \"Writing is rewriting.\" It turns out, **Claude can often improve the accuracy of its response when asked to do so**!\n",
|
||||
"\n",
|
||||
"There are many ways to prompt Claude to \"think again\". The ways that feel natural to ask a human to double check their work will also generally work for Claude. (Check out our [prompt chaining documentation](https://docs.anthropic.com/claude/docs/chain-prompts) for further examples of when and how to use prompt chaining.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"In this example, we ask Claude to come up with ten words... but one or more of them isn't a real word."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Asking Claude to make its answer more accurate** fixes the error! \n",
|
||||
"\n",
|
||||
"Below, we've pulled down Claude's incorrect response from above and added another turn to the conversation asking Claude to fix its previous answer."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"But is Claude revising its answer just because we told it to? What if we start off with a correct answer already? Will Claude lose its confidence? Here, we've placed a correct response in the place of `first_response` and asked it to double check again."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You may notice that if you generate a respnse from the above block a few times, Claude leaves the words as is most of the time, but still occasionally changes the words even though they're all already correct. What can we do to mitigate this? Per Chapter 8, we can give Claude an out! Let's try this one more time."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words. If all the words are real words, return the original list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Try generating responses from the above code a few times to see that Claude is much better at sticking to its guns now.\n",
|
||||
"\n",
|
||||
"You can also use prompt chaining to **ask Claude to make its responses better**. Below, we asked Claude to first write a story, and then improve the story it wrote. Your personal tastes may vary, but many might agree that Claude's second version is better.\n",
|
||||
"\n",
|
||||
"First, let's generate Claude's first version of the story."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Write a three-sentence short story about a girl who likes to run.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's have Claude improve on its first draft."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Make the story better.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This form of substitution is very powerful. We've been using substitution placeholders to pass in lists, words, Claude's former responses, and so on. You can also **use substitution to do what we call \"function calling,\" which is asking Claude to perform some function, and then taking the results of that function and asking Claude to do even more afterward with the results**. It works like any other substitution. More on this in the next appendix.\n",
|
||||
"\n",
|
||||
"Below is one more example of taking the results of one call to Claude and plugging it into another, longer call. Let's start with the first prompt (which includes prefilling Claude's response this time)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"\"\"Find all names from the below text:\n",
|
||||
"\n",
|
||||
"\"Hey, Jesse. It's me, Erin. I'm calling about the party that Joey is throwing tomorrow. Keisha said she would come and I think Mel will be there too.\"\"\"\n",
|
||||
"\n",
|
||||
"prefill = \"<names>\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's pass this list of names into another prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Alphabetize the list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill + \"\\n\" + first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that you've learned about prompt chaining, head over to Appendix 10.2 to learn how to implement function calling using prompt chaining."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"Name ten words that all end with the exact letters 'ab'.\"\n",
|
||||
"\n",
|
||||
"first_response = \"\"\"Here are 10 words that end with the letters 'ab':\n",
|
||||
"\n",
|
||||
"1. Cab\n",
|
||||
"2. Dab\n",
|
||||
"3. Grab\n",
|
||||
"4. Gab\n",
|
||||
"5. Jab\n",
|
||||
"6. Lab\n",
|
||||
"7. Nab\n",
|
||||
"8. Slab\n",
|
||||
"9. Tab\n",
|
||||
"10. Blab\"\"\"\n",
|
||||
"\n",
|
||||
"second_user = \"Please find replacements for all 'words' that are not real words. If all the words are real words, return the original list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initial prompt\n",
|
||||
"first_user = \"Write a three-sentence short story about a girl who likes to run.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Make the story better.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"first_user = \"\"\"Find all names from the below text:\n",
|
||||
"\n",
|
||||
"\"Hey, Jesse. It's me, Erin. I'm calling about the party that Joey is throwing tomorrow. Keisha said she would come and I think Mel will be there too.\"\"\"\n",
|
||||
"\n",
|
||||
"prefill = \"<names>\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Store and print Claude's response\n",
|
||||
"first_response = get_completion(messages)\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(first_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"second_user = \"Alphabetize the list.\"\n",
|
||||
"\n",
|
||||
"# API messages array\n",
|
||||
"messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": first_user\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": prefill + \"\\n\" + first_response\n",
|
||||
" \n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": second_user\n",
|
||||
" \n",
|
||||
" }\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Print Claude's response\n",
|
||||
"print(\"------------------------ Full messsages array with variable substutions ------------------------\")\n",
|
||||
"print(messages)\n",
|
||||
"print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
|
||||
"print(get_completion(messages))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,778 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.2: Tool Use\n",
|
||||
"\n",
|
||||
"- [Lesson](#lesson)\n",
|
||||
"- [Exercises](#exercises)\n",
|
||||
"- [Example Playground](#example-playground)\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Run the following setup cell to load your API key and establish the `get_completion` helper function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install anthropic\n",
|
||||
"\n",
|
||||
"# Import python's built-in regular expression library\n",
|
||||
"import re\n",
|
||||
"import anthropic\n",
|
||||
"\n",
|
||||
"# Retrieve the API_KEY variable from the IPython store\n",
|
||||
"%store -r API_KEY\n",
|
||||
"\n",
|
||||
"client = anthropic.Anthropic(api_key=API_KEY)\n",
|
||||
"\n",
|
||||
"# Rewrittten to call Claude 3 Sonnet, which is generally better at tool use, and include stop_sequences\n",
|
||||
"def get_completion(messages, system_prompt=\"\", prefill=\"\",stop_sequences=None):\n",
|
||||
" message = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" max_tokens=2000,\n",
|
||||
" temperature=0.0,\n",
|
||||
" system=system_prompt,\n",
|
||||
" messages=messages,\n",
|
||||
" stop_sequences=stop_sequences\n",
|
||||
" )\n",
|
||||
" return message.content[0].text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Lesson\n",
|
||||
"\n",
|
||||
"While it might seem conceptually complex at first, tool use, a.k.a. function calling, is actually quite simple! You already know all the skills necessary to implement tool use, which is really just a combination of substitution and prompt chaining.\n",
|
||||
"\n",
|
||||
"In previous substitution exercises, we substituted text into prompts. With tool use, we substitute tool or function results into prompts. Claude can't literally call or access tools and functions. Instead, we have Claude:\n",
|
||||
"1. Output the tool name and arguments it wants to call\n",
|
||||
"2. Halt any further response generation while the tool is called\n",
|
||||
"3. Then we reprompt with the appended tool results"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Function calling is useful because it expands Claude's capabilities and enables Claude to handle much more complex, multi-step tasks.\n",
|
||||
"Some examples of functions you can give Claude:\n",
|
||||
"- Calculator\n",
|
||||
"- Word counter\n",
|
||||
"- SQL database querying and data retrieval\n",
|
||||
"- Weather API"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can get Claude to do tool use by combining these two elements:\n",
|
||||
"\n",
|
||||
"1. A system prompt, in which we give Claude an explanation of the concept of tool use as well as a detailed descriptive list of the tools it has access to\n",
|
||||
"2. The control logic with which to orchestrate and execute Claude's tool use requests"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Tool use roadmap\n",
|
||||
"\n",
|
||||
"*This lesson teaches our current tool use format. However, we will be updating and improving tool use functionality in the near future, including:*\n",
|
||||
"* *A more streamlined format for function definitions and calls*\n",
|
||||
"* *More robust error handilgj and edge case coverage*\n",
|
||||
"* *Tighter integration with the rest of our API*\n",
|
||||
"* *Better reliability and performance, especially for more complex tool use tasks*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Examples\n",
|
||||
"\n",
|
||||
"To enable tool use in Claude, we start with the system prompt. In this special tool use system prompt, wet tell Claude:\n",
|
||||
"* The basic premise of tool use and what it entails\n",
|
||||
"* How Claude can call and use the tools it's been given\n",
|
||||
"* A detailed list of tools it has access to in this specific scenario \n",
|
||||
"\n",
|
||||
"Here's the first part of the system prompt, explaining tool use to Claude. This part of the system prompt is generalizable across all instances of prompting Claude for tool use. The tool calling structure we're giving Claude (`<function_calls> [...] </function_calls>`) is a structure Claude has been specifically trained to use, so we recommend that you stick with this."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_general_explanation = \"\"\"You have access to a set of functions you can use to answer the user's question. This includes access to a\n",
|
||||
"sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external\n",
|
||||
"resources, except by invoking the below functions.\n",
|
||||
"\n",
|
||||
"You can invoke one or more functions by writing a \"<function_calls>\" block like the following as part of your\n",
|
||||
"reply to the user:\n",
|
||||
"<function_calls>\n",
|
||||
"<invoke name=\"$FUNCTION_NAME\">\n",
|
||||
"<antml:parameter name=\"$PARAMETER_NAME\">$PARAMETER_VALUE</parameter>\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"<nvoke name=\"$FUNCTION_NAME2\">\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"</function_calls>\n",
|
||||
"\n",
|
||||
"String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that\n",
|
||||
"spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular\n",
|
||||
"expressions.\n",
|
||||
"\n",
|
||||
"The output and/or any errors will appear in a subsequent \"<function_results>\" block, and remain there as part of\n",
|
||||
"your reply to the user.\n",
|
||||
"You may then continue composing the rest of your reply to the user, respond to any errors, or make further function\n",
|
||||
"calls as appropriate.\n",
|
||||
"If a \"<function_results>\" does NOT appear after your function calls, then they are likely malformatted and not\n",
|
||||
"recognized as a call.\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here's the second part of the system prompt, which defines the exact tools Claude has access to in this specific situation. In this example, we will be giving Claude a calculator tool, which takes three parameters: two operands and an operator. \n",
|
||||
"\n",
|
||||
"Then we combine the two parts of the system prompt."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools = \"\"\"Here are the functions available in JSONSchema format:\n",
|
||||
"<tools>\n",
|
||||
"<tool_description>\n",
|
||||
"<tool_name>calculator</tool_name>\n",
|
||||
"<description>\n",
|
||||
"Calculator function for doing basic arithmetic.\n",
|
||||
"Supports addition, subtraction, multiplication\n",
|
||||
"</description>\n",
|
||||
"<parameters>\n",
|
||||
"<parameter>\n",
|
||||
"<name>first_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>First operand (before the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>second_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>Second operand (after the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>operator</name>\n",
|
||||
"<type>str</type>\n",
|
||||
"<description>The operation to perform. Must be either +, -, *, or /</description>\n",
|
||||
"</parameter>\n",
|
||||
"</parameters>\n",
|
||||
"</tool_description>\n",
|
||||
"</tools>\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now we can give Claude a question that requires use of the `calculator` tool. We will use `<function_calls\\>` in `stop_sequences` to detect if and when Claude calls the function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Multiply 1,984,135 by 9,343,116\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, we can extract out the parameters from Claude's function call and actually run the function on Claude's behalf.\n",
|
||||
"\n",
|
||||
"First we'll define the function's code."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def do_pairwise_arithmetic(num1, num2, operation):\n",
|
||||
" if operation == '+':\n",
|
||||
" return num1 + num2\n",
|
||||
" elif operation == \"-\":\n",
|
||||
" return num1 - num2\n",
|
||||
" elif operation == \"*\":\n",
|
||||
" return num1 * num2\n",
|
||||
" elif operation == \"/\":\n",
|
||||
" return num1 / num2\n",
|
||||
" else:\n",
|
||||
" return \"Error: Operation not supported.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then we'll extract the parameters from Claude's function call response. If all the parameters exist, we run the calculator tool."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def find_parameter(message, parameter_name):\n",
|
||||
" parameter_start_string = f\"name=\\\"{parameter_name}\\\">\"\n",
|
||||
" start = message.index(parameter_start_string)\n",
|
||||
" if start == -1:\n",
|
||||
" return None\n",
|
||||
" if start > 0:\n",
|
||||
" start = start + len(parameter_start_string)\n",
|
||||
" end = start\n",
|
||||
" while message[end] != \"<\":\n",
|
||||
" end += 1\n",
|
||||
" return message[start:end]\n",
|
||||
"\n",
|
||||
"first_operand = find_parameter(function_calling_response, \"first_operand\")\n",
|
||||
"second_operand = find_parameter(function_calling_response, \"second_operand\")\n",
|
||||
"operator = find_parameter(function_calling_response, \"operator\")\n",
|
||||
"\n",
|
||||
"if first_operand and second_operand and operator:\n",
|
||||
" result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)\n",
|
||||
" print(\"---------------- RESULT ----------------\")\n",
|
||||
" print(f\"{result:,}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that we have a result, we have to properly format that result so that when we pass it back to Claude, Claude understands what tool that result is in relation to. There is a set format for this that Claude has been trained to recognize:\n",
|
||||
"```\n",
|
||||
"<function_results>\n",
|
||||
"<result>\n",
|
||||
"<tool_name>{TOOL_NAME}</tool_name>\n",
|
||||
"<stdout>\n",
|
||||
"{TOOL_RESULT}\n",
|
||||
"</stdout>\n",
|
||||
"</result>\n",
|
||||
"</function_results>\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Run the cell below to format the above tool result into this structure."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_successful_function_run_injection_prompt(invoke_results):\n",
|
||||
" constructed_prompt = (\n",
|
||||
" \"<function_results>\\n\"\n",
|
||||
" + '\\n'.join(\n",
|
||||
" f\"<result>\\n<tool_name>{res['tool_name']}</tool_name>\\n<stdout>\\n{res['tool_result']}\\n</stdout>\\n</result>\"\n",
|
||||
" for res in invoke_results\n",
|
||||
" ) + \"\\n</function_results>\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return constructed_prompt\n",
|
||||
"\n",
|
||||
"formatted_results = [{\n",
|
||||
" 'tool_name': 'do_pairwise_arithmetic',\n",
|
||||
" 'tool_result': result\n",
|
||||
"}]\n",
|
||||
"function_results = construct_successful_function_run_injection_prompt(formatted_results)\n",
|
||||
"print(function_results)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now all we have to do is send this result back to Claude by appending the result to the same message chain as before, and we're good!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"full_first_response = function_calling_response + \"</function_calls>\"\n",
|
||||
"\n",
|
||||
"# Construct the full conversation\n",
|
||||
"messages = [multiplication_message,\n",
|
||||
"{\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": full_first_response\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": function_results\n",
|
||||
"}]\n",
|
||||
" \n",
|
||||
"# Print Claude's response\n",
|
||||
"final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(\"------------- FINAL RESULT -------------\")\n",
|
||||
"print(final_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Congratulations on running an entire tool use chain end to end!\n",
|
||||
"\n",
|
||||
"Now what if we give Claude a question that doesn't that doesn't require using the given tool at all?"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"non_multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Tell me the capital of France.\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Success! As you can see, Claude knew not to call the function when it wasn't needed.\n",
|
||||
"\n",
|
||||
"If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Exercises\n",
|
||||
"- [Exercise 10.2.1 - SQL](#exercise-1021---SQL)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise 10.2.1 - SQL\n",
|
||||
"In this exercise, you'll be writing a tool use prompt for querying and writing to the world's smallest \"database\". Here's the initialized database, which is really just a dictionary."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"db = {\n",
|
||||
" \"users\": [\n",
|
||||
" {\"id\": 1, \"name\": \"Alice\", \"email\": \"alice@example.com\"},\n",
|
||||
" {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@example.com\"},\n",
|
||||
" {\"id\": 3, \"name\": \"Charlie\", \"email\": \"charlie@example.com\"}\n",
|
||||
" ],\n",
|
||||
" \"products\": [\n",
|
||||
" {\"id\": 1, \"name\": \"Widget\", \"price\": 9.99},\n",
|
||||
" {\"id\": 2, \"name\": \"Gadget\", \"price\": 14.99},\n",
|
||||
" {\"id\": 3, \"name\": \"Doohickey\", \"price\": 19.99}\n",
|
||||
" ]\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"And here is the code for the functions that write to and from the database."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_user(user_id):\n",
|
||||
" for user in db[\"users\"]:\n",
|
||||
" if user[\"id\"] == user_id:\n",
|
||||
" return user\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def get_product(product_id):\n",
|
||||
" for product in db[\"products\"]:\n",
|
||||
" if product[\"id\"] == product_id:\n",
|
||||
" return product\n",
|
||||
" return None\n",
|
||||
"\n",
|
||||
"def add_user(name, email):\n",
|
||||
" user_id = len(db[\"users\"]) + 1\n",
|
||||
" user = {\"id\": user_id, \"name\": name, \"email\": email}\n",
|
||||
" db[\"users\"].append(user)\n",
|
||||
" return user\n",
|
||||
"\n",
|
||||
"def add_product(name, price):\n",
|
||||
" product_id = len(db[\"products\"]) + 1\n",
|
||||
" product = {\"id\": product_id, \"name\": name, \"price\": price}\n",
|
||||
" db[\"products\"].append(product)\n",
|
||||
" return product"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To solve the exercise, start by defining a system prompt like `system_prompt_tools_specific_tools` above. Make sure to include the name and description of each tool, along with the name and type and description of each parameter for each function. We've given you some starting scaffolding below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools_sql = \"\"\"\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools_sql"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When you're ready, you can try out your tool definition system prompt on the examples below. Just run the below cell!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"examples = [\n",
|
||||
" \"Add a user to the database named Deborah.\",\n",
|
||||
" \"Add a product to the database named Thingo\",\n",
|
||||
" \"Tell me the name of User 2\",\n",
|
||||
" \"Tell me the name of Product 3\"\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"for example in examples:\n",
|
||||
" message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": example\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" # Get & print Claude's response\n",
|
||||
" function_calling_response = get_completion([message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
" print(example, \"\\n----------\\n\\n\", function_calling_response, \"\\n*********\\n*********\\n*********\\n\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you did it right, the function calling messages should call the `add_user`, `add_product`, `get_user`, and `get_product` functions correctly.\n",
|
||||
"\n",
|
||||
"For extra credit, add some code cells and write parameter-parsing code. Then call the functions with the parameters Claude gives you to see the state of the \"database\" after the call."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"❓ If you want to see a possible solution, run the cell below!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from hints import exercise_10_2_1_solution; print(exercise_10_2_1_solution)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Congrats!\n",
|
||||
"\n",
|
||||
"Congratulations on learning tool use and function calling! Head over to the last appendix section if you would like to learn more about search & RAG."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Example Playground\n",
|
||||
"\n",
|
||||
"This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_general_explanation = \"\"\"You have access to a set of functions you can use to answer the user's question. This includes access to a\n",
|
||||
"sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external\n",
|
||||
"resources, except by invoking the below functions.\n",
|
||||
"\n",
|
||||
"You can invoke one or more functions by writing a \"<function_calls>\" block like the following as part of your\n",
|
||||
"reply to the user:\n",
|
||||
"<function_calls>\n",
|
||||
"<invoke name=\"$FUNCTION_NAME\">\n",
|
||||
"<antml:parameter name=\"$PARAMETER_NAME\">$PARAMETER_VALUE</parameter>\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"<nvoke name=\"$FUNCTION_NAME2\">\n",
|
||||
"...\n",
|
||||
"</invoke>\n",
|
||||
"</function_calls>\n",
|
||||
"\n",
|
||||
"String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that\n",
|
||||
"spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular\n",
|
||||
"expressions.\n",
|
||||
"\n",
|
||||
"The output and/or any errors will appear in a subsequent \"<function_results>\" block, and remain there as part of\n",
|
||||
"your reply to the user.\n",
|
||||
"You may then continue composing the rest of your reply to the user, respond to any errors, or make further function\n",
|
||||
"calls as appropriate.\n",
|
||||
"If a \"<function_results>\" does NOT appear after your function calls, then they are likely malformatted and not\n",
|
||||
"recognized as a call.\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"system_prompt_tools_specific_tools = \"\"\"Here are the functions available in JSONSchema format:\n",
|
||||
"<tools>\n",
|
||||
"<tool_description>\n",
|
||||
"<tool_name>calculator</tool_name>\n",
|
||||
"<description>\n",
|
||||
"Calculator function for doing basic arithmetic.\n",
|
||||
"Supports addition, subtraction, multiplication\n",
|
||||
"</description>\n",
|
||||
"<parameters>\n",
|
||||
"<parameter>\n",
|
||||
"<name>first_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>First operand (before the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>second_operand</name>\n",
|
||||
"<type>int</type>\n",
|
||||
"<description>Second operand (after the operator)</description>\n",
|
||||
"</parameter>\n",
|
||||
"<parameter>\n",
|
||||
"<name>operator</name>\n",
|
||||
"<type>str</type>\n",
|
||||
"<description>The operation to perform. Must be either +, -, *, or /</description>\n",
|
||||
"</parameter>\n",
|
||||
"</parameters>\n",
|
||||
"</tool_description>\n",
|
||||
"</tools>\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Multiply 1,984,135 by 9,343,116\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def do_pairwise_arithmetic(num1, num2, operation):\n",
|
||||
" if operation == '+':\n",
|
||||
" return num1 + num2\n",
|
||||
" elif operation == \"-\":\n",
|
||||
" return num1 - num2\n",
|
||||
" elif operation == \"*\":\n",
|
||||
" return num1 * num2\n",
|
||||
" elif operation == \"/\":\n",
|
||||
" return num1 / num2\n",
|
||||
" else:\n",
|
||||
" return \"Error: Operation not supported.\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def find_parameter(message, parameter_name):\n",
|
||||
" parameter_start_string = f\"name=\\\"{parameter_name}\\\">\"\n",
|
||||
" start = message.index(parameter_start_string)\n",
|
||||
" if start == -1:\n",
|
||||
" return None\n",
|
||||
" if start > 0:\n",
|
||||
" start = start + len(parameter_start_string)\n",
|
||||
" end = start\n",
|
||||
" while message[end] != \"<\":\n",
|
||||
" end += 1\n",
|
||||
" return message[start:end]\n",
|
||||
"\n",
|
||||
"first_operand = find_parameter(function_calling_response, \"first_operand\")\n",
|
||||
"second_operand = find_parameter(function_calling_response, \"second_operand\")\n",
|
||||
"operator = find_parameter(function_calling_response, \"operator\")\n",
|
||||
"\n",
|
||||
"if first_operand and second_operand and operator:\n",
|
||||
" result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)\n",
|
||||
" print(\"---------------- RESULT ----------------\")\n",
|
||||
" print(f\"{result:,}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_successful_function_run_injection_prompt(invoke_results):\n",
|
||||
" constructed_prompt = (\n",
|
||||
" \"<function_results>\\n\"\n",
|
||||
" + '\\n'.join(\n",
|
||||
" f\"<result>\\n<tool_name>{res['tool_name']}</tool_name>\\n<stdout>\\n{res['tool_result']}\\n</stdout>\\n</result>\"\n",
|
||||
" for res in invoke_results\n",
|
||||
" ) + \"\\n</function_results>\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return constructed_prompt\n",
|
||||
"\n",
|
||||
"formatted_results = [{\n",
|
||||
" 'tool_name': 'do_pairwise_arithmetic',\n",
|
||||
" 'tool_result': result\n",
|
||||
"}]\n",
|
||||
"function_results = construct_successful_function_run_injection_prompt(formatted_results)\n",
|
||||
"print(function_results)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"full_first_response = function_calling_response + \"</function_calls>\"\n",
|
||||
"\n",
|
||||
"# Construct the full conversation\n",
|
||||
"messages = [multiplication_message,\n",
|
||||
"{\n",
|
||||
" \"role\": \"assistant\",\n",
|
||||
" \"content\": full_first_response\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": function_results\n",
|
||||
"}]\n",
|
||||
" \n",
|
||||
"# Print Claude's response\n",
|
||||
"final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(\"------------- FINAL RESULT -------------\")\n",
|
||||
"print(final_response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"non_multiplication_message = {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Tell me the capital of France.\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"stop_sequences = [\"</function_calls>\"]\n",
|
||||
"\n",
|
||||
"# Get Claude's response\n",
|
||||
"function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)\n",
|
||||
"print(function_calling_response)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Appendix 10.3: Search & Retrieval\n",
|
||||
"\n",
|
||||
"Did you know you can use Claude to **search through Wikipedia for you**? Claude can find and retrieve articles, at which point you can also use Claude to summarize and synthesize them, write novel content from what it found, and much more. And not just Wikipedia! You can also search over your own docs, whether stored as plain text or embedded in a vector datastore.\n",
|
||||
"\n",
|
||||
"See our [RAG cookbook examples](https://github.com/anthropics/anthropic-cookbook/blob/main/third_party/Wikipedia/wikipedia-search-cookbook.ipynb) to learn how to supplement Claude's knowledge and improve the accuracy and relevance of Claude's responses with data retrieved from vector databases, Wikipedia, the internet, and more. There, you can also learn about how to use certain [embeddings](https://docs.anthropic.com/claude/docs/embeddings) and vector database tools.\n",
|
||||
"\n",
|
||||
"If you are interested in learning about advanced RAG architectures using Claude, check out our [Claude 3 technical presentation slides on RAG architectures](https://docs.google.com/presentation/d/1zxkSI7lLUBrZycA-_znwqu8DDyVhHLkQGScvzaZrUns/edit#slide=id.g2c736259dac_63_782)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,246 +0,0 @@
|
||||
exercise_1_1_hint = """The grading function in this exercise is looking for an answer that contains the exact Arabic numerals "1", "2", and "3".
|
||||
You can often get Claude to do what you want simply by asking."""
|
||||
|
||||
exercise_1_2_hint = """The grading function in this exercise is looking for answers that contain "soo" or "giggles".
|
||||
There are many ways to solve this, just by asking!"""
|
||||
|
||||
exercise_2_1_hint ="""The grading function in this exercise is looking for any answer that includes the word "hola".
|
||||
Ask Claude to reply in Spanish like you would when speaking with a human. It's that simple!"""
|
||||
|
||||
exercise_2_2_hint = """The grading function in this exercise is looking for EXACTLY "Michael Jordan".
|
||||
How would you ask another human to do this? Reply with no other words? Reply with only the name and nothing else? There are several ways to approach this answer."""
|
||||
|
||||
exercise_2_3_hint = """The grading function in this cell is looking for a response that is equal to or greater than 800 words.
|
||||
Because LLMs aren't great at counting words yet, you may have to overshoot your target."""
|
||||
|
||||
exercise_3_1_hint = """The grading function in this exercise is looking for an answer that includes the words "incorrect" or "not correct".
|
||||
Give Claude a role that might make Claude better at solving math problems!"""
|
||||
|
||||
exercise_4_1_hint = """The grading function in this exercise is looking for a solution that includes the words "haiku" and "pig".
|
||||
Don't forget to include the exact phrase "{TOPIC}" wherever you want the topic to be substituted in. Changing the "TOPIC" variable value should make Claude write a haiku about a different topic."""
|
||||
|
||||
exercise_4_2_hint = """The grading function in this exercise is looking for a response that includes the word "brown".
|
||||
If you surround "{QUESTION}" in XML tags, how does that change Claude's response?"""
|
||||
|
||||
exercise_4_3_hint = """The grading function in this exercise is looking for a response that includes the word "brown".
|
||||
Try removing one word or section of characters at a time, starting with the parts that make the least sense. Doing this one word at a time will also help you see just how much Claude can or can't parse and understand."""
|
||||
|
||||
exercise_5_1_hint = """The grading function for this exercise is looking for a response that includes the word "Warrior".
|
||||
Write more words in Claude's voice to steer Claude to act the way you want it to. For instance, instead of "Stephen Curry is the best because," you could write "Stephen Curry is the best and here are three reasons why. 1:"""
|
||||
|
||||
exercise_5_2_hint = """The grading function looks for a response of over 5 lines in length that includes the words "cat" and "<haiku>".
|
||||
Start simple. Currently, the prompt asks Claude for one haiku. You can change that and ask for two (or even more). Then if you run into formatting issues, change your prompt to fix that after you've already gotten Claude to write more than one haiku."""
|
||||
|
||||
exercise_5_3_hint = """The grading function in this exercise is looking for a response that contains the words "tail", "cat", and "<haiku>".
|
||||
It's helpful to break this exercise down to several steps.
|
||||
1. Modify the initial prompt template so that Claude writes two poems.
|
||||
2. Give Claude indicators as to what the poems will be about, but instead of writing in the subjects directly (e.g., dog, cat, etc.), replace those subjects with the keywords "{ANIMAL1}" and "{ANIMAL2}".
|
||||
3. Run the prompt and make sure that the full prompt with variable substitutions has all the words correctly substituted. If not, check to make sure your {bracket} tags are spelled correctly and formatted correctly with single moustache brackets."""
|
||||
|
||||
exercise_6_1_hint = """The grading function in this exercise is looking for the correct categorization letter + the closing parentheses and the first letter of the name of the category, such as "C) B" or "B) B" etc.
|
||||
Let's take this exercise step by step:
|
||||
1. How will Claude know what categories you want to use? Tell it! Include the four categories you want directly in the prompt. Be sure to include the parenthetical letters as well for easy classification. Feel free to use XML tags to organize your prompt and make clear to Claude where the categories begin and end.
|
||||
2. Try to cut down on superfluous text so that Claude immediately answers with the classification and ONLY the classification. There are several ways to do this, from speaking for Claude (providing anything from the beginning of the sentence to a single open parenthesis so that Claude knows you want the parenthetical letter as the first part of the answer) to telling Claude that you want the classification and only the classification, skipping the preamble.
|
||||
Refer to Chapters 2 and 5 if you want a refresher on these techniques.
|
||||
3. Claude may still be incorrectly categorizing or not including the names of the categories when it answers. Fix this by telling Claude to include the full category name in its answer.)
|
||||
4. Be sure that you still have {email} somewhere in your prompt template so that we can properly substitute in emails for Claude to evaluate."""
|
||||
|
||||
exercise_6_1_solution = """
|
||||
USER TURN
|
||||
Please classify this email into the following categories: {email}
|
||||
|
||||
Do not include any extra words except the category.
|
||||
|
||||
<categories>
|
||||
(A) Pre-sale question
|
||||
(B) Broken or defective item
|
||||
(C) Billing question
|
||||
(D) Other (please explain)
|
||||
</categories>
|
||||
|
||||
ASSISTANT TURN
|
||||
(
|
||||
"""
|
||||
|
||||
exercise_6_2_hint = """The grading function in this exercise is looking for only the correct letter wrapped in <answer> tags, such as "<answer>B</answer>". The correct categorization letters are the same as in the above exercise.
|
||||
Sometimes the simplest way to go about this is to give Claude an example of how you want its output to look. Just don't forget to wrap your example in <example></example> tags! And don't forget that if you prefill Claude's response with anything, Claude won't actually output that as part of its response."""
|
||||
|
||||
exercise_7_1_hint = """You're going to have to write some example emails and classify them for Claude (with the exact formatting you want). There are multiple ways to do this. Here are some guidelines below.
|
||||
1. Try to have at least two example emails. Claude doesn't need an example for all categories, and the examples don't have to be long. It's more helpful to have examples for whatever you think the trickier categories are (which you were asked to think about at the bottom of Chapter 6 Exercise 1). XML tags will help you separate out your examples from the rest of your prompt, although it's unnecessary.
|
||||
2. Make sure your example answer formatting is exactly the format you want Claude to use, so Claude can emulate the format as well. This format should make it so that Claude's answer ends in the letter of the category. Wherever you put the {email} placeholder, make sure that it's formatted exactly like your example emails.
|
||||
3. Make sure you still have the categories listed within the prompt itself, otherwise Claude won't know what categories to reference, as well as {email} as a placeholder for substitution."""
|
||||
|
||||
exercise_7_1_solution = """
|
||||
USER TURN
|
||||
Please classify emails into the following categories, and do not include explanations:
|
||||
<categories>
|
||||
(A) Pre-sale question
|
||||
(B) Broken or defective item
|
||||
(C) Billing question
|
||||
(D) Other (please explain)
|
||||
</categories>
|
||||
|
||||
Here are a few examples of correct answer formatting:
|
||||
<examples>
|
||||
Q: How much does it cost to buy a Mixmaster4000?
|
||||
A: The correct category is: A
|
||||
|
||||
Q: My Mixmaster won't turn on.
|
||||
A: The correct category is: B
|
||||
|
||||
Q: Please remove me from your mailing list.
|
||||
A: The correct category is: D
|
||||
</examples>
|
||||
|
||||
Here is the email for you to categorize: {email}
|
||||
|
||||
ASSISTANT TURN
|
||||
The correct category is:
|
||||
"""
|
||||
exercise_8_1_hint = """The grading function in this exercise is looking for a response that contains the phrase "I do not", "I don't", or "Unfortunately".
|
||||
What should Claude do if it doesn't know the answer?"""
|
||||
|
||||
exercise_8_2_hint = """The grading function in this exercise is looking for a response that contains the phrase "49-fold".
|
||||
Make Claude show its work and thought process first by extracting relevant quotes and seeing whether or not the quotes provide sufficient evidence. Refer back to the Chapter 8 Lesson if you want a refresher."""
|
||||
|
||||
exercise_9_1_solution = """
|
||||
You are a master tax acountant. Your task is to answer user questions using any provided reference documentation.
|
||||
|
||||
Here is the material you should use to answer the user's question:
|
||||
<docs>
|
||||
{TAX_CODE}
|
||||
</docs>
|
||||
|
||||
Here is an example of how to respond:
|
||||
<example>
|
||||
<question>
|
||||
What defines a "qualified" employee?
|
||||
</question>
|
||||
<answer>
|
||||
<quotes>For purposes of this subsection—
|
||||
(A)In general
|
||||
The term "qualified employee" means any individual who—
|
||||
(i)is not an excluded employee, and
|
||||
(ii)agrees in the election made under this subsection to meet such requirements as are determined by the Secretary to be necessary to ensure that the withholding requirements of the corporation under chapter 24 with respect to the qualified stock are met.</quotes>
|
||||
|
||||
<answer>According to the provided documentation, a "qualified employee" is defined as an individual who:
|
||||
|
||||
1. Is not an "excluded employee" as defined in the documentation.
|
||||
2. Agrees to meet the requirements determined by the Secretary to ensure the corporation's withholding requirements under Chapter 24 are met with respect to the qualified stock.</answer>
|
||||
</example>
|
||||
|
||||
First, gather quotes in <quotes></quotes> tags that are relevant to answering the user's question. If there are no quotes, write "no relevant quotes found".
|
||||
|
||||
Then insert two paragraph breaks before answering the user question within <answer></answer> tags. Only answer the user's question if you are confident that the quotes in <quotes></quotes> tags support your answer. If not, tell the user that you unfortunately do not have enough information to answer the user's question.
|
||||
|
||||
Here is the user question: {QUESTION}
|
||||
"""
|
||||
|
||||
exercise_9_2_solution = """
|
||||
You are Codebot, a helpful AI assistant who finds issues with code and suggests possible improvements.
|
||||
|
||||
Act as a Socratic tutor who helps the user learn.
|
||||
|
||||
You will be given some code from a user. Please do the following:
|
||||
1. Identify any issues in the code. Put each issue inside separate <issues> tags.
|
||||
2. Invite the user to write a revised version of the code to fix the issue.
|
||||
|
||||
Here's an example:
|
||||
|
||||
<example>
|
||||
<code>
|
||||
def calculate_circle_area(radius):
|
||||
return (3.14 * radius) ** 2
|
||||
</code>
|
||||
<issues>
|
||||
<issue>
|
||||
3.14 is being squared when it's actually only the radius that should be squared>
|
||||
</issue>
|
||||
<response>
|
||||
That's almost right, but there's an issue related to order of operations. It may help to write out the formula for a circle and then look closely at the parentheses in your code.
|
||||
</response>
|
||||
</example>
|
||||
|
||||
Here is the code you are to analyze:
|
||||
|
||||
<code>
|
||||
{CODE}
|
||||
</code>
|
||||
|
||||
Find the relevant issues and write the Socratic tutor-style response. Do not give the user too much help! Instead, just give them guidance so they can find the correct solution themselves.
|
||||
|
||||
Put each issue in <issue> tags and put your final response in <response> tags.
|
||||
"""
|
||||
|
||||
exercise_10_2_1_solution = """system_prompt = system_prompt_tools_general_explanation + \"""Here are the functions available in JSONSchema format:
|
||||
|
||||
<tools>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>get_user</tool_name>
|
||||
<description>
|
||||
Retrieves a user from the database by their user ID.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>user_id</name>
|
||||
<type>int</type>
|
||||
<description>The ID of the user to retrieve.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>get_product</tool_name>
|
||||
<description>
|
||||
Retrieves a product from the database by its product ID.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>product_id</name>
|
||||
<type>int</type>
|
||||
<description>The ID of the product to retrieve.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>add_user</tool_name>
|
||||
<description>
|
||||
Adds a new user to the database.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>name</name>
|
||||
<type>str</type>
|
||||
<description>The name of the user.</description>
|
||||
</parameter>
|
||||
<parameter>
|
||||
<name>email</name>
|
||||
<type>str</type>
|
||||
<description>The email address of the user.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
<tool_description>
|
||||
<tool_name>add_product</tool_name>
|
||||
<description>
|
||||
Adds a new product to the database.
|
||||
</description>
|
||||
<parameters>
|
||||
<parameter>
|
||||
<name>name</name>
|
||||
<type>str</type>
|
||||
<description>The name of the product.</description>
|
||||
</parameter>
|
||||
<parameter>
|
||||
<name>price</name>
|
||||
<type>float</type>
|
||||
<description>The price of the product.</description>
|
||||
</parameter>
|
||||
</parameters>
|
||||
</tool_description>
|
||||
|
||||
</tools>
|
||||
"""
|
||||
@@ -1,58 +0,0 @@
|
||||
# Welcome to Anthropic's Prompt Engineering Interactive Tutorial
|
||||
|
||||
## Course introduction and goals
|
||||
|
||||
This course is intended to provide you with a comprehensive step-by-step understanding of how to engineer optimal prompts within Claude.
|
||||
|
||||
**After completing this course, you will be able to**:
|
||||
- Master the basic structure of a good prompt
|
||||
- Recognize common failure modes and learn the '80/20' techniques to address them
|
||||
- Understand Claude's strengths and weaknesses
|
||||
- Build strong prompts from scratch for common use cases
|
||||
|
||||
## Course structure and content
|
||||
|
||||
This course is structured to allow you many chances to practice writing and troubleshooting prompts yourself. The course is broken up into **9 chapters with accompanying exercises**, as well as an appendix of even more advanced methods. It is intended for you to **work through the course in chapter order**.
|
||||
|
||||
**Each lesson has an "Example Playground" area** at the bottom where you are free to experiment with the examples in the lesson and see for yourself how changing prompts can change Claude's responses. There is also an [answer key](https://docs.google.com/spreadsheets/d/1jIxjzUWG-6xBVIa2ay6yDpLyeuOh_hR_ZB75a47KX_E/edit?usp=sharing).
|
||||
|
||||
Note: This tutorial uses our smallest, fastest, and cheapest model, Claude 3 Haiku. Anthropic has [two other models](https://docs.anthropic.com/claude/docs/models-overview), Claude 3 Sonnet and Claude 3 Opus, which are more intelligent than Haiku, with Opus being the most intelligent.
|
||||
|
||||
*This tutorial also exists on [Google Sheets using Anthropic's Claude for Sheets extension](https://docs.google.com/spreadsheets/d/19jzLgRruG9kjUQNKtCg1ZjdD6l6weA6qRXG5zLIAhC8/edit?usp=sharing). We recommend using that version as it is more user friendly.*
|
||||
|
||||
When you are ready to begin, go to `01_Basic Prompt Structure` to proceed.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
Each chapter consists of a lesson and a set of exercises.
|
||||
|
||||
### Beginner
|
||||
- **Chapter 1:** Basic Prompt Structure
|
||||
|
||||
- **Chapter 2:** Being Clear and Direct
|
||||
|
||||
- **Chapter 3:** Assigning Roles
|
||||
|
||||
### Intermediate
|
||||
- **Chapter 4:** Separating Data from Instructions
|
||||
|
||||
- **Chapter 5:** Formatting Output & Speaking for Claude
|
||||
|
||||
- **Chapter 6:** Precognition (Thinking Step by Step)
|
||||
|
||||
- **Chapter 7:** Using Examples
|
||||
|
||||
### Advanced
|
||||
- **Chapter 8:** Avoiding Hallucinations
|
||||
|
||||
- **Chapter 9:** Building Complex Prompts (Industry Use Cases)
|
||||
- Complex Prompts from Scratch - Chatbot
|
||||
- Complex Prompts for Legal Services
|
||||
- **Exercise:** Complex Prompts for Financial Services
|
||||
- **Exercise:** Complex Prompts for Coding
|
||||
- Congratulations & Next Steps
|
||||
|
||||
- **Appendix:** Beyond Standard Prompting
|
||||
- Chaining Prompts
|
||||
- Tool Use
|
||||
- Search & Retrieval
|
||||
1
real_world_prompting/.gitignore
vendored
Normal file
1
real_world_prompting/.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.env
|
||||
@@ -4,7 +4,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Lesson 2: A real-world prompt\n",
|
||||
"# Lesson 2: a real-world prompt\n",
|
||||
"\n",
|
||||
"In the previous lesson, we discussed several key prompting tips and saw an example of how to use each in isolation. Let's now try writing a much larger prompt that incorporates many of the techniques we just covered.\n",
|
||||
"\n",
|
||||
@@ -120,7 +120,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -291,7 +291,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -300,7 +300,7 @@
|
||||
"'\\nPatient Name: Lily Chen\\nAge: 8\\nMedical Record:\\n\\n2016 (Birth): Born at 34 weeks, diagnosed with Tetralogy of Fallot (TOF)\\n - Immediate surgery to place a shunt for increased pulmonary blood flow\\n2016 (3 months): Echocardiogram showed worsening right ventricular hypertrophy\\n2017 (8 months): Complete repair of TOF (VSD closure, pulmonary valve replacement, RV outflow tract repair)\\n2017 (10 months): Developed post-operative arrhythmias, started on amiodarone\\n2018 (14 months): Developmental delay noted, referred to early intervention services\\n2018 (18 months): Speech therapy initiated for delayed language development\\n2019 (2 years): Diagnosed with failure to thrive, started on high-calorie diet\\n2019 (2.5 years): Occupational therapy started for fine motor skill delays\\n2020 (3 years): Cardiac catheterization showed mild pulmonary stenosis\\n2020 (3.5 years): Diagnosed with sensory processing disorder (SPD)\\n2021 (4 years): Started integrated preschool program with IEP (Individualized Education Plan)\\n2021 (4.5 years): Hospitalized for RSV bronchiolitis, required brief oxygen support\\n2022 (5 years): Echocardiogram showed progression of pulmonary stenosis, balloon valvuloplasty performed\\n2022 (5.5 years): Diagnosed with attention-deficit/hyperactivity disorder (ADHD), started behavioral therapy\\n2023 (6 years): Cochlear implant surgery for sensorineural hearing loss\\n2023 (7 years): Started mainstream school with continued IEP support\\n2024 (7.5 years): Occupational therapy discontinued, met fine motor skill goals\\n2024 (8 years): Periodic cardiac follow-up shows stable pulmonary valve function\\n2024 (8 years): Speech development progressing well, ongoing therapy\\n '"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -330,7 +330,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -352,20 +352,70 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 98,
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Note: you may need to restart the kernel to use updated packages.\n",
|
||||
"Your browser has been opened to visit:\n",
|
||||
"\n",
|
||||
" https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login&state=1tuG7GpenqE0tnrhJMJBgmtqMdUItk&access_type=offline&code_challenge=yhf6A8DPqH88gqJR8yLR16ZX_qyDSpfqQmEB8vKECro&code_challenge_method=S256\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Credentials saved to file: [/Users/elie/.config/gcloud/application_default_credentials.json]\n",
|
||||
"\n",
|
||||
"These credentials will be used by any library that requests Application Default Credentials (ADC).\n",
|
||||
"\n",
|
||||
"Quota project \"anthropic\" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Updates are available for some Google Cloud CLI components. To install them,\n",
|
||||
"please run:\n",
|
||||
" $ gcloud components update\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from anthropic import Anthropic\n",
|
||||
"%pip install -U --quiet python-dotenv google-cloud-aiplatform \"anthropic[vertex]\"\n",
|
||||
"!gcloud auth application-default login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"just-aloe-430520-u6 us-central1\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from anthropic import AnthropicVertex\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"load_dotenv()\n",
|
||||
"client = Anthropic()\n",
|
||||
"\n",
|
||||
"project_id = os.environ.get(\"PROJECT_ID\")\n",
|
||||
"# Where the model is running. e.g. us-central1 or europe-west4 for haiku\n",
|
||||
"region = os.environ.get(\"REGION\")\n",
|
||||
"\n",
|
||||
"client = AnthropicVertex(project_id=project_id, region=region)\n",
|
||||
"\n",
|
||||
"print(project_id, region)\n",
|
||||
"\n",
|
||||
"def generate_summary_with_bad_prompt(patient_record):\n",
|
||||
" prompt_with_record = initial_prompt.format(record=patient_record)\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" max_tokens=4096,\n",
|
||||
" messages=[{\"role\": \"user\", \"content\": prompt_with_record}]\n",
|
||||
" )\n",
|
||||
@@ -382,7 +432,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -390,32 +440,15 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"===============================\n",
|
||||
"Here is a summary of Evelyn Thompson's 78-year-old medical record:\n",
|
||||
"Here is a summary of Evelyn Thompson's medical record:\n",
|
||||
"\n",
|
||||
"Chronic Conditions:\n",
|
||||
"- Type 2 diabetes (since 1985) - on metformin, dose increased in 2010\n",
|
||||
"- Hypertension (since 1992) - on lisinopril, dose increased in 2020\n",
|
||||
"- Hypothyroidism (since 2000) - on levothyroxine \n",
|
||||
"- Atrial fibrillation (since 2005) - on warfarin\n",
|
||||
"- Vitamin B12 deficiency (since 2008) - receiving monthly injections\n",
|
||||
"- Chronic kidney disease stage 3 (since 2015) - metformin adjusted\n",
|
||||
"- Mild cognitive impairment (since 2019) - on donepezil\n",
|
||||
"Evelyn is a 78-year-old female with a long history of chronic medical conditions including type 2 diabetes since 1985 (on metformin), hypertension since 1992 (on lisinopril), hypothyroidism since 2000 (on levothyroxine), and atrial fibrillation since 2005 (on warfarin and aspirin). She has also had osteoarthritis requiring a right total hip replacement in 1998 and a left total knee replacement in 2017. \n",
|
||||
"\n",
|
||||
"Surgical History:\n",
|
||||
"- Total hip replacement (1998) - right side, due to osteoarthritis\n",
|
||||
"- Cataract surgery (2003) - both eyes\n",
|
||||
"- Lumpectomy and radiation (2013) - for stage 2 breast cancer \n",
|
||||
"- Total knee replacement (2017) - left side, due to osteoarthritis\n",
|
||||
"In 2003, she underwent cataract surgery bilaterally. She was diagnosed with vitamin B12 deficiency in 2008 requiring monthly injections. In 2011, she had a transient ischemic attack. She was diagnosed with stage 2 breast cancer in 2013, treated with lumpectomy, radiation, and anastrozole.\n",
|
||||
"\n",
|
||||
"Cancer History: \n",
|
||||
"- Breast cancer (2013) - currently on anastrozole for recurrence prevention\n",
|
||||
"She developed chronic kidney disease stage 3 in 2015 requiring metformin adjustment. Other issues include pneumonia in 2018 requiring hospitalization, mild cognitive impairment since 2019 (on donepezil), refractory hypertension in 2020, and recurrent UTIs in 2021 treated with prophylactic antibiotics.\n",
|
||||
"\n",
|
||||
"Recent Issues:\n",
|
||||
"- Recurrent UTIs (2021) - on prophylactic antibiotics\n",
|
||||
"- Worsening kidney function per eGFR (2022)\n",
|
||||
"- Declining mobility (2023) - started physical therapy and home health aide\n",
|
||||
"\n",
|
||||
"Overall, an elderly patient with multiple chronic conditions requiring polypharmacy and close monitoring, especially for diabetes, hypertension, kidney disease, and cancer recurrence.\n"
|
||||
"Recent issues are worsening kidney function based on eGFR in 2022 and declining mobility in 2023 requiring physical therapy and home health aide.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1151,7 +1184,7 @@
|
||||
"def generate_summary_with_improved_prompt(patient_record):\n",
|
||||
" prompt_with_record = updated_prompt.format(record=patient_record) #use our rewritten prompt!\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" max_tokens=4096,\n",
|
||||
" system=system, #add in our system prompt!\n",
|
||||
" messages=[{\"role\": \"user\", \"content\": prompt_with_record}]\n",
|
||||
@@ -1687,7 +1720,7 @@
|
||||
" final_prompt_part = medical_record_input_prompt.format(record=patient_record) #add the medical record to the final prompt piece\n",
|
||||
" complete_prompt = updated_json_prompt + final_prompt_part\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" max_tokens=4096,\n",
|
||||
" system=system, #add in our system prompt!\n",
|
||||
" messages=[{\"role\": \"user\", \"content\": complete_prompt}]\n",
|
||||
@@ -1860,7 +1893,7 @@
|
||||
" final_prompt_part = medical_record_input_prompt.format(record=patient_record) #add the medical record to the final prompt piece\n",
|
||||
" complete_prompt = updated_json_prompt + final_prompt_part\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" max_tokens=4096,\n",
|
||||
" system=system, #add in our system prompt!\n",
|
||||
" messages=[{\"role\": \"user\", \"content\": complete_prompt}]\n",
|
||||
@@ -1959,7 +1992,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.6"
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Lesson 3: Prompt engineering\n",
|
||||
"# Lesson 3: Prompt Engineering\n",
|
||||
"\n",
|
||||
"In the first lesson, we quickly reviewed some key prompting tips. In the second lesson, we wrote a prompt that \"blindly\" applied all of those tips to a single prompt. Understanding these tips is critical, but it's equally important to understand the prompt engineering workflow and decision making framework.\n",
|
||||
"\n",
|
||||
|
||||
@@ -47,7 +47,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -71,7 +71,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -101,7 +101,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 44,
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -152,7 +152,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 45,
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -179,21 +179,54 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 46,
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Note: you may need to restart the kernel to use updated packages.\n",
|
||||
"Your browser has been opened to visit:\n",
|
||||
"\n",
|
||||
" https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login&state=1JlV7BunjhxeVP0sHTct8UQyia4vQW&access_type=offline&code_challenge=qx8gVXITZrRA8x4zSIulz7tYTCgNRtPLYti6p2dEna8&code_challenge_method=S256\n",
|
||||
"\n",
|
||||
"^C\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Command killed by keyboard interrupt\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%pip install --quiet -U python-dotenv google-cloud-aiplatform \"anthropic[vertex]\"\n",
|
||||
"!gcloud auth application-default login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from anthropic import Anthropic\n",
|
||||
"from anthropic import AnthropicVertex\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"load_dotenv()\n",
|
||||
"client = Anthropic()\n",
|
||||
"\n",
|
||||
"project_id = os.environ.get(\"PROJECT_ID\")\n",
|
||||
"# Where the model is running. e.g. us-central1 or europe-west4 for haiku\n",
|
||||
"region = os.environ.get(\"REGION\")\n",
|
||||
"\n",
|
||||
"client = AnthropicVertex(project_id=project_id, region=region)\n",
|
||||
"\n",
|
||||
"def summarize_call(transcript):\n",
|
||||
" final_prompt = prompt.format(transcript=transcript)\n",
|
||||
" # Make the API call\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" max_tokens=4096,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": final_prompt}\n",
|
||||
@@ -204,7 +237,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 47,
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -213,14 +246,13 @@
|
||||
"text": [
|
||||
"Here is a summary of the customer service call transcript:\n",
|
||||
"\n",
|
||||
"Main Issue:\n",
|
||||
"The customer was unable to turn on their Acme smart light bulb.\n",
|
||||
"Main Issue: The customer was unable to turn on their smart light bulb. \n",
|
||||
"\n",
|
||||
"Resolution:\n",
|
||||
"The service agent instructed the customer to reset the bulb by turning the power off for 5 seconds and then back on. This should reset the bulb and allow it to turn on.\n",
|
||||
"Resolution: The customer service agent instructed the customer to reset the bulb by turning the power off for 5 seconds and then back on, which should reset the bulb.\n",
|
||||
"\n",
|
||||
"Follow-Up:\n",
|
||||
"The agent told the customer to call back if they continued to have issues after trying the reset procedure. No other follow-up was mentioned.\n"
|
||||
"Follow-up: The agent told the customer to call back if they still needed further assistance after trying the reset procedure.\n",
|
||||
"\n",
|
||||
"Overall, it was a straightforward issue where the agent provided a simple troubleshooting step to potentially resolve the customer's problem with the smart light bulb not turning on. No major follow-up was required beyond checking if the reset worked or if the customer needed additional help.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -230,7 +262,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 48,
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -239,11 +271,11 @@
|
||||
"text": [
|
||||
"Summary:\n",
|
||||
"\n",
|
||||
"Main Issue: The customer's Acme SmartTherm thermostat was not maintaining the set temperature of 72°F, and the house was much warmer.\n",
|
||||
"Main Issue: The customer's Acme SmartTherm thermostat was not maintaining the set temperature correctly. The thermostat was set to 72°F, but the house was much warmer.\n",
|
||||
"\n",
|
||||
"Resolution: The agent guided the customer through the process of recalibrating the SmartTherm thermostat. This involved accessing the \"Calibration\" menu, adjusting the temperature to match the customer's room thermometer (79°F in this case), and confirming the new setting. The recalibration process may take a few minutes to complete.\n",
|
||||
"Resolution: The agent guided the customer through recalibrating the SmartTherm thermostat. This involved accessing the \"Calibration\" menu, adjusting the temperature to match a separate room thermometer reading of 79°F, and confirming the new calibration setting. The recalibration process may take a few minutes.\n",
|
||||
"\n",
|
||||
"Follow-up Required: The customer was advised to check the thermostat in an hour to see if the issue was resolved after the recalibration process completed.\n"
|
||||
"Follow-up: The agent instructed the customer to check back in an hour to see if the recalibration fixed the temperature issue with the SmartTherm thermostat maintaining the desired temperature setting.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -253,7 +285,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 49,
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -263,13 +295,13 @@
|
||||
"Here is a summary of the customer service call transcript:\n",
|
||||
"\n",
|
||||
"Main Issue:\n",
|
||||
"The customer was having an issue with their Acme SecureHome alarm system going off randomly in the middle of the night, even though all doors and windows were closed properly.\n",
|
||||
"The customer was having an issue with their Acme SecureHome security system, where the alarm kept going off randomly in the middle of the night, despite no apparent cause.\n",
|
||||
"\n",
|
||||
"How It Was Resolved:\n",
|
||||
"The customer service agent first had the customer check for any error messages on the control panel and confirm that the battery was not low. When those basic troubleshooting steps did not reveal the issue, the agent determined that one of the sensors may be malfunctioning and needed to transfer the customer to the technical support team for a full system diagnostic.\n",
|
||||
"How it was Resolved:\n",
|
||||
"The agent first checked if the issue could be caused by doors/windows not closing properly or a low battery in the control panel, but ruled those out based on the customer's responses. Unable to diagnose the issue further, the agent transferred the call to the technical team so they could run a full diagnostic on the system to identify the root cause, which was likely a malfunctioning sensor.\n",
|
||||
"\n",
|
||||
"Required Follow-Up:\n",
|
||||
"The technical support team needs to run a diagnostic on the customer's SecureHome system to identify which sensor(s) may be causing the false alarms and then repair or replace those components. The customer should be contacted again once the diagnostic is complete and the repair/replacement has been performed to ensure the random alarms have been resolved.\n"
|
||||
"The technical team needs to complete the system diagnostic on the customer's SecureHome system to pinpoint the faulty sensor or component causing the random alarms. Once identified, they can work to replace/repair that part to resolve the issue permanently for the customer.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -304,7 +336,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 50,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -342,7 +374,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 51,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -397,7 +429,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 52,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -445,7 +477,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 53,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -481,7 +513,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 56,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -537,7 +569,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 57,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -545,7 +577,7 @@
|
||||
" final_prompt = prompt.replace(\"[INSERT CALL TRANSCRIPT HERE]\", transcript)\n",
|
||||
" # Make the API call\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" system=system,\n",
|
||||
" max_tokens=4096,\n",
|
||||
" messages=[\n",
|
||||
@@ -564,7 +596,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 58,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -600,7 +632,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -642,7 +674,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -687,7 +719,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -783,7 +815,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 65,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -827,7 +859,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -862,7 +894,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -898,7 +930,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -930,7 +962,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 69,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1015,7 +1047,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 70,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -1046,7 +1078,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 71,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -1183,7 +1215,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 75,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -1394,7 +1426,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 76,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -1404,7 +1436,7 @@
|
||||
" final_prompt = prompt.replace(\"[INSERT CALL TRANSCRIPT HERE]\", transcript)\n",
|
||||
" # Make the API call\n",
|
||||
" response = client.messages.create(\n",
|
||||
" model=\"claude-3-sonnet-20240229\",\n",
|
||||
" model=\"claude-3-sonnet@20240229\",\n",
|
||||
" system=system,\n",
|
||||
" max_tokens=4096,\n",
|
||||
" messages=[\n",
|
||||
@@ -1430,7 +1462,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 77,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1456,7 +1488,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 78,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1489,7 +1521,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 79,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1526,7 +1558,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 80,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1545,7 +1577,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 82,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1564,7 +1596,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 83,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1590,7 +1622,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 84,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1609,7 +1641,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 85,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1680,7 +1712,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.6"
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Lesson 5: Customer support prompt\n",
|
||||
"## Lesson 5: customer support prompt\n",
|
||||
"\n",
|
||||
"In this lesson, we'll work on building a customer support chatbot prompt. Our goal is to build a virtual support bot called \"Acme Assistant\" for a fictional company called Acme Software Solutions. This fictional company sells a piece of software called AcmeOS, and the chatbot's job is to help answer customer questions around things like installation, error codes, troubleshooting, etc.\n",
|
||||
"\n",
|
||||
@@ -175,18 +175,33 @@
|
||||
"Next, let's write a function that we can use that will combine the various parts of the prompt and send a request to Claude."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install -U python-dotenv google-cloud-aiplatform \"anthropic[vertex]\"\n",
|
||||
"!gcloud auth application-default login"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from anthropic import Anthropic\n",
|
||||
"from anthropic import AnthropicVertex\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"import json\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"load_dotenv()\n",
|
||||
"client = Anthropic()\n",
|
||||
"\n",
|
||||
"project_id = os.environ.get(\"PROJECT_ID\")\n",
|
||||
"# Where the model is running. e.g. us-central1 or europe-west4 for haiku\n",
|
||||
"region = os.environ.get(\"REGION\")\n",
|
||||
"\n",
|
||||
"client = AnthropicVertex(project_id=project_id, region=region)\n",
|
||||
"\n",
|
||||
"def answer_question_first_attempt(question):\n",
|
||||
" system = \"\"\"\n",
|
||||
@@ -207,7 +222,7 @@
|
||||
" # Send a request to Claude\n",
|
||||
" response = client.messages.create(\n",
|
||||
" system=system,\n",
|
||||
" model=\"claude-3-haiku-20240307\",\n",
|
||||
" model=\"claude-3-haiku@20240307\",\n",
|
||||
" max_tokens=2000,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": final_prompt} \n",
|
||||
@@ -622,7 +637,7 @@
|
||||
" # Send a request to Claude\n",
|
||||
" response = client.messages.create(\n",
|
||||
" system=system,\n",
|
||||
" model=\"claude-3-haiku-20240307\",\n",
|
||||
" model=\"claude-3-haiku@20240307\",\n",
|
||||
" max_tokens=2000,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": final_prompt} \n",
|
||||
@@ -638,6 +653,13 @@
|
||||
"Let's start by making sure it still works when answering basic user questions:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
@@ -1038,7 +1060,7 @@
|
||||
" # Send a request to Claude\n",
|
||||
" response = client.messages.create(\n",
|
||||
" system=system,\n",
|
||||
" model=\"claude-3-haiku-20240307\",\n",
|
||||
" model=\"claude-3-haiku@20240307\",\n",
|
||||
" max_tokens=2000,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": final_prompt} \n",
|
||||
@@ -1250,7 +1272,7 @@
|
||||
" # Send a request to Claude\n",
|
||||
" response = client.messages.create(\n",
|
||||
" system=system,\n",
|
||||
" model=\"claude-3-haiku-20240307\",\n",
|
||||
" model=\"claude-3-haiku@20240307\",\n",
|
||||
" max_tokens=2000,\n",
|
||||
" messages=[\n",
|
||||
" {\"role\": \"user\", \"content\": final_prompt} \n",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Real world prompting course
|
||||
|
||||
Welcome to Anthropic's comprehensive real world prompting tutorial. This course is designed for experienced developers who have already dipped their toes into the world of prompt engineering, particularly those who have completed our comprehensive **[Prompt engineering interactive tutorial](../prompt_engineering_interactive_tutorial/README.md)**. If you haven't gone through that tutorial yet, we strongly recommend you do so before continuing, as it provides an in-depth exploration of various prompting techniques with hands-on exercises.
|
||||
Welcome to Anthropic's comprehensive real world prompting tutorial. This course is designed for experienced developers who have already dipped their toes into the world of prompt engineering.
|
||||
|
||||
Across five lessons, you will learn how to incorporate key prompting techniques into complex, real world prompts. We recommend that you start from the beginning with the [Prompting recap](./01_prompting_recap.ipynb) lesson, as each lesson builds on key concepts taught in previous ones.
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1,11 +0,0 @@
|
||||
# Tool use tutorial
|
||||
|
||||
Welcome to Anthropic's comprehensive tool use tutorial. Across six lessons, you will learn everything you need to know to implement tool use successfully in your workflows with Claude. We recommend that you start from the beginning with the [tool use overview](./01_tool_use_overview.ipynb), as each lesson builds on key concepts taught in previous ones.
|
||||
|
||||
## Table of contents
|
||||
* [Tool use overview](./01_tool_use_overview.ipynb)
|
||||
* [Your first simple tool](./02_your_first_simple_tool.ipynb)
|
||||
* [Forcing JSON with tool use](./03_structured_outputs.ipynb)
|
||||
* [The complete tool use workflow](./04_complete_workflow.ipynb)
|
||||
* [Tool choice](./05_tool_choice.ipynb)
|
||||
* [Building a chatbot with multiple tools](./06_chatbot_with_multiple_tools.ipynb)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user