New Insights from Cloud Tech Blogger
A loud bang echoed in the sky! Anthropic’s latest advanced foundational model, Claude 3 Sonnet, has officially landed on Amazon Bedrock! Cloud Tech Blogger: Florida Little Li Brother (Li Shaoyi) has already hands-on experienced Claude 3 and compiled a detailed tutorial, come and take a look! There are also certification exams and shortcuts to learn Amazon Bedrock at the end of the article, stay tuned for surprises, earning is gaining!
How to Interact with Claude 3 on Amazon Bedrock through API
Cloud Tech Blogger
Author: Li Shaoyi (Florida Little Li Brother)
“How to Interact with Claude 3 on Amazon Bedrock through API“
Published on: CSDN
1
How to call the API to access Claude 3 on Amazon Bedrock?
Statement: Amazon Bedrock and third-party models are only available in the Amazon Web Services overseas regions. Amazon Web Services China only recommends this service to help you understand cutting-edge industry technologies and develop overseas business.
Preparation for Testing: To call the Claude 3 model on Amazon Bedrock through the API, you need to meet the following conditions:
1
Have an Amazon Web Services overseas account.
2
The account has enabled access to the Claude 3 model on Amazon Bedrock.
3
Have permissions to manage IAM policies, users, and generate access keys.
4
Have downloaded the Amazon CLI command-line tool on your local machine.
1
Enable access to Claude 3 on Amazon Bedrock by entering the Amazon Bedrock service homepage and clicking “Model Access”.
2
After entering, click the “Manage Model Access” button in the upper right corner.
3
Find the Anthropic model series and click the “Submit use case details” button on the right.
4
Fill out the following form to apply for access to the Claude 3 model. Follow the requirements in the image to complete the form, and then click “Submit” in the lower right corner. The review will be completed within 5 minutes.
5
If you encounter the following error after submitting the model access application, it is because, according to Anthropic (Claude model) service area terms, Anthropic’s service area does not include mainland China and Hong Kong. For specific operations, please visit Little Li Brother’s CSDN account.
6
After the application review is successful, re-enter the “Manage Model Access” interface, and you will see that the box in front of the Claude 3 model can be checked. Select the Claude 3 model and click save in the lower right corner.
7
Next, we need to obtain API access permissions for the cloud model, which can be done in two ways. One is to access via access key on a personal computer (only suitable for personal development), and the other is to configure an IAM role for Amazon EC2 access (suitable for formal production environments). It is recommended to use the second method for more secure access to cloud services.
8
Since this tutorial is aimed at independent developers, we will access using the first method. Please ensure that you have installed the Amazon CLI on your personal computer.
9
Next, you need to create a policy based on the principle of least privilege in the IAM service. First, enter the IAM service and click to create a policy.
10
First, select Amazon Bedrock service as the access resource, then click Next.
11
Add fine-grained permissions for API access. Here, we select the “Invoke Model” operation, allowing everyone to call the large model to generate the required content.
12
Click “Add ARNs” and enter the region your account is located in, such as “us-east-1”, in the first box, and in the second red box, enter the Claude 3 model ID: “anthropic.claude-3-sonnet-20240229-v1:0”, and finally click to add.
13
Next, return to the authorization operation box and enter list, selecting ListFoundationModels.
14
Finally, scroll to the end of the interface and click Next.
15
Next, give the policy a name, such as “bedrock-testing”, and scroll to the bottom of the page to click “Create Policy” (not shown in the image).
16
Next, create a user that will be used as the entity for local access to the cloud API.
17
Give this user a name and click Next. Note that you do not need to check the option to allow console access.
18
Click the first red box “Attach Policies directly”, check the policy created in step 11, and finally click Next. After jumping to the next page, click “Create User” to complete user creation.
19
Next, enter the user you just created.
20
Click “security credentials” to create an access key for API access to cloud services.
21
Next, scroll to the “Access Key” section and click the red box “Create access key”.
22
Check the “Local Code” option and then click Next in the lower right corner.
23
Add a note to the Access Key and click create.
24
You need to copy the access key and secret access key and paste them into a text file, as these two keys will be used later (remember to delete the access key after testing). You can also click the “Download .csv file” below to download both access keys to your local machine.
25
Next, return to the command line console on your local computer and enter the configuration for accessing the cloud API using the Access Key. In the pop-up input box (4 lines in total), enter the copied “Access key”, “Secret access key”, and your account’s region, leaving the last line blank. After entering each line, press Enter to go to the next line.
26
Now comes the exciting part, this article will demonstrate how to use the Amazon Web Services Python Boto3 SDK to access the Claude 3 model on Amazon Bedrock. First, you need to install the Boto3 SDK locally by entering “pip install boto3” in the command line.
27
Next, enter your commonly used IDE, create an empty Python file, copy the following code and run it. This script can check the currently supported Anthropic Claude series models on Amazon Bedrock. Each different Claude 3 model has its own model ID, which is often long and hard to remember, and this script can help us quickly find the model ID we want to call.
import boto3
bedrock = boto3.client(service_name="bedrock")
response = bedrock.list_foundation_models(byProvider="anthropic")
for summary in response["modelSummaries"]:
print(summary["modelId"])28
After running the Python program, you will find the Claude 3 Sonnet model we want to call. The other ones are other Claude series models available on Amazon Bedrock. You can still experience them in the Cloud Laboratory.
29
Next, we will demonstrate how to call the API to access the Claude 3 model on Amazon Bedrock.
First, we will showcase a conversation scenario with Claude 3. We need to use the standard Message API provided by the Claude 3 model to define the input content of the request, and then use Boto3’s specified syntax to call the Claude 3 model hosted on Amazon Bedrock and engage in a conversation with the model. The explanation for the parameters of the Message API section is as follows:
– “max_tokens” indicates the maximum length of output from the large model
– “anthropic_version” indicates the API version, we fill in “bedrock-2023-05-31”
– “messages” is an array of conversation messages, where each message is represented by a JSON object. If it is a message sent by the user, the role is “user”; if it is a reply from the large model, the role is “assistant”; the key value of content represents the content of the conversation.
import boto3
import json
bedrock = boto3.client(service_name="bedrock-runtime")
# Call Claude 3 model's standard Message API
body = json.dumps({ "max_tokens": 256, "messages": [{"role": "user", "content": "Hello"}], "anthropic_version": "bedrock-2023-05-31"})
# Call the Claude 3 model on Amazon Bedrock to generate content
response = bedrock.invoke_model(body=body, modelId="anthropic.claude-3-sonnet-20240229-v1:0")
# Extract the large model's reply from the API response
response_body = json.loads(response.get("body").read())
print(response_body.get("content"))[
{"role": "user", "content": "Hello there."},
{"role": "assistant", "content": "Hi, I'm Claude. How can I help you?"},
{"role": "user", "content": "Can you explain LLMs in plain English?"},
]30
Copy the above code, paste it into an empty Python file, and run it to get Claude 3’s response.
31
The Claude 3 model is trained to understand different formats of structured and unstructured data, including not only language but also images, charts, technical diagrams, etc. Below is an example of calling the Claude 3 model via API to have the large model understand an image and generate a textual description. The image chosen by Little Li Brother is the official logo of Amazon Bedrock.
32
Copy the above code, paste it into an empty Python file, and run it to get Claude 3’s description of the image. Note that if it involves image processing tasks, the Content section of the Message API must specify the message type as image, and the input data should be the Base64 encoding of the image.
import json
import boto3
import base64
bedrock = boto3.client(service_name="bedrock-runtime")
def call_claude_sonet(base64_string):
prompt_config = {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": [
{
"type": "image", # The content type of the API request is image
"source": {
"type": "base64", # The input data for the API is the Base64 encoding of the image
"media_type": "image/png",
"data": base64_string,
},
},
{"type": "text", "text": "Provide a caption for this image"},
],
}
],
}
body = json.dumps(prompt_config)
modelId = "anthropic.claude-3-sonnet-20240229-v1:0"
accept = "application/json"
contentType = "application/json"
response = bedrock.invoke_model(
body=body, modelId=modelId, accept=accept, contentType=contentType
)
response_body = json.loads(response.get("body").read())
results = response_body.get("content")[0].get("text")
return results
if __name__ == "__main__":
image_path = 'bedrock.jpeg' # Modify here to read the desired image path
# Read the image and convert to Base64
with open(image_path, 'rb') as image_file:
image_data = image_file.read()
base64_string = base64.b64encode(image_data).decode('utf-8')
caption = call_claude_sonet(base64_string)
print(caption)33
It can be seen that Claude 3 accurately identified that this image is the Amazon Bedrock logo, and described more details about the logo’s position, color, and deeper meanings.
In future articles, Little Li Brother will continue to lead everyone to explore more features of Amazon Bedrock, such as “How to Build Secure, Compliant, and Responsible Large Models on Amazon Bedrock”, “How to Evaluate Custom Training Models on Amazon Bedrock”, “How to Train Custom Models on Amazon Bedrock”, and “How to Build Enterprise Knowledge Bases Using Amazon Bedrock”. Please follow the Cloud Tech Brother account for future articles from Little Li Brother!
Hope everyone can also pay attention to Little Li Brother’s Xiaohongshu and Douyin accounts: “Florida Little Li Brother”, to get more classic architectures and learning resources from Amazon Web Services.
After reading Little Li Brother’s sharing, do you want to try it yourself? For those who are not familiar with Amazon Bedrock, don’t worry, the surprises from Cloud Tech Brother are coming soon!
“Getting Started with Amazon Bedrock”
Want to try Claude 3 on Amazon Bedrock? Learn about the advantages, features, typical use cases, technical concepts, and costs of Amazon Bedrock, and review the architecture of building chatbot solutions using Amazon Bedrock and other Amazon Web Services products.
Learning Pathway
Amazon Web Services Official Learning Center
Scan to learn for free
* It is recommended to view from a desktop browser
Netease Cloud Classroom
Scan to learn for free
“One Chance to Fail, Free Retake” Limited Time Event is Live!
Want to improve your cloud skills but hesitate due to the “trial and error costs”? Amazon Web Services certification benefits are here! To help build career confidence for cloud computing talents, Amazon Web Services is launching a limited-time event of “One Chance to Fail, Free Retake”. During the event, you can get a free retake opportunity if you do not pass the first time.
Note: The first certification exam must be completed before April 15, 2024, and the free retake must be completed before June 30, 2024 to ensure the discount code can be effectively used.
“One Chance to Fail, Free Retake”
Scan to receive limited time discount code