Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add Amazon Bedrock Agents support #554

Open
wants to merge 96 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 95 commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
f6f8853
feat: appsync
massi-ang Dec 5, 2023
8a62b43
feat(appsync): API
massi-ang Dec 11, 2023
988c57f
feat(appsync): api
massi-ang Dec 12, 2023
67224b6
feat(appsync): poetry env
massi-ang Dec 12, 2023
facc201
feat(appsync): ongoing checkpoint
massi-ang Dec 13, 2023
91d1930
feat(appsync): checkpoint
massi-ang Dec 13, 2023
d11b1d1
feat(appsync): final with removal of ApiGw
massi-ang Dec 13, 2023
569072a
feat(appsync): working GraphQL
massi-ang Dec 14, 2023
9440095
feat(appsync): CF configuration
massi-ang Dec 14, 2023
e195072
feat(appsync): removal of CF in front of AppSync
massi-ang Dec 14, 2023
924fd09
feat(appsync): using AOI for CF
massi-ang Dec 14, 2023
807b20b
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Dec 14, 2023
031935d
feat(appsync): fix Delete session
massi-ang Dec 14, 2023
495c5a4
feat(appsync) : delete session result
massi-ang Dec 14, 2023
3ad9ebb
feat(appsync): moving handler function
massi-ang Dec 15, 2023
c015f39
feat(appsync): json decoder
massi-ang Dec 15, 2023
514f125
feat(appsync): cleanup
massi-ang Dec 15, 2023
b475fa3
feat(appsync): exclude autogenerated file from prettier linting
massi-ang Dec 15, 2023
badc9ac
feat(appsync): remove unused imports
massi-ang Dec 15, 2023
11e8de7
feat(appsync): remove unused imports
massi-ang Dec 15, 2023
384c3fb
feat(appsync): fix appsync merge errors
massi-ang Dec 15, 2023
c883541
feat(appsync): fixed semantic_search method based on graphql signature
massi-ang Dec 17, 2023
c3634e7
fix(appsync): correct logic for adding form fields to POST
massi-ang Dec 17, 2023
c4604d2
fix(appsync): merged api stability and cleanup
massi-ang Dec 19, 2023
9272cae
feat(appsync): schema improvements
massi-ang Dec 19, 2023
5e97f59
fix(appsync): reverting changes
massi-ang Dec 19, 2023
dcd665f
feat(appsync): single api
massi-ang Dec 19, 2023
433a647
fix(appsync): remove double resolver creation
massi-ang Dec 20, 2023
0ace107
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Dec 20, 2023
8c8694c
feat(agents): init
massi-ang Jan 8, 2024
e97cbcb
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Jan 8, 2024
8d16318
fix(appsync): start kendra data sync
massi-ang Jan 8, 2024
1128fb5
fix(appsync): correct field type
massi-ang Jan 8, 2024
da7dc3e
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Jan 10, 2024
13f568c
doc(appsync): update READMEs and diagrams
massi-ang Jan 10, 2024
c2e1c05
fix(appsync): correct subnet filtering
massi-ang Jan 10, 2024
1112a10
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Jan 10, 2024
29b66ad
Merge branch 'feat_appsync' into feat_bedrock_agents
massi-ang Jan 15, 2024
3c6ee87
feat(agents): wip
massi-ang Jan 15, 2024
486c311
feat(appsync): cleanup legacy apigw functionality
massi-ang Jan 15, 2024
ccf99cb
Merge branch 'feat_appsync' into feat_bedrock_agents
massi-ang Jan 15, 2024
33293a2
feat(agents): working poc
massi-ang Jan 15, 2024
775f735
feat(agents): group by provider and interface type
massi-ang Jan 15, 2024
aa9fcb9
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Jan 16, 2024
06f5cad
feat(agents): fix for the option groups
massi-ang Jan 23, 2024
f2ad37e
Merge branch 'main' into feat_bedrock_agents
massi-ang Jan 24, 2024
0c5d12e
feat(agents): refactoring of model IF constructs
massi-ang Jan 31, 2024
ba132fd
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Jan 31, 2024
8666b35
Merge branch 'main' into feat_bedrock_agents
massi-ang Feb 12, 2024
3a9a841
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Feb 12, 2024
df4dff2
feat(agents): added agent creation
massi-ang Feb 13, 2024
74bdf05
Merge branch 'main' of https://github.com/aws-samples/aws-genai-llm-c…
massi-ang Feb 19, 2024
e69386a
feat(agents): new backend API
massi-ang Feb 28, 2024
2894f95
feat(chat_ui): initial
massi-ang Feb 28, 2024
20414f1
feat(chat_ui): onFeedback and Avatar update
massi-ang Feb 28, 2024
83e3e9a
feat(chat_ui): refactor
massi-ang Feb 28, 2024
0d92e9f
feat(chat_ui): adding images as avatars
massi-ang Feb 28, 2024
4f4a9ff
feat(new_ui): refactor component API
massi-ang Feb 29, 2024
c080caa
feat(message_api): moving to message API for Claude models
massi-ang Mar 5, 2024
c9f20a4
feat(message_api): only yield on relevant chunks
massi-ang Mar 5, 2024
73b3317
fix(chat): fixing chat for streamed content
massi-ang Mar 5, 2024
53033d4
fix(chat): only send messages when token length > 0
massi-ang Mar 5, 2024
8c62424
fix(cleanup): remove unused code
massi-ang Mar 5, 2024
c7d9984
fix(streaming): fixing issues with streamed responses
massi-ang Mar 5, 2024
1426f59
fix(ui): replaced react-icons with cloudscape icons
massi-ang Mar 5, 2024
5c82d03
Merge branch 'main' into feat_bedrock_agents
massi-ang Mar 5, 2024
41214f3
Merge branch 'feat_message_api' into feat_bedrock_agents
massi-ang Mar 5, 2024
3e098eb
Merge branch 'feat_new_chat_ui' into feat_bedrock_agents
massi-ang Mar 5, 2024
be5baea
feat(agent): new UI and agent functionality
massi-ang Mar 5, 2024
246a5aa
fix: make the prop mandatory
massi-ang Mar 5, 2024
56233f2
Merge remote-tracking branch 'origin/main' into feat_bedrock_agents
massi-ang Mar 15, 2024
63bc6f5
fix(agents): correct generic type
massi-ang Mar 15, 2024
dc6a3e4
feat(agents): keep first letter as avatar
massi-ang Mar 15, 2024
2f75db8
Merge branch 'main' of github.com:aws-samples/aws-genai-llm-chatbot i…
massi-ang Apr 26, 2024
09a09b3
feat(agents): types
massi-ang Apr 26, 2024
1d0f22a
Merge branch 'main' of github.com:aws-samples/aws-genai-llm-chatbot i…
massi-ang May 14, 2024
5efc171
Merge branch 'main' of github.com:aws-samples/aws-genai-llm-chatbot i…
massi-ang Jun 13, 2024
5d2eed3
Merge branch 'main' of github.com:aws-samples/aws-genai-llm-chatbot
massi-ang Aug 12, 2024
9bf5e43
Merge branch 'main' into feat_bedrock_agents
massi-ang Aug 12, 2024
206f84b
chore: updated chat UI components to Cloudscape components
massi-ang Aug 12, 2024
287940d
fix: loading feedback and removed double handler
massi-ang Aug 12, 2024
674093d
fix: clean up and fix styles
massi-ang Aug 12, 2024
5ce94b5
fix: removed feedback buttons from "human" message
massi-ang Aug 12, 2024
fbb6159
fix: fix errors from ci pipeline
massi-ang Aug 12, 2024
8907882
fix: fix ci errors
massi-ang Aug 12, 2024
e9b49d7
fix: eslint issues
massi-ang Aug 12, 2024
c8afae0
Merge branch 'main' of github.com:aws-samples/aws-genai-llm-chatbot i…
massi-ang Aug 19, 2024
fa18001
test: updated snapshot
massi-ang Aug 19, 2024
6479bc0
test: Update cfn snapshot
charles-marion Aug 20, 2024
cb25874
Merge branch 'feat_bedrock_agents' of github.com:aws-samples/aws-gena…
massi-ang Aug 20, 2024
d3bac06
fix: linting errors
massi-ang Aug 20, 2024
d9bab73
Merge branch 'main' of github.com:aws-samples/aws-genai-llm-chatbot i…
massi-ang Aug 21, 2024
5d10c8d
chore: remove unnecessary files
massi-ang Aug 21, 2024
0edeb63
chore: remove unnecessary file
massi-ang Aug 21, 2024
3e09b8c
fix: double entry
massi-ang Aug 21, 2024
a1c44fd
chore: address review comments - first run
massi-ang Aug 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions cli/magic-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ const embeddingModels = [
options.bedrockRoleArn = config.bedrock?.roleArn;
options.guardrailsEnable = config.bedrock?.guardrails?.enabled;
options.guardrails = config.bedrock?.guardrails;
options.bedrockAgents = config.bedrock?.agents;
options.sagemakerModels = config.llms?.sagemaker ?? [];
options.enableSagemakerModels = config.llms?.sagemaker
? config.llms?.sagemaker.length > 0
Expand Down Expand Up @@ -343,6 +344,17 @@ async function processCreateOptions(options: any): Promise<void> {
},
initial: options.guardrails?.version ?? "DRAFT",
},
{
type: "confirm",
name: "bedrockAgents",
message: "Do you want to deploy a sample Bedrock Agent?",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add something like (Agent with access to workspace and weather .. backed by Claude [insert model]) just to give an idea of the out of the box one

initial: options.bedrockAgents ?? false,
skip() {
return !["us-east-1", "us-west-2"].includes(
(this as any).state.answers.bedrockRegion
);
},
},
{
type: "confirm",
name: "enableSagemakerModels",
Expand Down Expand Up @@ -461,8 +473,9 @@ async function processCreateOptions(options: any): Promise<void> {
{
type: "input",
name: "sagemakerCronStopSchedule",
hint: "This cron format is using AWS eventbridge cron syntax see docs for more information",
message: "Stop schedule for Sagmaker models expressed in AWS cron format",
hint: "This cron format is using AWS EventBridge cron syntax see docs for more information",
message:
"Stop schedule for Sagemaker models expressed in AWS cron format",
skip(): boolean {
return !(this as any).state.answers.enableCronFormat.includes("cron");
},
Expand Down Expand Up @@ -1151,6 +1164,7 @@ async function processCreateOptions(options: any): Promise<void> {
identifier: answers.guardrailsIdentifier,
version: answers.guardrailsVersion,
},
agents: answers.bedrockAgents,
}
: undefined,
llms: {
Expand Down
161 changes: 161 additions & 0 deletions lib/agents/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as lambda from "aws-cdk-lib/aws-lambda";
import * as iam from "aws-cdk-lib/aws-iam";
import * as path from "path";
import * as geo from "aws-cdk-lib/aws-location";
import { bedrock } from "@cdklabs/generative-ai-cdk-constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";
import { BucketDeployment, Source } from "aws-cdk-lib/aws-s3-deployment";
import { VectorIndex } from "@cdklabs/generative-ai-cdk-constructs/lib/cdk-lib/opensearch-vectorindex";
import { VectorCollection } from "@cdklabs/generative-ai-cdk-constructs/lib/cdk-lib/opensearchserverless";
import { NagSuppressions } from "cdk-nag";

export class BedrockWeatherAgent extends Construct {
constructor(scope: Construct, id: string) {
super(scope, id);

const powertools = lambda.LayerVersion.fromLayerVersionArn(
this,
"powertools",
`arn:aws:lambda:${
cdk.Stack.of(this).region
}:017000801446:layer:AWSLambdaPowertoolsPythonV2:58`
);

const placeIndex = new geo.CfnPlaceIndex(this, "place-index", {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: other classes use this case for construct ids

Suggested change
const placeIndex = new geo.CfnPlaceIndex(this, "place-index", {
const placeIndex = new geo.CfnPlaceIndex(this, "PlaceIndex", {

dataSource: "Esri",
indexName: "PlaceIndex",
pricingPlan: "RequestBasedUsage",
});

const vectorStore = new VectorCollection(this, "vector-collection");
const vectorIndex = new VectorIndex(this, "vector-index", {
collection: vectorStore,
indexName: "weather",
vectorField: "vector",
vectorDimensions: 1536,
mappings: [
{
mappingField: "AMAZON_BEDROCK_TEXT_CHUNK",
dataType: "text",
filterable: true,
},
{
mappingField: "AMAZON_BEDROCK_METADATA",
dataType: "text",
filterable: false,
},
],
});

const modulesLayer = new lambda.LayerVersion(this, "modules-layer", {
code: lambda.Code.fromDockerBuild(
path.join(__dirname, "weather/modules")
),
compatibleRuntimes: [lambda.Runtime.PYTHON_3_12],
description: "Layer with required modules for the agent",
});

const weather = new lambda.Function(this, "weather", {
runtime: lambda.Runtime.PYTHON_3_12,
description:
"Lambda function that implements APIs to retrieve weather data",
code: lambda.Code.fromAsset(path.join(__dirname, "weather")),
handler: "lambda.handler",
environment: {
LOCATION_INDEX: placeIndex.indexName,
},
memorySize: 512,
timeout: cdk.Duration.seconds(10),
layers: [powertools, modulesLayer],
});

const policy = new iam.Policy(this, "weather-policy", {
statements: [
new iam.PolicyStatement({
actions: ["geo:SearchPlaceIndexForText"],
resources: ["*"],
}),
],
});
weather.role?.attachInlinePolicy(policy);

const kb = new bedrock.KnowledgeBase(this, "weather-kb", {
embeddingsModel: bedrock.BedrockFoundationModel.TITAN_EMBED_TEXT_V1,
vectorField: "vector",
vectorIndex: vectorIndex,
vectorStore: vectorStore,
indexName: "weather",
instruction: "answers questions about WMO and metereology",
});

const bucket = new Bucket(this, "bedrock-kb-datasource-bucket", {
enforceSSL: true,
});

const keyPrefix = "my-docs";
new bedrock.S3DataSource(this, "my-docs-datasource", {
bucket: bucket,
dataSourceName: "my-docs",
knowledgeBase: kb,
inclusionPrefixes: [keyPrefix],
});

const agent = new bedrock.Agent(this, "weather-agent", {
foundationModel:
bedrock.BedrockFoundationModel.ANTHROPIC_CLAUDE_INSTANT_V1_2,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to switch to Sonnet 3 or Haiku now

instruction: `You are a weather expert and answer user question about weather in different places.
If you are asked to provide historical date, you need to answer with a summary of the weather over the period.
You answer the questions in the same language they have been asked.`,
description: "an agent to interact with a weather api",
knowledgeBases: [kb],
name: "WeatherTeller",
});

agent.addActionGroup({
actionGroupExecutor: weather,
apiSchema: bedrock.ApiSchema.fromAsset(
path.join(__dirname, "weather", "schema.json")
),
actionGroupState: "ENABLED",
});

agent.addActionGroup({
parentActionGroupSignature: "AMAZON.UserInput",
actionGroupState: "ENABLED",
actionGroupName: "UserInputAction",
});

new BucketDeployment(this, "my-docs-files", {
destinationBucket: bucket,
destinationKeyPrefix: keyPrefix,
sources: [Source.asset(path.join(__dirname, "my-documents"))],
});

new cdk.CfnOutput(this, "weather-function-name", {
value: weather.functionName,
});

NagSuppressions.addResourceSuppressions(weather.role!, [
{
id: "AwsSolutions-IAM4",
reason: "IAM role implicitly created by CDK.",
},
]);

NagSuppressions.addResourceSuppressions(policy, [
{
id: "AwsSolutions-IAM5",
reason: "IAM role implicitly created by CDK.",
},
]);

NagSuppressions.addResourceSuppressions(bucket, [
{
id: "AwsSolutions-S1",
reason: "Access logs not required",
},
]);
}
}
1 change: 1 addition & 0 deletions lib/agents/my-documents/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add your documents to this folder to get them uploaded
34 changes: 34 additions & 0 deletions lib/agents/weather/event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"messageVersion": "1.0",
"agent": {
"name": "alfa",
"id": "beta",
"alias": "alfa",
"version": "1.2"
},
"inputText": "any text",
"sessionId": "1234",
"actionGroup": "mygroup",
"apiPath": "/current_weather",
"httpMethod": "GET",
"parameters": [
{
"name": "place",
"type": "string",
"value": "Milan"
}
],
"requestBody": {
"content": {
"application/json": {
"properties": [
{
"name": "place",
"type": "string",
"value": "Milan"
}
]
}
}
}
}
140 changes: 140 additions & 0 deletions lib/agents/weather/lambda.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
from aws_lambda_powertools import Tracer
from aws_lambda_powertools.event_handler import BedrockAgentResolver
from aws_lambda_powertools.event_handler.openapi.params import Query
from pydantic import BaseModel
from typing import List
import boto3
import os
import requests
from typing import Annotated
import pandas as pd
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nip: I believe pandas is large and would cause cold start due to numpy. I would remove this dependency if it's easy (I believe Panda has smaller managed lambda layers)

import datetime

tracer = Tracer()


class Weather(BaseModel):
time: str
temperature: str
precipitation: str


class Period(BaseModel):
start_date: str
end_date: str


app = BedrockAgentResolver()


def get_metric_and_unit(w: dict, name: str) -> str:
return f"{w['current'][name]} {w['current_units'][name]}"


@app.get("/current_weather", description="get the current weather for a given place")
def get_current_weather(
place: Annotated[str, Query(description="the name of the place")]
) -> Weather:
resp = loc_client.search_place_index_for_text(
IndexName=os.environ.get(
"LOCATION_INDEX", os.environ.get("PLACE_INDEX", "Test")
),
Text=place,
)
[lon, lat] = resp["Results"][0]["Place"]["Geometry"]["Point"]
q = (
"https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}"
+ "&current=temperature_2m,precipitation"
)
w = requests.get(q.format(lat=lat, lon=lon)).json()

return Weather(
time=w["current"]["time"],
temperature=get_metric_and_unit(w, "temperature_2m"),
precipitation=get_metric_and_unit(w, "precipitation"),
)


@app.get(
"/absolute_period_dates",
description="get the absolute start and end date for a period in YYYY-MM-DD"
+ " format given the number of day difference from the today",
)
def get_absolute_period_dates(
startPeriodDeltaDays: Annotated[
int,
Query(
description="the difference in days from the today"
+ " to the start date of the period"
),
],
endPeriodDeltaDays: Annotated[
int,
Query(
description="the difference in days from the today"
+ " to the end date of the period"
),
],
) -> Period:
today = datetime.date.today()
p = Period()
p.start_date = (today - datetime.timedelta(startPeriodDeltaDays)).isoformat()
p.end_date = (today - datetime.timedelta(endPeriodDeltaDays)).isoformat()
return p


@app.get(
"/historical_weather",
description="get the historical daily mean temperature and precipitation"
+ " for a given place for a range of dates",
)
def get_historical_weather(
place: Annotated[str, Query(description="the name of the place")],
fromDate: Annotated[str, Query(description="starting date in YYYY-MM-DD format")],
toDate: Annotated[str, Query(description="ending date in YYYY-MM-DD format")],
) -> List[Weather]:
resp = loc_client.search_place_index_for_text(
IndexName=os.environ.get(
"LOCATION_INDEX", os.environ.get("PLACE_INDEX", "Test")
),
Text=place,
)
[lon, lat] = resp["Results"][0]["Place"]["Geometry"]["Point"]
q = (
"https://archive-api.open-meteo.com/v1/archive?latitude={lat}&longitude={lon}"
+ "&start_date={fromDate}&end_date={toDate}&hourly=temperature_2m,precipitation"
)
resp = requests.get(
q.format(lat=lat, lon=lon, fromDate=fromDate, toDate=toDate)
).json()
hourly_values = pd.DataFrame(resp["hourly"])
hourly_values["time"] = pd.to_datetime(hourly_values["time"])
hourly_values = hourly_values.set_index("time")
hourly_values = hourly_values.resample("D").agg(
{"temperature_2m": "mean", "precipitation": "sum"}
)
return [
Weather(
time=str(hourly_values.iloc[i].name).split(" ")[0],
temperature=hourly_values.iloc[i][0],
precipitation=hourly_values.iloc[i][1],
)
for i in range(hourly_values.shape[0])
]


def handler(event, context):
print(event)
resp = app.resolve(event, context)
print(resp)
return resp


if __name__ == "__main__":
with open("schema.json", "w") as f:
f.write(app.get_openapi_json_schema())
else:
sess = boto3.Session(
region_name=os.environ.get("LOCATION_AWS_REGION", os.environ.get("AWS_REGION"))
)
loc_client = sess.client("location")
7 changes: 7 additions & 0 deletions lib/agents/weather/modules/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM python:3.12-slim

RUN apt update -y && apt install zip -y

ADD requirements.txt .

RUN mkdir -p /asset/python && pip install -r requirements.txt -t /asset/python
2 changes: 2 additions & 0 deletions lib/agents/weather/modules/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
requests==2.29
pandas==2.2
Loading
Loading