How to use bedrock cross-region models that start with - apac. ?

[locobuzz-prathamesh]

so my model name is "apac.anthropic.claude-3-5-sonnet-20240620-v1:0"
and I try to put it in litellm like -

response = completion(
    model="bedrock/apac.anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"content": "Hello, how are you?", "role": "user"}]
)

and not matter what its not working. No matter how many combinations I try. What is right way to configure this? is this even supported?

[krrishdholakia]

Hi @locobuzz-prathamesh that should just work, can you please share the error / stacktrace you're seeing?

[locobuzz-prathamesh]

Hello @krrishdholakia here is the error im getting -

NotFoundError: litellm.NotFoundError: BedrockException - Bedrock HTTPX: Unknown provider=apac, model=apac.anthropic.claude-3-5-sonnet-20240620-v1:0
I tried removing that apac and putting anthoripic there and got this error instead -

BadRequestError: litellm.BadRequestError: BedrockException - {"message":"The provided model identifier is invalid."}

Thank you for helping me out here.

[krrishdholakia]

what version of litellm is this? If older, can you please bump to latest (v1.55.9) and try again?

[krrishdholakia]

bump on this? @locobuzz-prathamesh

can you please bump and let me know if the problem persists?

[locobuzz-prathamesh]

Yes you are correct. This was version issue. I was using version 1.51 but in 1.55.12; latest version its working as expected.

from litellm import completion

os.environ["AWS_ACCESS_KEY_ID"] = os.getenv("PRODUCTION_AWS_ACCESS_KEY")
os.environ["AWS_SECRET_ACCESS_KEY"] = os.getenv("PRODUCTION_AWS_SECRET_KEY")
os.environ["AWS_REGION_NAME"] = "ap-south-1"

response = completion(
    model="bedrock/apac.anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"content": "Hello, how are you?", "role": "user"}]
)

print(response)

I was using dspy's integration with litellm which is not upgraded to work with latest litellm. But When I upgrade it and use it as standalone without dspy it is working.
Thank you for helping me out. Sincere apologies for late reply @krrishdholakia