Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update utils.py- Fix error when using AzureOpenAI services #4254

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

genlin
Copy link
Contributor

@genlin genlin commented Nov 18, 2024

Why are these changes needed?

Seems _kwargs.update(kwargs) will cause errors when using the Azure OpenAI service. It forces the model set in the environment to change to "gpt-4o", which can cause issues (e.g., The API deployment for this resource does not exist.) in Azure.

The model name in Azure can be customized. For example, I created a model based on gpt-4o, and its name is my-gpt-4o that should be used in the API call. If we use gpt-4o as model in the completion API call, the error will occur.

Note I tested the code with the Azure OpenAI service, but I haven't had a chance to run it with OpenAI. Please double-check it.

Related issue number

Checks

Seems `_kwargs.update(kwargs)` will cause errors when using the Azure OpenAI service. It forces the model set in the environment to change to "gpt-4o", which can cause issues (e.g., `The API deployment for this resource does not exist.`). 

The model name in Azure can be customized. For example, I created a model based on gpt-4o, but its name is `my-gpt-4o`, and this name should be used in the API call.
@genlin
Copy link
Contributor Author

genlin commented Nov 18, 2024

#sign-off

@jackgerrits jackgerrits requested a review from afourney November 18, 2024 15:20
@afourney
Copy link
Member

I'm traveling today, but will take a look asap. @husseinmozannar is also familiar with this code, if there is urgency

@genlin
Copy link
Contributor Author

genlin commented Nov 22, 2024

@@husseinmozannar, I found that we have to use "gpt4-o" as the name of the model, unless it will cause unexpected issues when running the codes such as token limit calculation.

File "D:\autogen-main\autogen-main\python\packages\autogen-ext\src\autogen_ext\models\_openai\_model_info.py", line 122, in get_token_limit
   return _MODEL_TOKEN_LIMITS[resolved_model]
          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'gpt-4o-h1'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants