Unable to use any Azure Models other than OpenAI ones

[semidark]

Hi Folks,

I just got started with LiteLLM as a proxy because I needed an OpenAI-compatible API for my Azure AI Service. I began by working with OpenAI models (gpt-4o and gpt-4o-mini), and everything worked perfectly. So I thought, why not try some other models? I attempted to use Mistral-large.

Here’s my configuration:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/gpt-4o
      api_base: "https://censored.openai.azure.com/"
      api_key: "censored"
      api_version: "2024-05-01-preview"
  - model_name: gpt-4o-mini
    litellm_params:
      model: azure/gpt-4o-mini
      api_base: "https://censored.openai.azure.com/"
      api_key: "censored"
      api_version: "2024-05-01-preview"
  - model_name: Mistral-large
    litellm_params:
      model: azure_ai/Mistral-large
      api_base: "https://censored.services.ai.azure.com/"
      api_key: "censored"
      api_version: "2024-05-01-preview"

I tried a few variations of the model string (mistral-large,Mistral-large-latest,mistral-large), but nothing worked. I always receive the following error:

23:01:35 - LiteLLM Proxy:DEBUG: proxy_server.py:3494 - An error occurred: litellm.NotFoundError: NotFoundError: Azure_aiException - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Received Model Group=Mistral-large
Available Model Group Fallbacks=None
Model: azure_ai/Mistral-large
API Base: `https://censored.services.ai.azure.com/`
Messages: `[{'role': 'user', 'content': '### Task:\nGenerate 1-3 broad tags categorizing the main themes of the'}`
model_group: `Mistral-large`
deployment: `azure_ai/Mistral-large`

I checked the Metrics of the deployment in the ai.azure.com and it has a few succesfull requests, bot no input nor output tokens measured.

Any ideas on what I might be doing wrong here?