[Bug]: Some small inconsistencies in LiteLLM_SpendLogs -> api_base found

[stronk7]

What happened?

After adding a few more models to our staging instance and testing them, looking to the LiteLLM_SpendLogs contents, I've noticed a few inconsistencies. I think that they are not critical but, maybe, it would be great if they can be done more consistently:

1. Some models, like dall-e-2 and dall-e-3 (OpenAI) don't get the api_base in the table filled ever.
2. Some cache_hit = True requests, for some models, like gpt-4o and gpt-4o-mini (from OpenAI), or llama-3.1-8b-instant and llama-3.3-70b-versatile (from Groq) don't get the 'api_base' either. Also, perplexity's ones. In the other side, some providers like ollama_chat get the api_base for cache hits filled. Maybe it's something by provider, it would be great to get it managed consistently over all them.

Ciao :-)

Relevant log output

Not logs, but a query and its results, showing the behaviour described above. Hope it's illustrative:

SELECT
  to_char("startTime", 'YYYY-MM-DD') as day,
  model_group AS model_group,
  model AS model_name,
  CASE
    WHEN api_base ~ 'https://api.openai.com' THEN 'https://api.openai.com/...'
    WHEN api_base ~ 'https://api.anthropic.com' THEN 'https://api.anthropic.com/...'
    WHEN api_base ~ 'https://api.perplexity.ai' THEN 'https://api.perplexity.ai/...'
    WHEN api_base ~ 'https://api.groq.com' THEN 'https://api.groq.com/...'
    WHEN api_base ~ 'https://some.ollama.server' THEN 'https://ollama-chat/...'
    ELSE api_base
  END AS model_provider,
  call_type AS call_type,
  cache_hit
FROM "LiteLLM_SpendLogs"
WHERE to_char("startTime", 'YYYY-MM-DD') = '2024-12-19'
GROUP BY 1, 2, 3, 4, 5, 6 
ORDER BY 1, 2, 3, 4, 5, 6;

    day     |      model_group       |            model_name             |        model_provider         |     call_type     | cache_hit 
------------+------------------------+-----------------------------------+-------------------------------+-------------------+-----------
 2024-12-19 | all-minilm             | all-minilm                        | https://ollama-chat/...       | aembedding        | None
 2024-12-19 | all-minilm             | all-minilm                        | https://ollama-chat/...       | aembedding        | True
 2024-12-19 | bge-m3                 | bge-m3                            | https://ollama-chat/...       | aembedding        | None
 2024-12-19 | bge-m3                 | bge-m3                            | https://ollama-chat/...       | aembedding        | True
 2024-12-19 | dall-e-2               | dall-e-2                          |                               | aimage_generation | None
 2024-12-19 | dall-e-3               | dall-e-3                          |                               | aimage_generation | None
 2024-12-19 | gpt-4o                 | gpt-4o                            |                               | acompletion       | True
 2024-12-19 | gpt-4o                 | gpt-4o                            | https://api.openai.com/...    | acompletion       | None
 2024-12-19 | gpt-4o-mini            | gpt-4o-mini                       |                               | acompletion       | True
 2024-12-19 | gpt-4o-mini            | gpt-4o-mini                       | https://api.openai.com/...    | acompletion       | None
 2024-12-19 | llama3.1               | llama3.1                          | https://ollama-chat/...       | acompletion       | None
 2024-12-19 | llama3.1               | llama3.1                          | https://ollama-chat/...       | acompletion       | True
 2024-12-19 | llama3.1               | llama-3.1-8b-instant              |                               | acompletion       | True
 2024-12-19 | llama3.1               | llama-3.1-8b-instant              | https://api.groq.com/...      | acompletion       | None
 2024-12-19 | llama3.2-3b            | llama3.2                          | https://ollama-chat/...       | acompletion       | None
 2024-12-19 | llama3.3-70b           | llama-3.3-70b-versatile           |                               | acompletion       | True
 2024-12-19 | llama3.3-70b           | llama-3.3-70b-versatile           | https://api.groq.com/...      | acompletion       | None
 2024-12-19 | perplexity-online      | llama-3.1-sonar-small-128k-online |                               | acompletion       | True
 2024-12-19 | perplexity-online      | llama-3.1-sonar-small-128k-online | https://api.perplexity.ai/... | acompletion       | None
 2024-12-19 | qwen2.5-coder          | qwen2.5-coder                     | https://ollama-chat/...       | acompletion       | False
 2024-12-19 | qwen2.5-coder          | qwen2.5-coder                     | https://ollama-chat/...       | acompletion       | None
 2024-12-19 | qwen2.5-coder          | qwen2.5-coder                     | https://ollama-chat/...       | acompletion       | True
 2024-12-19 | text-embedding-3-small | text-embedding-3-small            |                               | aembedding        | True
 2024-12-19 | text-embedding-3-small | text-embedding-3-small            | https://api.openai.com/...    | aembedding        | None
(24 rows)

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.55.1

Twitter / LinkedIn details

No response