[Bug]: valid tokens do not longer work after 1.52.14

[chymian]

What happened?

A bug happened!
rel 1.52.15 and later do not accept valid tokens and render complete setups obsolete.

using litellm as a central api-proxy-hub for dozens of apps does not work any longer after updating beyond 1.52.14.

working version

version <= 1.42.14

curl   'https://api.sub.tld.org/models'   -H 'accept: application/json'   -H 'Authorization: Bearer sk-my-token'|jq|grep '"id":'
      "id": "groq/llama-3.1-8b-instant",
      "id": "groq/whisper-large-v3",
      "id": "groq/llama-3.3-70b-versatile",
      "id": "groq/llama-3.2-11b-vision-preview",
      "id": "jina_ai/colbert-v2",
…

not working

version >= 1.42.15

curl   'https://api.sub.tld.org/models'   -H 'accept: application/json'   -H 'Authorization: Bearer sk-my-token'
{"error":{"message":"Authentication Error, Invalid proxy server token passed. valid_token=None.","type":"auth_error","param":"None","code":"401"}}

nothing was changed in the setup but updated litellm-database (docker)

Relevant log output

13:18:55 - LiteLLM Proxy:ERROR: user_api_key_auth.py:1192 - litellm.proxy.proxy_server.user_api_key_auth(): Exception occured - Invalid proxy server token passed. valid_token=None.
Requester IP Address:172.21.0.6
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/auth/user_api_key_auth.py", line 1161, in user_api_key_auth
    _is_route_allowed = _is_allowed_route(
                        ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/auth/user_api_key_auth.py", line 188, in _is_allowed_route
    return _is_api_route_allowed(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/auth/user_api_key_auth.py", line 158, in _is_api_route_allowed
    raise Exception("Invalid proxy server token passed. valid_token=None.")
Exception: Invalid proxy server token passed. valid_token=None.
INFO:     172.21.0.6:40244 - "GET /models HTTP/1.1" 401 Unauthorized

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.53.15 and later

Twitter / LinkedIn details

No response

[ishaan-jaff]

hi @chymian - we've not seen other reports of this issue. Are you sure it's not related to your environment

[chymian]

@ishaan-jaff
as I have done nothing but updated litellm and apps like anythingllm, hoarder, etc stopped working I investigated and checked with different versions of litellm and found the last known good to be 1.42.14.
even the simple curl cmd. which I was using to check does not work anymore.

How can I help to clarify?

[ishaan-jaff]

• on 1.52.14 does it show the DB connected on /health/readiness
• is your proxy using the latest schema ? Do you see custom_llm_provider in SpendLogs ?
• Share the full strack trace on the proxy server when you see this error

[krrishdholakia]

@chymian can you confirm you see this on v1.55.4 https://github.com/BerriAI/litellm/releases/tag/v1.55.4

i think i remember seeing this a few weeks ago, but it was resolved shortly after - iirc it had to do with an issue in the get_data from the prisma client

[krrishdholakia]

bump on this? @chymian