Replies: 5 comments 4 replies
-
|
@Xerizaf i see issues with I don't see the litellm server raising any issues. What are the server logs? |
Beta Was this translation helpful? Give feedback.
-
|
@krrishdholakia Thank you for your answer. I try to load the model like this: litellm --model codellama-34b-instruct. It tells me Uvicorn running on port http://0.0.0.0:8000, but then when I try litellm --test it gives me Win error 10049.. Maybe there is another process running on port 8000? Please help me 😅 |
Beta Was this translation helpful? Give feedback.
-
|
Are you able to make normal curl requests to that endpoint? It looks like it's not running at port 8000 Check what gets outputted when you run Btw that also looks wrong to me. You need to specify the provider - https://docs.litellm.ai/docs/proxy/quick_start#supported-llms E.g. What docs did you look at? Will update them |
Beta Was this translation helpful? Give feedback.
-
|
No, I'm not even able to make curl requests. I specified the provider as perplexity, but then: curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535. Btw the output for server is: INFO: Started server process [5812] #------------------------------------------------------------# 'This product would be better if...'https://github.com/BerriAI/litellm/issues/new#------------------------------------------------------------# Thank you for using LiteLLM! - Krrish & Ishaan Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM: Test your local proxy with: "litellm --test" This runs an openai.ChatCompletion request to your proxy [In a new terminal tab] LiteLLM: Curl Command Test for your local proxy Docs: https://docs.litellm.ai/docs/simple_proxy See all Router/Swagger docs on http://0.0.0.0:8000 INFO: Application startup complete. |
Beta Was this translation helpful? Give feedback.
-
looks like your curl request was malformed |
Beta Was this translation helpful? Give feedback.
-
|
I tried curl --location 'http://0.0.0.0:8000/chat/completions' |
Beta Was this translation helpful? Give feedback.
-
|
Moving to discussion as this doesn't seem to be a litellm-specific bug. |
Beta Was this translation helpful? Give feedback.
-
|
Okk @krrishdholakia |
Beta Was this translation helpful? Give feedback.
-
|
@Xerizaf was this fixed for you? We can hop on a call and help get you setup with litellm proxy |
Beta Was this translation helpful? Give feedback.
-
What happened?
When I try to start a proxy it tells me running on port http://0.0.0.0:8000/, but when I try litellm --test I get Win error 10049. Can someone help me?
Relevant log output
LiteLLM: Making a test ChatCompletions request to your proxy. Model=gpt-3.5-turbo
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions
yield
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_backends\sync.py", line 206, in connect_tcp
sock = socket.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 851, in create_connection
raise exceptions[0]
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 836, in create_connection
sock.connect(sa)
OSError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 67, in map_httpcore_exceptions
yield
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 231, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection_pool.py", line 268, in handle_request
raise exc
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection_pool.py", line 251, in handle_request
response = connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 99, in handle_request
raise exc
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 76, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_backends\sync.py", line 205, in connect_tcp
with map_exceptions(exc_map):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 877, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 915, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 943, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 980, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 1016, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 230, in handle_request
with map_httpcore_exceptions():
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions
yield
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_backends\sync.py", line 206, in connect_tcp
sock = socket.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 851, in create_connection
raise exceptions[0]
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 836, in create_connection
sock.connect(sa)
OSError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 67, in map_httpcore_exceptions
yield
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 231, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection_pool.py", line 268, in handle_request
raise exc
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection_pool.py", line 251, in handle_request
response = connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 99, in handle_request
raise exc
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 76, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_backends\sync.py", line 205, in connect_tcp
with map_exceptions(exc_map):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 877, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 915, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 943, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 980, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 1016, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 230, in handle_request
with map_httpcore_exceptions():
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions
yield
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_backends\sync.py", line 206, in connect_tcp
sock = socket.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 851, in create_connection
raise exceptions[0]
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 836, in create_connection
sock.connect(sa)
OSError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 67, in map_httpcore_exceptions
yield
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 231, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection_pool.py", line 268, in handle_request
raise exc
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection_pool.py", line 251, in handle_request
response = connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 99, in handle_request
raise exc
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 76, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_backends\sync.py", line 205, in connect_tcp
with map_exceptions(exc_map):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 877, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 915, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 943, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 980, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_client.py", line 1016, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 230, in handle_request
with map_httpcore_exceptions():
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\httpx_transports\default.py", line 84, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [WinError 10049] Indirizzo richiesto non valido nel proprio contesto
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Scripts\litellm.exe_main.py", line 7, in
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\litellm\proxy\proxy_cli.py", line 336, in run_server
response = client.chat.completions.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_utils_utils.py", line 272, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai\resources\chat\completions.py", line 645, in create
return self._post(
^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 1088, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 853, in request
return self._request(
^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 896, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 958, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 896, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 958, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "C:\Users\Alessandro\AppData\Local\Programs\Python\Python311\Lib\site-packages\openai_base_client.py", line 905, in _request
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
Twitter / LinkedIn details
No response
Beta Was this translation helpful? Give feedback.
All reactions