text-embeddings-inference

[RRaphaell]

I want to add embedding model which is running using text-embeddings-inference

here is my config:

  - model_name: embedding
    litellm_params:
      model: huggingface/BAAI/bge-small-en-v1.5
      api_base: http://127.0.0.1:8080/v1

that's how I'm using it:

client = OpenAI(
    api_key="empty", 
    base_url="http://localhost:4000"
)
client.embeddings.create(
    model="embedding", 
    input="What is Deep Learning?", 
    encoding_format="float"
)

but it shows an error:

InternalServerError: Error code: 500 - {'error': {'message': 'litellm.APIConnectionError: HuggingfaceException - [Errno 111] Connection refused\nReceived Model Group=embedding\nAvailable Model Group Fallbacks=None', 'type': None, 'param': None, 'code': '500'}}