how to use custom huggingface embedding models?

[seoeaa]

how to specify and use a different text embedding model I wanted to change it to FlagEmbedding multilingual-e5-large

embed_model=HuggingFaceEmbedding(model_name='BAAI/bge-small-en', embed_batch_size=10, callback_manager=<llama_index.callbacks.base.CallbackManager object at 0x00000299D62FCFA0>,
tokenizer_name='BAAI/bge-small-en', max_length=512, pooling=<Pooling.CLS: 'cls'>, normalize='True', query_instruction=None, text_instruction=None, cache_folder=None),

[seoeaa]

It just doesn't work for other languages, I tested it, it doesn't answer the questions

[fcakyon]

It is expected if you are using the default HF model, which is fine-tuned on English data only.

[fcakyon]

We are working on a fix for using any HF model. It will be live, soon. Thank you for your patience while we work on that 🙏

[seoeaa]

when do you plan to do this?

[fcakyon]

In max 20 mins!

[fcakyon]

Its live with the v0.0.18 release @seoeaa ! #109

example usage:

AutoQueryEngine.from_parameters(embed_model='local:intfloat/multilingual-e5-large')

related docstring:

https://github.com/safevideo/autollm/blob/6b0b6104fcc1d04a5cf400b08f4eae33096ca535/autollm/auto/query_engine.py#L129-L130

[seoeaa]

wow fast

[seoeaa]

I'm already testing it!

[seoeaa]

Generating embeddings: 100%|█████████████████████████████████████████████████████████████████████████| 2594/2594 [11:21<00:00, 3.80it/s]
Traceback (most recent call last):
File "D:\Programm\AI\autollm\app.py", line 31, in
query_engine = AutoQueryEngine.from_parameters(
File "D:\Programm\AI\autollm\autollm\auto\query_engine.py", line 121, in from_parameters
return create_query_engine(
File "D:\Programm\AI\autollm\autollm\auto\query_engine.py", line 43, in create_query_engine
vector_store_index = AutoVectorStoreIndex.from_defaults(
File "D:\Programm\AI\autollm\autollm\auto\vector_store_index.py", line 89, in from_defaults
index = VectorStoreIndex.from_documents(
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\indices\base.py", line 102, in from_documents
return cls(
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\indices\vector_store\base.py", line 49, in init
super().init(
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\indices\base.py", line 71, in init
index_struct = self.build_index_from_nodes(nodes)
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\indices\vector_store\base.py", line 238, in build_index_from_nodes
return self._build_index_from_nodes(nodes)
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\indices\vector_store\base.py", line 226, in _build_index_from_nodes
self._add_nodes_to_index(
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\indices\vector_store\base.py", line 187, in _add_nodes_to_index
new_ids = self._vector_store.add(nodes)
File "D:\Programm\AI\autollm\myenv\lib\site-packages\llama_index\vector_stores\lancedb.py", line 122, in add
tbl.add(data)
File "D:\Programm\AI\autollm\myenv\lib\site-packages\lancedb\table.py", line 628, in add
data = _sanitize_data(
File "D:\Programm\AI\autollm\myenv\lib\site-packages\lancedb\table.py", line 68, in _sanitize_data
data = _sanitize_schema(
File "D:\Programm\AI\autollm\myenv\lib\site-packages\lancedb\table.py", line 1060, in _sanitize_schema
return pa.Table.from_arrays(
File "pyarrow\table.pxi", line 3969, in pyarrow.lib.Table.from_arrays
File "pyarrow\table.pxi", line 1463, in pyarrow.lib._sanitize_arrays
File "pyarrow\array.pxi", line 371, in pyarrow.lib.asarray
File "pyarrow\table.pxi", line 565, in pyarrow.lib.ChunkedArray.cast
File "D:\Programm\AI\autollm\myenv\lib\site-packages\pyarrow\compute.py", line 404, in cast
return call_function("cast", [arr], options, memory_pool)
File "pyarrow_compute.pyx", line 590, in pyarrow._compute.call_function
File "pyarrow_compute.pyx", line 385, in pyarrow._compute.Function.call
File "pyarrow\error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow\error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowTypeError: Size of FixedSizeList is not the same. input list: fixed_size_list<item: float>[1024] output list: fixed_size_list<item: float>[384]
(myenv) PS D:\Programm\AI\autollm>

[seoeaa]

I'll try again with a new database

[seoeaa]

my script

import os
import logging
import gradio as gr
from autollm import AutoQueryEngine
from autollm.utils.document_reading import read_github_repo_as_documents, read_files_as_documents
import lancedb

from pydantic import BaseModel, Field

logging.basicConfig(level=logging.DEBUG)

os.environ["OPENROUTER_API_KEY"] = "sk-or-v1-0c8"
model = "openrouter/openai/gpt-3.5-turbo"

relative_folder_path = "examples/data"
uri = "tmp/lancedb"
db = lancedb.connect(uri=uri)
db.drop_database()

documents = read_files_as_documents(input_dir="examples/data" , recursive=True)

service_context_params = {
"chunk_size": 512,
"embed_model":'local:intfloat/multilingual-e5-large'
}

llm_params = {
"model": model,
}

query_engine = AutoQueryEngine.from_parameters(
documents=documents,
system_prompt='..............',
query_wrapper_prompt='..................',
enable_cost_calculator=True,
llm_params=llm_params,
service_context_params=service_context_params,
vector_store_params={
"vector_store_type": "LanceDBVectorStore",
"uri": "/tmp/lancedb2",
"table_name": "lancedb",
"nprobs": 20
},
query_engine_params={"similarity_top_k": 10},
)

query = "..................."
response = query_engine.query(query)

print(response.response)

def greet(query):
return query_engine.query(query).response

demo = gr.Interface(fn=greet, inputs="text", outputs="text")
demo.launch()

[SeeknnDestroy]

hi @seoeaa. Thank you so much for being one of our early testers, we deeply appreciate it! The issue in this script is the uri you are using while trying to remove the database is not the same with the uri you are creating the database.

Can you change those lines:

"vector_store_type": "LanceDBVectorStore",
"uri": "/tmp/lancedb2",
"table_name": "lancedb",
"nprobs": 20
},

with this:

"vector_store_type": "LanceDBVectorStore",
"uri": "tmp/lancedb2",
"table_name": "lancedb",
"nprobs": 20
},

for compatibility? The change here is from /tmp/lancedb to tmp/lancedb as you are trying to remove database with this line uri = "tmp/lancedb"

[seoeaa]

everything works with the new database, didn’t change anything, slash didn’t test everything and it works like that

[seoeaa]

Is it possible to run Generating embeddings asynchronously?