document loading error #99
-
|
documents = read_files_as_documents(input_dir="examples/data" , recursive=True) PS D:\Programm\AI\autollm> & C:/Users/Aleksandr/AppData/Local/Programs/Python/Python310/python.exe d:/Programm/AI/autollm/app.py |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 5 replies
-
|
Hi @seoeaa, Thank you for your patience. We've addressed the issue in our latest release. The problem was due to inconsistent metadata fields in the documents being loaded into the database. We've now standardized the metadata fields across different file types to prevent this kind of error in the future. To resolve the issue on your end, please follow these steps:
Here's a quick snippet to help you clean the old database: import lancedb
# Provide the correct URI to your lancedb database
uri = "tmp/lancedb"
db = lancedb.connect(uri=uri)
# This will drop the current database so you can start fresh
db.drop_database()After cleaning the database, you can proceed with reading the documents again. If you encounter any further issues or have questions, please don't hesitate to reach out. Best, |
Beta Was this translation helpful? Give feedback.
-
|
import os from pydantic import BaseModel, Field logging.basicConfig(level=logging.DEBUG) os.environ["OPENROUTER_API_KEY"] = "sk-or-v1-0c86a53" #git_repo_url = "https://github.com/ultralytics/ultralytics.git" relative_folder_path = "examples/data" documents = read_files_as_documents(input_dir="examples/data" , recursive=True) #documents = read_github_repo_as_documents( git_repo_url=git_repo_url,relative_folder_path=relative_folder_path,required_exts=required_exts#) service_context_params = { llm_params = { query_engine = AutoQueryEngine.from_parameters( query = ".........." print(response.response) def greet(query): demo = gr.Interface(fn=greet, inputs="text", outputs="text") (myenv) PS D:\Programm\AI\autollm> python app.py |
Beta Was this translation helpful? Give feedback.
-
|
Please refer to those steps. This step-by-step guilde will solve your problem #99 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Also, as we mentioned in this discussion you need to give |
Beta Was this translation helpful? Give feedback.
-
|
I set it to 512, and updated to 0.17, and still an error, I completed the basic steps, now tit will look at the documents but everything seems to be correct there, as Vile apparently finds and loads |
Beta Was this translation helpful? Give feedback.
-
|
Are we sure that you run the following script first, to ensure we have our old database deleted? The root of the problem is coming from the old database. import lancedb
# Provide the correct URI to your lancedb database
uri = "tmp/lancedb"
db = lancedb.connect(uri=uri)
# This will drop the current database so you can start fresh
db.drop_database()After that please run |
Beta Was this translation helpful? Give feedback.
-
import os from pydantic import BaseModel, Field logging.basicConfig(level=logging.DEBUG) os.environ["OPENROUTER_API_KEY"] = "sk-or-v1-0c86a53ee9" relative_folder_path = "examples/data" documents = read_files_as_documents(input_dir="examples/data" , recursive=True) service_context_params = { llm_params = { query_engine = AutoQueryEngine.from_parameters( query = ".........." print(response.response) def greet(query): demo = gr.Interface(fn=greet, inputs="text", outputs="text") |
Beta Was this translation helpful? Give feedback.
-
|
import os from pydantic import BaseModel, Field logging.basicConfig(level=logging.DEBUG) os.environ["OPENROUTER_API_KEY"] = "sk-or-v1-0c" relative_folder_path = "examples/data" documents = read_files_as_documents(input_dir="examples/data" , recursive=True) service_context_params = { llm_params = { query_engine = AutoQueryEngine.from_parameters( query = ".........." print(response.response) def greet(query): demo = gr.Interface(fn=greet, inputs="text", outputs="text") (myenv) PS D:\Programm\AI\autollm> pip show autollm |
Beta Was this translation helpful? Give feedback.
-
|
changed the name of the db file and it worked |
Beta Was this translation helpful? Give feedback.
-
|
Amazing to hear that @seoeaa ! Feel free to ping for more questions. We are ready to help you 👍 |
Beta Was this translation helpful? Give feedback.
Hi @seoeaa,
Thank you for your patience. We've addressed the issue in our latest release. The problem was due to inconsistent metadata fields in the documents being loaded into the database. We've now standardized the metadata fields across different file types to prevent this kind of error in the future.
To resolve the issue on your end, please follow these steps:
pip install -U autollmlancedbdatabases that were created with the old version, as the metadata fields may not be compatible.read_files_as_documentsfunction.Here's a quick snippet to help you clean the old database: