-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
getting error "conversation not found" #147
Comments
Looks like this is a bug. I think I have addressed this and published a new version - |
@adhityan Thank you! it works now! However, shouldn't any subsequent run (with the same query and same conversation id) be much faster due to the cache? I see different outputs from the llm and it's "slow", so it looks like no cache is being used. |
I see what you mean and perhaps the documentation should make this clearer / we should really rename this feature. The cache is not used to cache the query response against a query seen already. It's used to store the metadata around loaders, conversations, etc. It makes sure that a document or URL (or any already seen loader) is not re-processed. It also stores info around if a particular chunk is already seen and avoid creating new embedding for it. Apart from this, it stores conversation data so that users can get more contextual response. A better name from cache is database and maybe we should really rename it down the line. The app sends all conversations in a topic (upto a limit after which it prunes older entries) to the LLM as history so the LLM knows what the user is talking about. This is useful if you ask follow-up questions. By default, if no optional Tl;dr - the more you chat with the app, the more the content sent to the LLM and thus the slower it gets. The next version of embedJs will allow an option to turn off the conversations feature altogether - allowing you to have no history. Hope this was helpful. |
@adhityan Thank you for the clarification, that was indeed very helpful. But then something else, maybe I'm understanding it completely wrong. Currently, each query (regardless of whether cache is active or not, regardless of whether it's a new conversationId or not) with the code above takes about 2 seconds, or even longer. I came across this library through some searching and it looked very promising. Because I actually want to use it to build my own "knowledge bases" for a twilio voice bot (e.g. with the realtimeAPI). Thanks again for a response and information regarding this as well. |
It should not be re-creating any embedding again when you have the cache set. I will check if there is a bug here and if so address it with a quick bugfix release tom. Will keep this ticket open in the meantime. The response from the LLM can take some time but its usually fast. There is a work in progress to allow streaming responses from the LLM which should bring the time to first words lower. |
Thank you for looking into it! Is there a way to debug
You're support is really kind. Really appreciate it! 🙏 |
You can get a lot of debug logs by setting the following environment variable I pushed across an old refactor topic which renames caches to stores along with a number of internal changes. This issue made me realize its important to address this before new loaders / databases are added in. If you update to version |
Thank you very much @adhityan! |
🐛 Describe the bug
I'll use a simple example:
Unfortunately, I get the following error when executing:
what am I doing wrong?
The text was updated successfully, but these errors were encountered: