New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: context warning message #1952
Comments
tjbck
changed the title
Too much context fails without showing error
feat: context warning message
May 3, 2024
1 task
I am in favour of having an environment variable to disable the ability for users to change the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I decided to test gradient's 1 million context llama3 model by adjusting the context parameter accordingly. However, as can be seen in this server log I run out of memory trying to store all this context:
The issue I had was the openwebui didn't show or explain this error. It just tries to generate the response forever before erroring out with a message about connection issues to ollama. It would be very helpful if the ui could pop up a message indicating your context length caused an out of memory issue, preferably with the amount of memory it was trying to use to make it easy for users to tune how much context their system can handle.
The text was updated successfully, but these errors were encountered: