-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Max Token/Message Control #292
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A big reason why the pickup on the Assistants API has been so slow, is the token use can get out of control. With the new V2 support you added, this can be addressed.
"Context window management
The Assistants API automatically manages the truncation to ensure it stays within the model's maximum context length. You can customize this behavior by specifying the maximum tokens you'd like a run to utilize and/or the maximum number of recent messages you'd like to include in a run."
So some UI to let us set the max messages, and tokens a run will use. It will also fix another issue where in very long conversations the bots tend to go off message. With a maximum number of recent messages, we can limit this.
The text was updated successfully, but these errors were encountered: