Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Omit/Modify Conversation History When Triggering Request in useChat #1466

Open
DarioPanada opened this issue Apr 29, 2024 · 0 comments
Open
Labels

Comments

@DarioPanada
Copy link

DarioPanada commented Apr 29, 2024

Feature Description

Hi Team,

Firstly, thank you so much for supporting this library, I have been finding it really useful!

I have a usecase where I need to limit the payload size when invoking an inference service. Because of this, it would be useful that the handSubmit function exposed by the useChat hook allowed to either omit the conversation history from the request (and only pass the latest user prompt), or allowed to override the conversation history that is passed. (So that, for example, I could only pass the first 15 words of each message.)

Right now, I see the library's API doesn't allow for that, but I was wondering if in principle it is something you would be willing to support? I made a sample PR, happy to take feedback :) #1468

Use Case

Limiting payload size when back-end service imposes this kind of limit. Eg: AWS CloudFront when intercepting requests with ViewerRequest Lambda@Edge function associations imposes a 40kb limit after which the request is truncated.

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants