You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now, if a stop word is reached, because StoppingStream is a wrap over the completion stream, the stop word itself is still added to the context and to the SessionId. This causes the following prompt for the same session to need a whole new session to be created, because the original one with the stopping word does not match, and may cause issues with inference due to the extra token that should be getting ignored.
The text was updated successfully, but these errors were encountered:
Right now, if a stop word is reached, because
StoppingStream
is a wrap over the completion stream, the stop word itself is still added to the context and to theSessionId
. This causes the following prompt for the same session to need a whole new session to be created, because the original one with the stopping word does not match, and may cause issues with inference due to the extra token that should be getting ignored.The text was updated successfully, but these errors were encountered: