-
Notifications
You must be signed in to change notification settings - Fork 316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot generate parentheses in JSON strings #838
Comments
lapp0
added a commit
to lapp0/outlines
that referenced
this issue
May 17, 2024
lapp0
added a commit
to lapp0/outlines
that referenced
this issue
May 18, 2024
lapp0
added a commit
to lapp0/outlines
that referenced
this issue
May 18, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the issue as clearly as possible:
I am fine tuning a model to convert non-standard JSONs to a standard schema. I'm using vLLM with Outlines to constrain the output generation so it will follow my desired schema. I have noticed that after a recent Outlines update, the model seems to no longer be able to generate parentheses in JSON strings. Some example diffs are below (minus sign represents the expected line and plus represents the actual output)
As you can see above, it will it will either drop parentheses, replace them with a similar character like square bracket, or completely skip text in parentheses. I have seen about 100 other similar examples.
This previously was not an issue for me, so I check recent PRs in this repo to see if one of them affected relevant code. This one seems to be the culprit: #829
Specifically the following change:
It looks like for some reason, opening and closing parentheses were added to the prohibited characters for strings.
I'm not confident to submit a PR to fix this bug because I'm not sure of the motivation behind the PR that caused the bug.
Steps/code to reproduce the bug:
Expected result:
Error message:
No response
Outlines/Python version information:
0.0.40
Python 3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0]
Context for the issue:
No response
The text was updated successfully, but these errors were encountered: