-
-
Notifications
You must be signed in to change notification settings - Fork 257
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature] Generating constant str rules #718
Comments
I'm thinking whether other feature (e.g. node/branch tagging) could be hijacked/reused for this use case in some way. For the path 2, you can check out https://github.com/pest-parser/pest/tree/master/meta/src/optimizer -- e.g. this https://github.com/pest-parser/pest/blob/master/meta/src/optimizer/concatenator.rs will turn e.g. |
Is the node/branch tagging you are referring to discussed somewhere? |
#550 -- it's primarily for the parser state and results, but it could perhaps be exposed in the generated rules code |
Problem
Often the grammar requires some constant strings in it (keywords, operators, etc). These can be introduced as a separate rule, example:
and then reused in some other rules. When parsing and then generating the AST, these constant tokens are lost (which is obviously a good thing). But when we want to serialize the AST back to its code form we need these constant tokens. Since pest does not generate these constant strings as public (they are inlined), one could not reuse them. The final solution is to redefine them, for example by attaching them to the AST nodes:
This leads to a problem when we want to change a constant string; we need to remember to change it in both places.
Proposal
Pest could detect const rules and generate constants for them, for example:
I see two paths that can be taken here:
Path 2 is obviously preferred, but harder to implement: then there would be a need of some const eval engine. For instance, a rule is constant if all rules used inside are constant, if the repetition count is constant, etc. Then a the rule would be have to be evaluated to a single final constant string.
I don't know the codebase at all, but I'm pretty sure I would be able to implement path 1 and send a PR for it if it is wanted. I would only worry about whether such a primitive feature would feel incomplete compared to path 2.
The text was updated successfully, but these errors were encountered: