Unrecognised rules with grammars split across multiple files

[tomtau]

When rules defined in one pest file are used in a different pest file, as per https://pest.rs/book/grammars/grammars.html#load-multiple-grammars
( e.g. https://github.com/pest-parser/pest/blob/master/derive/examples/base.pest which is then used in https://github.com/pest-parser/pest/blob/master/derive/examples/calc.pest ), the extension will report errors of undefined rules for the names defined in a different file (even though there is no problem when building it thanks to pest-parser/pest#758 )

[Jamalam360]

This is hard because there isn't an import statement in the pest files, it is defined in the attribute in Rust...hmm. It is a similar issue as the inline grammar problem - we don't have infrastructure to communicate with Rust Analyzer and iirc there are open issues for that on the RA repo.

If I did remember that correctly, we might have to put in a temporary solution like adding a //@pest-ls include <file path> but that's not ideal :/

[tomtau]

Can there be a plugin config for users to specify whether or which grammars to load together?

[Jamalam360]

Maybe yeah

[tomtau]

Maybe a project-level/workspace-level setting, e.g.: {"loadTogether": [["base.pest", "calc.pest"], ["base.pest", "othergrammar.pest"]]} could be a workaround?

https://code.visualstudio.com/api/references/contribution-points
https://code.visualstudio.com/api/language-extensions/language-server-extension-guide#using-configuration-settings-in-the-server

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_configuration

[Jamalam360]

We already have some config so this will be fine.

I'd like to keep a track of those RA issues that might allow a kind of plugin system though

[Jamalam360]

As far as I can tell, linked grammars work by just doing grammar_one_string + grammar_two_string. This is difficult for the LS:

// one
test = { "a" ~ test_two }

// two
test_two = { "b" }

Result:

// one
test = { "a" ~ test_two }
// two
test_two = { "b" }

Imagine we want to rename test_two. This will cause edits in file one like this:

// one
test = { "a" ~ test_two }
renamed_name

Since the LSP thinks that test_two is there. I'm not sure how we want to handle this.

We might need to rethink the pest_meta parser API (to take a Vec<Pairs<_>>?) and/or the way linked grammars work, unless someone has a suggestion.

[tomtau]

Would the meta-grammar extension ("use base") help? One issue for that is how to interpret it in the file-less pest_vm environments.

[Jamalam360]

If there was an explicit import statement, the LSP would probably not require any modifications for this issue to be fixed. I'm not sure what the best way forward is. Maybe a poll in the server again

[tomtau]

@huacnlee any thoughts?

[Jamalam360]

It would also make more semantic sense to have a dedicated statement arguably

[Jamalam360]

This should probably be revisited with Pest V3, it seems too awkward to fix for now