Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Herestring not recognized _after_ stdout redirect. #232

Open
danfuzz opened this issue Oct 31, 2023 · 4 comments
Open

Herestring not recognized _after_ stdout redirect. #232

danfuzz opened this issue Oct 31, 2023 · 4 comments

Comments

@danfuzz
Copy link
Contributor

danfuzz commented Oct 31, 2023

It looks like the parser has trouble with herestrings that come after a redirect of stdout. For example, a parse of the following indicates an error on the here-string token <<<:

cat >x <<< 'x'

Parse tree:

(program [0, 0] - [1, 0]
  (redirected_statement [0, 0] - [0, 14]
    body: (command [0, 0] - [0, 3]
      name: (command_name [0, 0] - [0, 3]
        (word [0, 0] - [0, 3])))
    redirect: (file_redirect [0, 4] - [0, 6]
      destination: (word [0, 5] - [0, 6]))
    (ERROR [0, 7] - [0, 9])
    redirect: (file_redirect [0, 9] - [0, 14]
      destination: (raw_string [0, 11] - [0, 14]))))

Compare that to the following, which works:

cat <<< 'x'

Parse tree:

(program [0, 0] - [1, 0]
  (command [0, 0] - [0, 11]
    name: (command_name [0, 0] - [0, 3]
      (word [0, 0] - [0, 3]))
    redirect: (herestring_redirect [0, 4] - [0, 11]
      (raw_string [0, 8] - [0, 11]))))
@danfuzz
Copy link
Contributor Author

danfuzz commented Oct 31, 2023

I think the root cause here is that the grammar in this project has explicitly different rules for where herestrings are accepted vs. other redirects, whereas the actual shell grammar doesn't make that distinction (other than heredocs, which are of course super special).

@danfuzz
Copy link
Contributor Author

danfuzz commented Oct 31, 2023

My branch https://github.com/danfuzz/tree-sitter-bash/tree/danfuzz-redirects has a proposed fix, though (as I write this) it doesn't produce identical output for the Herestrings test. So, perhaps it's more of a "suggested direction" at this point.

Clarification (after looking more closely): I think my proposed change improves the parse output in most cases of the test, specifically wrapping herestring-redirected statements in (redirected_statement ...) nodes. However, it does fail to parse a case where redirections and arguments are mixed, e.g. (simplified) x <<<x x >x.

@danfuzz danfuzz changed the title Here-string not recognized _after_ stdout redirect. Herestring not recognized _after_ stdout redirect. Oct 31, 2023
@danfuzz
Copy link
Contributor Author

danfuzz commented Oct 31, 2023

Update: I think the failure I noted above highlights a pre-existing bug in the grammar. In particular, it looks like (non-redirection) arguments after a file redirection are taken to be redirections. For example:

x <x a b c

Parse tree:

(program [0, 0] - [1, 0]
  (redirected_statement [0, 0] - [0, 10]
    body: (command [0, 0] - [0, 1]
      name: (command_name [0, 0] - [0, 1]
        (word [0, 0] - [0, 1])))
    redirect: (file_redirect [0, 2] - [0, 10]
      destination: (word [0, 3] - [0, 4])
      destination: (word [0, 5] - [0, 6])
      destination: (word [0, 7] - [0, 8])
      destination: (word [0, 9] - [0, 10]))))

@amaanq
Copy link
Member

amaanq commented Feb 10, 2024

I've fixed the x <x a b c case but the first case in your issue is pretty difficult, throwing herestring_redirect under the 'redirect' field in the repeat1 body fixes it, but makes some parsing output in the tests worse, notably the same case I've fixed. But maybe that's desired, it'd be like this:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants