How to discern nodes based on their order?

[debugloop]

I've been playing around with using ast-grep for Go's structured logging library slog to tame a growing zoo of different formats in log field names. In addition to predeclaring fields, it also supports variadically adding log fields by invoking slog.Info("msg", "key", "value", "morekeys", "values"). I'd like to enforce kebab case on every key, but I can't get it to work.

The following Playground link demonstrates how to fix the first key and preserve the latter arguments. I'd be happy already if it would fix the last non-conforming key (second to last element in the argument_list) instead of the first. That would allow me to fix all bad fields on consecutive runs of ast-grep. My idea has been too add a $$$PREV_VALUES before the constrained $KEY, but it appears not greedy enough and always matches zero nodes.

Even better would be the option to fix all keys in one go, without iterating executions of ast-grep to get the last non-conforming key, but as I said, I'd be happy with the first already. I've read about util rules and the possibility to go recursive, but I'm just not there yet understanding-wise to tackle this, but I feel this might be a way to go...

Playground Link

On a similar note: The amount of args should always be odd 🤔

[HerringtonDarkholme]

Hi @debugloop

Thanks for you valuable feedback! I appreciate all your questions and reporting and they gave me many insights on how users are using ast-grep.

For the use case in this post, I think it may be impossible to fix multiple nodes or nodes. At the moment, ast-grep fixing can only fix one target ast node at a time. And telling node's order is not possible at the moment either. :(

I didn't support them because it may be too complex for the rule YAML.

I can try using YAML rule to implement this change. But I would recommend to use programmatic API.

Sadly, currently python API is still under development. #672

NodeJS API has not supported the Python language yet.

[debugloop]

Thanks for still trying, I'd rather stick to YAML as I intend to use it as a pipeline component where others can add there rule in a simple and straightforward way, without modifying/extending the pipeline by further calls or executables.

[HerringtonDarkholme]

Since util rule is recursive, we can always roll out order-specific rules by hand. (alas, peano-number)

In this case, we can define the key node as either

1. follows the first argument message
2. follows a value argument which itself follows another key node.

id: slog-kebab-dynamic-call
message: slog field does not use kebab-case
language: go
utils:
  inside-slog:
    pattern:
      context: '{ slog.$FUNC($$$) }'
      selector: argument_list
  first-key:
    not:
      follows:
        pattern: $_
        stopBy:
          pattern: $_
  at-key:
    kind: interpreted_string_literal
    follows:
      stopBy:
        pattern: $_
      pattern: $_
      any:
        - matches: first-key
        - follows:
            matches: at-key
            stopBy:
              pattern: $_
rule:
  matches: at-key
  pattern: $KEY
  inside:
    matches: inside-slog
constraints:
  FUNC:
    regex: '^(Debug|Info|Warn|Error)$'
  KEY:
    regex: '^".*?[A-Z._ /].*?"$'
transform:
  BARE_KEY:
    substring:
      source: $KEY
      startChar: 1
      endChar: -1
  KEBAB_KEY:
    convert:
      source: $BARE_KEY
      toCase: kebabCase
fix: '"$KEBAB_KEY"'

Rule Playground

I would think about how to express the "quantity/order" concept in YAML rule. But currently this is a workaround.

[debugloop]

Thanks so much, that get's me a lot farther in my quest 🙏🏻 I kinda don't understand why the outermost pattern (inside-slog) does not really restrict the results any more, and also the FUNC constraint seems to be ineffective. I've tried pulling the (inside-slog) match into the top level pattern, but it seems the ineffectiveness of the constraints is not related to the invoke through a match... See here.

[HerringtonDarkholme]

I forgot to update the selector field! I changed it to other things.

https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiIoJCQkKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogc2xvZy1rZWJhYi1keW5hbWljLWNhbGxcbm1lc3NhZ2U6IHNsb2cgZmllbGQgZG9lcyBub3QgdXNlIGtlYmFiLWNhc2Vcbmxhbmd1YWdlOiBnb1xudXRpbHM6XG4gIGluc2lkZS1zbG9nOlxuICAgIHBhdHRlcm46XG4gICAgICBjb250ZXh0OiAneyBzbG9nLiRGVU5DKCQkJCkgfSdcbiAgICAgIHNlbGVjdG9yOiBjYWxsX2V4cHJlc3Npb25cbiAgZmlyc3Qta2V5OlxuICAgIG5vdDpcbiAgICAgIGZvbGxvd3M6XG4gICAgICAgIHBhdHRlcm46ICRfXG4gICAgICAgIHN0b3BCeTpcbiAgICAgICAgICBwYXR0ZXJuOiAkX1xuICBhdC1rZXk6XG4gICAga2luZDogaW50ZXJwcmV0ZWRfc3RyaW5nX2xpdGVyYWxcbiAgICBmb2xsb3dzOlxuICAgICAgc3RvcEJ5OlxuICAgICAgICBwYXR0ZXJuOiAkX1xuICAgICAgcGF0dGVybjogJF9cbiAgICAgIGFueTpcbiAgICAgICAgLSBtYXRjaGVzOiBmaXJzdC1rZXlcbiAgICAgICAgLSBmb2xsb3dzOlxuICAgICAgICAgICAgbWF0Y2hlczogYXQta2V5XG4gICAgICAgICAgICBzdG9wQnk6XG4gICAgICAgICAgICAgIHBhdHRlcm46ICRfXG5ydWxlOlxuICBtYXRjaGVzOiBhdC1rZXlcbiAgcGF0dGVybjogJEtFWVxuICBpbnNpZGU6XG4gICAgbWF0Y2hlczogaW5zaWRlLXNsb2dcbiAgICBzdG9wQnk6XG4gICAgICBraW5kOiBjYWxsX2V4cHJlc3Npb25cbmNvbnN0cmFpbnRzOlxuICBGVU5DOlxuICAgIHJlZ2V4OiAnXihEZWJ1Z3xJbmZvfFdhcm58RXJyb3IpJCdcbiAgS0VZOlxuICAgIHJlZ2V4OiAnXlwiLio/W0EtWi5fIC9dLio/XCIkJ1xudHJhbnNmb3JtOlxuICBCQVJFX0tFWTpcbiAgICBzdWJzdHJpbmc6XG4gICAgICBzb3VyY2U6ICRLRVlcbiAgICAgIHN0YXJ0Q2hhcjogMVxuICAgICAgZW5kQ2hhcjogLTFcbiAgS0VCQUJfS0VZOlxuICAgIGNvbnZlcnQ6XG4gICAgICBzb3VyY2U6ICRCQVJFX0tFWVxuICAgICAgdG9DYXNlOiBrZWJhYkNhc2VcbmZpeDogJ1wiJEtFQkFCX0tFWVwiJ1xuIiwic291cmNlIjoiZnVuYyBtYWluKCkge1xuICBzbG9nLldhcm4oIFwiVGhpcyBpcyB0aGUgbWVzc2FnZSFcIiwgXCJGaWVsZEtleU9uZVwiLCBcImZpZWxkLWNvbnRlbnQtb25lXCIsIFwiZmllbGQta2V5X3R3b1wiLCBcInNkZnNkZlwiLCBcInNkZnNkZlwiLCBcInNkZGZzZGZzZFwiKVxuICBzbG9nLldhcm4oXCJUaGlzIGlzIHRoZSBtZXNzYWdlIVwiLCBcImZpZWxkLWtleS1vbmVcIiwgXCJmaWVsZC1jb250ZW50X29uZVwiLCBcImZpZWxkLWtleV90d29cIiwgMy4xNClcbiAgb3RoZXIuTGliRnVuY3Rpb24oXCJUaGlzIGlzIHRoZSBtZXNzYWdlIVwiLCBcImZpZWxkLWtleS1vbmVcIiwgXCJmaWVsZC1jb250ZW50X29uZVwiLCBcImZpZWxkLWtleV90d29cIiwgMy4xNCkgIFxufSJ9

[debugloop]

Thank you again, got my head around it and managed to adapt it to work on WarnContext(ctx, ... calls as well 🎉 I got one more question though: Performance wise this seems way worse than just a plain lookup for slog.$FUNC, why is that? Does the recursion run on all functions regardless of the outer match? I.e. the inside part is checked only after the initial match is through?

Rewriting to the following does not seem to change things, even though I can specify order this way:

rule:
  all:
    - inside:
        matches: inside-slog
        stopBy:
          kind: call_expression
    - matches: at-key
    - pattern: $KEY

[HerringtonDarkholme]

Using all can specify rule order so it is generally faster than using object style rule. (Object style rule is for more concise rule)

So in this case, it will first run the recursion rule. Using all can let us choose which rule to run first!

Haha this is an advanced usage of all rule (currently it is not documented). Glad you find it!

[debugloop]

Yes I thought it would be this way, but it's not really different I'm afraid. It seems like the fixed order does not have an effect sadly...