config parsing interpose envvars early, sometimes causing invalid yaml

[hdhoang]

We discovered this by having a password env-var starting with grawlix @!*%..., and worked around by always ""-enclosing variable expansion. yq conversion didn't use the quotes.

I suggest that the config-convert subcommand eagerly wrap strings in quote for defense.

The symptom is syntax error in yaml parsing, pointing at ${ character. Example config, without component-specific interpolation:

timezone: ${T}
#2345678901
# workaround: timezone: "${T}"

# irrelevant minimal pipe
data_dir: "."
sources:
  stdin:
    type: stdin
sinks:
  blackhole:
    type: blackhole
    inputs: [stdin]

No issue:

❯ env T=Etc/GMT vector validate --config-yaml config.yaml
√ Loaded ["config.yaml"]
√ Component configuration
√ Health check "blackhole"
--------------------------
                 Validated

YAML parse error:

❯ env T=@ vector validate --config-yaml config.yaml
Failed to load ["config.yaml"]
------------------------------
x found character that cannot start any token at line 1 column 11, while scanning for the next token

With ""-wrapping, sensible config error:

❯ env T=@ vector validate --config-yaml config.yaml
Failed to load ["config.yaml"]
------------------------------
x No such time zone

cheers!

[jszwedko]

👍 this rough edge is also noted here: #17343 . We have discussed moving the env var interpolation to later in the config loading process, but it will be quite a bit of work.