New Feature: onyo new --yaml (with supporting flags)

[aqw]

This is a WIP. Saving now. Will finish later:

---

This issue is about adding the --yaml flag to onyo new, which has multiple knock-on effects that need to be addressed at the same time.

I have made the following modifications from our whiteboarding session:

• I inverted --yaml-preserve-comments to --yaml-ignore-comments
• I inverted the behavior of comment merge conflicts. Comments are overwritten like key-values. When there's conflicts, then the new comments are used. This makes --yaml-ignore-comments more meaningful. And one can use --yaml with comments to mass update comments.

Relevant parts of onyo new --help:

Asset contents are populated in a waterfall pattern and overwrite
values from previous steps:

  1) --clone or --template
  2) --yaml
  3) --keys
  4) --edit (i.e. manual user input)

YAML files containing multiple (N) documents will create multiple
assets. Likewise, keys passed multiple (N) times to --keys will also
create multiple assets. N must be the same everywhere (or 1).
As an illustration:

     b1         d1  (result: a + b1 + c + d1)
a -> b2 -> c -> d2  (result: a + b2 + c + d2)
     b3         d3  (result: a + b3 + c + d3)

See --keys and --yaml for more information.

  -c, --clone CLONE
                        Path of an asset to clone. The equivalent of --recursive --yaml.
                        
                        This cannot be used with the --template flag nor the
                        onyo.new.template Pseudo Key.
  -t, --template TEMPLATE [...]
                        Path of a template to populate the contents of new assets.
                        The equivalent of --recursive  --yaml .onyo/templates/.
                        
                        This cannot be used with the --clone flag nor the
                        onyo.new.template Pseudo Key.
  -y, --yaml YAML [...]
                        Path to a YAML file (or asset or template) to populate the
                        contents of new assets. When an asset/template directory
                        is given, only the top-level is used. See also --recursive.
                        
                        Each YAML file can contain either 1 or N documents (where
                        N is the number of assets to be created) separated by a
                        `---` line. A file with one document will apply to all new
                        assets, otherwise each will be applied to each new asset
                        in the order they are declared.

                        Comments are written like key/values, where a new
                        comment will overwrite the previous comment.
                        See also --yaml-ignore-comments.
  -i, --yaml-ignore-comments
                        Do not set comments.
  -r, --recursive
                        Recurse into Asset/Template Directories. All child assets,
                        templates, and directories are enumerated and behave as if
                        they are a multi-document YAML file

                        Can only be used with the --yaml flag.

Tasks:

• --template only operates on templates, and can accept multiple templates
• --template is only relative to the template dir. It no longer tries CWD as a fallback.
• --clone only operates on assets, and only one asset
• --yaml (and thus --template and --clone) can accept multi document YAML files (see design: multi document file support #724)
• --yaml-ignore-comments will disable comments for all files, including the first one (e.g. --template). If someone wants the comments of the first, but not later in --yaml, then they should remove the comments from the --yaml file. It's trivial to do so.

Open question:

1. the onyo.new.template pseudo key makes things complicated. --template or --clone disallow the pseudo key (meaning they're not truly aliases of --yaml). --yaml accepts it, unless --clone or --template have also been used.
2. can a template create a directory with only the content (think multi-doc files):

    onyo.is.directory: True
    onyo.is.asset: False
   

3. Define the sort criteria for recursive behavior.
4. write design issue for stdin (cat multiple templates/assets into a multi-doc and then into new)
5. write design issue for comment overwrite behavior
6. should --yaml-ignore-comments truly ignore them, or rather disable overwriting comments? Ignoring comments is "wrong", because we do want the comments for the first file (usually template). And if the user doesn't want comments later, they can remove them with sed.

Notes:

• The design issue for this feature is Design: allow onyo new to accept values in YAML files #703
• This depends on design: multi document file support #724 (will turn into a feature request)
• Related: new --tsv will be removed: Design: replace onyo new --tsv with onyo tsv-to-yaml #722