Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Gomplate as a Template Manager #589

Open
osterman opened this issue Aug 27, 2019 · 11 comments
Open

Feature Request: Gomplate as a Template Manager #589

osterman opened this issue Aug 27, 2019 · 11 comments
Assignees
Milestone

Comments

@osterman
Copy link

@hairyhenderson I understand this may be a long shot feature request! Anyways, just wanted to get it on your radar, in case it sounds cool.

what

  • support a way to create templatized projects, e.g. a full-fledged template manager for boilerplate code
  • provide a common way to reference project templates e.g. using github repos
  • support a project config file with properties (variables) and a github project source for the template.

why

  • Sometimes more "abstractions" is just more convoluted when simple template generation does the job. Template generation is very well understood. It allows projects to diverge, but also provides a way to systematically update by regenerating templates
  • The boilr project appears abandoned
  • Gomplate is insanely powerful for templating

our use-cases

Don't get me wrong, gomplate can do this today with some scripting. We do it already to some degree at cloudposse. Just I would love if gomplate became a convention for how to structure templatized projects, rather than everyone creating their own convention. E.g. today, we use a Makefile with a bunch of custom scripting.

  1. We generate our README.md from a README.yaml
  2. Generate the LICENSE file from the projects configuration
  3. Automatically create .github/workflows (or travis/circle/codefresh pipelines) for a project with template parameters
  4. Create the scaffolding for "terraform" modules

prior art

@hairyhenderson
Copy link
Owner

Hey @osterman! Thanks for this, it's really interesting 🤔

First off, I'm going to assume that full boilr functionality isn't what you're looking for necessarily - is the interactive prompting stuff important? If so I'm thinking that probably belongs in a separate higher-level gomplate-based app. Guiding users interactively is useful to be sure, but IMO it doesn't make sense with the gomplate flow right now...

Barring the interactivity, this sounds to me like a smarter kind of --input-dir that can understand a conventional layout that includes a set of default values that don't need to be explicitly provided with a --context or --datasource argument. An interesting feature of boilr is that it understands templated filenames too - something that gomplate currently doesn't - that could be part of this, though TBH I can't see why it couldn't just as a matter of course. Though it'd be pretty weird to have a files named {{ .foo | regexp.Replace "(foo)bar" "$1" }}.

I'm not sure --input-dir is right, but maybe a new --input-project argument is all that's needed, something like:

# grab the project from the filesystem
$ gomplate --input-project ./sample-project
...
# grab the project from GitHub
$ gomplate --input-project github.com/hairyhenderson/sample-project
...
# or some other git repo
$ gomplate --input-project [email protected]:hairyhenderson/sample-project.git
...

These would all use the defaults provided in the project, probably using that file as the . context, I guess, and then overriding seems fairly natural:

$ gomplate --context .=./my_overrides.yaml --input-project github.com/hairyhenderson/sample-project

Am I missing anything?

@osterman
Copy link
Author

@hairyhenderson yes, I think what you describe is pretty accurate.

I'm going to assume that full boilr functionality isn't what you're looking for necessarily - is the interactive prompting stuff important?

True, but I think it does look fancy and demos well =)

Barring the interactivity, this sounds to me like a smarter kind of --input-dir

Agree.

that includes a set of default values that don't need to be explicitly provided

Yes, some sane defaults, like values.yaml in helm.

You suggest: (I agree)

gomplate --context .=./my_overrides.yaml --input-project github.com/hairyhenderson/sample-project

Now, the only modifications I would make are....

  1. Add a --config=GOMPLATE_CONFIG parameter that defaults to something. GOMPLATE_CONFIG is an environment variable and if not defined, defaults to .gomplate.yaml or something.
  2. We can specify all arguments in this config.
  3. Support subpaths and tags for version pinning and monorepos like in helmfile or terraform. e.g.
  • --input-project github.com/hairyhenderson/sample-project@foo/bar?ref=1.2.3 (helmfile syntax)
  • --input-project github.com/hairyhenderson/sample-project//foo/bar?ref=1.2.3 (terraform syntax)

For example, .gomplate.yaml:

context: ./my_overrides.yaml 
input-project: github.com/hairyhenderson/sample-project//foo/bar?ref=1.2.3
output-dir: .
overwrite: true

Now, in a project, we can just run gomplate and it will render the project, overwriting any files that t conflict. Running git diff (assuming committed code) will show us what changed. We can always revert or cherry-pick changes.

An interesting feature of boilr is that it understands templated filenames too - something that gomplate currently doesn't

I think this is nice, but could maybe come later?

@hairyhenderson
Copy link
Owner

(eep, I thought I responded days ago - somehow this didn't get posted!)

@osterman ok, yeah, that all sounds awesome. I've been thinking about a gomplate config file recently, and it'd fit nicely into this.

I'm going to try to break this down into bite-sized chunks, which should mostly be able to be implemented atomically. I'll create separate issues for these as I go to make them easier to work on:

  • Support a gomplate config file to set commandline (and other!) params
    • defaults to .gomplate.yaml, overridden by GOMPLATE_CONFIG, and finally by --config=
  • Support for boilerplate projects with --input-project (or maybe just --project?)
    • layout of templates/ folder plus defaults.yaml file
  • Support reading projects from git repositories, including subpaths and git refs
    • maybe this should extend to --file and --input-dir too?
    • maybe this should be more generic support for reading from URLs (using the same schemes as for datasources)?

@rossedman
Copy link

rossedman commented Sep 30, 2019

@hairyhenderson 👋 also very interested in this. was going to recommend the above (config file, reading projects from git, etc). I was wondering if this work had begun or what was the status of it. Thanks.

@hairyhenderson
Copy link
Owner

@rossedman hi! I've taken a bit of a break from gomplate for the past month or so - not for any particular reason, just lots of "life" things taking precedence 😉.

All that to say, no I haven't begun working on this, though I was just thinking about it the other day. I won't promise any kind of timeline, but I will promise that I'll get started on this sometime soon. I ran into a few situations over the past couple weeks where at least the git support would've been useful.

@hairyhenderson
Copy link
Owner

Just a bit of an update - support for git datasources, which is tangentially related to this feature, is pretty close to dropping here: #661 (comments welcome, especially on the docs!). It was a bit more of a pain to implement than I expected 😂

@lorengordon
Copy link

This would tie in very nicely to a gomplate github action also... Have a git project managing the templates, and a github action where gomplate uses that project as the template source, templates the files in the current repo, and opens a pull request with any changes.

@github-actions
Copy link

This issue is stale because it has been open for 60 days with no activity. Remove stale label or comment or this will be automatically closed in a few days.

@osterman
Copy link
Author

osterman commented May 5, 2023

Would still love to see this! =)

@github-actions
Copy link

github-actions bot commented Jul 5, 2023

This issue is stale because it has been open for 60 days with no
activity. If it is no longer relevant or necessary, please close it.
Given no action, it will be closed in 14 days.

If it's still relevant, one of the following will remove the stale
marking:

  • A maintainer can add this issue to a milestone to indicate that
    it's been accepted and will be worked on
  • A maintainer can remove the stale label
  • Anyone can post an update or other comment

@github-actions github-actions bot added the Stale label Jul 5, 2023
@hairyhenderson hairyhenderson self-assigned this Jul 5, 2023
@hairyhenderson hairyhenderson added this to the future milestone Jul 5, 2023
@hairyhenderson
Copy link
Owner

I've assigned this to the future milestone so this shouldn't go stale again. This might be fulfilled by v4, but I'm not certain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants