How can we improve CI integration?

[tim775]

I’ve been talking with people about how they integrated infracost into their CI pipeline and I’ve been getting some great suggestions on how to make it easier/better. I’m still trying to figure out GH discussions, so this time I’m going try posting the different ideas as comments.

Feel free to post your own ideas and if you see something you like, uproote, thumbsup or reply!

[tim775]

Make the GH Action more composable

Currently the infracost action runs the ‘scripts/ci/diff.sh’ bash script. That performs several steps: runs terraform, generates a diff, puts a comment on the pr. Each of these could be exposed as individual steps making it easier to, for example, upload the diff as an artifact.

[tim775]

Don’t run Terraform in the Infracost action, use the setup_terraform action for that

Since Infracost really just needs a plan json to do its thing, let people setup up and run Terraform using the official Hashicorp actions. Same goes for vault or whatever else is needed.

[tim775]

Add CI specific --format options

The CLI breakdown, diff, and output commands support the --format flag so you can choose the output style as table output, json, or html. Why not add --format ghaction or --format cicomment?

[snordmann]

Makes sense, though I think that means that you would need to support different Markdown flavors. And also the question whether the output should be escaped already.

[PatMyron]

need to support different flavors

https://sarifweb.azurewebsites.net/ aims to solve that, Github already has some SARIF support (cc @NickLiffen)

[NickLiffen]

Agreed 👍 SARIF aims to make these sorts of things standardised to an extent. If you suppose SARIF, it means developers could use GitHub Code Scanning. Happy to help in here if you would like an assist in getting SARIF supported 👍

[tim775]

Support for monorepos

Automatically detect nested tf projects when running. Or maybe automatically generate an Infracost config file that lists all nested projects.

[tim775]

Better multiproject output

When posting to a PR comment, show a table/matrix summarizing the diff for each project. Or only show projects that have changes.

[tim775]

GH action library

The docs currently show two ways to set up gh actions, but there are a lot of different ways to do it. Could we have a section of the docs for "user submissions" so I can see more examples? For example it would be cool to see an example of how to setup a Terragrunt monorepo that gets required credentials from Vault.

[tim775]

Ditch the bash script

The scripts have outgrown bash, should we move the logic to composable go commands or nodejs?

[snordmann]

There was already some discussion around that in #962.
Possible options:

1. Integrate notifications channel configuration in Infracost Dashboard
2. Analyze CI use-cases and rewrite CI integration in go, typescript, whatever
3. Refactor bash scripts so that they are easier to work with

My preferred approach would be 2, because you could:

• leverage existing SDKs for the CI integration (eg. https://github.com/xanzy/go-gitlab for GitLab, https://github.com/slack-go/slack for slack, etc.)
• is easy to use (no self-hosted setup needed like option 1)
• can be configured easily, eg. using infracost config (unlike option 3)
• type safety (unlike option 3)

Some downsides are:

• if integrated with main infracost cli, it will make the code base more bloated
• difficult to prototype and adjust for individuals
• if it is an extra application, there might be weird dependencies between terragrunt, terraform, infracost and the infracost-ci app

So basically, even a rewrite in go or typescript is no silver bullet.

[tim775]

Default to "Update" for pull request comments

Having an Infracost comment on every commit can be noisy. Should we make "one comment per PR" the default behavior?

[tim775]

Sparkline in comments

Put a sparkline in the comments showing how the total cost has changed with each commit.

[tim775]

Use the GH "Check" API to show a one line status on the PR

We could show something like "Total $xxx. +/-$yyy this Pull Request" and link to an artifact with the full breakdown.

[alikhajeh1]

Most of the ideas in this discussion went into action!

See https://www.infracost.io/docs/guides/actions_migration/

Thanks to everyone who contributed to the discussion