CLI flag to return JSON from percy build:wait

[sandstrom]

Summary

We don't want to use the Percy Github integration, due to security concerns. When you get hacked, as most companies eventually are (Solarwinds and Heroku are two recent examples), we don't want all our source code to leak.

percy build:wait --build 123 should have a --json flag that returns json data that looks like this:

{ "status" : "changes", "change_count" : 8, "build_url" : "https://percy.io/…builds/27676" }

Problem

For security, we'd like to use the Percy CLI, which we can pin with npm and checksums (fairly secure). Your current CLI is great and gets us most of the way.

We'll run something like this in Github Actions:

percy exec -- node percy-test.js 
[percy] Percy has started!
[percy] Running "node percy-test.js"
[percy] Snapshot taken: Website
[percy] Finalized build #4: https://percy.io/57827…/website/builds/27676

After trigger the build, I need to wait for it to complete, before we can report back in the Github Action check (CI flow).

Do do that, you have the build:wait command. Great! When running percy build:wait --build XXX I get relevant data back.

But it's just text, so not easy to parse.

$ percy build:wait --build 27676
[percy] Build #3 finished! https://percy.io/57827…/website/builds/27676
[percy] Found 8 changes

Would be great with a --json option, that we can parse programmatically.

Because we need to manually grab the URL and the status. Otherwise we cannot update the PR properly (because we're not using your Github integration, for security reasons we talked about).

Ideally we'd get something like this:

$ percy build:wait --build 27676 --json
{ "status" : "changes", "change_count" : 8, "build_url" : "https://percy.io/…builds/27676" }

# Response with no changes
{ "status" : "no_changes", "build_url" : "https://percy.io/…builds/27676" }

# Failure
{ "status" : "failed", "build_url" : "https://percy.io/…builds/27676" }

Maybe you'll want to give the keys other names, obviously. But you get the idea.

This would be enough for us to then call the Github API and add a comment to the PR, so basically we'd have the same flow as with a Github integration, but without any of the security concerns.

Pull-request linking

On this page you have details about the ENV variable PERCY_PULL_REQUEST ("The pull request number to associate to the build, if any").

But how would you know the base path? I'm thinking you'd rather want it to be something like PERCY_PULL_REQUEST_URL where we enter the full URL to the PR. That way, you can link to that PR from within your UI, without actually having read access to it (it's just a link).

Or, you'd need to let us set the repo base path in your settings. We'd set it to e.g. https://github.com/my-org/my-repo.