Support GitLab's build and release instances as OIDC providers

[balasankarc]

Summary

We at GitLab (https://about.gitlab.com) are looking at integrating cosign to our official build and release workflow and are very much interested in using the keyless signing flow for it. However, our builds and releases happen not at GitLab.com, but at two separate GitLab instances - dev.gitlab.org and ops.gitlab.net. This PR adds support for those instances as valid OIDC token providers so that keyless signing and verification can be done in pipelines running on those instances.

As the first step, we will be signing our cloud native Docker images using cosign, and are also thinking about signing other build artifacts in the future.

PS: I referred to #1214 for this PR.

Release Note

• Support GitLab's build and release instances as OIDC providers

Documentation

[balasankarc]

cc @haydentherapper

[codecov]

Codecov Report

Merging #1327 (7174b50) into main (46be165) will increase coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1327      +/-   ##
==========================================
+ Coverage   58.40%   58.49%   +0.09%     
==========================================
  Files          50       50              
  Lines        3053     3053              
==========================================
+ Hits         1783     1786       +3     
+ Misses       1112     1110       -2     
+ Partials      158      157       -1     

see 1 file with indirect coverage changes

[haydentherapper]

Thanks! Can you update the order to be alphabetized?

[balasankarc]

@haydentherapper Sure. Done.

[haydentherapper]

@balasankarc Can you confirm how the gitlab.com issuer is used?

[balasankarc]

@haydentherapper I'm trying to setup a local fulcio/rekor instance and imitate the test in #1214 (comment)

Config file of fulcio is as follows

{
  "OIDCIssuers": {
    "https://ops.gitlab.net": {
      "IssuerURL": "https://ops.gitlab.net",
      "ClientID": "sigstore",
      "Type": "gitlab-pipeline"
    }
  }
}

$ cosign sign --yes --oidc-issuer="https://ops.gitlab.net" --fulcio-url="http://fulcio.balasankarc.in:5555" --rekor-url="http://fulcio.balasankarc.in:3000" --insecure-skip-verify "docker://${CI_REGISTRY_IMAGE}:${CI_JOB_ID}@${image_digest}" 
Generating ephemeral keys...
Retrieving signed certificate...
Error: signing [docker://registry.ops.gitlab.net/balasankarc/test-fulcio:10937947@sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33]: getting signer: getting key from Fulcio: retrieving cert: POST http://fulcio.balasankarc.in:5555/api/v1/signingCert returned 400 Bad Request: "{\"code\":3,\"message\":\"There was an error processing the identity token\",\"details\":[]}"
main.go:74: error during command execution: signing [docker://registry.ops.gitlab.net/balasankarc/test-fulcio:10937947@sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33]: getting signer: getting key from Fulcio: retrieving cert: POST http://fulcio.balasankarc.in:5555/api/v1/signingCert returned 400 Bad Request: "{\"code\":3,\"message\":\"There was an error processing the identity token\",\"details\":[]}"

In the fulcio logs, I am seeing

2023-08-18T05:25:34.693Z        ERROR   server/error.go:45      returning with error    {"requestID": "-3sDJtZU", "code": "InvalidArgument", "clientMessage": "There was an error processing the identity token", "error": "failed to match issuer URL https:/
/accounts.google.com from token with any configured providers"}

Looks like I might be getting hit with sigstore/cosign#1258 as I am not seeing the OIDC issuer I specified being used by cosign, but google is being used (this is being run on a GCP VM, spun up by a GitLab CI job, which has SIGSTORE_ID_TOKEN env variable set as expected)

[haydentherapper]

Can you set --oidc-provider=envvar? This should force cosign to use the env var token rather than GCP.

[haydentherapper]

@balasankarc Did you have any updates on if this is working as intended for you?

Also just wanted to confirm that the gitlab.com issuer is still in use?

[haydentherapper]

Just wanted to bump this - I'm good to merge this and get these into staging once you confirm this is working for you locally and the gitlab.com issuer is needed.

[balasankarc]

@haydentherapper Apologies - I was OOO for last two weeks and away from my work computer and am just back today. I will continue with testing it locally and get back to you.

Also, to answer your other question

Also just wanted to confirm that the gitlab.com issuer is still in use?

Yes. It is definitely still in use.

[balasankarc]

@haydentherapper Confirmed that this works locally. --oidc-provider=envvar helped

/ # COSIGN_EXPERIMENTAL=1 cosign sign --yes --fulcio-url="http://fulcio.balasankarc.in:5555" --insecure-skip-verify "registry.ops.gitlab.net/balasankarc/test-fulcio:11097770@sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33"
Generating ephemeral keys...
Retrieving signed certificate...

        The sigstore service, hosted by sigstore a Series of LF Projects, LLC, is provided pursuant to the Hosted Project Tools Terms of Use, available at https://lfprojects.org/policies/hosted-project-tools-terms-of-use/.
        Note that if your submission includes personal data associated with this signed artifact, it will be part of an immutable record.
        This may include the email address associated with the account with which you authenticate your contractual Agreement.
        This information will be used for signing this artifact and will be stored in public transparency logs and cannot be removed later, and is subject to the Immutable Record notice at https://lfprojects.org/policies/hosted-project-tools-immutable-re
cords/.

By typing 'y', you attest that (1) you are not submitting the personal data of any other person; and (2) you understand and agree to the statement and the Agreement terms at the URLs listed above.
tlog entry created with index: 34424008
Pushing signature to: registry.ops.gitlab.net/balasankarc/test-fulcio
/ # curl --silent http://fulcio.balasankarc.in:5555/api/v1/rootCert > /tmp/root.pem
/ # export SIGSTORE_ROOT_FILE=/tmp/root.pem
/ # COSIGN_EXPERIMENTAL=1 cosign verify registry.ops.gitlab.net/balasankarc/test-fulcio@sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33 --certificate-identity="https://ops.gitlab.net/balasankarc/test-fulcio//.gitlab-ci.yml@refs/he
ads/main" --certificate-oidc-issuer "https://ops.gitlab.net" --insecure-ignore-sct

Verification for registry.ops.gitlab.net/balasankarc/test-fulcio@sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates

<snipped JSON output>

I believe this PR is good to go. I've rebased against latest main to resolve conflicts.

[balasankarc]

@haydentherapper Thanks for the merge. Just to confirm - did these changes already get deployed to the staging/public instances (and thus can be used already) or is that to be done manually?

[haydentherapper]

They’ll be deployed within the week, I’ll ping you here once it’s rolled out.

[balasankarc]

They’ll be deployed within the week, I’ll ping you here once it’s rolled out.

Thank you. Much appreciated.

[haydentherapper]

@balasankarc Changes are deployed to the prod and staging environments, let me know if you have any issues.