Vault Patches

Project which takes HashiCorp Vault sources, and add Gitlab interconnexion on it.

Warning : Before v1.15.6, policies had different name format.
Before migration to v1.15.6, don't forget to duplicate all your policies with the new formatted names.
After migration, you'll be able to remove old policies.

Please note that the GitLab connector is only available with a full-Vault build starting 1.18.0.

Building the image

How it works

• The only thing you have to do is to run the patch.sh script provided, with the specified version you want ( eg: patch.sh v1.15.1)
  • The script starts by cloning Vault sources from the specified version to the vault/ directory
  • Then, the git diff in patches/ folder matching the specified version is applied
• Patched sources now resides in vault/ folder. You can now edit or build the Vault project with our modifications!
• For convenience, you will also find a ready-to-go docker file in docker/ folder. You can build the project using docker build -t customized_vault:latest -f docker/Dockerfile .

Run it

This command is only designed to test your Vault build. DO NOT use this configuration on production!

docker run --rm -p 8200:8200 -e VAULT_API_ADDR="http://localhost" --cap-add=IPC_LOCK customized_vault:latest

How to make a new patch ?

While updating Vault to a new version, we strongly suggest you to start by copying the previous version folder patches and use it as a base.

Vault APIs are quite stable, so you (theoretically) will not spend a lot of times on migration.

To patch and ignore errors, you can run git apply --reject --whitespace=fix ../patches/<DESIRED_VERSION>.patch. Then fix all files that were rejected. Run a git add . && git commit -m "new version" in vault directory, then a git diff HEAD~1 > ../patches/v<DESIRED_VERSION>.patch to save the full diff.

Starting v1.18.0, you can use the patch.sh command by adding a --dev argument at the end of the command. For example: ./patch.sh v1.18.0 --dev.

---

Using the connector (as an administrator)

For the upcoming chapter, we consider that you have a customized Vault running. For a configuration breakdown & best practices, please refer to the official HashiCorp Vault documentation.

GitLab configuration

• First of all, you will need to create a GitLab token that can impersonate users. We need it in order to transform our CI_JOB_TOKEN into user rights. As CI_JOB_TOKEN could not access directly to the GitLab APIs we are interested in, we have to create an impersonation token, that is immediately removed when authentication succeed.
  • Log-in to GitLab with a user that is able to impersonate (we suggest to use root or another "service account", as this account is not a personal account and thus will never be disabled) ;
  • Go to https://<your_gitlab_instance>/-/profile/personal_access_tokens, and generate a Personal Access Token with ( at least) the following rights:
    • api
    • read_api
    • read_user
    • sudo
  • Caution: Starting Gitlab 16.0, Personal Access Token must have an expiration date. Please, be aware of the expiration date of the token, and do the appropriate action to generate a new one before it expires.
  • Keep the generated PAT in a safe place, we'll use it later in Vault configuration
• Then, we have to create an OAuth2 application that will enable the possibility for users to log-in seamlessly to Vault using GitLab callbacks.
  • Go to https://<your_gitlab_instance>/admin/applications, and create a new one
    • Name: what you want
      • eg: Vault
    • Redirect URI: The URI of your (yet to configure) Vault Server, with /v1/auth/gitlab/oauth as final path
      • eg: https://my_vault_server.local/v1/auth/gitlab/oauth
    • You can define "Trusted" parameter as you want. If trusted is set to true, people will not have a prompt from GitLab to confirm that user wants to connect to GitLab.
    • Set the "confidential" parameter to "true"
    • Scopes: you must, at least, give the following scopes to the application:
      • read_api
      • read_user
      • openid
      • profile
  • Keep the created Application ID and Secret in a safe place.

Congratulations, GitLab is now fully configured! Let's move on to the Vault side.

Vault

• Log-in with the Vault root Token.
• Go to Access -> Authentication Method, and add a new method. Select the "GitLab" method.
  • Leave all variables by default, and click on "Enable method".
• Then, go to the freshly created authentication method, click on "Configure", and then on "Gitlab Options". You have a few fields to fill:
  • Base URL : The address of your GitLab instance.
    • eg: https://my_gitlab_server.local.
  • Minimal Access Level: This is the minimum access that will be used to match user rights.
    • For example, if this value is set to "maintainer", only "maintainer" and "owner" roles will be parsed.
    • This variable could help when you have to deal with users who have a lot of projects to speed up their connection flow.
    • By default, you can safely keep the option to guest, or reporter.
  • OAuth Application ID & OAuth Application Secret: put in the previously generated values from GitLab.
  • OAuth callback URL: It should be the root address of your Vault instance.
    • eg: https://my_vault_server.local.
  • CI token: put in the previously generated PAT from GitLab.
  • (Starting v1.18.1) (Optional) Vault Service Token: an optional Vault service token that will be used to read loaded policies in your Vault . See VST Configuration for more details.

Vault is now configured, you should be able to log in to Vault using your GitLab credentials now!

Setting-up policies

When logging-in a user from GitLab, Vault will retrieve the list of granted gitlab projects and groups for the user. This enables us to use these information as an automated policy loader.

Basically, Vault will replace all non-alphanumeric characters from the group/project path to underscores (_), and concatenate the user role after it. A few examples to fully understand it:

• A user have access to group/project1 with role maintainer
  • The matching policy will be named group:project1:maintainer.
• A user have access to group-one/ with role owner, and group-two/very/long/pa-th/project with role reporter
  • The matching policies will be named group_one:owner and group_two:very:long:pa_th:project:reporter

Every matching Vault policy will be loaded to the user token.

Vault Service Token Configuration

Added in v1.18.1 and onwards, the Vault Service Token allows you to filter the generated policies based on your GitLab access, by removing any policy that does not really exist in Vault. This feature drastically reduce token size, and also your audit logs if you enabled them.

You must provide a token that never expires OR a token that can be renewed indefinitely. The backend will refresh every two minutes the list of ACLs registered in Vault, and will also renew the token accordingly. We highly recommend to create a dedicated token for this usage with a very specific policy, in order to limit any risk if this token got stolen.

Create an ACL policy named token-viewer, with the following content:

# Allow people to list all ACLs
path "sys/policies/acl" {
  capabilities = ["list"]
}

# Allow tokens to look up their own properties
path "auth/token/lookup-self" {
  capabilities = ["read"]
}

# Allow tokens to renew themselves
path "auth/token/renew-self" {
  capabilities = ["update"]
}

# Allow tokens to revoke themselves
path "auth/token/revoke-self" {
  capabilities = ["update"]
}

This policy is very restrictive, and only allows token to list ACL on Vault (but not see them), and perform basic operations on its own token (lookup/renew/revoke).

Then, using a root account, create a new token with this policy only : vault token create -policy="token-viewer" -period=24h -no-default-policy

This token will have no privileges but the ones you defined in your token-viewer policy, ensuring a very limited risk for your passwords.

If no Vault Service Token is provided, the program will load all the policies based on your rights.

If you provide a token, please note that users must have to log-out and login again when privileges changes.

---

Using the connector (as a user)

Connecting to Vault UI

• Go to your Vault instance, and select GitLab authentication method. Multiple options are available:
  • The most simple way: log-in using OAuth2: it will redirect you to your GitLab instance to prove your identity, and log you back into Vault.
  • You can also provide a Gitlab Personal Access Token, or a Gitlab username & password.

Connecting to Vault using APIs

To log in to Vault using your Gitlab profile, you can use :

• A GitLab Personal Access Token
• Or a temporary and automatically generated CI_JOB_TOKEN (see GitLab documentation about it)

In both case, you will have to do a request to your Vault server to generate a Vault token from your Gitlab token. Then, you will be able to use your Vault token for all subsequent calls to Vault.

GitLab PAT (Personal Access Token)

Using a GitLab PAT, you must send a POST request to https://<your_vault_instance>/v1/auth/gitlab/login, with the following payload:

{
  "token": "YOUR_GITLAB_PAT_HERE"
}

If authentication succeed, you will receive a 201 CREATED response from Vault server, and you will be able to find your token in the returned JSON payload, in auth/client_token.

GitLab CI_JOB_TOKEN

Using a GitLab PAT, you must send a POST request to https://<your_vault_instance>/v1/auth/gitlab/ci, with the following payload:

{
  "token": "YOUR_GITLAB_CI_JOB_TOKEN_HERE"
}

If authentication succeed, you will receive a 201 CREATED response from Vault server, and you will be able to find your token in the returned JSON payload, in auth/client_token.

Examples

You will be able to find some examples of scripts using these connection methods in examples folder.

---

Contributing & licence

All contributions are welcome, as you agree with the licence defined by this project. Feel free to open a PR, we will have a close look on it (and we thank you in advance for your participation)!

This project is licenced under MIT. Please remember that HashiCorp Vault is under BSL, therefore our licence only applies to patch files. You still have to reach HashiCorp team if you want to sell this software as a service.