The purpose of this step is to bootstrap a GCP organization, creating all the required resources & permissions to start using the Cloud Foundation Toolkit (CFT). This step also guides you on how to configure a CI/CD project to host a Jenkins Agent, which connects to your existing Jenkins Controller infrastructure & your own Git repos (which might live on-prem). The Jenkins Agent will run CI/CD Pipelines for foundations code in subsequent stages.
Another CI/CD option is to use Cloud Build & Cloud Source Repos. If you don't have a Jenkins implementation and don't want one, then we recommend you to use the Cloud Build module instead.
Disclaimer: Jenkins support will be deprecated in a future release. Consider using the Cloud Build module instead.
The objective of the instructions below is to configure the infrastructure that allows you to run CI/CD deployments for the next stages (1-org, 2-environments, 3-networks, 4-projects
) using Jenkins. The infrastructure consists in two Google Cloud Platform projects (prj-b-seed
and prj-b-cicd
) and VPN configuration to connect to your on-prem environment.
It is a best practice to have two separate projects here (prj-b-seed
and prj-b-cicd
) for separation of concerns. On one hand, prj-b-seed
stores terraform state and has the Service Account able to create / modify infrastructure. On the other hand, the deployment of that infrastructure is coordinated by Jenkins, which is implemented in prj-b-cicd
and connected to your Controller on-prem.
After following the instructions below, you will have:
-
The
prj-b-seed
project, which contains:- Terraform state bucket.
- Custom Service Accounts used by Terraform to create new resources in GCP.
-
The
prj-b-cicd
project, which contains:- GCE Instance for the Jenkins Agent, connected to your current Jenkins Controller using SSH.
- VPC to connect the Jenkins GCE Instance to.
- FW rules to allow communication over port 22.
- VPN connection with on-prem (or where ever your Jenkins Controller is located).
- Custom service account
[email protected]
for the GCE instance.- This service account is granted the access to generate tokens on the Terraform custom service accounts in the
prj-b-seed
project.
- This service account is granted the access to generate tokens on the Terraform custom service accounts in the
-
Note: these instructions do not indicate how to create a Jenkins Controller. To deploy a Jenkins Controller, you should follow Jenkins Architecture recommendations.
If you don't have a Jenkins implementation and don't want one, then we recommend you to use the Cloud Build module instead.
Please see the requirements of Software, Infrastructure and Permissions before following the instructions below.
Note: If you are using MacOS, replace cp -RT
with cp -R
in the relevant
commands. The -T
flag is needed for Linux, but causes problems for MacOS.
You arrived to these instructions because you are using the jenkins_bootstrap
to run the 0-bootstrap step instead of cloudbuild_bootstrap
. Please follow the indications below:
- Make sure you cover all the requirements of Software, Infrastructure and Permissions before following the instructions below.
- Required information:
- Access to the Jenkins Controller host to run
ssh-keygen
command - Access to the Jenkins Controller Web UI
- SSH Agent Jenkins plugin installed in your Jenkins Controller
- Private IP address for the Jenkins Agent: usually assigned by your network administrator. You will use this IP for the GCE instance that will be created in the
prj-b-cicd
GCP Project in step II. Create the SEED and CI/CD projects using Terraform. - Access to create five Git repositories, one for each directory in this monorepo (
gcp-bootstrap, gcp-org, gcp-environments, gcp-networks, gcp-projects
). These are usually private repositories that might be on-prem.
- Access to the Jenkins Controller host to run
-
Generate a SSH key pair. In the Jenkins Controller host, use the
ssh-keygen
command to generate a SSH key pair.- You will need this key pair to enable authentication between the Controller and Agent. Although the key pair can be generated in any linux machine, it is recommended not to copy the secret private key from one host to another, so you probably want to do this in the Jenkins Controller host command line.
- Note the
ssh-keygen
command uses the-N
option to protect the private key with a password. In this example, we are using-N "my-password"
. This is important because you will need both, the private key and the password when configuring the SSH Agent in you Jenkins Controller Web UI.
SSH_LOCAL_CONFIG_DIR="$HOME/.ssh" JENKINS_USER="jenkins" JENKINS_AGENT_NAME="AgentGCE1" SSH_KEY_FILE_PATH="$SSH_LOCAL_CONFIG_DIR/$JENKINS_USER-${JENKINS_AGENT_NAME}_rsa" mkdir -p "$SSH_LOCAL_CONFIG_DIR" ssh-keygen -t rsa -m PEM -N "my-password" -C $JENKINS_USER -f $SSH_KEY_FILE_PATH cat $SSH_KEY_FILE_PATH
-
Configure a new SSH Jenkins Agent in the Jenkins Controller’s Web UI. You need the following information:
- SSH Agent Jenkins plugin installed in your Controller
- SSH private key you just generated in the previous step
- Passphrase that protects the private key (the one you used in the
-N
option) - Jenkins Agent’s private IP address (usually assigned by your Network Administrator. In the provided examples this IP is "172.16.1.6"). This private IP will be reachable through the VPN connection that you will create later.
-
Create five individual Git repositories in your Git server (This might be a task delegated to your infrastructure team)
- Note that although this infrastructure code is distributed to you as a monorepo, you will store the code in five different repositories, one for each directory:
./gcp-bootstrap ./gcp-org ./gcp-environments ./gcp-networks ./gcp-projects
- For simplicity, let's name your five repositories as follows:
YOUR_NEW_REPO-gcp-bootstrap YOUR_NEW_REPO-gcp-org YOUR_NEW_REPO-gcp-environments YOUR_NEW_REPO-gcp-networks YOUR_NEW_REPO-gcp-projects
- Note: Towards the end of these instructions, you will configure your Jenkins Controller with new automatic pipelines only for the following repositories:
YOUR_NEW_REPO-gcp-org YOUR_NEW_REPO-gcp-environments YOUR_NEW_REPO-gcp-networks YOUR_NEW_REPO-gcp-projects
- Note: there is no automatic pipeline needed for
YOUR_NEW_REPO-gcp-bootstrap
- In this 0-bootstrap section we only work with your new repository that is a copy of the directory
./0-bootstrap
(YOUR_NEW_REPO-gcp-bootstrap
)
-
Clone this mono-repository with:
git clone https://github.com/terraform-google-modules/terraform-example-foundation
-
Clone the repository you created to host the
0-bootstrap
directory with:git clone <YOUR_NEW_REPO-gcp-bootstrap> gcp-bootstrap
-
Navigate into the freshly cloned repo and change to a new branch
cd gcp-bootstrap git checkout -b plan
-
Copy contents of foundation to new repo (modify accordingly based on your current directory).
mkdir -p envs/shared cp -RT ../terraform-example-foundation/0-bootstrap/ ./envs/shared cd ./envs/shared
-
Activate the Jenkins module and disable the Cloud Build module. This implies manually editing the following files:
- Rename file
./cb.tf
to./cb.tf.example
mv ./cb.tf ./cb.tf.example
- Rename file
./jenkins.tf.example
to./jenkins.tf
mv ./jenkins.tf.example ./jenkins.tf
- Un-comment the
jenkins_bootstrap
variables in./variables.tf
- Un-comment the
jenkins_bootstrap
outputs in./outputs.tf
- Comment-out the
cloudbuild_bootstrap
outputs in./outputs.tf
- Rename file
-
Rename
terraform.example.tfvars
toterraform.tfvars
and update the file with values from your environment.mv ./terraform.example.tfvars ./terraform.tfvars
-
One of the value to supply (variable
jenkins_agent_gce_ssh_pub_key
) is the public SSH key you generated in the first step.- Note: this is not the secret private key. The public SSH key can be in your repository code.
-
Show the public key using
cat "${SSH_KEY_FILE_PATH}.pub"
-
You will copy / paste it in the
terraform.tfvars
file (variablejenkins_agent_gce_ssh_pub_key
). -
Provide the rest of the values needed in
terraform.tfvars
-
Use the helper script validate-requirements.sh to validate your environment:
../../../terraform-example-foundation/scripts/validate-requirements.sh -o <ORGANIZATION_ID> -b <BILLING_ACCOUNT_ID> -u <END_USER_EMAIL> -e
Note: The script is not able to validate if the user is in a Cloud Identity or Google Workspace group with the required roles.
-
Commit changes:
cd ../.. git add . git commit -m 'Bootstrap configuration using jenkins_module'
-
Push
plan
branch to your repository YOUR_NEW_REPO-gcp-bootstrap withgit push --set-upstream origin plan cd ./envs/shared
- Required information:
- Terraform version 1.3.0 - See Requirements section for more details.
- The
terraform.tfvars
file with all the necessary values.
-
Get the appropriate credentials: run the following command with an account that has the necessary permissions.
gcloud auth application-default login
-
Open the link in your browser and accept.
-
Run terraform commands.
- After the credentials are configured, we will create the
prj-b-seed
project (which contains the GCS state bucket and Terraform custom service account) and theprj-b-cicd
project (which contains the Jenkins Agent, its custom service account and where we will add VPN configuration) - Use Terraform 1.3.0 to run the terraform script with the commands below
terraform init terraform plan terraform apply
- The Terraform script will take about 10 to 15 minutes. Once it finishes, note that communication between on-prem and the
prj-b-cicd
project won’t happen yet - you will configure the VPN network connectivity in step III. Create VPN connection.
- After the credentials are configured, we will create the
-
Move Terraform State to the GCS bucket created in the Seed Project
- Get tfstate bucket name
export backend_bucket=$(terraform output -raw gcs_bucket_tfstate) echo "backend_bucket = ${backend_bucket}"
- Rename
backend.tf.example
tobackend.tf
and update with your tfstate bucket name
mv backend.tf.example backend.tf for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_ME/${backend_bucket}/" $i; done
-
Re-run
terraform init
and agree to copy state to gcs when promptedterraform init
- (Optional) Run
terraform apply
to verify state is configured correctly. You can confirm the terraform state is now in that bucket by visiting the bucket url in your Seed Project.
- (Optional) Run
-
Commit changes
git add backend.tf git commit -m 'Terraform Backend configuration using GCS'
-
Push
plan
branch to your repository YOUR_NEW_REPO-gcp-bootstrap withgit push
Here you will configure a VPN Network tunnel to enable connectivity between the prj-b-cicd
project and your on-prem environment. Learn more about a VPN tunnel in GCP.
- Required information:
- On-prem VPN public IP Address
- Jenkins Controller’s network CIDR (the example code uses "10.1.0.0/24")
- Jenkins Agent network CIDR (the example code uses "172.16.1.0/24")
- VPN PSK (pre-shared secret key)
-
Check in the
prj-b-cicd
project for the VPN gateway static IP addresses which have been reserved. These addresses are required by the Network Administrator for the configuration of the on-prem side of the VPN tunnels to GCP.- Assuming your network administrator already configured the on-prem end of the VPN, the CI/CD end of the VPN might show the message
First Handshake
for around 5 minutes. - When the VPN is ready, the status will show
Tunnel is up and running
. At this point, your Jenkins Controller (on-prem) and Jenkins Agent (inprj-b-cicd
project) must have network connectivity through the VPN.
- Assuming your network administrator already configured the on-prem end of the VPN, the CI/CD end of the VPN might show the message
-
Test a pipeline using the Jenkins Controller Web UI:
- Note: this section is considered out of the scope of this document. Since there are multiple options on how to configure the Git repositories and Multibranch Pipeline in your Jenkins Controller, here we can only provide some guidance that you should keep in mind while completing this step. Visit the Jenkins website for more information, there are plenty of Jenkins Plugins that could help with the task.
- You need to configure a "Multibranch Pipeline". Note that the
Jenkinsfile
andtf-wrapper.sh
files use the$BRANCH_NAME
environment variable. the$BRANCH_NAME
variable is only available in Jenkins' Multibranch Pipelines.
- You need to configure a "Multibranch Pipeline". Note that the
- Jenkinsfile: A Jenkinsfile has been included which closely aligns with the Cloud Build pipeline. Additionally, the stage
TF wait for approval
which lets you confirm via Jenkins UI before proceeding withterraform apply
has been disabled by default. It can be enabled by un-commenting that stage in the file.
-
Create Multibranch pipelines for your new repos (
YOUR_NEW_REPO-gcp-org, YOUR_NEW_REPO-gcp-environments, YOUR_NEW_REPO-gcp-networks, YOUR_NEW_REPO-gcp-projects
).- DO NOT configure an automatic pipeline for your
YOUR_NEW_REPO-gcp-bootstrap
repository
- DO NOT configure an automatic pipeline for your
-
In your Jenkins Controller Web UI, create Multibranch Pipelines only for the following repositories:
YOUR_NEW_REPO-gcp-org YOUR_NEW_REPO-gcp-environments YOUR_NEW_REPO-gcp-networks YOUR_NEW_REPO-gcp-projects
-
Assuming your new Git repositories are private, you may need to configure new credentials In your Jenkins Controller web UI, so it can connect to the repositories.
-
You will also want to configure automatic triggers in each one of the Jenkins Multibranch Pipelines, unless you want to run the pipelines manually from the Jenkins Web UI after each commit to your repositories.
-
You can now move to the instructions for step 1-org.
-
Clone the repo you created manually in 0-bootstrap instructions.
git clone <YOUR_NEW_REPO-gcp-org> gcp-org
-
Navigate into the repo and change to a nonproduction branch. All subsequent steps assume you are running them from the
gcp-org
directory. If you run them from another directory, adjust your copy paths accordingly.cd gcp-org git checkout -b plan
-
Copy contents of foundation to new repo.
cp -RT ../terraform-example-foundation/1-org/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/Jenkinsfile . cp ../terraform-example-foundation/build/tf-wrapper.sh . chmod 755 ./tf-wrapper.sh
-
Update the variables located in the
environment {}
section of theJenkinsfile
with values from gcp-bootstrap:_TF_SA_EMAIL _STATE_BUCKET_NAME _PROJECT_ID (the CI/CD project ID)
-
You can re-run
terraform output
in the gcp-bootstrap directory to find these values.BACKEND_STATE_BUCKET_NAME=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "_STATE_BUCKET_NAME = ${BACKEND_STATE_BUCKET_NAME}" sed -i'' -e "s/BACKEND_STATE_BUCKET_NAME/${BACKEND_STATE_BUCKET_NAME}/" ./Jenkinsfile TERRAFORM_SA_EMAIL=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw organization_step_terraform_service_account_email) echo "_TF_SA_EMAIL = ${TERRAFORM_SA_EMAIL}" sed -i'' -e "s/TERRAFORM_SA_EMAIL/${TERRAFORM_SA_EMAIL}/" ./Jenkinsfile CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo "_PROJECT_ID = ${CICD_PROJECT_ID}" sed -i'' -e "s/CICD_PROJECT_ID/${CICD_PROJECT_ID}/" ./Jenkinsfile
-
Rename
./envs/shared/terraform.example.tfvars
to./envs/shared/terraform.tfvars
mv ./envs/shared/terraform.example.tfvars ./envs/shared/terraform.tfvars
-
Check if a Security Command Center Notification with the default name, scc-notify, already exists. If it exists, choose a different value for the
scc_notification_name
variable in the./envs/shared/terraform.tfvars
file.export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -json common_config | jq '.org_id' --raw-output) gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}
-
Check if your organization already has an Access Context Manager Policy.
export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)") echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}"
-
Update the
envs/shared/terraform.tfvars
file with values from your environment and 0-bootstrap step. If the previous step showed a numeric value, make sure to un-comment the variablecreate_access_context_manager_access_policy = false
. See the shared folder README.md for additional information on the values in theterraform.tfvars
file.export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "remote_state_bucket = ${backend_bucket}" sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./envs/shared/terraform.tfvars if [ ! -z "${ACCESS_CONTEXT_MANAGER_ID}" ]; then sed -i'' -e "s=//create_access_context_manager_access_policy=create_access_context_manager_access_policy=" ./envs/shared/terraform.tfvars; fi
-
Commit changes.
git add . git commit -m 'Initialize org repo'
-
Push your plan branch.
- Assuming you configured an automatic trigger in your Jenkins Controller (see Jenkins sub-module README), this will trigger a plan. You can also trigger a Jenkins job manually. Given the many options to do this in Jenkins, it is out of the scope of this document see Jenkins website for more details.
git push --set-upstream origin plan
-
Review the plan output in your Controller's web UI.
-
Merge changes to production branch.
git checkout -b production git push --set-upstream origin production
-
Review the apply output in your Controller's web UI. (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Clone the repo you created manually in 0-bootstrap.
git clone <YOUR_NEW_REPO-gcp-environments> gcp-environments
-
Navigate into the repo and change to a nonproduction branch. All subsequent steps assume you are running them from the
gcp-environments
directory. If you run them from another directory, adjust your copy paths accordingly.cd gcp-environments git checkout -b plan
-
Copy contents of foundation to new repo.
cp -RT ../terraform-example-foundation/2-environments/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/Jenkinsfile . cp ../terraform-example-foundation/build/tf-wrapper.sh . chmod 755 ./tf-wrapper.sh
-
Update the variables located in the
environment {}
section of theJenkinsfile
with values from your environment:_TF_SA_EMAIL _STATE_BUCKET_NAME _PROJECT_ID (the CI/CD project ID)
-
You can re-run
terraform output
in the gcp-bootstrap directory to find these values.BACKEND_STATE_BUCKET_NAME=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "_STATE_BUCKET_NAME = ${BACKEND_STATE_BUCKET_NAME}" sed -i'' -e "s/BACKEND_STATE_BUCKET_NAME/${BACKEND_STATE_BUCKET_NAME}/" ./Jenkinsfile TERRAFORM_SA_EMAIL=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw environment_step_terraform_service_account_email) echo "_TF_SA_EMAIL = ${TERRAFORM_SA_EMAIL}" sed -i'' -e "s/TERRAFORM_SA_EMAIL/${TERRAFORM_SA_EMAIL}/" ./Jenkinsfile CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo "_PROJECT_ID = ${CICD_PROJECT_ID}" sed -i'' -e "s/CICD_PROJECT_ID/${CICD_PROJECT_ID}/" ./Jenkinsfile
-
Rename
terraform.example.tfvars
toterraform.tfvars
and update the file with values from your environment and 0-bootstrap.mv terraform.example.tfvars terraform.tfvars
-
You can re-run
terraform output
in the gcp-bootstrap directory to find these values. See any of the envs folder README.md files for additional information on the values in theterraform.tfvars
file.export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "remote_state_bucket = ${backend_bucket}" sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./terraform.tfvars
-
Commit changes.
git add . git commit -m 'Initialize environments repo'
-
Push your plan branch.
git push --set-upstream origin plan
- Assuming you configured an automatic trigger in your Jenkins Controller (see Jenkins sub-module README), this will trigger a plan. You can also trigger a Jenkins job manually. Given the many options to do this in Jenkins, it is out of the scope of this document see Jenkins website for more details.
-
Review the plan output in your Controller's web UI.
-
Merge changes to development.
git checkout -b development git push --set-upstream origin development
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Merge changes to nonproduction with.
git checkout -b nonproduction git push --set-upstream origin nonproduction
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Merge changes to production branch.
git checkout -b production git push --set-upstream origin production
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
You can now move to the instructions in the next step, go to Deploying step 3-networks-dual-svpc to use the Dual Shared VPC mode, or go to Deploying step 3-networks-hub-and-spoke to use the Hub and Spoke network mode.
-
Clone the repo you created manually in 0-bootstrap.
git clone <YOUR_NEW_REPO-gcp-networks> gcp-networks
-
Navigate into the repo and change to a nonproduction branch. All subsequent steps assume you are running them from the
gcp-networks
directory. If you run them from another directory, adjust your copy paths accordingly.cd gcp-networks git checkout -b plan
-
Copy contents of foundation to new repo.
cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/Jenkinsfile . cp ../terraform-example-foundation/build/tf-wrapper.sh . chmod 755 ./tf-wrapper.sh
-
Update the variables located in the
environment {}
section of theJenkinsfile
with values from your environment:_TF_SA_EMAIL _STATE_BUCKET_NAME _PROJECT_ID (the CI/CD project ID)
-
You can re-run
terraform output
in the gcp-bootstrap directory to find these values.BACKEND_STATE_BUCKET_NAME=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "_STATE_BUCKET_NAME = ${BACKEND_STATE_BUCKET_NAME}" sed -i'' -e "s/BACKEND_STATE_BUCKET_NAME/${BACKEND_STATE_BUCKET_NAME}/" ./Jenkinsfile TERRAFORM_SA_EMAIL=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw networks_step_terraform_service_account_email) echo "_TF_SA_EMAIL = ${TERRAFORM_SA_EMAIL}" sed -i'' -e "s/TERRAFORM_SA_EMAIL/${TERRAFORM_SA_EMAIL}/" ./Jenkinsfile CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo "_PROJECT_ID = ${CICD_PROJECT_ID}" sed -i'' -e "s/CICD_PROJECT_ID/${CICD_PROJECT_ID}/" ./Jenkinsfile
-
Rename
common.auto.example.tfvars
tocommon.auto.tfvars
, renameshared.auto.example.tfvars
toshared.auto.tfvars
and renameaccess_context.auto.example.tfvars
toaccess_context.auto.tfvars
.mv common.auto.example.tfvars common.auto.tfvars mv shared.auto.example.tfvars shared.auto.tfvars mv access_context.auto.example.tfvars access_context.auto.tfvars
-
Update
common.auto.tfvars
file with values from your environment and bootstrap. See any of the envs folder README.md files for additional information on the values in thecommon.auto.tfvars
file. -
Update
shared.auto.tfvars
file with thetarget_name_server_addresses
. -
Update
access_context.auto.tfvars
file with theaccess_context_manager_policy_id
. -
Use
terraform output
to get the backend bucket and networks step Terraform Service Account values from gcp-bootstrap output.export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -json common_config | jq '.org_id' --raw-output) export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)") echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}" sed -i'' -e "s/ACCESS_CONTEXT_MANAGER_ID/${ACCESS_CONTEXT_MANAGER_ID}/" ./access_context.auto.tfvars export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "remote_state_bucket = ${backend_bucket}" sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars
-
Commit changes.
git add . git commit -m 'Initialize networks repo'
-
You must manually plan and apply the
shared
environment (only once) since thedevelopment
,nonproduction
andproduction
environments depend on it. -
To use the
validate
option of thetf-wrapper.sh
script, please follow the instructions to install the terraform-tools component. -
Also update
backend.tf
with your backend bucket from gcp-bootstrap output.for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_ME/${backend_bucket}/" $i; done
-
Use
terraform output
to get the Cloud Build project ID and the networks step Terraform Service Account from gcp-bootstrap output. An environment variableGOOGLE_IMPERSONATE_SERVICE_ACCOUNT
will be set using the Terraform Service Account to enable impersonation.export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo ${CICD_PROJECT_ID} export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw networks_step_terraform_service_account_email) echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}
-
Run
init
andplan
and review output for environment shared../tf-wrapper.sh init shared ./tf-wrapper.sh plan shared
-
Run
validate
and check for violations../tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID}
-
Run
apply
shared../tf-wrapper.sh apply shared
-
Push your plan branch.
git push --set-upstream origin plan
- Assuming you configured an automatic trigger in your Jenkins Controller (see Jenkins sub-module README), this will trigger a plan. You can also trigger a Jenkins job manually. Given the many options to do this in Jenkins, it is out of the scope of this document see Jenkins website for more details.
-
Review the plan output in your Controller's web UI.
-
Merge changes to production branch.
git checkout -b production git push --set-upstream origin production
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
After production has been applied, apply development and nonproduction.
-
Merge changes to development
git checkout -b development git push --set-upstream origin development
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Merge changes to nonproduction.
git checkout -b nonproduction git push --set-upstream origin nonproduction
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Clone the repo you created manually in 0-bootstrap.
git clone <YOUR_NEW_REPO-gcp-networks> gcp-networks
-
Navigate into the repo and change to a nonproduction branch. All subsequent steps assume you are running them from the
gcp-networks
directory. If you run them from another directory, adjust your copy paths accordingly.cd gcp-networks git checkout -b plan
-
Copy contents of foundation to new repo.
cp -RT ../terraform-example-foundation/3-networks-hub-and-spoke/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/Jenkinsfile . cp ../terraform-example-foundation/build/tf-wrapper.sh . chmod 755 ./tf-wrapper.sh
-
Update the variables located in the
environment {}
section of theJenkinsfile
with values from your environment:_TF_SA_EMAIL _STATE_BUCKET_NAME _PROJECT_ID (the CI/CD project ID)
-
You can re-run
terraform output
in the gcp-bootstrap directory to find these values.BACKEND_STATE_BUCKET_NAME=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "_STATE_BUCKET_NAME = ${BACKEND_STATE_BUCKET_NAME}" sed -i'' -e "s/BACKEND_STATE_BUCKET_NAME/${BACKEND_STATE_BUCKET_NAME}/" ./Jenkinsfile TERRAFORM_SA_EMAIL=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw networks_step_terraform_service_account_email) echo "_TF_SA_EMAIL = ${TERRAFORM_SA_EMAIL}" sed -i'' -e "s/TERRAFORM_SA_EMAIL/${TERRAFORM_SA_EMAIL}/" ./Jenkinsfile CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo "_PROJECT_ID = ${CICD_PROJECT_ID}" sed -i'' -e "s/CICD_PROJECT_ID/${CICD_PROJECT_ID}/" ./Jenkinsfile
-
Rename
common.auto.example.tfvars
tocommon.auto.tfvars
, renameshared.auto.example.tfvars
toshared.auto.tfvars
and renameaccess_context.auto.example.tfvars
toaccess_context.auto.tfvars
.mv common.auto.example.tfvars common.auto.tfvars mv shared.auto.example.tfvars shared.auto.tfvars mv access_context.auto.example.tfvars access_context.auto.tfvars
-
Update
common.auto.tfvars
file with values from your environment and bootstrap. See any of the envs folder README.md files for additional information on the values in thecommon.auto.tfvars
file. -
Update
shared.auto.tfvars
file with thetarget_name_server_addresses
. -
Update
access_context.auto.tfvars
file with theaccess_context_manager_policy_id
. -
Use
terraform output
to get the backend bucket value from gcp-bootstrap output.export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -json common_config | jq '.org_id' --raw-output) export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)") echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}" sed -i'' -e "s/ACCESS_CONTEXT_MANAGER_ID/${ACCESS_CONTEXT_MANAGER_ID}/" ./access_context.auto.tfvars export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "remote_state_bucket = ${backend_bucket}" sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars
-
Commit changes.
git add . git commit -m 'Initialize networks repo'
-
You must manually plan and apply the
shared
environment (only once) since thedevelopment
,nonproduction
andproduction
environments depend on it. -
To use the
validate
option of thetf-wrapper.sh
script, please follow the instructions to install the terraform-tools component. -
Also update
backend.tf
with your backend bucket from gcp-bootstrap output.for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_ME/${backend_bucket}/" $i; done
-
Use
terraform output
to get the Cloud Build project ID and the networks step Terraform Service Account from gcp-bootstrap output. An environment variableGOOGLE_IMPERSONATE_SERVICE_ACCOUNT
will be set using the Terraform Service Account to enable impersonation.export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo ${CICD_PROJECT_ID} export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw networks_step_terraform_service_account_email) echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}
-
Run
init
andplan
and review output for environment shared../tf-wrapper.sh init shared ./tf-wrapper.sh plan shared
-
Run
validate
and check for violations../tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID}
-
Run
apply
shared../tf-wrapper.sh apply shared
-
Push your plan branch.
git push --set-upstream origin plan
- Assuming you configured an automatic trigger in your Jenkins Controller (see Jenkins sub-module README), this will trigger a plan. You can also trigger a Jenkins job manually. Given the many options to do this in Jenkins, it is out of the scope of this document see Jenkins website for more details.
-
Review the plan output in your Controller's web UI.
-
Merge changes to production branch.
git checkout -b production git push --set-upstream origin production
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
After production has been applied, apply development and nonproduction.
-
Merge changes to development
git checkout -b development git push --set-upstream origin development
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Merge changes to nonproduction.
git checkout -b nonproduction git push --set-upstream origin nonproduction
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
Clone the repo you created manually in 0-bootstrap.
git clone <YOUR_NEW_REPO-gcp-projects> gcp-projects
-
Navigate into the repo and change to a nonproduction branch. All subsequent steps assume you are running them from the
gcp-projects
directory. If you run them from another directory, adjust your copy paths accordingly.cd gcp-projects git checkout -b plan
-
Copy contents of foundation to new repo.
cp -RT ../terraform-example-foundation/4-projects/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/Jenkinsfile . cp ../terraform-example-foundation/build/tf-wrapper.sh . chmod 755 ./tf-wrapper.sh
-
Update the variables located in the
environment {}
section of theJenkinsfile
with values from your environment:_TF_SA_EMAIL _STATE_BUCKET_NAME _PROJECT_ID (the CI/CD project ID)
-
You can re-run
terraform output
in the gcp-bootstrap directory to find these values.BACKEND_STATE_BUCKET_NAME=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "_STATE_BUCKET_NAME = ${BACKEND_STATE_BUCKET_NAME}" sed -i'' -e "s/BACKEND_STATE_BUCKET_NAME/${BACKEND_STATE_BUCKET_NAME}/" ./Jenkinsfile TERRAFORM_SA_EMAIL=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw projects_step_terraform_service_account_email) echo "_TF_SA_EMAIL = ${TERRAFORM_SA_EMAIL}" sed -i'' -e "s/TERRAFORM_SA_EMAIL/${TERRAFORM_SA_EMAIL}/" ./Jenkinsfile CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo "_PROJECT_ID = ${CICD_PROJECT_ID}" sed -i'' -e "s/CICD_PROJECT_ID/${CICD_PROJECT_ID}/" ./Jenkinsfile
-
Rename
auto.example.tfvars
files toauto.tfvars
.mv common.auto.example.tfvars common.auto.tfvars mv shared.auto.example.tfvars shared.auto.tfvars mv development.auto.example.tfvars development.auto.tfvars mv nonproduction.auto.example.tfvars nonproduction.auto.tfvars mv production.auto.example.tfvars production.auto.tfvars
-
See any of the envs folder README.md files for additional information on the values in the
common.auto.tfvars
,development.auto.tfvars
,nonproduction.auto.tfvars
, andproduction.auto.tfvars
files. -
See any of the shared folder README.md files for additional information on the values in the
shared.auto.tfvars
file. -
Use
terraform output
to get the backend bucket value from gcp-bootstrap output.export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) echo "remote_state_bucket = ${backend_bucket}" sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars
-
Commit changes.
git add . git commit -m 'Initialize projects repo'
-
Also update
backend.tf
with your backend bucket from gcp-bootstrap output.for i in `find . -name 'backend.tf'`; do sed -r -i "s/UPDATE_ME|UPDATE_PROJECTS_BACKEND/${backend_bucket}/" $i; done
-
You need to manually plan and apply only once the
shared
environments sincedevelopment
,nonproduction
, andproduction
depend on it. -
Use
terraform output
to get the Cloud Build project ID and the projects step Terraform Service Account from gcp-bootstrap output. An environment variableGOOGLE_IMPERSONATE_SERVICE_ACCOUNT
will be set using the Terraform Service Account to enable impersonation.export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw cicd_project_id) echo ${CICD_PROJECT_ID} export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw projects_step_terraform_service_account_email) echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}
-
Run
init
andplan
and review output for environmentsshared
../tf-wrapper.sh init shared ./tf-wrapper.sh plan shared
-
Run
validate
and check for violations../tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID}
-
Run
apply
shared../tf-wrapper.sh apply shared
-
Push your plan branch.
git push --set-upstream origin plan
- Assuming you configured an automatic trigger in your Jenkins Controller (see Jenkins sub-module README), this will trigger a plan. You can also trigger a Jenkins job manually. Given the many options to do this in Jenkins, it is out of the scope of this document see Jenkins website for more details.
-
Review the plan output in your Controller's web UI.
-
Merge changes to production branch.
git checkout -b production git push --set-upstream origin production
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
After production has been applied, apply development.
-
Merge changes to development branch.
git checkout -b development git push --set-upstream origin development
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
-
After development has been applied, apply nonproduction.
-
Merge changes to nonproduction branch.
git checkout -b nonproduction git push --set-upstream origin nonproduction
-
Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
Refer to the contribution guidelines for information on contributing to this module.