Cloud Foundry Management (cf-mgmt) is automatic way to managing orgs, spaces, users (from ldap groups or internal store) mapping to roles, quotas, application security groups and private-domains that can be driven from concourse pipeline and GIT managed metadata
cf-mgmt tool is composed by 2 CLIs, cf-mgmt and cf-mgmt-config, providing the features to declarativelly manage orgs, spaces, users mapping to roles, quotas, application security groups and private-domains.
cf-mgmt-config CLI is resposible for creating the configuration files that represent the desired state of your foundation and provides the set of commands for you to manage this configuration.
cf-mgmt CLI is resposible to apply the configuration generated by cf-mgmt-config tool to the foundation. It provides commands to apply the config as a whole or only parts of it.
A common use of cf-mgmt is to generate a Concourse pipeline that apply the configuration generated by cf-mgmt-config to a foundation. This is done by using a git repository as a resouce for the configuration and having the pipeline to read from there and apply the changes. cf-mgmt provides a command to generate this pipeline. See more at the Gettting Started section below.
Compiled releases are available on Github.
Download the binary for your platform and place it somewhere on your path.
Don't forget to chmod +x the file on Linux and macOS.
- Centralize Credhub vault services: All concourse pipeline will fetch data from centralize credhub Vault server.
- Credhub path naming convention: /concourse//<PIPELINE_NAME>/<JOB_NAME>/
WIP
- cf-cmgt-config
- cf-mgmt
- fly (binary to connect to concourse web server)
- concourse web server url
- concourse login
- concourse password
- cf (connect to Ops Manager)
- uaac (create client for cf-mgmt to interact with)
- LDAP USER
- LDAP PASSWORD
- jumpbox access: This server must have access to concourse web server and ops manager This server must have all of the requirements binaries
cf-mgmt needs a uaa client to be able to interact with cloud controller and uaa for create, updating, deleting, and listing entities.
To create a non-admin client execute the following command with Cloud Foundry UAA Client. Recent addition of 2 authorities needed to setup shared domains with tcp routing routing.router_groups.read
uaac target uaa.<system-domain>
uaac token client get <adminuserid> -s <admin-client-secret>
uaac client add cf-mgmt \
--name cf-mgmt \
--secret <cf-mgmt-secret> \
--authorized_grant_types client_credentials,refresh_token \
--authorities cloud_controller.admin,scim.read,scim.write,routing.router_groups.read
fly -t optci login -c <CONCOURSE_WEB_URL> -u <CI_LOGIN> -p <CI_PASSWORD>
-- STOP!!!! - CONCOURSE CI/CD pipeline already setup -- Proceed to Operators and Developers section
- Create new foundation in github under "projects"
- Name: "[foundation]-cf-mgmt" a. Example: lab-aws-cf-mgmt
- After repo creation, we need to enable branch protection a. Setting → Branches → Add Rules b. Branch name pattern → "master" -----> Enable → Require pull request reviews before merging
### Create code folder
mkdir code
cd code
## Checkout repo from step 3b
git checkout https://github.com/__project__/[foundation]-cf-mgmt
## Change directory to repo
cd "[foundation]-cf-mgmt"
## Create dev branch
git checkout -b dev
## create gitignore
echo vars.yml >> .gitignore
If there's already a configured foundation that you want to start using cf-mgmt on, cf-mgmt export-config command will export the current foundation configs and generate the files for cf-mgmt usage. See more in the docs.
# Please use configuration from step 1
cf-mgmt-linux export-config \
--system-domain=<FOUNDATION>.anthonydo.com \
--user-id=<user_id> --client-secret=<secret_token>
## Generate concourse pipeline
cf-mgmt-config-linux generate-concourse-pipeline
# your git repo uri
git_repo_uri: "https://<APPLICATION_TOKEN>@github.com/projects/<FOUNDATION>-cf-mgmt.git"
git_repo_branch: master
# your cf system domain
system_domain: "<FOUNDATION>.github.com"
# user account with permission to create orgs/spaces
user_id: "<USER_ID_FROM_STEP1>"
# DEPRECATED - Use client_secret - password of user account with permission to create orgs/spaces
password: ""
# client secret for uaa for user_id
client_secret: "<SECRET_FROM_STEP1>"
# logging level for cf-mgmt commands in the pipeline
log_level: INFO
# time interval to trigger update/delete jobs on
time-trigger: 15m
# configuration directory
config_dir: config
# allow specifying ldap server in pipeline vs in ldap.yml only needed if using LDAP
ldap_server: "ldap://<LDAP_URL>"
# allow specifying ldap bind user in pipeline vs in ldap.yml only needed if using LDAP
ldap_user: '<LDAP_USER>'
# password to bind to ldap - only needed if using LDAP
ldap_password: "<LDAP_PASSWORD>"
## Deploy pipeline to concourse web
fly -t optci set-pipeline \
-p cf-mgmt -c pipeline.yml \
--load-vars-from=config/vars.yml
You should be able to visit http://concourse_web and see pipeline call cf-mgmt
Example:
Open Platform Engineering > Cloud Foundry Management (cf-mgmt) > pipeline.JPG
git add -A .
git commit -m "init rev for <FOUNDATION>"
git push origin dev
- Login to repo https://github.com/projects/-cf-mgmt/pulls a. Click on "New pull request" b. Base: Master Compare: Dev c. Mention/Include: [Applicaton Owner] as approval d. Create pull request e. Send pull request URL to approval
- Howto: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/merging-a-pull-request)
- Login to pull request: a. Review Change b. Approve/Reject as need c. Approve and Merge
Open browser to CONCOURSE_URL:
Example concourse URL: http://[concourse_url]/teams/main/pipelines/cf-mgmt
- cf-cmgt-config binaries - Don NOT forget to chmod +x the file on Linux and macOS.
- Please read the cf-mgmt-config doc and familiarize yourself with the syntax **
git clone --branch dev [email protected]:[account]/[project]/[FOUNDATION]-cf-mgmt.git
## How to add org
cf-mgmt-config-linux org --org=anthonydo-org
2020/01/28 09:24:05 I0128 09:24:05.076215 9687 yaml_config.go:229] Adding org: anthonydo-org The org [anthonydo-org] has been created
cf-mgmt-config-linux space \
--org=anthonydo-org \
--space=anthony-space1
2020/01/28 09:43:08 I0128 09:43:08.27724 1359 yaml_config.go:293] Adding space: anthony-space1 The org/space [anthonydo-org/anthony-space1] has been created
Look at output of git status command and see what files needs to commit and push
git status
# On branch dev # Changed but not updated: # (use "git add ..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # modified: config/orgs.yml # # Untracked files: # (use "git add ..." to include in what will be committed) # # config/anthonydo-org/ no changes added to commit (use "git add" and/or "git commit -a")
Let's add, commit and push
## add orgs
git add config/orgs.yml
## add orgs directory
git add config/anthonydo-org/
## commit
git commit -m "add anthonydo-org and space"
## push to dev
git push origin dev
Login to https://github.com/[project]/[FOUNDATION]-cf-mgmt/pulls
- Click on "New pull request"
- Base: Master <= Compare: Dev
- Mention/Include: Cloud Foundry Management (cf-mgmt) as approval
- Create pull request
- Send pull request URL to approval
Login to https://github.com/[project]/[FOUNDATION]-cf-mgmt/pulls
- Click on "Pull requests"
- Select the pull request for review
- Review Change
- Approve/Reject as need
- Approve and Merge
Check for update orgs/spaces/user