Using Github With AMO.md

Using Github With AMO

The steps below are required in order to deploy a model using the AutoMLOps package. The process below assumes that the AutoMLOps package is already installed in the environment where the AutoMLOps execution cells are being run.

Create a Repository

Go to Github, and create a new repository leaving fields as default. Let’s call it automlops-test.

[Optional] Create repository via Terraform

resource "github_repository" "my_repo" {
  name        = "my_cool_repo"
  description = "My awesome codebase"
  visibility  = "private"
}

Set up SSH connection

Follow this guide on how to add your SSH keys to the environment from where you are running the AutoMLOps commands.

Pre-requisites

Ensure that you have the following IAM Roles to provision an Identity Pool:

• roles/iam.workloadIdentityPoolAdmin

Set up Workload Identity Federation (WIF) for GitHub Actions

1. Create an Identity Pool in GCP

- Enter a pool name and click ‘Continue’. Example details: - Name: automlops-1-github-pool - Pool ID: automlops-1-github-pool - \

2. Create an Identity Provider and add it to the pool:

• Fill in the attributes and conditions for the provider with the following details:

  • Provider type: OpenID Connect (OIDC)

  • Provider Name: automlops-github-provider

  • Provider ID: automlops-github-provider

  • Issuer (URL): https://token.actions.githubusercontent.com/

  • JWK file (JSON): <leave empty>

  • Audiences: <select Default audience>

  • Attribute Mappings:

    • Google 1: google.subject
    • OIDC 1: assertion.sub
    • Google 2: attribute.actor
    • OIDC 2: assertion.actor
    • Google 3: attribute.repository
    • OIDC 3: assertion.repository

  • Attribute Conditions: google.subject.contains("<my_repo_name>")

    • i.e. google.subject.contains("Akhan221/automlops-test")

3. Create a service account
   Go to the service accounts tab and click on new service account in the top right. Provide a unique name for the service account (SA).

[Optional] Create the Service Account using Terraform

resource "google_service_account" "service_account" {
  account_id   = "service-account-id"
  display_name = "Service Account"
}

4. Grant necessary permission to the Service Account
   1. roles/artifactregistry.writer
   2. roles/pubsub.publisher
   3. roles/iam.workloadIdentityUser
   4. roles/storage.admin

5. Grant identity pool access to the SA, following the instructions at this link.
   1. Option 1
      1. Select “+ Grant Access” at the top of the Identity Pool view
      2. Check “Grant access using Service Account impersonation”
      3. Select the service account you just created
      4. For the attribute name, select “repository”
      5. For the attribute value, write full name of your repo e.g. “Akhan221/automlops-test”
      6. Click Save
   2. Option 2
      1. Navigate to IAM -> Service Accounts, select the desired service account from the list
      2. Select “Permissions” from the top menu
      3. Select “+ Grant Access”
      4. Add the principal that aligns to the Identity Pool
         1. principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.repository/FULL_REPO_STRING
   3. Option 3 (grant full access to the pool)

      1. Run the following command:

         gcloud iam service-accounts add-iam-policy-binding "FULL_SERVICE_ACCOUNT_STRING" \

         --project="PROJECT_ID" \

         --role="roles/iam.workloadIdentityUser" \

         --member=principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/*

Run the Generate Step with the WIF Details

When going through the AutoMLOps example notebook, on the generate step make sure to include these fields along with the other required parameters:

AutoMLOps.generate(
   ...
   use_ci=True,
   deployment_framework='github-actions',
   project_number='<project_number>', # ie. 99800000432
   source_repo_type='github',
   source_repo_name='source/repo/string', # ie. Akhan221/automlops-test            
   workload_identity_pool='identity_pool_string',           # ID string
   workload_identity_provider='identity_provider_string',   # ID string            
   workload_identity_service_account='workload_identity_sa' # full SA string
)

For the source_repo_name, note that this is referring to the full name of the repo including the owner prefix. For example, Akhan221/my-test-repo.

Run the provision and deploy steps

Finish running the remainder of the commands and verify that the pipeline actually gets created in Vertex AI.