Lab_3_Vertex_AI_Model_Deployment.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39aa0466",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2023 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6693e950-502b-4f4e-aaac-34b44b015bff",
   "metadata": {
    "id": "6693e950-502b-4f4e-aaac-34b44b015bff"
   },
   "source": [
    "# **Lab 3:** Vertex AI Model Deployment\n",
    "This lab deploys our trained BQML model to Vertex AI. We will then submit our inference requests for prediction in real time!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdkKACtoezI2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fdkKACtoezI2",
    "outputId": "75eab7b5-3f9e-49bc-acf3-a7bddda79f3b"
   },
   "outputs": [],
   "source": [
    "! pip install --quiet --upgrade google-cloud-aiplatform "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "TOdUTvelK0gn",
   "metadata": {
    "id": "TOdUTvelK0gn"
   },
   "outputs": [],
   "source": [
    "project_id   = \"\"\n",
    "team_name    = \"\" \n",
    "location     = \"us\" #This is currently necessary\n",
    "region       = \"us-central1\"\n",
    "\n",
    "dataset_name = \"datathon_ds_{}\".format(team_name)\n",
    "bucket_name  = \"gs://{}_{}\".format(project_id,dataset_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "209ec9e2-e26c-4f02-84ec-206bcb1a3fcb",
   "metadata": {
    "id": "209ec9e2-e26c-4f02-84ec-206bcb1a3fcb"
   },
   "outputs": [],
   "source": [
    "from typing import Dict, List, Union\n",
    "from google.cloud import aiplatform\n",
    "from google.protobuf import json_format\n",
    "from google.protobuf.struct_pb2 import Value\n",
    "from google.cloud import bigquery\n",
    "from google.cloud.bigquery import Client, QueryJobConfig\n",
    "import json\n",
    "\n",
    "client = bigquery.Client(project=project_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fY_ubLYAE0GD",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fY_ubLYAE0GD",
    "outputId": "850be5af-8682-4d46-c6fd-4df2e231c989"
   },
   "outputs": [],
   "source": [
    "! gcloud config set project $project_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "rV9V-nXyfMU1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rV9V-nXyfMU1",
    "outputId": "5aa6d93f-e012-4c69-d0af-acbf619e25b8"
   },
   "outputs": [],
   "source": [
    "! gsutil mb -l $region $bucket_name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Uo1fHGRCNbDm",
   "metadata": {
    "id": "Uo1fHGRCNbDm"
   },
   "source": [
    "Enable the **Vertex AI API**: https://console.cloud.google.com/marketplace/product/google/aiplatform.googleapis.com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "g3EpRiiZLtNC",
   "metadata": {
    "id": "g3EpRiiZLtNC"
   },
   "source": [
    "## Deploying your BQML model to Vertex"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lYuUgC-8L5qW",
   "metadata": {
    "id": "lYuUgC-8L5qW"
   },
   "source": [
    "**Step 1**: Export BQML model to GCS bucket\n",
    "\n",
    "To export your trained BQML model to the previously created bucket by follow the steps provided [[here]](https://cloud.google.com/bigquery/docs/exporting-models#export)\n",
    "\n",
    "**Step 2** Import model to vertex model registry\n",
    "\n",
    "We will now import the model to vertex ai model registry by clicking on the ```import``` button and following the steps mentioned [[here]](https://cloud.google.com/vertex-ai/docs/model-registry/import-model#import_a_model_using)\n",
    "\n",
    "Under model settings select:\n",
    "*   Model framework -> tensorflow\n",
    "*   Model framework version -> 1.15\n",
    "*   Accelerator type -> None\n",
    "*   leave all other settings as default\n",
    "\n",
    "**Step 3** Deploy model to vertex endpoint (~15 mins)\n",
    "\n",
    "We will now deploy our model to an endpoint following the steps provided [[here]](https://cloud.google.com/vertex-ai/docs/predictions/get-predictions#deploy_a_model_to_an_endpoint)\n",
    "\n",
    "Under model settings select:\n",
    "*   machine type -> n1-standard-2\n",
    "*   leave all other settings as default\n",
    "*   skip model monitoring section\n",
    "\n",
    "Note that the model deployment will take several minutes (~15 mins)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "IGsM97-6ZU3K",
   "metadata": {
    "id": "IGsM97-6ZU3K"
   },
   "source": [
    "## Run inference on Deployed Model in real time"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "KUSe8g_gR2CT",
   "metadata": {
    "id": "KUSe8g_gR2CT"
   },
   "source": [
    "To run your first real time prediction:\n",
    "*   Select your model endpoint under deployments. \n",
    "*   Select the ```DEPLOY & TEST``` sub-menu.\n",
    "*   Under the ```Test your model``` section paste the payload provided below\n",
    "\n",
    "\n",
    "```\n",
    "{\"instances\": [{\"country\":\"India\",\"operating_system\":\"ANDROID\",\"language\":\"en-us\",\"cnt_user_engagement\": 72,\"cnt_level_start_quickplay\": 0,\"cnt_level_end_quickplay\": 6,\"cnt_level_complete_quickplay\": 3,\"cnt_level_reset_quickplay\": 1,\"cnt_post_score\": 9,\"cnt_spend_virtual_currency\": 0,\"cnt_ad_reward\": 0,\"cnt_challenge_a_friend\": 0,\"cnt_completed_5_levels\": 1,\"cnt_use_extra_steps\": 0,\"user_first_engagement\": 1533434460293005}\n",
    "]}\n",
    "```\n",
    "\n",
    "Congrats!!!! You have successfully recived your first prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "-8f4x8XqTTf9",
   "metadata": {
    "id": "-8f4x8XqTTf9"
   },
   "source": [
    "## Sending larger requests to the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "-k4LG3mG27kf",
   "metadata": {
    "id": "-k4LG3mG27kf"
   },
   "outputs": [],
   "source": [
    "query = f\"\"\"SELECT * FROM `{dataset_name}.cc_eval_dataset`\"\"\"\n",
    "job = client.query(query)\n",
    "df = job.to_dataframe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "GEqzxYQu2eTD",
   "metadata": {
    "id": "GEqzxYQu2eTD"
   },
   "outputs": [],
   "source": [
    "df = df.drop(['user_pseudo_id', 'churned'], axis=1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "y6y284lo4UFJ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 193
    },
    "id": "y6y284lo4UFJ",
    "outputId": "ab89899b-10e5-4f58-e7bf-b2a520de165f"
   },
   "outputs": [],
   "source": [
    "df.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7qp4IhmZ5tEJ",
   "metadata": {
    "id": "7qp4IhmZ5tEJ"
   },
   "outputs": [],
   "source": [
    "df = df.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5_kfUXRZ4hbv",
   "metadata": {
    "id": "5_kfUXRZ4hbv"
   },
   "outputs": [],
   "source": [
    "# format the dataframe for the endpoint\n",
    "payload = json.loads(df.to_json(orient=\"records\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ca043cd-bb79-4cc6-9c71-cc736f63f8db",
   "metadata": {
    "id": "re7okpHfTs5F"
   },
   "outputs": [],
   "source": [
    "def predict_custom_trained_model_sample(\n",
    "    project: str,\n",
    "    endpoint_id: str,\n",
    "    instances: Union[Dict, List[Dict]],\n",
    "    location: str = \"us-central1\",\n",
    "    api_endpoint: str = \"us-central1-aiplatform.googleapis.com\",\n",
    "):\n",
    "    \"\"\"\n",
    "    `instances` can be either single instance of type dict or a list\n",
    "    of instances.\n",
    "    \"\"\"\n",
    "    # The AI Platform services require regional API endpoints.\n",
    "    client_options = {\"api_endpoint\": api_endpoint}\n",
    "    # Initialize client that will be used to create and send requests.\n",
    "    # This client only needs to be created once, and can be reused for multiple requests.\n",
    "    client = aiplatform.gapic.PredictionServiceClient(client_options=client_options)\n",
    "    # The format of each instance should conform to the deployed model's prediction input schema.\n",
    "    instances = instances if type(instances) == list else [instances]\n",
    "    instances = [\n",
    "        json_format.ParseDict(instance_dict, Value()) for instance_dict in instances\n",
    "    ]\n",
    "    parameters_dict = {}\n",
    "    parameters = json_format.ParseDict(parameters_dict, Value())\n",
    "    endpoint = client.endpoint_path(\n",
    "        project=project, location=location, endpoint=endpoint_id\n",
    "    )\n",
    "    response = client.predict(\n",
    "        endpoint=endpoint, instances=instances, parameters=parameters\n",
    "    )\n",
    "    print(\"response\")\n",
    "    print(\" deployed_model_id:\", response.deployed_model_id)\n",
    "    # The predictions are a google.protobuf.Value representation of the model's predictions.\n",
    "    predictions = response.predictions\n",
    "    for prediction in predictions:\n",
    "        print(\" prediction:\", dict(prediction))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c2d8ea7-d519-44df-a0f9-7fc28deb3ecf",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "### Submit multiple prediction requests to model endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "UggAB16XTs8r",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UggAB16XTs8r",
    "outputId": "4e06c785-3767-4d2d-fd63-c4299b16708b"
   },
   "outputs": [],
   "source": [
    "predict_custom_trained_model_sample(\n",
    "    project=\"<your-project-id>\",\n",
    "    endpoint_id=\"your-vertex-endpoint-id\",\n",
    "    location=\"us-central1\",\n",
    "    instances=payload\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ogP_mYV1UtWe",
   "metadata": {
    "id": "ogP_mYV1UtWe"
   },
   "source": [
    "## [Optional] Deploy a second version to the same endpoint\n",
    "\n",
    "The next exercises are optional. You can continue to iterate on your BQML models for improving the ROC metric. Once you are comfortable with your models, feel free to explore the advanced features of vertex endpoints in the below sections.\n",
    "\n",
    "Your second model was able to increase the accuracy of the ROC score. However, you probably don't want to update your application to point to a new endpoint URL, and you don't want to create sudden change in your application. You can add the new model to the same endpoint, serving a small percentage of traffic, and gradually increase the traffic split for the new model until it is serving 100% of the traffic.\n",
    "\n",
    "Now lets deploy a second version of our logistic model to the same endpoint and split the traffic 50% to each model. [[docs]](https://cloud.google.com/vertex-ai/docs/general/deployment#models-endpoint)\n",
    "\n",
    "Note that deploying a new version to the same endpoint takes fewer minutes (~5 min)\n",
    "\n",
    "You can now submit your payload and monitor the traffic to both models via the endpoints page :)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7arsB8RNVEGV",
   "metadata": {
    "id": "7arsB8RNVEGV"
   },
   "source": [
    "## [Optional] Lets monitor for prediction skew for incoming requests in our endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf4dc7e3-c1ff-4dfc-ab6d-6d2158ab6a08",
   "metadata": {},
   "source": [
    "Select ```Edit settings``` tab for your model endpoint \n",
    "\n",
    "Under model monitoring:\n",
    "*  select monitoring interval -> 1 hour\n",
    "*  select monitoring data window -> 1 hour\n",
    "*  select sampling rate -> 100%\n",
    "\n",
    "Under monitoring objectives: \n",
    "*  select prediction drift detection\n",
    "*  Set the alerts to the following below\n",
    "\n",
    "```\n",
    "{\"cnt_user_engagement\":0.01,\"country\":0.01,\"cnt_spend_virtual_currency\":0.01,\"user_first_engagement\":0.01,\"language\":0.01,\"cnt_level_complete_quickplay\":0.01,\"cnt_challenge_a_friend\":0.01,\"cnt_use_extra_steps\":0.01,\"cnt_level_start_quickplay\":0.01,\"cnt_ad_reward\":0.01,\"cnt_level_reset_quickplay\":0.01,\"cnt_level_end_quickplay\":0.01,\"operating_system\":0.01,\"cnt_completed_5_levels\":0.01,\"cnt_post_score\":0.01}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ZnDz11TD7NFS",
   "metadata": {
    "id": "ZnDz11TD7NFS"
   },
   "outputs": [],
   "source": [
    "# create a seperate dataframe for data skew\n",
    "df_data_skew = df.copy(deep=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "AO8606h17rZQ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 243
    },
    "id": "AO8606h17rZQ",
    "outputId": "94b596e6-5fac-49c6-fc1a-aa4e895a82fd"
   },
   "outputs": [],
   "source": [
    "from random import randrange\n",
    "\n",
    "# add skew to the integer fields of our evaluation dataset \n",
    "df_data_skew['cnt_spend_virtual_currency'] = [ randrange(10000,100000)  for k in df_data_skew.index]\n",
    "df_data_skew['cnt_user_engagement'] = [ randrange(2000,5000)  for k in df_data_skew.index]\n",
    "df_data_skew['cnt_challenge_a_friend'] = [ randrange(10,20)  for k in df_data_skew.index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "LIEMUCF09Fz1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 364
    },
    "id": "LIEMUCF09Fz1",
    "outputId": "074abccb-a5f2-41eb-9908-943cd57c6724"
   },
   "outputs": [],
   "source": [
    "df_data_skew.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2M7wdd2x_HkZ",
   "metadata": {
    "id": "2M7wdd2x_HkZ"
   },
   "outputs": [],
   "source": [
    "skew_payload = json.loads(df_data_skew.to_json(orient=\"records\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "gmwFSTLWbI2f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gmwFSTLWbI2f",
    "outputId": "d41df9fa-2068-4df5-8d84-df2780beda24"
   },
   "outputs": [],
   "source": [
    "predict_custom_trained_model_sample(\n",
    "    project=\"<your-project-id>\",\n",
    "    endpoint_id=\"your-vertex-endpoint-id\",\n",
    "    location=\"us-central1\",\n",
    "    instances=skew_payload\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "environment": {
   "kernel": "python3",
   "name": "common-cpu.m107",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/base-cpu:m107"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}