notebooks/official/automl/automl_image_classification_batch_prediction.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title"
      },
      "source": [
        "# AutoML training image classification model for batch prediction\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_classification_batch_prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fautoml%2Fautoml_image_classification_batch_prediction.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>    \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/automl/automl_image_classification_batch_prediction.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_classification_batch_prediction.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "overview:automl"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create image classification models and do batch prediction using a Google Cloud [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model.\n",
        "\n",
        "Learn more about [Get predictions from an image classification model](https://cloud.google.com/vertex-ai/docs/image-data/classification/get-predictions)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "objective:automl,training,batch_prediction"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you create an AutoML image classification model from a Python script, and then do a batch prediction using the Vertex SDK. You can alternatively create and deploy models using the `gcloud` command-line tool or online using the Cloud Console.\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a Vertex dataset resource.\n",
        "- Train the model.\n",
        "- View the model evaluation.\n",
        "- Make a batch prediction.\n",
        "\n",
        "There's one key difference between using batch prediction and using online prediction:\n",
        "\n",
        "**Online Prediction Service**: Does an on-demand prediction for the entire set of instances (i.e., one or more data items) and returns the results in real-time.\n",
        "\n",
        "**Batch Prediction Service**: Does a queued (batch) prediction for the entire set of instances in the background and stores the results in a Cloud Storage bucket when ready."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:flowers,icn"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the [Flowers dataset](https://www.tensorflow.org/datasets/catalog/tf_flowers) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). The version of the dataset you use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the type of flower an image is from a class of five flowers: *daisy*, *dandelion*, *rose*, *sunflower*, or *tulip*."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "61RBz8LLbxCR"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "No17Cw5hgx12"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other required packages\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "install_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
        "                                 tensorflow==2.15.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R5Xep4W9lq-Z"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XRvKdaPDTznN"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SbmM4z7FOBpM"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmWOrTJ3gx13"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "Authenticate your environment on Google Colab.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NyKGtVQjgx13"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DF4l8DTdWgPY"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project. Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "set_project_id"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l {LOCATION} -p {PROJECT_ID} {BUCKET_URI}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "Next, set up some variables used throughout the tutorial.\n",
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "from google.cloud import aiplatform"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "## Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tutorial_start:automl"
      },
      "source": [
        "## Tutorial\n",
        "\n",
        "Now you're ready to start creating your own AutoML image classification model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_file:u_dataset,csv"
      },
      "source": [
        "#### Location of Cloud Storage training data.\n",
        "\n",
        "Now set the variable `IMPORT_FILE` to the location of the CSV index file in Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_file:flowers,csv,icn"
      },
      "outputs": [],
      "source": [
        "IMPORT_FILE = \"gs://cloud-samples-data/ai-platform/flowers/flowers.csv\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "quick_peek:csv"
      },
      "source": [
        "#### Quick peek at your data\n",
        "\n",
        "This tutorial uses a version of the Flowers dataset that is stored in a public Cloud Storage bucket, using a CSV index file.\n",
        "\n",
        "Begin by taking a quick look at the data. First, count the number of examples by determining the number of rows in the CSV index file using the (`wc -l`) command. Then, preview the first few rows of the file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "quick_peek:csv"
      },
      "outputs": [],
      "source": [
        "if \"IMPORT_FILES\" in globals():\n",
        "    FILE = IMPORT_FILES[0]\n",
        "else:\n",
        "    FILE = IMPORT_FILE\n",
        "\n",
        "count = ! gsutil cat $FILE | wc -l\n",
        "print(\"Number of Examples\", int(count[0]))\n",
        "\n",
        "print(\"First 10 rows\")\n",
        "! gsutil cat $FILE | head"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:image,icn"
      },
      "source": [
        "### Create the Dataset\n",
        "\n",
        "Next, create the dataset resource using the `create` method for the `ImageDataset` class, which takes the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the dataset resource.\n",
        "- `gcs_source`: A list of one or more dataset index files to import the data items into the dataset resource.\n",
        "- `import_schema_uri`: The data labeling schema for the data items.\n",
        "\n",
        "This operation may take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_dataset:image,icn"
      },
      "outputs": [],
      "source": [
        "dataset = aiplatform.ImageDataset.create(\n",
        "    display_name=\"Flowers\",\n",
        "    gcs_source=[IMPORT_FILE],\n",
        "    import_schema_uri=aiplatform.schema.dataset.ioformat.image.single_label_classification,\n",
        ")\n",
        "\n",
        "print(dataset.resource_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:image,icn"
      },
      "source": [
        "### Create and run training pipeline\n",
        "\n",
        "To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.\n",
        "\n",
        "#### Create training pipeline\n",
        "\n",
        "An AutoML training pipeline is created with the `AutoMLImageTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the training job resource.\n",
        "- `prediction_type`: The type of task for which the model is trained.\n",
        "  - `classification`: An image classification model.\n",
        "  - `object_detection`: An image object detection model.\n",
        "- `multi_label`: For a classification task, specify whether it's multi-labeled (`True`) or single-labeled (`False`).\n",
        "- `model_type`: The type of model for deployment.\n",
        "  - `CLOUD`: Deployment on Google Cloud\n",
        "  - `CLOUD_HIGH_ACCURACY_1`: Optimized for accuracy over latency for deployment on Google Cloud.\n",
        "  - `CLOUD_LOW_LATENCY_`: Optimized for latency over accuracy for deployment on Google Cloud.\n",
        "  - `MOBILE_TF_VERSATILE_1`: Deployment on an edge device.\n",
        "  - `MOBILE_TF_HIGH_ACCURACY_1`:Optimized for accuracy over latency for deployment on an edge device.\n",
        "  - `MOBILE_TF_LOW_LATENCY_1`: Optimized for latency over accuracy for deployment on an edge device.\n",
        "- `base_model`: (optional) Transfer learning from existing model resource -- supported for image classification only.\n",
        "\n",
        "The instantiated object is the DAG (directed acyclic graph) for the training job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_automl_pipeline:image,icn"
      },
      "outputs": [],
      "source": [
        "dag = aiplatform.AutoMLImageTrainingJob(\n",
        "    display_name=\"flowers\",\n",
        "    prediction_type=\"classification\",\n",
        "    multi_label=False,\n",
        "    model_type=\"CLOUD\",\n",
        "    base_model=None,\n",
        ")\n",
        "\n",
        "print(dag)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "source": [
        "#### Run the training pipeline\n",
        "\n",
        "Next, run the DAG to start the training job by invoking the `run` method, with the following parameters:\n",
        "\n",
        "- `dataset`: The dataset resource to train the model.\n",
        "- `model_display_name`: The human readable name for the trained model.\n",
        "- `training_fraction_split`: The percentage of the dataset to use for training.\n",
        "- `test_fraction_split`: The percentage of the dataset to use for test (holdout data).\n",
        "- `validation_fraction_split`: The percentage of the dataset to use for validation.\n",
        "- `budget_milli_node_hours`: (optional) Maximum training time specified in unit of millihours (1000 = hour).\n",
        "- `disable_early_stopping`: By default, the model training stops early if the model performance doesn't improve. Setting `disable_early_stopping` = `True` overrides this behavior, allowing the model to train for the entire specified duration.\n",
        "\n",
        "The `run` method, upon completion, returns the model resource"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "outputs": [],
      "source": [
        "model = dag.run(\n",
        "    dataset=dataset,\n",
        "    model_display_name=\"flowers\",\n",
        "    training_fraction_split=0.8,\n",
        "    validation_fraction_split=0.1,\n",
        "    test_fraction_split=0.1,\n",
        "    budget_milli_node_hours=8000,\n",
        "    disable_early_stopping=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "## Review model evaluation scores\n",
        "\n",
        "After your model training is complete, you can review the evaluation scores for it using the `list_model_evaluations()` method. This method returns an iterator for each evaluation slice."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_prediction"
      },
      "source": [
        "## Send a batch prediction request\n",
        "\n",
        "Send a batch prediction to your deployed model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "model_evaluations = model.list_model_evaluations()\n",
        "\n",
        "for model_evaluation in model_evaluations:\n",
        "    print(model_evaluation.to_dict())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_test_items:batch_prediction"
      },
      "source": [
        "### Get test item(s)\n",
        "\n",
        "Now generate a batch prediction for your Vertex AI model. Use arbitrary examples from the dataset as test items. Don't be concerned that the example was likely used in training the model -- the point is to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_test_items:automl,icn,csv"
      },
      "outputs": [],
      "source": [
        "test_items = !gsutil cat $IMPORT_FILE | head -n2\n",
        "if len(str(test_items[0]).split(\",\")) == 3:\n",
        "    _, test_item_1, test_label_1 = str(test_items[0]).split(\",\")\n",
        "    _, test_item_2, test_label_2 = str(test_items[1]).split(\",\")\n",
        "else:\n",
        "    test_item_1, test_label_1 = str(test_items[0]).split(\",\")\n",
        "    test_item_2, test_label_2 = str(test_items[1]).split(\",\")\n",
        "\n",
        "print(test_item_1, test_label_1)\n",
        "print(test_item_2, test_label_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "copy_test_items:batch_prediction"
      },
      "source": [
        "### Copy test item(s)\n",
        "\n",
        "For the batch prediction, copy the test items over to your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copy_test_items:batch_prediction"
      },
      "outputs": [],
      "source": [
        "file_1 = test_item_1.split(\"/\")[-1]\n",
        "file_2 = test_item_2.split(\"/\")[-1]\n",
        "\n",
        "! gsutil cp $test_item_1 $BUCKET_URI/$file_1\n",
        "! gsutil cp $test_item_2 $BUCKET_URI/$file_2\n",
        "\n",
        "test_item_1 = BUCKET_URI + \"/\" + file_1\n",
        "test_item_2 = BUCKET_URI + \"/\" + file_2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_file:automl,image"
      },
      "source": [
        "### Create the batch input file\n",
        "\n",
        "Now make a batch input file, which is stored in your local Cloud Storage bucket. The batch input file can be either CSV or JSONL. Use JSONL in this tutorial. For JSONL file, make one dictionary entry per line for each data item (instance). The dictionary contains the key/value pairs:\n",
        "\n",
        "- `content`: The Cloud Storage path to the image.\n",
        "- `mime_type`: The content type. In our example, it's a `jpeg` file.\n",
        "\n",
        "For example:\n",
        "\n",
        "                        {'content': '[your-bucket]/file1.jpg', 'mime_type': 'jpeg'}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_batch_file:automl,image"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "gcs_input_uri = BUCKET_URI + \"/test.jsonl\"\n",
        "with tf.io.gfile.GFile(gcs_input_uri, \"w\") as f:\n",
        "    data = {\"content\": test_item_1, \"mime_type\": \"image/jpeg\"}\n",
        "    f.write(json.dumps(data) + \"\\n\")\n",
        "    data = {\"content\": test_item_2, \"mime_type\": \"image/jpeg\"}\n",
        "    f.write(json.dumps(data) + \"\\n\")\n",
        "\n",
        "print(gcs_input_uri)\n",
        "! gsutil cat $gcs_input_uri"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "source": [
        "### Make the batch prediction request\n",
        "\n",
        "Now that your model resource is trained, make a batch prediction by invoking the `batch_predict()` method, with the following parameters:\n",
        "\n",
        "- `job_display_name`: The human readable name for the batch prediction job.\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction results.\n",
        "- `sync`: Set `True` to wait until the completion of the job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job = model.batch_predict(\n",
        "    job_display_name=\"flowers\",\n",
        "    gcs_source=gcs_input_uri,\n",
        "    gcs_destination_prefix=BUCKET_URI,\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "print(batch_predict_job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "### Wait for completion of batch prediction job\n",
        "\n",
        "Next, wait for the batch job to complete. Alternatively, you can set the parameter `sync` to `True` in the `batch_predict()` method to block until the batch prediction job is completed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,icn"
      },
      "source": [
        "### Get the predictions\n",
        "\n",
        "Next, get the results from the completed batch prediction job.\n",
        "\n",
        "The results are written to the Cloud Storage output bucket specified in the batch prediction request. Call the `iter_outputs()` method to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a JSON format:\n",
        "\n",
        "- `content`: The prediction request.\n",
        "- `prediction`: The prediction response.\n",
        " - `ids`: The internally assigned unique identifiers for each prediction request.\n",
        " - `displayNames`: The class names for each class label.\n",
        " - `confidences`: The predicted confidence, between 0 and 1, per class label."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_batch_prediction:mbsdk,icn"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "bp_iter_outputs = batch_predict_job.iter_outputs()\n",
        "\n",
        "prediction_results = list()\n",
        "for blob in bp_iter_outputs:\n",
        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
        "        prediction_results.append(blob.name)\n",
        "\n",
        "tags = list()\n",
        "for prediction_result in prediction_results:\n",
        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\"\n",
        "    with tf.io.gfile.GFile(name=gfile_name, mode=\"r\") as gfile:\n",
        "        for line in gfile.readlines():\n",
        "            line = json.loads(line)\n",
        "            print(line)\n",
        "            break"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "# Delete the dataset using the Vertex dataset object\n",
        "dataset.delete()\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the AutoML trainig job\n",
        "dag.delete()\n",
        "\n",
        "# Delete the batch prediction job\n",
        "batch_predict_job.delete()\n",
        "\n",
        "# Delete the cloud storage bucket\n",
        "delete_bucket = False  # set True for deletion\n",
        "if delete_bucket:\n",
        "    ! gsutil rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "automl_image_classification_batch_prediction.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}