Keras_Flowers_on_TPU_(playground).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Keras_Flowers_on_TPU_(playground).ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.3"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Yd4z24wngWau"
      },
      "source": [
        "Bu not defterini bir GPU'da deneyebilirsiniz, ancak hızlı bir şekilde TPU'ya geçeceksiniz. Bu model için buna değer."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "89B27-TGiDNB"
      },
      "source": [
        "## Yüklemeler"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "9u3d4Z7uQsmp",
        "outputId": "6c392b86-1b89-4fbe-c3fe-6ba9fe0d3f84",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "import os, math, json, random\n",
        "import numpy as np\n",
        "from matplotlib import pyplot as plt\n",
        "import tensorflow as tf\n",
        "print(\"Tensorflow version \" + tf.__version__)\n",
        "AUTO = tf.data.experimental.AUTOTUNE\n",
        "\n",
        "# little wrinkle: Keras models do not yet work on TPU if eager mode is enabled\n",
        "# tf.enable_eager_execution()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Tensorflow version 1.13.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Lzd6Qi464PsA"
      },
      "source": [
        "## Colab kimlik doğrulama \n",
        "Özel GCS hücrelerine erişiyor olsaydınız buna ihtiyaç duyardınız. Bu alıştırma için eğitim verileri, kimlik doğrulamanın gerekli olmadığı halka açık bir hücrededir."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "both",
        "colab_type": "code",
        "id": "MPx0nvyUnvgT",
        "colab": {}
      },
      "source": [
        "# IS_COLAB_BACKEND = 'COLAB_GPU' in os.environ  # this is always set on Colab, the value is 0 or 1 depending on GPU presence\n",
        "# if IS_COLAB_BACKEND:\n",
        "#   from google.colab import auth\n",
        "#   auth.authenticate_user()  # not necessary to access a public bucket but you will probably want to access your private buckets too"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "UaKGHPjWkcVj"
      },
      "source": [
        "## TPU algılama"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tmv6p137kgob",
        "colab": {}
      },
      "source": [
        "# TPUClusterResolver() automatically detects a connected TPU on all Gooogle's\n",
        "# platforms: Colaboratory, AI Platform (ML Engine), Kubernetes and Deep Learning\n",
        "# VMs created through the 'ctpu up' utility. If auto-detection is not available,\n",
        "# you can pass the name of your TPU explicitly:\n",
        "# tf.contrib.cluster_resolver.TPUClusterResolver('MY_TPU_NAME')\n",
        "# tip: on a VM created with \"ctpu up\" the TPU has the same name as the VM.\n",
        "\n",
        "try:\n",
        "  tpu = tf.contrib.cluster_resolver.TPUClusterResolver() # TPU detection\n",
        "  \n",
        "  print('Running on TPU ', tpu.cluster_spec().as_dict()['worker'])  \n",
        "except ValueError:\n",
        "  print(\"Running on GPU or CPU\")\n",
        "  tpu = None"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "w9S3uKC_iXY5"
      },
      "source": [
        "## Yapılandırma"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "M3G-2aUBQJ-H",
        "colab": {}
      },
      "source": [
        "GCS_PATTERN = 'gs://flowers-public/tfrecords-jpeg-192x192-2/*.tfrec'\n",
        "IMAGE_SIZE = [192, 192]\n",
        "\n",
        "if tpu:\n",
        "  BATCH_SIZE = 128  # On TPU in Keras, this is the per-core batch size. The global batch size is 8x this.\n",
        "else:\n",
        "  BATCH_SIZE = 32  # On Colab/GPU, a higher batch size does not help and sometimes does not fit on the GPU (OOM)\n",
        "\n",
        "VALIDATION_SPLIT = 0.19\n",
        "CLASSES = ['daisy', 'dandelion', 'roses', 'sunflowers', 'tulips'] # do not change, maps to the labels in the data (folder names)\n",
        "\n",
        "# splitting data files between training and validation\n",
        "filenames = tf.gfile.Glob(GCS_PATTERN)\n",
        "random.shuffle(filenames)\n",
        "split = int(len(filenames) * VALIDATION_SPLIT)\n",
        "training_filenames = filenames[split:]\n",
        "validation_filenames = filenames[:split]\n",
        "print(\"Pattern matches {} data files. Splitting dataset into {} training files and {} validation files\".format(len(filenames), len(training_filenames), len(validation_filenames)))\n",
        "validation_steps = int(3670 // len(filenames) * len(validation_filenames)) // BATCH_SIZE\n",
        "steps_per_epoch = int(3670 // len(filenames) * len(training_filenames)) // BATCH_SIZE\n",
        "print(\"With a batch size of {}, there will be {} batches per training epoch and {} batch(es) per validation run.\".format(BATCH_SIZE, steps_per_epoch, validation_steps))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "tMy0zz6FXnJY",
        "colab": {}
      },
      "source": [
        "#@title display utilities [ÇALIŞTIR]\n",
        "\n",
        "def dataset_to_numpy_util(dataset, N):\n",
        "  dataset = dataset.batch(N)\n",
        "  \n",
        "  if tf.executing_eagerly():\n",
        "    # In eager mode, iterate in the Datset directly.\n",
        "    for images, labels in dataset:\n",
        "      numpy_images = images.numpy()\n",
        "      numpy_labels = labels.numpy()\n",
        "      break;\n",
        "      \n",
        "  else: # In non-eager mode, must get the TF note that \n",
        "        # yields the nextitem and run it in a tf.Session.\n",
        "    get_next_item = dataset.make_one_shot_iterator().get_next()\n",
        "    with tf.Session() as ses:\n",
        "      numpy_images, numpy_labels = ses.run(get_next_item)\n",
        "\n",
        "  return numpy_images, numpy_labels\n",
        "\n",
        "def title_from_label_and_target(label, correct_label):\n",
        "  label = np.argmax(label, axis=-1)  # one-hot to class number\n",
        "  correct_label = np.argmax(correct_label, axis=-1) # one-hot to class number\n",
        "  correct = (label == correct_label)\n",
        "  return \"{} [{}{}{}]\".format(CLASSES[label], str(correct), ', shoud be ' if not correct else '',\n",
        "                              CLASSES[correct_label] if not correct else ''), correct\n",
        "\n",
        "def display_one_flower(image, title, subplot, red=False):\n",
        "    plt.subplot(subplot)\n",
        "    plt.axis('off')\n",
        "    plt.imshow(image)\n",
        "    plt.title(title, fontsize=16, color='red' if red else 'black')\n",
        "    return subplot+1\n",
        "  \n",
        "def display_9_images_from_dataset(dataset):\n",
        "  subplot=331\n",
        "  plt.figure(figsize=(13,13))\n",
        "  images, labels = dataset_to_numpy_util(dataset, 9)\n",
        "  for i, image in enumerate(images):\n",
        "    title = CLASSES[np.argmax(labels[i], axis=-1)]\n",
        "    subplot = display_one_flower(image, title, subplot)\n",
        "    if i >= 8:\n",
        "      break;\n",
        "              \n",
        "  plt.tight_layout()\n",
        "  plt.subplots_adjust(wspace=0.1, hspace=0.1)\n",
        "  plt.show()\n",
        "  \n",
        "def display_9_images_with_predictions(images, predictions, labels):\n",
        "  subplot=331\n",
        "  plt.figure(figsize=(13,13))\n",
        "  for i, image in enumerate(images):\n",
        "    title, correct = title_from_label_and_target(predictions[i], labels[i])\n",
        "    subplot = display_one_flower(image, title, subplot, not correct)\n",
        "    if i >= 8:\n",
        "      break;\n",
        "              \n",
        "  plt.tight_layout()\n",
        "  plt.subplots_adjust(wspace=0.1, hspace=0.1)\n",
        "  plt.show()\n",
        "  \n",
        "def display_training_curves(training, validation, title, subplot):\n",
        "  if subplot%10==1: # set up the subplots on the first call\n",
        "    plt.subplots(figsize=(10,10), facecolor='#F0F0F0')\n",
        "    plt.tight_layout()\n",
        "  ax = plt.subplot(subplot)\n",
        "  ax.set_facecolor('#F8F8F8')\n",
        "  ax.plot(training)\n",
        "  ax.plot(validation)\n",
        "  ax.set_title('model '+ title)\n",
        "  ax.set_ylabel(title)\n",
        "  ax.set_xlabel('epoch')\n",
        "  ax.legend(['train', 'valid.'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "kvPXiovhi3ZZ"
      },
      "source": [
        "## TFRecords'dan görüntüleri ve etiketleri okuma"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "LtAVr-4CP1rp",
        "colab": {}
      },
      "source": [
        "def read_tfrecord(example):\n",
        "    features = {\n",
        "        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n",
        "        \"class\": tf.io.FixedLenFeature([], tf.int64),  # shape [] means scalar\n",
        "        \"one_hot_class\": tf.io.VarLenFeature(tf.float32),\n",
        "    }\n",
        "    example = tf.parse_single_example(example, features)\n",
        "    image = tf.image.decode_jpeg(example['image'], channels=3)\n",
        "    image = tf.cast(image, tf.float32) / 255.0  # convert image to floats in [0, 1] range\n",
        "    image = tf.reshape(image, [*IMAGE_SIZE, 3]) # explicit size will be needed for TPU\n",
        "    class_label = tf.cast(example['class'], tf.int32)\n",
        "    one_hot_class = tf.sparse.to_dense(example['one_hot_class'])\n",
        "    one_hot_class = tf.reshape(one_hot_class, [5])\n",
        "    return image, one_hot_class\n",
        "\n",
        "def load_dataset(filenames):\n",
        "  # read from TFRecords. For optimal performance, use \"interleave(tf.data.TFRecordDataset, ...)\"\n",
        "  # to read from multiple TFRecord files at once and set the option experimental_deterministic = False\n",
        "  # to allow order-altering optimizations.\n",
        "\n",
        "  option_no_order = tf.data.Options()\n",
        "  option_no_order.experimental_deterministic = False\n",
        "\n",
        "  dataset = tf.data.Dataset.list_files(GCS_PATTERN)\n",
        "  dataset = dataset.with_options(option_no_order)\n",
        "  #dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=16)\n",
        "  dataset = dataset.interleave(tf.data.TFRecordDataset, cycle_length=16, num_parallel_calls=AUTO) # faster\n",
        "  dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTO)\n",
        "  return dataset"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "xb-b4PRz-V6O",
        "colab": {}
      },
      "source": [
        "display_9_images_from_dataset(load_dataset(training_filenames))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "22rVDTx8wCqE"
      },
      "source": [
        "## eğitim ve doğrulama veri kümeleri"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "7wxKyCklR4Gh",
        "colab": {}
      },
      "source": [
        "def get_batched_dataset(filenames):\n",
        "  dataset = load_dataset(filenames)\n",
        "  dataset = dataset.cache() # This dataset fits in RAM\n",
        "  dataset = dataset.repeat()\n",
        "  dataset = dataset.batch(BATCH_SIZE, drop_remainder=True) # drop_remainder will be needed on TPU\n",
        "  dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n",
        "  # should shuffle too but this dataset was well shuffled on disk already\n",
        "  # For proper ordering of map/batch/repeat/prefetch, see Dataset performance guide: https://www.tensorflow.org/guide/performance/datasets\n",
        "  return dataset\n",
        "\n",
        "def get_training_dataset():\n",
        "  return get_batched_dataset(training_filenames)\n",
        "\n",
        "def get_validation_dataset():\n",
        "  return get_batched_dataset(validation_filenames)\n",
        "\n",
        "some_flowers, some_labels = dataset_to_numpy_util(load_dataset(validation_filenames), 8*20)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ALtRUlxhw8Vt"
      },
      "source": [
        "## Model [ÇALIŞMASI GEREKLİ]\n",
        "1.Tek bir evrişimli tabaka ile modeli olduğu gibi eğitin.\n",
        " * %55 Doğruluk ... İyi değil.\n",
        "2. ek evrişim katmanlar araya eklenmiş maksimum ortaklama katmanları ekleyin. Ayrıca ikinci bir yoğun katman eklemeyi deneyin. Örneğin:<br/>\n",
        "**`conv 3x3, 16 filters, relu`**<br/>\n",
        "**`conv 3x3, 30 filters, relu`**<br/>\n",
        "**`max pool 2x2`**<br/>\n",
        "**`conv 3x3, 50 filters, relu`**<br/>\n",
        "**`max pool 2x2`**<br/>\n",
        "**`conv 3x3, 70 filters, relu`**<br/>\n",
        "**`flatten`**<br/>\n",
        "**`dense 90 relu, relu`**<br/>\n",
        "**`dense 10 softmax`**<br/>\n",
        " * %62 Doğruluk... nispeten başarılı. Ancak bu model 3M'den parametreden daha fazla ve çarpıcı biçimde üst üste geliyor (aşırı öğrenme = eval loss yerine aşağı doğru artıyor).\n",
        "3. Vektörleştirme katmanını global ortalama ortaklama katmanı ile değiştirin.\n",
        " * %68 Doğruluk... Model mütevazı 50K parametrelerine geri döndü, öncekinden daha iyi çalışıyor ve artık aşırı öğrenme olmuyor.\n",
        "4. 1x1 evrişimleri de denemeyi deneyin. Genellikle 3x3 evrişim izler ve filtre sayısını azaltır. Ayrıca yoğun katmanlar arasına seyreltme/dropout da ekleyebilirsiniz. Örneğin:\n",
        "**`conv 3x3, 20 filters, relu`**<br/>\n",
        "**`conv 3x3, 50 filters, relu`**<br/>\n",
        "**`max pool 2x2`**<br/>\n",
        "**`conv 3x3, 70 filters, relu`**<br/>\n",
        "**`conv 1x1, 50 filters, relu`**<br/>\n",
        "**`max pool 2x2`**<br/>\n",
        "**`conv 3x3, 100 filters, relu`**<br/>\n",
        "**`conv 1x1, 70 filters, relu`**<br/>\n",
        "**`max pool 2x2`**<br/>\n",
        "**`conv 3x3, 120 filters, relu`**<br/>\n",
        "**`conv 1x1, 80 filters, relu`**<br/>\n",
        "**`max pool 2x2`**<br/>\n",
        "**`global average pooling`**<br/>\n",
        "**`dense 70 relu, relu`**<br/>\n",
        "**`dense 10 softmax`**<br/>\n",
        " * %73 Doğruluk\n",
        "5.Hedef %80 doğruluk! İyi şanslar. (Oraya ulaşmak için 20 epoch eğitmek isteyebilirsin. Daha uzun süren eğitime değip değmeyeceğini görmek için eğitimlerin eğrilerini bak.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "XLJNVGwHUDy1",
        "colab": {}
      },
      "source": [
        "model = tf.keras.Sequential([\n",
        "    # little wrinkle: specifying the input shape as a Keras InputLayer does not\n",
        "    # work on TPU yet. Please add an input shape on your first layer instead. \n",
        "    # l.InputLayer(input_shape=[*IMAGE_SIZE, 3]),\n",
        "    \n",
        "    ###\n",
        "    tf.keras.layers.Conv2D(kernel_size=3, filters=20, padding='same', activation='relu', input_shape=[*IMAGE_SIZE, 3]),\n",
        "    #\n",
        "    # YOUR LAYERS HERE\n",
        "    #\n",
        "    # LAYERS TO TRY:\n",
        "    # Conv2D(kernel_size=3, filters=30, padding='same', activation='relu')\n",
        "    # MaxPooling2D(pool_size=2)\n",
        "    # GlobalAveragePooling2D() / Flatten()\n",
        "    # Dense(90, activation='relu')\n",
        "    #\n",
        "    tf.keras.layers.Flatten(),\n",
        "    tf.keras.layers.Dense(5, activation='softmax')\n",
        "    ###\n",
        "])\n",
        "\n",
        "def accuracy_metric(y_true, y_pred):\n",
        "  predictions  = tf.math.argmax(y_pred, axis=-1, output_type=tf.int32)\n",
        "  ground_truth = tf.squeeze(y_true, axis=-1)\n",
        "  ground_truth = tf.cast(ground_truth, tf.int32)\n",
        "  correct = tf.math.equal(predictions, ground_truth)\n",
        "  accuracy = tf.reduce_mean(tf.cast(correct, tf.int32))\n",
        "  return accuracy\n",
        "\n",
        "model.compile(\n",
        "  optimizer='adam',\n",
        "  loss='categorical_crossentropy',\n",
        "  metrics=['accuracy'])\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "dMfenMQcxAAb"
      },
      "source": [
        "## Eğitim"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "H7QwBi6_ri4x",
        "colab": {}
      },
      "source": [
        "if tpu:\n",
        "  # tpu = tf.contrib.cluster_resolver.TPUClusterResolver()\n",
        "  strategy = tf.contrib.tpu.TPUDistributionStrategy(tpu)\n",
        "  tpu_model = tf.contrib.tpu.keras_to_tpu_model(model, strategy=strategy)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "M-ID7vP5mIKs",
        "colab": {}
      },
      "source": [
        "EPOCHS = 10\n",
        "\n",
        "if tpu:\n",
        "  history = tpu_model.fit(get_training_dataset, steps_per_epoch=steps_per_epoch, epochs=EPOCHS,\n",
        "                          validation_data=get_validation_dataset, validation_steps=validation_steps)\n",
        "  # Little wrinkle: reading directly from dataset object not yet implemented\n",
        "  # for Keras/TPU. Please use a function that returns a dataset.\n",
        "else:\n",
        "  history = model.fit(get_training_dataset(), steps_per_epoch=steps_per_epoch, epochs=EPOCHS,\n",
        "                      validation_data=get_validation_dataset(), validation_steps=validation_steps)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "VngeUBIdyJ1T",
        "colab": {}
      },
      "source": [
        "print(history.history.keys())\n",
        "display_training_curves(history.history['acc'], history.history['val_acc'], 'accuracy', 211)\n",
        "display_training_curves(history.history['loss'], history.history['val_loss'], 'loss', 212)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MKFMWzh0Yxsq"
      },
      "source": [
        "## Tahminleme"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ExtZuDlh2Lem",
        "colab": {}
      },
      "source": [
        "inference_model = model\n",
        "if tpu:\n",
        "  inference_model.set_weights(tpu_model.get_weights()) # this copies the weights from TPU to CPU"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ehlsvY46Hs9z",
        "colab": {}
      },
      "source": [
        "# randomize the input so that you can execute multiple times to change results\n",
        "permutation = np.random.permutation(8*20)\n",
        "some_flowers, some_labels = (some_flowers[permutation], some_labels[permutation])\n",
        "\n",
        "predictions = inference_model.predict(some_flowers, batch_size=16)\n",
        "evaluations = inference_model.evaluate(some_flowers, some_labels, batch_size=16)\n",
        "  \n",
        "print(np.array(CLASSES)[np.argmax(predictions, axis=-1)].tolist())\n",
        "print('[val_loss, val_acc]', evaluations)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "qzCCDL1CZFx6",
        "colab": {}
      },
      "source": [
        "display_9_images_with_predictions(some_flowers, predictions, some_labels)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SVY1pBg5ydH-"
      },
      "source": [
        "## License"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "hleIN5-pcr0N"
      },
      "source": [
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "author: Martin Gorner<br>\n",
        "twitter: @martin_gorner\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "Copyright 2018 Google LLC\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "you may not use this file except in compliance with the License.\n",
        "You may obtain a copy of the License at\n",
        "\n",
        "    http://www.apache.org/licenses/LICENSE-2.0\n",
        "\n",
        "Unless required by applicable law or agreed to in writing, software\n",
        "distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "See the License for the specific language governing permissions and\n",
        "limitations under the License.\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "This is not an official Google product but sample code provided for an educational purpose\n"
      ]
    }
  ]
}