C4_W2_Lab_1_FirstAutoEncoder.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "C4_W2_Lab_1_FirstAutoEncoder.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nlG-1dWDVwS9"
      },
      "source": [
        "# Ungraded Lab: First Autoencoder\n",
        "\n",
        "In this lab, you will build your first simple autoencoder. This will take in three-dimensional data, encodes it to two dimensions, and decodes it back to 3D."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "al-wVtulWPDe"
      },
      "source": [
        "## Imports"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LCQpIDns-Tj8"
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "%matplotlib inline"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QJqm-GedWRBa"
      },
      "source": [
        "## Prepare and preview the dataset\n",
        "\n",
        "You will first create a synthetic dataset to act as input to the autoencoder. You can do that with the function below."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.empty?"
      ],
      "metadata": {
        "id": "334aBXiaLs9Q"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q-OdghfC_D1P"
      },
      "source": [
        "def generate_data(m):\n",
        "    '''plots m random points on a 3D plane'''\n",
        "\n",
        "    angles = np.random.rand(m) * 3 * np.pi / 2 - 0.5\n",
        "    data = np.empty((m, 3))\n",
        "    data[:,0] = np.cos(angles) + np.sin(angles)/2 + 0.1 * np.random.randn(m)/2\n",
        "    data[:,1] = np.sin(angles) * 0.7 + 0.1 * np.random.randn(m) / 2\n",
        "    data[:,2] = data[:, 0] * 0.1 + data[:, 1] * 0.3 + 0.1 * np.random.randn(m)\n",
        "    \n",
        "    return data"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v1PTW9_N-ubv"
      },
      "source": [
        "# use the function above to generate data points\n",
        "X_train = generate_data(100)\n",
        "X_train = X_train - X_train.mean(axis=0, keepdims=0)\n",
        "\n",
        "# preview the data\n",
        "ax = plt.axes(projection='3d')\n",
        "ax.scatter3D(X_train[:, 0], X_train[:, 1], X_train[:, 2], cmap='RdBl');"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YqTs5ytvW-d9"
      },
      "source": [
        "## Build the Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GkLJdYqeA2lI"
      },
      "source": [
        "Now you will build the simple encoder-decoder model. Notice the number of neurons in each Dense layer. The model will contract in the encoder then expand in the decoder."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JKhKySrG_bIu"
      },
      "source": [
        "encoder = keras.models.Sequential([keras.layers.Dense(2, input_shape=[3])])\n",
        "decoder = keras.models.Sequential([keras.layers.Dense(3, input_shape=[2])])\n",
        "\n",
        "autoencoder = keras.models.Sequential([encoder, decoder])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mhqBpSZSXDpU"
      },
      "source": [
        "## Compile the Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ob6L7Sb4Bb_s"
      },
      "source": [
        "You can then setup the model for training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sk0a1N2mXCvS"
      },
      "source": [
        "autoencoder.compile(loss=\"mse\", optimizer=keras.optimizers.SGD(lr=0.1))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b-6eGoIrXIrV"
      },
      "source": [
        "## Train the Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cda0bq1WLFpV"
      },
      "source": [
        "You will configure the training to also use the input data as your target output. In our example, that will be `X_train`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f_sVqEE8_j9K"
      },
      "source": [
        "history = autoencoder.fit(X_train, X_train, epochs=200)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "39udYj_uXXjR"
      },
      "source": [
        "## Plot the encoder output"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MvavjilWDAL8"
      },
      "source": [
        "As mentioned, you can use the encoder to compress the input to two dimensions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fgMtTYrnHL8v"
      },
      "source": [
        "# encode the data\n",
        "codings = encoder.predict(X_train)\n",
        "\n",
        "# see a sample input-encoder output pair\n",
        "print(f'input point: {X_train[0]}')\n",
        "print(f'encoded point: {codings[0]}')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4uqLlpGkHPdV"
      },
      "source": [
        "# plot all encoder outputs\n",
        "fig = plt.figure(figsize=(4,3))\n",
        "plt.plot(codings[:,0], codings[:, 1], \"b.\")\n",
        "plt.xlabel(\"$z_1$\", fontsize=18)\n",
        "plt.ylabel(\"$z_2$\", fontsize=18, rotation=0)\n",
        "plt.grid(True)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WsczM7RtXrru"
      },
      "source": [
        "## Plot the Decoder output"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yz4UIGBSEwu8"
      },
      "source": [
        "The decoder then tries to reconstruct the original input. See the outputs below. You will see that although not perfect, it still follows the general shape of the original input."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oiSZA2NBE6Xw"
      },
      "source": [
        "# decode the encoder output\n",
        "decodings = decoder.predict(codings)\n",
        "\n",
        "# see a sample output for a single point\n",
        "print(f'input point: {X_train[0]}')\n",
        "print(f'encoded point: {codings[0]}')\n",
        "print(f'decoded point: {decodings[0]}')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jOhdRBd_30iK"
      },
      "source": [
        "# plot the decoder output\n",
        "ax = plt.axes(projection='3d')\n",
        "ax.scatter3D(decodings[:, 0], decodings[:, 1], decodings[:, 2], c=decodings[:, 0], cmap='Reds');"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NW6y8830Gbcr"
      },
      "source": [
        "That's it for this simple demonstration of the autoencoder!"
      ]
    }
  ]
}