The_Big_Sleep_BigGANxCLIP.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "The Big Sleep: BigGANxCLIP.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "KAcixx9Z3XYH",
        "UlIUu0jK3S19",
        "b3DmDF7Dq5uh"
      ],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Chad2000/Big-Sleep/blob/main/The_Big_Sleep_BigGANxCLIP.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NRUouuHptoRp"
      },
      "source": [
        "# Description\n",
        "\n",
        "This notebook utilizes CLIP to steer BigGAN towards images that match a textual prompt.\n",
        "\n",
        "It also dings."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "reChJi_vrNI_"
      },
      "source": [
        "## Using this notebook\n",
        "\n",
        "\n",
        "\n",
        "Of note: this notebook comes with no guarantees that it will work for your prompt or at all, but it's free! There are many magical numbers and I'm sure there are better ways to regularize the latent code. But it's free!\n",
        "\n",
        "**To use the notebook: edit the prompt text found in the \"Parameters\" cell to reflect what image you want to generate. The prompt starts as '''a cityscape in the style of Van Gogh'''. After setting the prompt, follow directions in the cell labeled \"Restart the kernel...\"**\n",
        "\n",
        "Images will be shown routinely in the \"Train\" cell after a \"ding.\"\n",
        "\n",
        "Feel free to send inquiries to https://twitter.com/advadnoun, though I'm sometimes busy, so I can't promise I'll be able to get to you about your specific issue.\n",
        "\n",
        "\n",
        "One last thing: feel free to use, modify, and distribute this -- just please be cool and cite me (using my twitter lol) if you use it!\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mZi8SIIXsZgU"
      },
      "source": [
        "## Donations\n",
        "\n",
        "If you're feeling generous, you can donate to my venmo @rynnn while your image loads :) "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2nD1n0xEBcko"
      },
      "source": [
        "# Restart the kernel...\n",
        "\n",
        " and rerun all cells after running the cell below and setting your prompt.\n",
        "\n",
        "\n",
        "**In the top menu press \\<Runtime\\> then \\<Restart and run all\\> to restart everything after this cell stops running**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "etzxXVZ_r-Nf",
        "outputId": "7b3b77c3-b467-44f8-f7b5-646024ad344c"
      },
      "source": [
        "import subprocess\n",
        "\n",
        "CUDA_version = [s for s in subprocess.check_output([\"nvcc\", \"--version\"]).decode(\"UTF-8\").split(\", \") if s.startswith(\"release\")][0].split(\" \")[-1]\n",
        "print(\"CUDA version:\", CUDA_version)\n",
        "\n",
        "if CUDA_version == \"10.0\":\n",
        "    torch_version_suffix = \"+cu100\"\n",
        "elif CUDA_version == \"10.1\":\n",
        "    torch_version_suffix = \"+cu101\"\n",
        "elif CUDA_version == \"10.2\":\n",
        "    torch_version_suffix = \"\"\n",
        "else:\n",
        "    torch_version_suffix = \"+cu110\"\n",
        "\n",
        "!pip install torch==1.7.1{torch_version_suffix} torchvision==0.8.2{torch_version_suffix} -f https://download.pytorch.org/whl/torch_stable.html ftfy regex"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CUDA version: 11.0\n",
            "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
            "Collecting torch==1.7.1+cu110\n",
            "tcmalloc: large alloc 1156751360 bytes == 0x55a069de6000 @  0x7f5ce22041e7 0x55a0669daf48 0x55a0669a59c7 0x55a066b24655 0x55a066abe828 0x55a0669a9292 0x55a066a876ae 0x55a0669a8ee9 0x55a066a9a99d 0x55a066a1cfe9 0x55a0669aa69a 0x55a066a18c9e 0x55a0669aa69a 0x55a066a18c9e 0x55a066a17e0d 0x55a0669ab02c 0x55a0669ab231 0x55a066a1a1e6 0x55a066a17b0e 0x55a0669ab02c 0x55a0669ab231 0x55a066a1a1e6 0x55a066a17b0e 0x55a0669ab02c 0x55a0669ab231 0x55a066a1a1e6 0x55a066a17e0d 0x55a0669ab02c 0x55a0669ab231 0x55a066a1a1e6 0x55a066a17b0e\n",
            "tcmalloc: large alloc 1445945344 bytes == 0x55a0aed10000 @  0x7f5ce2205615 0x55a0669a606c 0x55a066a85eba 0x55a0669a8e8d 0x55a066a9a99d 0x55a066a1cfe9 0x55a066a17b0e 0x55a0669aa77a 0x55a066a18c9e 0x55a066a17b0e 0x55a0669aa77a 0x55a066a18c9e 0x55a066a17b0e 0x55a0669aa77a 0x55a066a18c9e 0x55a066a17b0e 0x55a0669aa77a 0x55a066a18c9e 0x55a066a17b0e 0x55a0669aa77a 0x55a066a18c9e 0x55a0669aa69a 0x55a066a18c9e 0x55a066a17b0e 0x55a0669aa77a 0x55a066a1986a 0x55a066a17b0e 0x55a0669aa77a 0x55a066a18c9e 0x55a066a17e0d 0x55a0669aa77a\n",
            "  Using cached https://download.pytorch.org/whl/cu110/torch-1.7.1%2Bcu110-cp37-cp37m-linux_x86_64.whl\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KAcixx9Z3XYH"
      },
      "source": [
        "# Many Imports\n",
        "\n",
        "Import the usual libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "piJOg9MY7khd",
        "trusted": true
      },
      "source": [
        "import torch\n",
        "import numpy as np\n",
        "import torchvision\n",
        "import torchvision.transforms.functional as TF\n",
        "\n",
        "import PIL\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import os\n",
        "import random\n",
        "import imageio\n",
        "from IPython import display\n",
        "from IPython.core.interactiveshell import InteractiveShell\n",
        "InteractiveShell.ast_node_interactivity = \"all\"\n",
        "\n",
        "import glob\n",
        "\n",
        "from google.colab import output\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EUOAL4UcytFb",
        "outputId": "558df765-1bc4-467e-bab7-ec6cfbbe3b7f"
      },
      "source": [
        "!nvidia-smi -L\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JeMCHwDdHIcu"
      },
      "source": [
        "# Perceptor\n",
        "\n",
        "Get and load up CLIP"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tm3_VmxpAiB1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 562
        },
        "outputId": "a0b96465-b179-40c0-d9f0-495c8dbe209d"
      },
      "source": [
        "\n",
        "\n",
        "%cd /content/\n",
        "\n",
        "!git clone https://github.com/openai/CLIP.git\n",
        "\n",
        "\n",
        "%cd /content/CLIP/\n",
        "\n",
        "!pip install ftfy\n",
        "\n",
        "import os\n",
        "import clip\n",
        "import torch\n",
        "\n",
        "\n",
        "import numpy as np\n",
        "\n",
        "# Load the model\n",
        "perceptor, preprocess = clip.load('ViT-B/32')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content\n",
            "Cloning into 'CLIP'...\n",
            "remote: Enumerating objects: 16, done.\u001b[K\n",
            "remote: Counting objects: 100% (16/16), done.\u001b[K\n",
            "remote: Compressing objects: 100% (11/11), done.\u001b[K\n",
            "remote: Total 90 (delta 6), reused 11 (delta 4), pack-reused 74\u001b[K\n",
            "Unpacking objects: 100% (90/90), done.\n",
            "/content/CLIP\n",
            "Processing /root/.cache/pip/wheels/22/8b/08/7d1c17849e10371206a262304973b5a9f45e8b9d0a2179f465/ftfy-6.0-cp37-none-any.whl\n",
            "Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from ftfy) (0.2.5)\n",
            "Installing collected packages: ftfy\n",
            "Successfully installed ftfy-6.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|███████████████████████████████████████| 354M/354M [00:15<00:00, 22.7MiB/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "error",
          "ename": "RuntimeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-4-ec6ae9d1c968>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     18\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     19\u001b[0m \u001b[0;31m# Load the model\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0mperceptor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpreprocess\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclip\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'ViT-B/32'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/content/CLIP/clip/clip.py\u001b[0m in \u001b[0;36mload\u001b[0;34m(name, device, jit)\u001b[0m\n\u001b[1;32m    131\u001b[0m                     \u001b[0mnode\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcopyAttributes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdevice_node\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    132\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 133\u001b[0;31m     \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpatch_device\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    134\u001b[0m     \u001b[0mpatch_device\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencode_image\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    135\u001b[0m     \u001b[0mpatch_device\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencode_text\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    471\u001b[0m         \"\"\"\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 473\u001b[0;31m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    474\u001b[0m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    471\u001b[0m         \"\"\"\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 473\u001b[0;31m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    474\u001b[0m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    471\u001b[0m         \"\"\"\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 473\u001b[0;31m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    474\u001b[0m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    471\u001b[0m         \"\"\"\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 473\u001b[0;31m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    474\u001b[0m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    471\u001b[0m         \"\"\"\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 473\u001b[0;31m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    474\u001b[0m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    471\u001b[0m         \"\"\"\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 473\u001b[0;31m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    474\u001b[0m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mapply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m    472\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    473\u001b[0m             \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 474\u001b[0;31m         \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    475\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    476\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/content/CLIP/clip/clip.py\u001b[0m in \u001b[0;36mpatch_device\u001b[0;34m(module)\u001b[0m\n\u001b[1;32m    122\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    123\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mpatch_device\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 124\u001b[0;31m         \u001b[0mgraphs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"graph\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    125\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"forward1\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    126\u001b[0m             \u001b[0mgraphs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward1\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/torch/jit/_script.py\u001b[0m in \u001b[0;36mgraph\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    447\u001b[0m             \u001b[0;31m`\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;31m`\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m.\u001b[0m \u001b[0mSee\u001b[0m \u001b[0;34m:\u001b[0m\u001b[0mref\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0minterpreting\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mgraphs\u001b[0m\u001b[0;31m`\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mdetails\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    448\u001b[0m             \"\"\"\n\u001b[0;32m--> 449\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_c\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_method\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"forward\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    450\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    451\u001b[0m         \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mRuntimeError\u001b[0m: Method 'forward' is not defined."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vw3cxCJZDw4w"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mw0KBLebMywW"
      },
      "source": [
        "# Parameters\n",
        "\n",
        "Don't bother changing anything but the prompt below if you're not using a different type of BigGAN"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Nq0wA-wc-P-s",
        "trusted": true
      },
      "source": [
        "im_shape = [512, 512, 3]\n",
        "sideX, sideY, channels = im_shape\n",
        "\n",
        "tx = clip.tokenize('''Living Simulation''')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UlIUu0jK3S19"
      },
      "source": [
        "# Define\n",
        "\n",
        "Some helper functions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gCj1wr8pOqOe",
        "trusted": true
      },
      "source": [
        "def displ(img, pre_scaled=True):\n",
        "  img = np.array(img)[:,:,:]\n",
        "  img = np.transpose(img, (1, 2, 0))\n",
        "  if not pre_scaled:\n",
        "    img = scale(img, 48*4, 32*4)\n",
        "  imageio.imwrite(str(3) + '.png', np.array(img))\n",
        "  return display.Image(str(3)+'.png')\n",
        "\n",
        "def gallery(array, ncols=2):\n",
        "    nindex, height, width, intensity = array.shape\n",
        "    nrows = nindex//ncols\n",
        "    assert nindex == nrows*ncols\n",
        "    # want result.shape = (height*nrows, width*ncols, intensity)\n",
        "    result = (array.reshape(nrows, ncols, height, width, intensity)\n",
        "              .swapaxes(1,2)\n",
        "              .reshape(height*nrows, width*ncols, intensity))\n",
        "    return result\n",
        "\n",
        "def card_padded(im, to_pad=3):\n",
        "  return np.pad(np.pad(np.pad(im, [[1,1], [1,1], [0,0]],constant_values=0), [[2,2], [2,2], [0,0]],constant_values=1),\n",
        "            [[to_pad,to_pad], [to_pad,to_pad], [0,0]],constant_values=0)\n",
        "\n",
        "def get_all(img):\n",
        "  img = np.transpose(img, (0,2,3,1))\n",
        "  cards = np.zeros((img.shape[0], sideX+12, sideY+12, 3))\n",
        "  for i in range(len(img)):\n",
        "    cards[i] = card_padded(img[i])\n",
        "  print(img.shape)\n",
        "  cards = gallery(cards)\n",
        "  imageio.imwrite(str(3) + '.png', np.array(cards))\n",
        "  return display.Image(str(3)+'.png')\n",
        "  "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GHB7BcR1zNLZ"
      },
      "source": [
        "# Generator\n",
        "\n",
        "Alter BigGAN for multiple class & latent inputs\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7EuUz-ICNKUr",
        "cellView": "both",
        "trusted": true
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "def checkin(loss):\n",
        "  print(loss)\n",
        "  best = torch.topk(loss[2], k=1, largest=False)[1]\n",
        "  with torch.no_grad():\n",
        "    al = model(*lats(), 1)[best:best+1].cpu().numpy()\n",
        "  for allls in al:\n",
        "    displ(allls)\n",
        "    display.display(display.Image(str(3)+'.png'))\n",
        "    print('\\n')\n",
        "  output.eval_js('new Audio(\"https://freesound.org/data/previews/80/80921_1022651-lq.ogg\").play()')\n",
        "\n",
        "\n",
        "def ascend_txt():\n",
        "  out = model(*lats(), 1)\n",
        "\n",
        "  cutn = 128\n",
        "  p_s = []\n",
        "  for ch in range(cutn):\n",
        "    size = int(sideX*torch.zeros(1,).normal_(mean=.8, std=.3).clip(.5, .95))\n",
        "    offsetx = torch.randint(0, sideX - size, ())\n",
        "    offsety = torch.randint(0, sideX - size, ())\n",
        "    apper = out[:, :, offsetx:offsetx + size, offsety:offsety + size]\n",
        "    apper = torch.nn.functional.interpolate(apper, (224,224), mode='nearest')\n",
        "    p_s.append(apper)\n",
        "  into = torch.cat(p_s, 0)\n",
        "  # into = torch.nn.functional.interpolate(out, (224,224), mode='nearest')\n",
        "\n",
        "  into = nom((into + 1) / 2)\n",
        "  \n",
        "  iii = perceptor.encode_image(into)\n",
        "\n",
        "\n",
        "  llls = lats()\n",
        "\n",
        "  \n",
        "  lat_l = torch.abs(1 - torch.std(llls[0], dim=1)).mean() + \\\n",
        "  torch.abs(torch.mean(llls[0])).mean() + \\\n",
        "  4*torch.max(torch.square(llls[0]).mean(), lats.thrsh_lat)\n",
        "\n",
        "\n",
        "  for array in llls[0]:\n",
        "    mean = torch.mean(array)\n",
        "    diffs = array - mean\n",
        "    var = torch.mean(torch.pow(diffs, 2.0))\n",
        "    std = torch.pow(var, 0.5)\n",
        "    zscores = diffs / std\n",
        "    skews = torch.mean(torch.pow(zscores, 3.0))\n",
        "    kurtoses = torch.mean(torch.pow(zscores, 4.0)) - 3.0\n",
        "\n",
        "    lat_l = lat_l + torch.abs(kurtoses) / llls[0].shape[0] + torch.abs(skews) / llls[0].shape[0]\n",
        "\n",
        "\n",
        "  \n",
        "\n",
        "  cls_l = ((50*torch.topk(llls[1],largest=False,dim=1,k=999)[0])**2).mean()\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "  return [lat_l, cls_l, -100*torch.cosine_similarity(t, iii, dim=-1).mean()]\n",
        "\n",
        "def train(epoch, i):\n",
        "  loss1 = ascend_txt()\n",
        "  loss = loss1[0] + loss1[1] + loss1[2]\n",
        "  optimizer.zero_grad()\n",
        "  loss.backward()\n",
        "  optimizer.step()\n",
        "  \n",
        "\n",
        "  \n",
        "  \n",
        "  if itt % 100 == 0:\n",
        "    checkin(loss1)\n",
        "\n",
        "\n",
        "itt = 0\n",
        "for epochs in range(10000):\n",
        "  for i in range(50000):\n",
        "    train(eps, i)\n",
        "    itt+=1\n",
        "  eps+=1\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WtlDVVMvzMUd"
      },
      "source": [
        "!pip install pytorch-pretrained-biggan\n",
        "\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import math\n",
        "import json\n",
        "import logging\n",
        "import os\n",
        "import shutil\n",
        "import tempfile\n",
        "from functools import wraps\n",
        "from hashlib import sha256\n",
        "import sys\n",
        "from io import open\n",
        "\n",
        "import boto3\n",
        "import requests\n",
        "from botocore.exceptions import ClientError\n",
        "from tqdm import tqdm\n",
        "\n",
        "try:\n",
        "    from urllib.parse import urlparse\n",
        "except ImportError:\n",
        "    from urlparse import urlparse\n",
        "\n",
        "try:\n",
        "    from pathlib import Path\n",
        "    PYTORCH_PRETRAINED_BIGGAN_CACHE = Path(os.getenv('PYTORCH_PRETRAINED_BIGGAN_CACHE',\n",
        "                                                   Path.home() / '.pytorch_pretrained_biggan'))\n",
        "except (AttributeError, ImportError):\n",
        "    PYTORCH_PRETRAINED_BIGGAN_CACHE = os.getenv('PYTORCH_PRETRAINED_BIGGAN_CACHE',\n",
        "                                              os.path.join(os.path.expanduser(\"~\"), '.pytorch_pretrained_biggan'))\n",
        "\n",
        "logger = logging.getLogger(__name__)  # pylint: disable=invalid-name\n",
        "\n",
        "\n",
        "def url_to_filename(url, etag=None):\n",
        "    \"\"\"\n",
        "    Convert `url` into a hashed filename in a repeatable way.\n",
        "    If `etag` is specified, append its hash to the url's, delimited\n",
        "    by a period.\n",
        "    \"\"\"\n",
        "    url_bytes = url.encode('utf-8')\n",
        "    url_hash = sha256(url_bytes)\n",
        "    filename = url_hash.hexdigest()\n",
        "\n",
        "    if etag:\n",
        "        etag_bytes = etag.encode('utf-8')\n",
        "        etag_hash = sha256(etag_bytes)\n",
        "        filename += '.' + etag_hash.hexdigest()\n",
        "\n",
        "    return filename\n",
        "\n",
        "\n",
        "def filename_to_url(filename, cache_dir=None):\n",
        "    \"\"\"\n",
        "    Return the url and etag (which may be ``None``) stored for `filename`.\n",
        "    Raise ``EnvironmentError`` if `filename` or its stored metadata do not exist.\n",
        "    \"\"\"\n",
        "    if cache_dir is None:\n",
        "        cache_dir = PYTORCH_PRETRAINED_BIGGAN_CACHE\n",
        "    if sys.version_info[0] == 3 and isinstance(cache_dir, Path):\n",
        "        cache_dir = str(cache_dir)\n",
        "\n",
        "    cache_path = os.path.join(cache_dir, filename)\n",
        "    if not os.path.exists(cache_path):\n",
        "        raise EnvironmentError(\"file {} not found\".format(cache_path))\n",
        "\n",
        "    meta_path = cache_path + '.json'\n",
        "    if not os.path.exists(meta_path):\n",
        "        raise EnvironmentError(\"file {} not found\".format(meta_path))\n",
        "\n",
        "    with open(meta_path, encoding=\"utf-8\") as meta_file:\n",
        "        metadata = json.load(meta_file)\n",
        "    url = metadata['url']\n",
        "    etag = metadata['etag']\n",
        "\n",
        "    return url, etag\n",
        "\n",
        "\n",
        "def cached_path(url_or_filename, cache_dir=None):\n",
        "    \"\"\"\n",
        "    Given something that might be a URL (or might be a local path),\n",
        "    determine which. If it's a URL, download the file and cache it, and\n",
        "    return the path to the cached file. If it's already a local path,\n",
        "    make sure the file exists and then return the path.\n",
        "    \"\"\"\n",
        "    if cache_dir is None:\n",
        "        cache_dir = PYTORCH_PRETRAINED_BIGGAN_CACHE\n",
        "    if sys.version_info[0] == 3 and isinstance(url_or_filename, Path):\n",
        "        url_or_filename = str(url_or_filename)\n",
        "    if sys.version_info[0] == 3 and isinstance(cache_dir, Path):\n",
        "        cache_dir = str(cache_dir)\n",
        "\n",
        "    parsed = urlparse(url_or_filename)\n",
        "\n",
        "    if parsed.scheme in ('http', 'https', 's3'):\n",
        "        # URL, so get it from the cache (downloading if necessary)\n",
        "        return get_from_cache(url_or_filename, cache_dir)\n",
        "    elif os.path.exists(url_or_filename):\n",
        "        # File, and it exists.\n",
        "        return url_or_filename\n",
        "    elif parsed.scheme == '':\n",
        "        # File, but it doesn't exist.\n",
        "        raise EnvironmentError(\"file {} not found\".format(url_or_filename))\n",
        "    else:\n",
        "        # Something unknown\n",
        "        raise ValueError(\"unable to parse {} as a URL or as a local path\".format(url_or_filename))\n",
        "\n",
        "\n",
        "def split_s3_path(url):\n",
        "    \"\"\"Split a full s3 path into the bucket name and path.\"\"\"\n",
        "    parsed = urlparse(url)\n",
        "    if not parsed.netloc or not parsed.path:\n",
        "        raise ValueError(\"bad s3 path {}\".format(url))\n",
        "    bucket_name = parsed.netloc\n",
        "    s3_path = parsed.path\n",
        "    # Remove '/' at beginning of path.\n",
        "    if s3_path.startswith(\"/\"):\n",
        "        s3_path = s3_path[1:]\n",
        "    return bucket_name, s3_path\n",
        "\n",
        "\n",
        "def s3_request(func):\n",
        "    \"\"\"\n",
        "    Wrapper function for s3 requests in order to create more helpful error\n",
        "    messages.\n",
        "    \"\"\"\n",
        "\n",
        "    @wraps(func)\n",
        "    def wrapper(url, *args, **kwargs):\n",
        "        try:\n",
        "            return func(url, *args, **kwargs)\n",
        "        except ClientError as exc:\n",
        "            if int(exc.response[\"Error\"][\"Code\"]) == 404:\n",
        "                raise EnvironmentError(\"file {} not found\".format(url))\n",
        "            else:\n",
        "                raise\n",
        "\n",
        "    return wrapper\n",
        "\n",
        "\n",
        "@s3_request\n",
        "def s3_etag(url):\n",
        "    \"\"\"Check ETag on S3 object.\"\"\"\n",
        "    s3_resource = boto3.resource(\"s3\")\n",
        "    bucket_name, s3_path = split_s3_path(url)\n",
        "    s3_object = s3_resource.Object(bucket_name, s3_path)\n",
        "    return s3_object.e_tag\n",
        "\n",
        "\n",
        "@s3_request\n",
        "def s3_get(url, temp_file):\n",
        "    \"\"\"Pull a file directly from S3.\"\"\"\n",
        "    s3_resource = boto3.resource(\"s3\")\n",
        "    bucket_name, s3_path = split_s3_path(url)\n",
        "    s3_resource.Bucket(bucket_name).download_fileobj(s3_path, temp_file)\n",
        "\n",
        "\n",
        "def http_get(url, temp_file):\n",
        "    req = requests.get(url, stream=True)\n",
        "    content_length = req.headers.get('Content-Length')\n",
        "    total = int(content_length) if content_length is not None else None\n",
        "    progress = tqdm(unit=\"B\", total=total)\n",
        "    for chunk in req.iter_content(chunk_size=1024):\n",
        "        if chunk: # filter out keep-alive new chunks\n",
        "            progress.update(len(chunk))\n",
        "            temp_file.write(chunk)\n",
        "    progress.close()\n",
        "\n",
        "\n",
        "def get_from_cache(url, cache_dir=None):\n",
        "    \"\"\"\n",
        "    Given a URL, look for the corresponding dataset in the local cache.\n",
        "    If it's not there, download it. Then return the path to the cached file.\n",
        "    \"\"\"\n",
        "    if cache_dir is None:\n",
        "        cache_dir = PYTORCH_PRETRAINED_BIGGAN_CACHE\n",
        "    if sys.version_info[0] == 3 and isinstance(cache_dir, Path):\n",
        "        cache_dir = str(cache_dir)\n",
        "\n",
        "    if not os.path.exists(cache_dir):\n",
        "        os.makedirs(cache_dir)\n",
        "\n",
        "    # Get eTag to add to filename, if it exists.\n",
        "    if url.startswith(\"s3://\"):\n",
        "        etag = s3_etag(url)\n",
        "    else:\n",
        "        response = requests.head(url, allow_redirects=True)\n",
        "        if response.status_code != 200:\n",
        "            raise IOError(\"HEAD request failed for url {} with status code {}\"\n",
        "                          .format(url, response.status_code))\n",
        "        etag = response.headers.get(\"ETag\")\n",
        "\n",
        "    filename = url_to_filename(url, etag)\n",
        "\n",
        "    # get cache path to put the file\n",
        "    cache_path = os.path.join(cache_dir, filename)\n",
        "\n",
        "    if not os.path.exists(cache_path):\n",
        "        # Download to temporary file, then copy to cache dir once finished.\n",
        "        # Otherwise you get corrupt cache entries if the download gets interrupted.\n",
        "        with tempfile.NamedTemporaryFile() as temp_file:\n",
        "            logger.info(\"%s not found in cache, downloading to %s\", url, temp_file.name)\n",
        "\n",
        "            # GET file object\n",
        "            if url.startswith(\"s3://\"):\n",
        "                s3_get(url, temp_file)\n",
        "            else:\n",
        "                http_get(url, temp_file)\n",
        "\n",
        "            # we are copying the file before closing it, so flush to avoid truncation\n",
        "            temp_file.flush()\n",
        "            # shutil.copyfileobj() starts at the current position, so go to the start\n",
        "            temp_file.seek(0)\n",
        "\n",
        "            logger.info(\"copying %s to cache at %s\", temp_file.name, cache_path)\n",
        "            with open(cache_path, 'wb') as cache_file:\n",
        "                shutil.copyfileobj(temp_file, cache_file)\n",
        "\n",
        "            logger.info(\"creating metadata file for %s\", cache_path)\n",
        "            meta = {'url': url, 'etag': etag}\n",
        "            meta_path = cache_path + '.json'\n",
        "            with open(meta_path, 'w', encoding=\"utf-8\") as meta_file:\n",
        "                json.dump(meta, meta_file)\n",
        "\n",
        "            logger.info(\"removing temp file %s\", temp_file.name)\n",
        "\n",
        "    return cache_path\n",
        "\n",
        "\n",
        "def read_set_from_file(filename):\n",
        "    '''\n",
        "    Extract a de-duped collection (set) of text from a file.\n",
        "    Expected file format is one item per line.\n",
        "    '''\n",
        "    collection = set()\n",
        "    with open(filename, 'r', encoding='utf-8') as file_:\n",
        "        for line in file_:\n",
        "            collection.add(line.rstrip())\n",
        "    return collection\n",
        "\n",
        "\n",
        "def get_file_extension(path, dot=True, lower=True):\n",
        "    ext = os.path.splitext(path)[1]\n",
        "    ext = ext if dot else ext[1:]\n",
        "    return ext.lower() if lower else ext\n",
        "\n",
        "\n",
        "PRETRAINED_MODEL_ARCHIVE_MAP = {\n",
        "    'biggan-deep-128': \"https://s3.amazonaws.com/models.huggingface.co/biggan/biggan-deep-128-pytorch_model.bin\",\n",
        "    'biggan-deep-256': \"https://s3.amazonaws.com/models.huggingface.co/biggan/biggan-deep-256-pytorch_model.bin\",\n",
        "    'biggan-deep-512': \"https://s3.amazonaws.com/models.huggingface.co/biggan/biggan-deep-512-pytorch_model.bin\",\n",
        "}\n",
        "\n",
        "PRETRAINED_CONFIG_ARCHIVE_MAP = {\n",
        "    'biggan-deep-128': \"https://s3.amazonaws.com/models.huggingface.co/biggan/biggan-deep-128-config.json\",\n",
        "    'biggan-deep-256': \"https://s3.amazonaws.com/models.huggingface.co/biggan/biggan-deep-256-config.json\",\n",
        "    'biggan-deep-512': \"https://s3.amazonaws.com/models.huggingface.co/biggan/biggan-deep-512-config.json\",\n",
        "}\n",
        "\n",
        "WEIGHTS_NAME = 'pytorch_model.bin'\n",
        "CONFIG_NAME = 'config.json'\n",
        "import copy\n",
        "import json\n",
        "\n",
        "class BigGANConfig(object):\n",
        "    \"\"\" Configuration class to store the configuration of a `BigGAN`. \n",
        "        Defaults are for the 128x128 model.\n",
        "        layers tuple are (up-sample in the layer ?, input channels, output channels)\n",
        "    \"\"\"\n",
        "    def __init__(self,\n",
        "                 output_dim=128,\n",
        "                 z_dim=128,\n",
        "                 class_embed_dim=128,\n",
        "                 channel_width=128,\n",
        "                 num_classes=1000,\n",
        "                 layers=[(False, 16, 16),\n",
        "                         (True, 16, 16),\n",
        "                         (False, 16, 16),\n",
        "                         (True, 16, 8),\n",
        "                         (False, 8, 8),\n",
        "                         (True, 8, 4),\n",
        "                         (False, 4, 4),\n",
        "                         (True, 4, 2),\n",
        "                         (False, 2, 2),\n",
        "                         (True, 2, 1)],\n",
        "                 attention_layer_position=8,\n",
        "                 eps=1e-4,\n",
        "                 n_stats=51):\n",
        "        \"\"\"Constructs BigGANConfig. \"\"\"\n",
        "        self.output_dim = output_dim\n",
        "        self.z_dim = z_dim\n",
        "        self.class_embed_dim = class_embed_dim\n",
        "        self.channel_width = channel_width\n",
        "        self.num_classes = num_classes\n",
        "        self.layers = layers\n",
        "        self.attention_layer_position = attention_layer_position\n",
        "        self.eps = eps\n",
        "        self.n_stats = n_stats\n",
        "\n",
        "    @classmethod\n",
        "    def from_dict(cls, json_object):\n",
        "        \"\"\"Constructs a `BigGANConfig` from a Python dictionary of parameters.\"\"\"\n",
        "        config = BigGANConfig()\n",
        "        for key, value in json_object.items():\n",
        "            config.__dict__[key] = value\n",
        "        return config\n",
        "\n",
        "    @classmethod\n",
        "    def from_json_file(cls, json_file):\n",
        "        \"\"\"Constructs a `BigGANConfig` from a json file of parameters.\"\"\"\n",
        "        with open(json_file, \"r\", encoding='utf-8') as reader:\n",
        "            text = reader.read()\n",
        "        return cls.from_dict(json.loads(text))\n",
        "\n",
        "    def __repr__(self):\n",
        "        return str(self.to_json_string())\n",
        "\n",
        "    def to_dict(self):\n",
        "        \"\"\"Serializes this instance to a Python dictionary.\"\"\"\n",
        "        output = copy.deepcopy(self.__dict__)\n",
        "        return output\n",
        "\n",
        "    def to_json_string(self):\n",
        "        \"\"\"Serializes this instance to a JSON string.\"\"\"\n",
        "        return json.dumps(self.to_dict(), indent=2, sort_keys=True) + \"\\n\"\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "def snconv2d(eps=1e-12, **kwargs):\n",
        "    return nn.utils.spectral_norm(nn.Conv2d(**kwargs), eps=eps)\n",
        "\n",
        "def snlinear(eps=1e-12, **kwargs):\n",
        "    return nn.utils.spectral_norm(nn.Linear(**kwargs), eps=eps)\n",
        "\n",
        "def sn_embedding(eps=1e-12, **kwargs):\n",
        "    return nn.utils.spectral_norm(nn.Embedding(**kwargs), eps=eps)\n",
        "\n",
        "class SelfAttn(nn.Module):\n",
        "    \"\"\" Self attention Layer\"\"\"\n",
        "    def __init__(self, in_channels, eps=1e-12):\n",
        "        super(SelfAttn, self).__init__()\n",
        "        self.in_channels = in_channels\n",
        "        self.snconv1x1_theta = snconv2d(in_channels=in_channels, out_channels=in_channels//8,\n",
        "                                        kernel_size=1, bias=False, eps=eps)\n",
        "        self.snconv1x1_phi = snconv2d(in_channels=in_channels, out_channels=in_channels//8,\n",
        "                                      kernel_size=1, bias=False, eps=eps)\n",
        "        self.snconv1x1_g = snconv2d(in_channels=in_channels, out_channels=in_channels//2,\n",
        "                                    kernel_size=1, bias=False, eps=eps)\n",
        "        self.snconv1x1_o_conv = snconv2d(in_channels=in_channels//2, out_channels=in_channels,\n",
        "                                         kernel_size=1, bias=False, eps=eps)\n",
        "        self.maxpool = nn.MaxPool2d(2, stride=2, padding=0)\n",
        "        self.softmax  = nn.Softmax(dim=-1)\n",
        "        self.gamma = nn.Parameter(torch.zeros(1))\n",
        "\n",
        "    def forward(self, x):\n",
        "        _, ch, h, w = x.size()\n",
        "        # Theta path\n",
        "        theta = self.snconv1x1_theta(x)\n",
        "        theta = theta.view(-1, ch//8, h*w)\n",
        "        # Phi path\n",
        "        phi = self.snconv1x1_phi(x)\n",
        "        phi = self.maxpool(phi)\n",
        "        phi = phi.view(-1, ch//8, h*w//4)\n",
        "        # Attn map\n",
        "        attn = torch.bmm(theta.permute(0, 2, 1), phi)\n",
        "        attn = self.softmax(attn)\n",
        "        # g path\n",
        "        g = self.snconv1x1_g(x)\n",
        "        g = self.maxpool(g)\n",
        "        g = g.view(-1, ch//2, h*w//4)\n",
        "        # Attn_g - o_conv\n",
        "        attn_g = torch.bmm(g, attn.permute(0, 2, 1))\n",
        "        attn_g = attn_g.view(-1, ch//2, h, w)\n",
        "        attn_g = self.snconv1x1_o_conv(attn_g)\n",
        "        # Out\n",
        "        out = x + self.gamma*attn_g\n",
        "        return out\n",
        "\n",
        "\n",
        "class BigGANBatchNorm(nn.Module):\n",
        "    \"\"\" This is a batch norm module that can handle conditional input and can be provided with pre-computed\n",
        "        activation means and variances for various truncation parameters.\n",
        "        We cannot just rely on torch.batch_norm since it cannot handle\n",
        "        batched weights (pytorch 1.0.1). We computate batch_norm our-self without updating running means and variances.\n",
        "        If you want to train this model you should add running means and variance computation logic.\n",
        "    \"\"\"\n",
        "    def __init__(self, num_features, condition_vector_dim=None, n_stats=51, eps=1e-4, conditional=True):\n",
        "        super(BigGANBatchNorm, self).__init__()\n",
        "        self.num_features = num_features\n",
        "        self.eps = eps\n",
        "        self.conditional = conditional\n",
        "\n",
        "        # We use pre-computed statistics for n_stats values of truncation between 0 and 1\n",
        "        self.register_buffer('running_means', torch.zeros(n_stats, num_features))\n",
        "        self.register_buffer('running_vars', torch.ones(n_stats, num_features))\n",
        "        self.step_size = 1.0 / (n_stats - 1)\n",
        "\n",
        "        if conditional:\n",
        "            assert condition_vector_dim is not None\n",
        "            self.scale = snlinear(in_features=condition_vector_dim, out_features=num_features, bias=False, eps=eps)\n",
        "            self.offset = snlinear(in_features=condition_vector_dim, out_features=num_features, bias=False, eps=eps)\n",
        "        else:\n",
        "            self.weight = torch.nn.Parameter(torch.Tensor(num_features))\n",
        "            self.bias = torch.nn.Parameter(torch.Tensor(num_features))\n",
        "\n",
        "    def forward(self, x, truncation, condition_vector=None):\n",
        "        # Retreive pre-computed statistics associated to this truncation\n",
        "        coef, start_idx = math.modf(truncation / self.step_size)\n",
        "        start_idx = int(start_idx)\n",
        "        if coef != 0.0:  # Interpolate\n",
        "            running_mean = self.running_means[start_idx] * coef + self.running_means[start_idx + 1] * (1 - coef)\n",
        "            running_var = self.running_vars[start_idx] * coef + self.running_vars[start_idx + 1] * (1 - coef)\n",
        "        else:\n",
        "            running_mean = self.running_means[start_idx]\n",
        "            running_var = self.running_vars[start_idx]\n",
        "\n",
        "        if self.conditional:\n",
        "            running_mean = running_mean.unsqueeze(0).unsqueeze(-1).unsqueeze(-1)\n",
        "            running_var = running_var.unsqueeze(0).unsqueeze(-1).unsqueeze(-1)\n",
        "\n",
        "            weight = 1 + self.scale(condition_vector).unsqueeze(-1).unsqueeze(-1)\n",
        "            bias = self.offset(condition_vector).unsqueeze(-1).unsqueeze(-1)\n",
        "\n",
        "            out = (x - running_mean) / torch.sqrt(running_var + self.eps) * weight + bias\n",
        "        else:\n",
        "            out = F.batch_norm(x, running_mean, running_var, self.weight, self.bias,\n",
        "                               training=False, momentum=0.0, eps=self.eps)\n",
        "\n",
        "        return out\n",
        "\n",
        "\n",
        "class GenBlock(nn.Module):\n",
        "    def __init__(self, in_size, out_size, condition_vector_dim, reduction_factor=4, up_sample=False,\n",
        "                 n_stats=51, eps=1e-12):\n",
        "        super(GenBlock, self).__init__()\n",
        "        self.up_sample = up_sample\n",
        "        self.drop_channels = (in_size != out_size)\n",
        "        middle_size = in_size // reduction_factor\n",
        "\n",
        "        self.bn_0 = BigGANBatchNorm(in_size, condition_vector_dim, n_stats=n_stats, eps=eps, conditional=True)\n",
        "        self.conv_0 = snconv2d(in_channels=in_size, out_channels=middle_size, kernel_size=1, eps=eps)\n",
        "\n",
        "        self.bn_1 = BigGANBatchNorm(middle_size, condition_vector_dim, n_stats=n_stats, eps=eps, conditional=True)\n",
        "        self.conv_1 = snconv2d(in_channels=middle_size, out_channels=middle_size, kernel_size=3, padding=1, eps=eps)\n",
        "\n",
        "        self.bn_2 = BigGANBatchNorm(middle_size, condition_vector_dim, n_stats=n_stats, eps=eps, conditional=True)\n",
        "        self.conv_2 = snconv2d(in_channels=middle_size, out_channels=middle_size, kernel_size=3, padding=1, eps=eps)\n",
        "\n",
        "        self.bn_3 = BigGANBatchNorm(middle_size, condition_vector_dim, n_stats=n_stats, eps=eps, conditional=True)\n",
        "        self.conv_3 = snconv2d(in_channels=middle_size, out_channels=out_size, kernel_size=1, eps=eps)\n",
        "\n",
        "        self.relu = nn.ReLU()\n",
        "\n",
        "    def forward(self, x, cond_vector, truncation):\n",
        "        x0 = x\n",
        "\n",
        "        x = self.bn_0(x, truncation, cond_vector)\n",
        "        x = self.relu(x)\n",
        "        x = self.conv_0(x)\n",
        "\n",
        "        x = self.bn_1(x, truncation, cond_vector)\n",
        "        x = self.relu(x)\n",
        "        if self.up_sample:\n",
        "            x = F.interpolate(x, scale_factor=2, mode='nearest')\n",
        "        x = self.conv_1(x)\n",
        "\n",
        "        x = self.bn_2(x, truncation, cond_vector)\n",
        "        x = self.relu(x)\n",
        "        x = self.conv_2(x)\n",
        "\n",
        "        x = self.bn_3(x, truncation, cond_vector)\n",
        "        x = self.relu(x)\n",
        "        x = self.conv_3(x)\n",
        "\n",
        "        if self.drop_channels:\n",
        "            new_channels = x0.shape[1] // 2\n",
        "            x0 = x0[:, :new_channels, ...]\n",
        "        if self.up_sample:\n",
        "            x0 = F.interpolate(x0, scale_factor=2, mode='nearest')\n",
        "\n",
        "        out = x + x0\n",
        "        return out\n",
        "\n",
        "class Generator(nn.Module):\n",
        "    def __init__(self, config):\n",
        "        super(Generator, self).__init__()\n",
        "        self.config = config\n",
        "        ch = config.channel_width\n",
        "        condition_vector_dim = config.z_dim * 2\n",
        "\n",
        "        self.gen_z = snlinear(in_features=condition_vector_dim,\n",
        "                              out_features=4 * 4 * 16 * ch, eps=config.eps)\n",
        "\n",
        "        layers = []\n",
        "        for i, layer in enumerate(config.layers):\n",
        "            if i == config.attention_layer_position:\n",
        "                layers.append(SelfAttn(ch*layer[1], eps=config.eps))\n",
        "            layers.append(GenBlock(ch*layer[1],\n",
        "                                   ch*layer[2],\n",
        "                                   condition_vector_dim,\n",
        "                                   up_sample=layer[0],\n",
        "                                   n_stats=config.n_stats,\n",
        "                                   eps=config.eps))\n",
        "        self.layers = nn.ModuleList(layers)\n",
        "\n",
        "        self.bn = BigGANBatchNorm(ch, n_stats=config.n_stats, eps=config.eps, conditional=False)\n",
        "        self.relu = nn.ReLU()\n",
        "        self.conv_to_rgb = snconv2d(in_channels=ch, out_channels=ch, kernel_size=3, padding=1, eps=config.eps)\n",
        "        self.tanh = nn.Tanh()\n",
        "\n",
        "    def forward(self, cond_vector, truncation):\n",
        "        z = self.gen_z(cond_vector[0].unsqueeze(0))\n",
        "\n",
        "        # We use this conversion step to be able to use TF weights:\n",
        "        # TF convention on shape is [batch, height, width, channels]\n",
        "        # PT convention on shape is [batch, channels, height, width]\n",
        "        z = z.view(-1, 4, 4, 16 * self.config.channel_width)\n",
        "        z = z.permute(0, 3, 1, 2).contiguous()\n",
        "\n",
        "        for i, layer in enumerate(self.layers):\n",
        "            if isinstance(layer, GenBlock):\n",
        "                z = layer(z, cond_vector[i+1].unsqueeze(0), truncation)\n",
        "                # z = layer(z, cond_vector[].unsqueeze(0), truncation)\n",
        "            else:\n",
        "                z = layer(z)\n",
        "\n",
        "        z = self.bn(z, truncation)\n",
        "        z = self.relu(z)\n",
        "        z = self.conv_to_rgb(z)\n",
        "        z = z[:, :3, ...]\n",
        "        z = self.tanh(z)\n",
        "        return z\n",
        "\n",
        "class BigGAN(nn.Module):\n",
        "    \"\"\"BigGAN Generator.\"\"\"\n",
        "\n",
        "    @classmethod\n",
        "    def from_pretrained(cls, pretrained_model_name_or_path, cache_dir=None, *inputs, **kwargs):\n",
        "        if pretrained_model_name_or_path in PRETRAINED_MODEL_ARCHIVE_MAP:\n",
        "            model_file = PRETRAINED_MODEL_ARCHIVE_MAP[pretrained_model_name_or_path]\n",
        "            config_file = PRETRAINED_CONFIG_ARCHIVE_MAP[pretrained_model_name_or_path]\n",
        "        else:\n",
        "            model_file = os.path.join(pretrained_model_name_or_path, WEIGHTS_NAME)\n",
        "            config_file = os.path.join(pretrained_model_name_or_path, CONFIG_NAME)\n",
        "\n",
        "        try:\n",
        "            resolved_model_file = cached_path(model_file, cache_dir=cache_dir)\n",
        "            resolved_config_file = cached_path(config_file, cache_dir=cache_dir)\n",
        "        except EnvironmentError:\n",
        "            logger.error(\"Wrong model name, should be a valid path to a folder containing \"\n",
        "                         \"a {} file and a {} file or a model name in {}\".format(\n",
        "                         WEIGHTS_NAME, CONFIG_NAME, PRETRAINED_MODEL_ARCHIVE_MAP.keys()))\n",
        "            raise\n",
        "\n",
        "        logger.info(\"loading model {} from cache at {}\".format(pretrained_model_name_or_path, resolved_model_file))\n",
        "\n",
        "        # Load config\n",
        "        config = BigGANConfig.from_json_file(resolved_config_file)\n",
        "        logger.info(\"Model config {}\".format(config))\n",
        "\n",
        "        # Instantiate model.\n",
        "        model = cls(config, *inputs, **kwargs)\n",
        "        state_dict = torch.load(resolved_model_file, map_location='cpu' if not torch.cuda.is_available() else None)\n",
        "        model.load_state_dict(state_dict, strict=False)\n",
        "        return model\n",
        "\n",
        "    def __init__(self, config):\n",
        "        super(BigGAN, self).__init__()\n",
        "        self.config = config\n",
        "        self.embeddings = nn.Linear(config.num_classes, config.z_dim, bias=False)\n",
        "        self.generator = Generator(config)\n",
        "\n",
        "    def forward(self, z, class_label, truncation):\n",
        "        assert 0 < truncation <= 1\n",
        "\n",
        "        embed = self.embeddings(class_label)\n",
        "        cond_vector = torch.cat((z, embed), dim=1)\n",
        "\n",
        "        z = self.generator(cond_vector, truncation)\n",
        "        return z\n",
        "\n",
        "\n",
        "\n",
        "model = BigGAN.from_pretrained('biggan-deep-512')\n",
        "model = model.cuda().eval()\n",
        "\n",
        "\n",
        "def one_hot_from_int(int_or_list, batch_size=1):\n",
        "    \"\"\" Create a one-hot vector from a class index or a list of class indices.\n",
        "        Params:\n",
        "            int_or_list: int, or list of int, of the imagenet classes (between 0 and 999)\n",
        "            batch_size: batch size.\n",
        "                If int_or_list is an int create a batch of identical classes.\n",
        "                If int_or_list is a list, we should have `len(int_or_list) == batch_size`\n",
        "        Output:\n",
        "            array of shape (batch_size, 1000)\n",
        "    \"\"\"\n",
        "    if isinstance(int_or_list, int):\n",
        "        int_or_list = [int_or_list]\n",
        "\n",
        "    if len(int_or_list) == 1 and batch_size > 1:\n",
        "        int_or_list = [int_or_list[0]] * batch_size\n",
        "\n",
        "    assert batch_size == len(int_or_list)\n",
        "\n",
        "    array = np.zeros((batch_size, 1000), dtype=np.float32)\n",
        "    for i, j in enumerate(int_or_list):\n",
        "        array[i, j] = 1.0\n",
        "    return array\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1-aCuJ7SD7Oh"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XaocGDQXz3Zx"
      },
      "source": [
        "# Latent coordinate\n",
        "\n",
        "Choose a place to start in BigGAN (it'll be a dog. Probably a hound lol)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GdCh2D8Dt8Xd"
      },
      "source": [
        "class Pars(torch.nn.Module):\n",
        "    def __init__(self):\n",
        "        super(Pars, self).__init__()\n",
        "        # params = torch.zeros(32, 1).normal_(std=1)\n",
        "\n",
        "        # self.stds = torch.nn.Parameter(params)\n",
        "\n",
        "        # params = torch.zeros(32, 1).normal_(std=.1)\n",
        "        # self.means = torch.nn.Parameter(params)\n",
        "        \n",
        "        self.normu = torch.nn.Parameter(torch.zeros(32, 128).normal_(std=1).cuda())\n",
        "\n",
        "\n",
        "        params_other = torch.zeros(32, 1000).normal_(-3.9, .3)\n",
        "        self.cls = torch.nn.Parameter(params_other)\n",
        "\n",
        "        self.thrsh_lat = torch.tensor(1).cuda()\n",
        "\n",
        "        self.thrsh_cls = torch.tensor(1.9).cuda()\n",
        "\n",
        "\n",
        "\n",
        "    def forward(self):\n",
        "      # return self.ff2(self.ff1(self.latent_code)), torch.softmax(1000*self.ff4(self.ff3(self.cls)), -1)\n",
        "      return self.normu, torch.sigmoid(self.cls)\n",
        "\n",
        "\n",
        "torch.manual_seed(0)\n",
        "\n",
        "\n",
        "lats = Pars().cuda()\n",
        "optimizer = torch.optim.Adam(lats.parameters(), .07)\n",
        "eps = 0\n",
        "\n",
        "nom = torchvision.transforms.Normalize((0.48145466, 0.4578275, 0.40821073), (0.26862954, 0.26130258, 0.27577711))\n",
        "t = perceptor.encode_text(tx.cuda()).detach().clone()\n",
        "\n",
        "with torch.no_grad():\n",
        "  al = (model(*lats(), 1).cpu()).numpy()\n",
        "  print(lats())\n",
        "  for allls in al:\n",
        "    displ(allls)\n",
        "    print('\\n')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yJkNIRhU2qIf"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WztSrRF23Rqg"
      },
      "source": [
        "# Train\n",
        "\n",
        "Train and generate samples"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RLWOiUNuWu_X"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b3DmDF7Dq5uh"
      },
      "source": [
        "# Diagnostics\n",
        "\n",
        "If things don't work, try tweaking the learning rate. If that doesn't work, tweak the losses or even try changing the torch manual seed. The graphs below show the distributions of the class and latent vectors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7iWSEKsV9SJv"
      },
      "source": [
        "with torch.no_grad():\n",
        "  al = model(lats()[0], torch.zeros_like(lats()[1]), 1).cpu().numpy()\n",
        "for allls in al:\n",
        "  displ(allls)\n",
        "  print('\\n')\n",
        "\n",
        "with torch.no_grad():\n",
        "  al = model(torch.zeros_like(lats()[0]).normal_(), lats()[1], 1).cpu().numpy()\n",
        "for allls in al:\n",
        "  displ(allls)\n",
        "  print('\\n')\n",
        "\n",
        "\n",
        "with torch.no_grad():\n",
        "  these = tuple((lats()[0][:1].expand(32, -1), lats()[1][:1].expand(32, -1)))\n",
        "  al = model(*these, 1).cpu().numpy()\n",
        "for allls in al:\n",
        "  displ(allls)\n",
        "  print('\\n')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2H-ftPmUD9Bd"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZbjKZoZsw_Mm"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "with torch.no_grad():\n",
        "  plt.close()\n",
        "  th = plt.hist(lats()[0].detach().cpu().numpy().tolist())\n",
        "  plt.show(th)\n",
        "  plt.close()\n",
        "\n",
        "  plt.show(plt.hist(lats()[1].detach().cpu().numpy().tolist()))\n",
        "  print(torch.mean(lats()[0]))\n",
        "  print(torch.mean(lats()[1]))\n",
        "  print(lats()[0])\n",
        "  print(lats()[1])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cDUNftahX1lc"
      },
      "source": [
        "with torch.no_grad():\n",
        "  al = nom((1 + model(*lats(), 1).cpu()) / 2).numpy()\n",
        "  for allls in al:\n",
        "    displ(allls)\n",
        "    print('\\n')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1a_fjw3NTgTm"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d83p_35Frdb-"
      },
      "source": [
        "# Citations\n",
        "\n",
        "Thanks to the creators of BigGAN and to huggingface!\n",
        "\n",
        "https://github.com/huggingface/pytorch-pretrained-BigGAN\n",
        "\n",
        "@inproceedings{\n",
        "brock2018large,\n",
        "title={Large Scale {GAN} Training for High Fidelity Natural Image Synthesis},\n",
        "author={Andrew Brock and Jeff Donahue and Karen Simonyan},\n",
        "booktitle={International Conference on Learning Representations},\n",
        "year={2019},\n",
        "url={https://openreview.net/forum?id=B1xsqj09Fm},\n",
        "}\n",
        "\n",
        "\n",
        "Thanks to the authors below & OpenAI for sharing CLIP! https://github.com/openai/CLIP\n",
        "\n",
        "Alec Radford \\* Jong Wook Kim \\* Chris Hallacy Aditya Ramesh Gabriel Goh Sandhini Agarwal\n",
        "Girish Sastry Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger\n",
        "Ilya Sutskever\n",
        "\n",
        "As a good launching point for future directions and to find more related work, see https://distill.pub/2017/feature-visualization/\n",
        "\n",
        "Thanks to Alex Mordvintsev for creating Deep Dream, which this pulls on in terms of name and concept as well.\n",
        "\n",
        "Props to Ceyuan Yang, Yujun Shen, and Bolei Zhou for separating the latent vector by layer! https://openreview.net/forum?id=Syxp-1HtvB\n",
        "\n",
        "& GANspace as well\n",
        "\n",
        "@inproceedings{härkönen2020ganspace,\n",
        "  title     = {GANSpace: Discovering Interpretable GAN Controls},\n",
        "  author    = {Erik Härkönen and Aaron Hertzmann and Jaakko Lehtinen and Sylvain Paris},\n",
        "  booktitle = {Proc. NeurIPS},\n",
        "  year      = {2020}\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qM82-7XLbDIS"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YQ3AK_iIjwaw"
      },
      "source": [
        "# Bottom"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ctw6C-W-aUoa"
      },
      "source": [
        "\n",
        "tx = clip.tokenize(\"eros\")\n",
        "im = (torch.tensor(np.array(PIL.Image.open('/content/eros.png'))).unsqueeze(0)/255).permute(0,3,1,2)\n",
        "im = (torch.nn.functional.interpolate(im, (224, 224)) - .45) / .22\n",
        "perceptor(im.cuda(), tx.cuda())[0]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W1CTnhtx8M5A"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oxwI3qbk750V"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KWu4h17q5pxT"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}