01_2_Data_Handling_PyGeometric.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.10"
    },
    "colab": {
      "name": "01.2: Data_Handling_PyGeometric.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/gulabpatel/Graph_Neural_Network/blob/main/01.2%3A%20Data_Handling_PyGeometric.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "13e65620"
      },
      "source": [
        "Video Walkthrough the code : https://www.youtube.com/watch?v=Q5T-JdyVCfs"
      ],
      "id": "13e65620"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ea9fa3b3"
      },
      "source": [
        "### Custom PyG dataset"
      ],
      "id": "ea9fa3b3"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7fd0ada3"
      },
      "source": [
        "In the first part of the notebook we will see how to create a custom dataset in PyG. \n",
        "\n",
        "The dataset we'll load is called FRANKENSTEIN, the files can be downloaded from the \n",
        "[networkrepository](http://networkrepository.com/FRANKENSTEIN.php) site. The dataset was originally presented in the paper titled [Graph Invariant Kernels](https://www.ijcai.org/Proceedings/15/Papers/528.pdf).\n",
        "\n",
        "The dataset is a collection of graphs representing molecules, but the atom symbols of the vertices are substituted with MNIST digits. Each graph is associated with a label, indicating the mutagenicity of the molecule."
      ],
      "id": "7fd0ada3"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9OlnSWpUyhvK",
        "outputId": "acc4bb4a-4394-4781-d9ec-31ac3bd2bc71"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "id": "9OlnSWpUyhvK",
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "758e3554"
      },
      "source": [
        "The following is the README of the dataset:\n",
        "\n",
        ">FRANKENSTEIN contains the following comma separated text files:\n",
        ">\n",
        ">n: total number of nodes\n",
        ">m: total number of edges\n",
        ">N: number of graphs\n",
        ">\n",
        ">\n",
        ">*.node_attrs (n lines) \n",
        "\tmatrix of node attributes,\n",
        "\tthe comma seperated values in the i-th line is the attribute vector of the node with node_id i\n",
        ">\n",
        ">*.edges (m lines) \n",
        "\tsparse (block diagonal) adjacency matrix for all graphs,\n",
        "\teach line corresponds to (row, col) resp. (node_id, node_id)\n",
        ">\n",
        ">*.graph_labels (N lines)\n",
        "\tclass labels for all graphs in the dataset,\n",
        "\tthe value in the i-th line is the class label of the graph with graph_id i\n",
        ">\n",
        ">*.graph_idx (n lines)\n",
        "\tcolumn vector of graph identifiers for all nodes of all graphs,\n",
        "\tthe value in the i-th line is the graph_id of the node with node_id i"
      ],
      "id": "758e3554"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bbfe4ccb"
      },
      "source": [
        "The dataset is composed as follows:\n",
        "\n",
        "    Nr. of graphs:         4337\n",
        "    Total nr. of nodes:    73283 x 780 (weird, we'll change it later to 784)\n",
        "    Total nr. of edges:    155068"
      ],
      "id": "bbfe4ccb"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fzzcU6W0OJuZ",
        "outputId": "235b5689-88b8-46fc-d66c-d7fd0b2aae12"
      },
      "source": [
        "# Install required packages.\n",
        "!pip install -q torch-scatter -f https://pytorch-geometric.com/whl/torch-1.9.0+cu102.html\n",
        "!pip install -q torch-sparse -f https://pytorch-geometric.com/whl/torch-1.9.0+cu102.html\n",
        "!pip install -q torch-geometric"
      ],
      "id": "fzzcU6W0OJuZ",
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 2.6 MB 4.9 MB/s \n",
            "\u001b[K     |████████████████████████████████| 1.4 MB 4.1 MB/s \n",
            "\u001b[K     |████████████████████████████████| 222 kB 6.8 MB/s \n",
            "\u001b[K     |████████████████████████████████| 376 kB 8.9 MB/s \n",
            "\u001b[K     |████████████████████████████████| 45 kB 2.6 MB/s \n",
            "\u001b[?25h  Building wheel for torch-geometric (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "43d69805"
      },
      "source": [
        "import torch\n",
        "import os\n",
        "import pandas as pd\n",
        "from torch_geometric.data import InMemoryDataset, Data, download_url, extract_zip\n",
        "from torch_geometric.utils.convert import to_networkx\n",
        "import networkx as nx\n",
        "from tqdm import tqdm\n",
        "import matplotlib.pyplot as plt"
      ],
      "id": "43d69805",
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b3685c21"
      },
      "source": [
        "To create the dataset we need to convert the raw information into a ```Data``` object (a graph) in PyG.\n",
        "\n",
        "The first step is to load the csv files, this can be done manually or using some data library as Pandas:"
      ],
      "id": "b3685c21"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e642519c"
      },
      "source": [
        "raw_dir = \"/content/drive/MyDrive/dataset/FRANKENSTEIN\"\n",
        "\n",
        "path = os.path.join(raw_dir, 'FRANKENSTEIN.node_attrs')\n",
        "node_attrs = pd.read_csv(path, sep=',', header=None)\n",
        "node_attrs.index += 1\n",
        "\n",
        "path = os.path.join(raw_dir, 'FRANKENSTEIN.edges')\n",
        "edge_index = pd.read_csv(path, sep=',', names=['source', 'target'])\n",
        "edge_index.index += 1\n",
        "\n",
        "path = os.path.join(raw_dir, 'FRANKENSTEIN.graph_idx')\n",
        "graph_idx = pd.read_csv(path, sep=',', names=['idx'])\n",
        "graph_idx.index += 1\n",
        "\n",
        "path = os.path.join(raw_dir, 'FRANKENSTEIN.graph_labels')\n",
        "graph_labels = pd.read_csv(path, sep=',', names=['label'])\n",
        "graph_labels.index += 1"
      ],
      "id": "e642519c",
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c24354dd"
      },
      "source": [
        "Graph ids go from 1 to 4337, let's extract the information for a single graph (id: 2345)"
      ],
      "id": "c24354dd"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ec9399bc"
      },
      "source": [
        "g_idx=2345\n",
        "\n",
        "node_ids = graph_idx.loc[graph_idx['idx']==g_idx].index\n",
        "            \n",
        "# Node features\n",
        "attributes = node_attrs.loc[node_ids, :]\n",
        "\n",
        "# Edges info\n",
        "edges = edge_index.loc[edge_index['source'].isin(node_ids)]\n",
        "edges_ids = edges.index\n",
        "\n",
        "# Graph label\n",
        "label = graph_labels.loc[g_idx]"
      ],
      "id": "ec9399bc",
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5f0e4182",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ac6faad0-d623-456f-8ef9-677d61680f89"
      },
      "source": [
        "print(\"Nodes:\", node_ids.shape)\n",
        "print(\"Attributes:\", attributes.shape)\n",
        "print(\"Edges:\", edges.shape)\n",
        "print(\"Label:\", label.shape)"
      ],
      "id": "5f0e4182",
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Nodes: (18,)\n",
            "Attributes: (18, 780)\n",
            "Edges: (38, 2)\n",
            "Label: (1,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f4873f45",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0b491533-5168-4971-fbd8-bad77ea0e26a"
      },
      "source": [
        "print(\"Nodes:\", node_ids)\n",
        "print(\"Attributes:\", attributes)\n",
        "print(\"Edges:\", edges)\n",
        "print(\"Label:\", label)"
      ],
      "id": "f4873f45",
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Nodes: Int64Index([38571, 38572, 38573, 38574, 38575, 38576, 38577, 38578, 38579,\n",
            "            38580, 38581, 38582, 38583, 38584, 38585, 38586, 38587, 38588],\n",
            "           dtype='int64')\n",
            "Attributes:        0    1    2    3    4    5    6    ...  773  774  775  776  777  778  779\n",
            "38571    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38572    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38573    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38574    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38575    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38576    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38577    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38578    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38579    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38580    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38581    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38582    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38583    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38584    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38585    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38586    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38587    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "38588    0    0    0    0    0    0    0  ...  0.0  0.0  0.0  0.0  0.0  0.0    0\n",
            "\n",
            "[18 rows x 780 columns]\n",
            "Edges:        source  target\n",
            "81573   38571   38572\n",
            "81574   38572   38571\n",
            "81575   38571   38573\n",
            "81576   38573   38571\n",
            "81577   38571   38574\n",
            "81578   38574   38571\n",
            "81579   38572   38575\n",
            "81580   38575   38572\n",
            "81581   38572   38576\n",
            "81582   38576   38572\n",
            "81583   38573   38577\n",
            "81584   38577   38573\n",
            "81585   38574   38578\n",
            "81586   38578   38574\n",
            "81587   38575   38579\n",
            "81588   38579   38575\n",
            "81589   38575   38580\n",
            "81590   38580   38575\n",
            "81591   38577   38581\n",
            "81592   38581   38577\n",
            "81593   38578   38581\n",
            "81594   38581   38578\n",
            "81595   38581   38582\n",
            "81596   38582   38581\n",
            "81597   38582   38583\n",
            "81598   38583   38582\n",
            "81599   38583   38584\n",
            "81600   38584   38583\n",
            "81601   38583   38585\n",
            "81602   38585   38583\n",
            "81603   38584   38586\n",
            "81604   38586   38584\n",
            "81605   38585   38587\n",
            "81606   38587   38585\n",
            "81607   38586   38588\n",
            "81608   38588   38586\n",
            "81609   38587   38588\n",
            "81610   38588   38587\n",
            "Label: label    1\n",
            "Name: 2345, dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "689ae1fc"
      },
      "source": [
        "At this stage the indices in the ```edges``` variable are not normalized for the single graph, e.g. they do not start from 0"
      ],
      "id": "689ae1fc"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e295e6b5"
      },
      "source": [
        "edge_idx = torch.tensor(edges.to_numpy().transpose(), dtype=torch.long)\n",
        "map_dict = {v.item():i for i,v in enumerate(torch.unique(edge_idx))}\n",
        "map_edge = torch.zeros_like(edge_idx)\n",
        "for k,v in map_dict.items():\n",
        "    map_edge[edge_idx==k] = v"
      ],
      "id": "e295e6b5",
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0965b446",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7d45e9fa-5ddf-4a1a-acbd-95e8f918ba2b"
      },
      "source": [
        "map_dict, map_edge, map_edge.shape"
      ],
      "id": "0965b446",
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "({38571: 0,\n",
              "  38572: 1,\n",
              "  38573: 2,\n",
              "  38574: 3,\n",
              "  38575: 4,\n",
              "  38576: 5,\n",
              "  38577: 6,\n",
              "  38578: 7,\n",
              "  38579: 8,\n",
              "  38580: 9,\n",
              "  38581: 10,\n",
              "  38582: 11,\n",
              "  38583: 12,\n",
              "  38584: 13,\n",
              "  38585: 14,\n",
              "  38586: 15,\n",
              "  38587: 16,\n",
              "  38588: 17},\n",
              " tensor([[ 0,  1,  0,  2,  0,  3,  1,  4,  1,  5,  2,  6,  3,  7,  4,  8,  4,  9,\n",
              "           6, 10,  7, 10, 10, 11, 11, 12, 12, 13, 12, 14, 13, 15, 14, 16, 15, 17,\n",
              "          16, 17],\n",
              "         [ 1,  0,  2,  0,  3,  0,  4,  1,  5,  1,  6,  2,  7,  3,  8,  4,  9,  4,\n",
              "          10,  6, 10,  7, 11, 10, 12, 11, 13, 12, 14, 12, 15, 13, 16, 14, 17, 15,\n",
              "          17, 16]]),\n",
              " torch.Size([2, 38]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "73e16cab"
      },
      "source": [
        "As final step we convert the ```DataFrames``` to torch tensors. The node features are basically MNIST images, therefore their size should be 784 (28x28), but for some reason the files provide vectors of length 780. To adjust this, we simply add a padding of 4 zeros at the end of the vector (it will not affect the digits representation, as we'll see later)."
      ],
      "id": "73e16cab"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "df63102c"
      },
      "source": [
        "attrs = torch.tensor(attributes.to_numpy(), dtype=torch.float)\n",
        "pad = torch.zeros((attrs.shape[0], 4), dtype=torch.float)\n",
        "x = torch.cat((attrs, pad), dim=-1)\n",
        "\n",
        "edge_idx = map_edge.long()\n",
        "\n",
        "np_lab = label.to_numpy()\n",
        "y = torch.tensor(np_lab if np_lab[0] == 1 else [0], dtype=torch.long)"
      ],
      "id": "df63102c",
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "973a11c3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2b913a70-105a-4bf1-f3da-3d91baf861d5"
      },
      "source": [
        "x.shape"
      ],
      "id": "973a11c3",
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([18, 784])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "673f7499"
      },
      "source": [
        "Then we create the ```Data``` object representing the graph"
      ],
      "id": "673f7499"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "29d1f29d"
      },
      "source": [
        "graph = Data(x=x, edge_index=edge_idx,  y=y)"
      ],
      "id": "29d1f29d",
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6093b140"
      },
      "source": [
        "Let's visualize the graph!"
      ],
      "id": "6093b140"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "28f8a075",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 607
        },
        "outputId": "7052798a-1ecb-4ef3-c740-570cc4d51712"
      },
      "source": [
        "vis = to_networkx(graph)\n",
        "plt.figure(1,figsize=(8,8)) \n",
        "nx.draw(vis, cmap=plt.get_cmap('Set3'),node_size=70,linewidths=6)\n",
        "plt.show()"
      ],
      "id": "28f8a075",
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAJOCAYAAABBWYj1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde1zUVeL/8TcjIKAIoSKICqiYJGKaqAWS4SUlM/NWWlkZ1lezdmvVNN3WynLL7KLd1lCzLMvULmuopXgJ8ppXTFMSUFG8IYgC4jDz+6PH+tstzQEZPjPD6/lX5ZyZ9zwqeXvO+ZzjZrVarQIAAMBVmYwOAAAA4CwoTgAAADaiOAEAANiI4gQAAGAjihMAAICNKE4AAAA2ojgBAADYiOIEAABgI4oTAACAjShOAAAANqI4AQAA2IjiBAAAYCOKEwAAgI0oTgAAADaiOAEAANiI4gQAAGAjihMAAICNKE4AAAA2ojgBAADYiOIEAABgI4oTAACAjShOAAAANqI4AQAA2IjiBAAAYCOKEwAAgI0oTgAAADaiOAEAANiI4gQAAGAjihMAAICNKE4AAAA2ojgBAADYiOIEAABgI4oTAACAjShOAAAANqI4AQAA2IjiBAAAYCOKEwAAgI0oTgAAADaiOAEAANiI4gQAAGAjihMAAICNKE4AAAA2ojgBAADYiOIEAABgI4oTAACAjShOAAAANqI4AQAA2IjiBAAAYCOKEwAAgI3cjQ4A4DfFZWal7M7T1px8FZZclJ+3hzqGBiixbZB8PPlfFQAcgZvVarUaHQKoySwWq5LTDmrWmkwVlZr/8Ou+Xu564raWSoprLpPJzYCEAID/oDgBBrJYrBq7eKeWbs+96msHtA/Ra4PaUZ4AwEDscQIMlJx20KbSJElLt+dqTnqWnRMBAP4MxQkwSHGZWbPWZFZozMzUAyou++NyHgCgelCcAIOk7M677J6mP1NUatbyjDw7JQIAXA3FCTDI1pz8So3bkl25cQCAa0dxAgxSWHKxWscBAK4dxQkwiJ+3R7WOAwBcO4oTYJCOoQGVGhcTVrlxAIBrR3ECDJLYNki+XhU7EdzXy12JUcF2SgQAuBqKE2AQH8/fTgSviCcTIuTtWctOiQAAV0NxAgyUFNdcA9qH2PTaAe1D9EhsuJ0TAQD+DFeuAAazWKyak56lmakHrnhX3ZMJEXokNpzrVgDAYBQnwEEUl5n10Zrdem7WR7J6eMvtYrHu7HKD3h7/CMtzAOAgKE6AAxkzZozee+89WSwWSZK3t7fy8vJUr149g5MBACT2OAEOpWnTpurXr5/c3d3Vs2dPvfDCC/Lw4NwmAHAUzDgBDqh+/fp6+umnNWnSJKOjAAD+CzNOgAPy9vZWfj530gGAo6E4AQ6I4gQAjqlixxYDqBZ16tRRQUGB0TFwFcVlZqXsztPWnHwVllyUn7eHOoYGKLFtkHw8+e0VcEXscQIcUNeuXeXh4aHU1FSjo+AyLBarktMOataazCuevfXEbS2VFNecs7cAF8MfiQAH5Ovrq5MnTxodA5dhsVg1dvFOLd2ee8XXFJWa9fLyfdqXV6TXBrWjPAEuhD1OgAPy9/fX+fPnjY6By0hOO/inpem/Ld2eqznpWXZOBKA6UZwAB3TdddepuLjY6Bj4neIys2atyazQmJmpB1Rc9sflPADOieIEOKCAgACKk4MoLCzUmTNnJEkpu/Muu6fpzxSVmrU8I88e0QAYgOIEOKAGDRrowoULRseApOeff17169fXTTfdpLn/Xlep99iSzdESgKugOAEOqEGDBrp48aLRMWo8i8Uik8mkWrVqadu2bdqyc0+l3qewhH+XgKvgqTrAAQUFBVGcqpHFYtGOHTu0cuVKbdq0ST///LOOHj2q8+fPy83t/z8RZ71QuQ37ft7cNwi4CooT4IAaNWqk8vJyo2O4pL1792r58uXasGGD9uzZo9zcXBUVFUmS/Pz81LRpU8XExCg2NlZ9+vSRn5+fAgMD5enpqbtuidLqSnSnmLCAKv4WAIxCcQIcUOPGjcXZtNcmKytLy5cvV3p6unbv3q3Dhw+rsLBQ0m/nZIWEhKhNmzZKSkrS7bffrjZt2lzxvUaOHKmkpCRFtm2nztNWV2iDuK+XuxKjgq/5+wBwDJwcDjggi8WiWrVqqaioSHXr1jU6jkM7evSoli9frrS0NO3atUs5OTk6c+aMLBaL6tSpo8aNG+uGG25Q586ddfvtt+vGG2+UyVT57Z2z1/+ql5fvs/n1kxIjNbJr80p/HgDHwowT4IBMJpPc3Nx07NgxRUREGB3HIeTn52v58uVav369duzYoezsbJ0+fVrl5eXy9vZWcHCwrr/+et15553q0aOHunTpInf3qv8tLimuufblFdl0COaA9iF6JDa8yjMAMA4zToCDcnd315o1a9S1a1ejo1Src+fOaeXKlVq7dq22b9+uX3/9VadPn9bFixdVu3ZtNWrUSBEREYqJiVH37t0VHx8vT0/Pas1osVg1Jz1LM1MPXPGuuicTIvRIbDjXrQAuhhknwEG5u7vr+PHjRsewm9LSUqWmpio1NVU//fSTMjMzdeLECZWVlcnT01MNGzZUixYtNHToUHXr1k09evSQj4+P0bElSSaTm0Z2ba77OjfT8ow8bcnOV2HJRfl5eygmLECJUcHy9qxldEwAdsCME+BgnnvuOX3yySfKysqSp6enTCaT9uzZo/Bw51zyMZvNWr9+vVatWqWtW7dq//79On78uEpLS+Xu7q4GDRooPDxc7du3V7du3dSrVy/5+fkZHRsALosZJ8DBBAcHKzc3V1arVRcuXFBwcLBCQ0ONjnVVFotFmzZt0vfff69NmzZp3759ysvLU3FxsWrVqqWAgACFhoaqR48eio+PV+/evRUYGGh0bACoEGacAAdjNpvVqlUrZWVlyd3dXTNmzNCTTz5pdKxLLBaLdu3apZUrV2rjxo2XDos8d+6cTCaT/P391axZM0VHRysuLk69e/dW06ZNjY4NAFWC4gQ4mOIys177PFUzFy6TyctXD9wzQDe3bKTEtkHy8azeSeJffvlFK1asUHp6un7++WcdPnz40mGR9erVU9OmTRUVFaVbbrlFffr0UcuWLas1HwBUN4oT4CAsFquS0w5q1prMKz6p9cRtLZUU17zKn9TKycm5dBZSRkaGDh06pMLCQlkslkuHRd5www26+eab1bt3b91www3XdBYSADgrihPgACwWq8Yu3mnz2UCvDWonk8lN5eXlslgs8vCw7S6048ePa8WKFVq3bp127dql7OxsFRQUqLy8/NJhkddff726dOmiXr166aabbqIgAcB/YXM44ACS0w7aVJokaen2XEUG11O3oHLdeeedio+P1+zZs//nNQUFBZfOQtqxY4cOHjyo/Px8mc1meXt7KygoSK1atVJiYqJ69OihW265xS6HRQKAq2HGCTBYcZm5wvefeZksOvLOQyo+e0ZNmzZV//79tX37dmVmZurUqVOXDosMDAxURESEOnbsqISEBN12223VflgkALgSihNgsMU/HdHYxTsrPO7Ustd1PiNV0m9HGLRs2VI33XSTunXrpu7du3PHHQDYAXPzgMG25uRXapxPaFtd3J+m8vJybd26VY0bN67iZACA36M4AQYrLLlYqXE9Eu9SZOf6SklJ0cWLlXsPAEDF8LgMYDA/b9ueiPu9ZkH19fe//10bNmxwipPFAcAVUJwAA+zevVtt2rRRkyZNNPeVyZV6j5iwgCpOBQC4GooTYICQkBBlZWUpNzdXp7Z/J0vpuQqN9/VyV2JUsJ3SAQCuhOIEGGDlypX6zwOt1osXpD0rKzT+yYQIeXvWskc0AMCfoDgB1Wjbtm2KiIjQ/fffr/79+ys4OFgeHh76/IVHNaB9iE3vMaB9iB6JDbdzUgDA5fBUHVANTpw4oXvvvVdr165Vp06dlJqaqqZNm2rt2rXauXOnbu7SRZ07WRUZXE8zUw9c8a66JxMi9EhseJXfVQcAsA0HYAJ2ZDabNXr0aM2dO1eNGzfWggULFB8f/6djisvMWp6Rpy3Z+SosuSg/bw/FhAUoMSqY5TkAMBjFCbCTmTNnasKECTKZTHr99df16KOPGh0JAHCNWKoDqtjq1as1fPhwHT9+XKNHj9abb74pk4nthADgCihOQBXJysrS4MGDtW3bNvXq1Uu7d+9WQABnLQGAK+GPwcA1Ki4u1qBBg9SiRQudP39eu3bt0ooVKyhNAOCCKE5AJVksFj333HPy9/fXmjVr9MUXX2jv3r2KiooyOhoAwE5YqgMq4fPPP9eoUaN0/vx5TZ48WZMnT2YfEwDUABQnoAJ27Nihe+65R5mZmRoyZIjmzZsnLy8vo2MBAKoJf0QGbHDq1Cn16NFDHTp0kL+/v7KysrRw4UJKEwDUMBQn4E+YzWaNGjVKQUFB+uWXX5SamqpNmzapWbNmRkcDABjAZZfqisvMStmdp605///05Y6hAUpsGyQfT5f92qhC77zzjsaPHy83NzfNmjVLo0aNMjoSAMBgLndyuMViVXLaQc1ak3nF+76euK2lkuKac98XLmvNmjV64IEHlJeXp8cee0xvvfWW3N0p2wAAFytOFotVYxfv1NLtuVd97YD2IXptUDvKEy7JycnRoEGD9NNPP6l79+76/PPPOYsJAPA/XGqPU3LaQZtKkyQt3Z6rOelZdk4EZ1BSUqIhQ4YoPDxcRUVF2rFjh77//ntKEwDgD1ymOBWXmTVrTWaFxsxMPaDisj8u56FmsFgsmjJlivz9/bVq1Sp99tln2rdvn6Kjo42OBgBwUC5TnFJ25112T9OfKSo1a3lGnp0SwZEtWbJEDRs21LRp0zRx4kSdOnVKQ4YMMToWAMDBuUxx2pqTX6lxW7LzdezYMa1atUpmM7NPrm7Xrl2KjIzU4MGDlZCQoDNnzmjKlCmc+g0AsInL/LQoLLlYqXELPl+q8PBw9ezZUydOnKjiVHAU+fn5uv3223XjjTeqTp06+vXXX/XFF1/Ix8fH6GgAACfiMsXJz9ujUuNKzubrwoUL8vb21pkzZ6o4FYxmsVg0ZswYBQYGas+ePVq1apW2bt2q8PBwo6MBAJyQyxSnjqGVewLKfGyfJMlqtSoqKkpeXl5q166dJk6cqKwsnrpzZv/6179Ur149zZs3T2+++aaOHDmihIQEo2MBAJyYyxSnxLZB8vWq2CGFvrXd1TW0rtzc3DR37lyVlpbqX//6l0JCQvTBBx+oefPm8vHxUUxMjJ5//nkdPXrUTulRldavX69mzZrp8ccf1wMPPKDCwkKNGTPG6FgAABfgUgdgzl7/q15evs/m109KjNQjsWGaN2+ehgwZIl9f3//59XPnzumjjz7SkiVLtG3bNhUUFKhu3bqKjo5W//799cgjj3DWjwM5dOiQBg8erC1btighIUGfffaZGjRoYHQsAIALcaniZO+Tw/Pz8zVv3jx9+eWX2rVrl4qKiuTn56cOHTpo0KBBGj58uOrWrXstXwGVUFpaqocffliLFi1SixYt9Nlnn6lDhw5GxwIAuCCXKk7Sb+VpTnqWZqYeuOJddU8mROiR2PBrvm7l6NGjSk5O1rJly7Rnzx4VFxerfv36iomJ0ZAhQzR06FB5eXld02fgz7344ot66aWX5O3trffff1/33HOP0ZEAAC7M5YrTfxSX/Xa45ZbsfBWWXJSft4diwgKUGBUsb89advnMrKwszZ49W8uXL9e+fft04cIFNWrUSF26dNGwYcM0YMAALoutIl9++aVGjhyps2fP6plnntHzzz/PWUwAALtz2eLkCDIyMpScnKzvvvtOmZmZMpvNaty4seLi4jR8+HD17t2bH/YVlJGRoSFDhmjfvn26++679fHHH3MWEwCg2lCcqtGWLVs0Z84crV69WllZWbJarWratKm6deumESNGKD4+3uiIDqugoED33nuvvvvuO7Vv3/7SfiYAAKoTxckgFotF69ev17x587Ru3TodPnxYbm5uat68uRISEpSUlKSOHTsaHdNwFotFTz31lN555x0FBgZq/vz56tmzp9GxAAA1FMXJQVgsFqWkpOjjjz9Wenq6jh49Knd3d0VERKhXr15KSkpSmzZtjI5ZrZKTk/XUU0+pvLxc06ZN01/+8hejIwEAajiKk4Mym81asmSJFi5cqA0bNujEiRPy8vJS69at1adPHz322GMKDQ01OqZdpKWl6b777tORI0f0yCOP6N1332VTPQDAIVCcnERpaak+/fRTLVq0SFu2bFF+fr58fHzUpk0b9evXT0lJSQoKCjI65jU5cuSIhgwZoo0bN+rWW2/V559/rsDAQKNjAQBwCcXJSZ09e/bSqebbt29XYWGhfH191a5dO919990aMWKE/P39jY5pk9LSUiUlJWnhwoVq3ry5Fi5cyP4uAIBDoji5iFOnTmnu3Ln66quvtHv3bp07d07+/v666aabNHjwYD3wwAMO+dj+tGnT9Pzzz8vLy0vvvPOO7rvvPqMjAQBwRRQnF3XkyJFLp5r//PPPKikpUYMGDdSpUyfdc889uvfee+Xp6Vkln1VcZlbK7jxtzfn/h412DA1QYtsg+Xj+tjepsLBQb731liZPniyTyaRvvvlGSUlJKigo0NixYzV16lTOtAIAODyKUw1x4MABffDBB1qxYoX279+vsrIyNWrUSDfffLPuu+8+3XXXXRXegG2xWJWcdlCz1mRe8XqbJ25rqaS45rr//vv02Wef6eWXX9aCBQv0888/q3///vroo4+43w8A4DQoTjXUrl27lJycrO+//16ZmZkqLy9XSEiIunbtqgcffFA9e/b8wwzQ4sWL1apVK0VHR1foQuVODa1aNvkelZQUS5Kio6O1dOlSDrAEADgdihMkSRs2bNDcuXOVmpqqnJwcWa1WhYaGqlu3bnrkkUcUGxurhg0b6vz581qyZIkO12mll5fvs/n981PnqGjzl/L09NTEiRM1ZcoU+30ZAADshOKEP7BYLFqzZo3mzZun9evX68iRIzKZTLJYLLJarfKqU0/NnlygC1bb9yRZSs/pZPJj8nJ306233qqvvvrKjt8AAAD7oDjhqiwWi5588km9//77Ki8vV52oBDXo+3SF32fG4HYa2KGJHRICAFA9eIwJV2UymZSfny9J8vb2VtMO3Sr1Pluy86swFQAA1Y/iBJu89NJL2rNnj86dO6duvRIr9R6FJRerOBUAANWLC8Bgk/Dw8Et/7eftUan3qOw4AAAcBTNOqLCOoQGVGhcTVrlxAAA4CooTKiyxbZB8vSo2Wenr5a7EqGA7JQIAoHpQnFBhPp6/nQheEU8mRMjbs5adEgEAUD0oTqiUpLjmGtA+xKbXDmgfokdiw6/+QgAAHBzFCZViMrnpb3GBMu38SpbSc5d9ja+XuyYlRuq1Qe1kMrlVc0IAAKoeB2CiUnJyctSpUyedOnVK1loe8ovqpvufmiKzyUN+3h6KCQtQYlQwy3MAAJfCcQSolDNnzqhu3bq/FaeLF1T2yw96uI2nOnToYHQ0AADshqU6VMqNN96offv2yc3NTXFxcZKkCxcuGJwKAAD7YsYJlfbmm2/K09NT69atk5ubm9zc2McEAHBt7HFCpYWGhqpTp0764osvjI4CAEC1YMYJlXLgwAEdOnRIa9euNToKAADVhhknVMqAAQO0fft2ZWVlGR0FAIBqw+ZwVJjFYlFKSor+8pe/GB0FAIBqRXFChc2fP18Wi0VPPvmk0VEAAKhWLNWhwiIjI9WsWTOtXLnS6CgAAFQrihMqJC8vT40bN9bWrVs57BIAUOOwVIcKefbZZ9WoUSNKEwCgRqI4oUIWL16spKQko2MAAGAIlupgs2+++UZ33323zp8/Ly8vL6PjAABQ7ShOsNlNN92k2rVr68cffzQ6CgAAhuDkcNikqKhI27dv1/fff290FAAADMMeJ9hkypQp8vPzU/fu3Y2OAgCAYShOsMn8+fM1bNgwo2MAAGAo9jjhqtLS0hQfH6/8/Hz5+/sbHQcAAMNQnHBVt956q86cOaNdu3YZHQUAAEOxORx/qqysTOnp6fr000+NjgIAgOHY44Q/NX36dNWuXVtDhgwxOgoAAIZjqQ5/qkmTJoqPj2fGCQAAUZzwJ/bs2aOoqCgdPnxYTZo0MToOAACGozjhiu68807t3btXmZmZRkcBAMAhsDkcl2WxWPTdd9/pjTfeMDoKAAAOg83huKz3339fkvR///d/BicBAMBxsFSHy4qIiND111+vZcuWGR0FAACHwVId/uDw4cPKzMzUV199ZXQUAAAcCjNO+IP7779fa9asUW5urtFRAABwKOxxwh98+eWXeuyxx4yOAQCAw2HGCf9j0aJFGjZsmIqLi+Xp6Wl0HAAAHArFCf+jXbt28vf317p164yOAgCAw2FzOC45c+aMdu/erfXr1xsdBQAAh8QeJ1zy97//XQEBAYqLizM6CgAADonihEs++eQTPfDAA0bHAADAYbHHCZKk1NRU9ejRQwUFBapXr57RcQAAcEgUJ0iSYmNjVVpaqp9++snoKAAAOCw2h0OlpaXauHGjli5danQUAAAcGnucoJdeekk+Pj666667jI4CAIBDozhBycnJGjhwoNExAABweOxxqqEsFousVqt2796tDh066OjRowoKCjI6FgAADo0Zpxpq6tSpql+/vvr376+wsDBKEwAANqA41VB169bV+fPnlZOToyNHjqhTp04qKioyOhYAAA6N4lRDBQUFqVatWpIkDw8P+fv7y9vb2+BUAAA4No4jqCGKy8xK2Z2nrTn5Kiy5qHP5AfJoFafyAxs0cOBAzZ07V+7u/OcAAMCfYXO4i7NYrEpOO6hZazJVVGr+w6+7Wy5q3B1tNTKuuUwmNwMSAgDgPChOLsxisWrs4p1auj33qq8d0D5Erw1qR3kCAOBPsMfJhSWnHbSpNEnS0u25mpOeZedEAAA4N4qTiyouM2vWmswKjZmZekDFZX9czgMAAL+hOLmolN15l93T9GeKSs1anpFnp0QAADg/ipOL2pqTX6lxW7IrNw4AgJqA4uSiCksuVus4AABqAoqTi/Lz9qjWcQAA1AQUJxdktVoVXrdyp0xcH8AhmAAAXAnFycVMnDhRPj4+empQN1lKz1dorKX0vEb17aJ69epp9uzZdkoIAIDzoji5mFtvvVVWq1UlRQUq/PGzCo0t+PEzXThfpLKyMnXq1MlOCQEAcF4UJxezY8cOXbhwQSaTSeaM79TG55xN45pczNWFHSlyc3PTxYsXtX//fjsnBQDA+XDliosoLi5W9+7dtXnzZr388sv6+OOPdeLECR06dFgfb8nVzNQDlz3XydfLXU8mRKjf9XXVrGlT3XbbbQoLC9Ps2bM1ZMgQffrppzKZ6NcAAEgUJ5fwww8/KDExUd7e3lq3bp0iIyOVl5enEydOKDo6WtJvJ4kvz8jTlux8FZZclJ+3h2LCApQYFSxvz1qSpPT0dEVFRcnPz0/fffed+vfvr+uuu05paWkKDw838isCAOAQKE5O7m9/+5veeOMN3XnnnVqyZInc3avuqbizZ88qPj5eGRkZevfdd/Xoo49W2XsDAOCMKE5OKj8/X3FxcTpw4ICSk5P14IMP2u2zJk2apGnTpqlXr15atmxZlZYzAACcCcXJCX399de65557FBQUpLS0NDVp0sTun7lp0yb16tVLHh4eWrt2raKiouz+mQAAOBp2/ToRi8Wi4cOH6+6779awYcN08ODBailNktS5c2cdP35ckZGRateunV555ZVq+VwAABwJM05O4tChQ4qLi9PJkyf1xRdfqG/fvoZlmT59uiZMmKAuXbpo1apV8vb2NiwLAADViRknJzBnzhy1aNFCfn5+ys3NNbQ0SdK4ceO0c+dO/fLLL2rUqJF+/PFHQ/MAAFBdKE4OzGw264477tDIkSP19NNPa/fu3QoICDA6liQpKipKeXl56tq1q+Li4jRhwgSjIwEAYHcs1TmojIwMdevWTWVlZVq+fLliY2ONjnRFycnJGjVqlCIjI7V+/Xr5+/sbHQkAALtgxskBvfrqq2rXrp1at26tEydOOHRpkqSkpCRlZmYqPz9fjRs31ooVK4yOBACAXVCcHEhxcbFuueUWTZw4Ua+++qrS0tLk5eVldCybhIaG6tChQ7r77ruVmJioxx57zOhIAABUOZbqHERaWpr69Okjb29vrVmzRm3atDE6UqUtWbJEw4YNU5MmTZSenq6goCCjIwEAUCWYcXIA48aNU3x8vLp166ajR486dWmSpIEDB+rw4cOqVauWmjVrps8//9zoSAAAVAmKk4Hy8/PVpk0bvfnmm5o7d67+/e9/u8x1JoGBgdq/f79GjhypoUOHasiQIbJYLEbHAgDgmrBUZ5B///vfGjx4sBo1aqS0tDQ1bdrU6Eh2s2rVKvXr10/+/v5KS0tT8+bNjY4EAEClMONUzSwWix588EHddddduvfee5WVleXSpUmSevTooby8PDVq1EitWrXSv/71L6MjAQBQKcw4VaNDhw6pa9euOnHihBYtWqQ777zT6EjVbvLkyXr55ZfVs2dPffvtty6zNAkAqBmYcaom8+fPV4sWLeTr66vc3NwaWZokaerUqdq4caM2btyoRo0aadeuXUZHAgDAZhQnOzObzerbt68efvhh/fWvf1VGRobDXJtilE6dOun48eNq06aN2rdvr2nTphkdCQAAm7BUZ0d79uxRt27dVFpaqhUrVjj8CeBGeP311zVu3Dh16tRJq1evlo+Pj9GRAAC4Imac7GT69OmKjo5Wq1atdPz4cUrTFTz99NPatWuXMjMz1ahRI6WnpxsdCQCAK6I4VbHS0lLFxsZqwoQJmjZtmtLT05lFuYo2bdro2LFjuvXWW9W1a1eNHz/e6EgAAFwWS3VVKD09XX369JGnp6fWrVvn9CeAG2Hu3Ll67LHH1Lp1a/3www/y9/c3OhIAAJcw41RFxo8fr65duyo+Pl55eXmUpkoaMWKEMjMzVVBQoODgYKWkpBgdCQCASyhO1yg/P19RUVF64403NGfOHC1btoyzia5RaGiocnJyNHDgQPXt21dJSYysnEIAACAASURBVElc1wIAcAgs1V2DZcuWafDgwWrYsKHS0tLUrFkzoyO5nKVLl2ro0KEKCQlRWlqaGjdubHQkAEANxoxTJVgsFj388MPq16+fhgwZouzsbEqTnQwYMEBHjhyRh4eHwsLCtHDhQklSeXm5fvzxR4PTAQBqGmacKujIkSOKi4tTXl6eFi1apH79+hkdqcZ44okn9M477+juu+/WDTfcoKlTp2rnzp2Kjo6WJBWXmZWyO09bc/JVWHJRft4e6hgaoMS2QfLxZPkUAHDtKE4VMH/+fCUlJalVq1b64YcfavwJ4EZYvXq17rjjDl24cEGS1LNnT61YsVLJaQc1a02mikrNfxjj6+WuJ25rqaS45jKZ3Ko7MgDAhVCcrsBqtcrN7bcfsmazWQMGDNCyZcv09NNP67XXXjM4Xc11+vRptWrVSvn5+ZIkDw9PDZuZorU5pVcdO6B9iF4b1I7yBACoNNYvLsNsNis6OlrPPvusOnTooG7duqmkpETr169XXFyc0fFqtK1bt+rChQvy8vKS1WpV7XaJNpUmSVq6PVeRwfU0smtzO6cEALiqGjnjdLW9MHPnztXjjz+u8vJylZeXq3Pnzlq1ahUngDsIq9WqY8eO6aedu/XUuhKZ3TxsHuvr5a5NE7uz5wkAUCk1qjhZLNar7oUZHR+uf9zTVSdPnpAkNWjQQEeOHFHt2rWrOy6uYvFPRzR28c4Kj5sxuJ0Gdmhih0QAAFdXY44jsFisGrt4p15evu+ypUmSikrNeuW7A7J0uk+Sm2rVqqX8/Hy988471RsWNtmak1+pcVuyKzcOAIAas16RnHZQS7fn2vTaum27K6Khj+6M8FHbtm0VHx9v53SojMKSi9U6DgCAGlGcisvMmrUms0JjzoV11V/HshfGkfl52763qSrGAQBQI5bqUnbnXXF57kqKSs1anpFnp0SoCh1DK3eOVkwY528BACqnRhQn9sK4psS2QfL1qtiMoK+XuxKjgu2UCADg6mpEcWIvjGvy8fztRPCKeDIhQt6eteyUCADg6mpEcWIvjOtKimuuAe1DbHrtgPYheiQ23M6JAACurEYUJ/bCuC6TyU2vDWqnSYmRV1y2s1w4r8Eta3HdCgDgmtWIAzCLy8zqPG11hTaI+3q5a/PEHizrOJHist829G/J/u1E+GVLF+nM/q0q3pcuD5NV8+bN07Bhw4yOCQBwYjXiWfv/7IV5efk+m8ewF8b5+Hi6a2CHJpdOBf90dILOnz0rSQoICpKnp6eR8QAALqBGLNVJ7IWpiWrVqqWbb75ZJpNJU6dO1aBBg4yOBABwcjWmONm0F6b0nIrWz1fT3LXshXEBJ06c0I8//qg+ffropZdeMjoOAMAF1Ig9Tr/3+70wJ47kaO0XyTqzM1W1VC6LxaJDhw4pJMS2GSo4tl9//VURERHasWOHoqOjjY4DAHBiNbI4/d6bb76pv/3tb3Jzc1NgYKA2b96sJk2aGB0LVSgyMlLBwcFKTU01OgoAwIlRnCQdPHhQe/fulYeHh3r37q2CggLVq1fP6FioQkuWLNGQIUNUWFiounXrGh0HAOCkKE6/U79+fQ0bNkyzZs0yOgqq2HXXXaf777+ff7cAgEqrMZvDbfXQQw/p448/NjoG7GDkyJH68MMPjY4BAHBizDj9TnFxsXx9ffX111+rb9++RsdBFSotLVXdunX14Ycf6v777zc6DgDACVGcLiM2NlbFxcXavn270VFQxfr06aMDBw4oMzPT6CgAACdEcbqMtLQ0xcfH69SpUwoI4L46V3LgwAFdf/312r17t9q0aWN0HACAk6E4XUFgYKDuuusuffDBB0ZHQRVr1aqVQkND9f333xsdBQDgZNgcfgUjR47UZ599ZnQM2MGLL76o1NRUFRcXGx0FAOBkmHG6gv9sJF64cKEGDx5sdBxUMT8/P40YMUJvvPGG0VEAAE6E4vQnbrvtNp08eVIZGRlGR0EVe/rppzVnzhwVFhYaHQUA4EQoTn9iy5Yt6ty5s44ePaqgoCCj46AKlZSUyNfXVwsWLNC9995rdBwAgJOgOF1FcHCwevbsqY8++sjoKKhiPXv2VE5Ojvbv3290FACAk2Bz+FWMGjVKS5YsMToG7GDmzJk6cOCAfvnlF6OjAACcBDNOV1FWViYfHx/NnTtXw4cPNzoOqljLli3VsmVLrVixwugoAAAnQHGyQa9evZSTk8PMhAtasGCBHnroIZ07d05eXl5GxwEAODiKkw127dqldu3aKScnR82aNTM6DqpYvXr19Nhjj2n69OlGRwEAODj2ONkgOjpaISEhGj9+vNFRYAcPPfQQJ8QDAGxCcbLRX/7yF3399deyWCxGR0EV++c//6mioiItXrzY6CgAAAfHUp2NzGazfHx89Pbbb+vRRx81Og6qWEJCgo4dO6a9e/caHQUA4MAoThXQt29f7du3T5mZmUZHQRXLyMhQdHS0Dhw4oBYtWhgdBwDgoChOFfDLL7+odevW+vXXX9W8eXOj46CKNW/eXDfccIOWLVtmdBQAgIOiOFVQWFiY2rdvry+//NLoKKhiH374oUaOHKnz58/L09PT6DgAAAfE5vAK+tvf/qaUlBQ2ibughx56SLVr19aUKVOMjgIAcFAUpwp6/PHHJUlvv/22wUlgD8OHD9f777+vr7/+WmPGjDE6DgDAwbBUVwkDBgzQtm3blJ2dbXQUVKGCggLNnDlT//jHP1S7dm1ZLBaVlZUZHQsA4EAoTpWQlZWl5s2b6+eff1ZkZKTRcVBFxo8frxkzZlxahvXz81NBQYHBqQAAjoTiVEktW7bU9ddfr2+//dboKKgiZWVlevDBB/XVV1/pQrlVQZ0SNfzpKSosuSg/bw91DA1QYtsg+Xi6Gx0VAGAQilMlJScna/To0SouLpa7Oz9IXUV5uUVDX0jWxrN+MnnV/cOv+3q564nbWioprrlMJjcDEgIAjMTm8EoaMWKEatWqpddff93oKKgiFotV45bs0uaykMuWJkkqKjXr5eX7NHbxTlks/JkDAGoailMlmUwm3XnnnfrnP/+pDh06qFevXkZHwjVKTjuopdtzbXrt0u25mpOeZedEAABHQ3GqpClTpiglJUVnzpzR9u3bVVpaanQkXIPiMrNmranYVTozUw+ouMxsp0QAAEdEcaqkU6dOqby8/NLfBwUFGZgG1ypld56KSitWgopKzVqekWenRAAAR8Su5kqaNWuWgoOD9fzzz8ssk8qadNCEpbt4AstJbc3Jr9S4Ldn5GtihSRWnAQA4Kn6qV5Kbm5smTnxWh+u0UkqORTtq19WOLYcv/fpnWw7r+WV7eALLSRSWXKzWcQAA58RSXSVZLFaNXbxTK4778ASWC/Dz9qjWcQAA50RxqiSewHItHUMDKjUuJqxy4wAAzoniVAk8geV6EtsGyderYivXvl7uSowKtlMiAIAjojhVAk9guR4fz99OBK+IJxMi5O1Zy06JAACOiOJUCdfyBBYcV1Jccw1oH2LTawe0D9EjseF2TgQAcDQ8VVcJ1/oEltVqlZsbT9k5mpKSYmV+8g+dPempup0HXfGuuicTIvRIbDhPSgJADURxqoTKPkn1zeLPNOfhm5WYmKivvvqqilPhWvXs2VObN29WeXm5Crb+W3FDn9Atdz1w6WyumLAAJUYFszwHADUYxakSOoYG6LP/OrPJVoWZ2+Tm5qaePXvaIRWu1fTp0zVy5Ejt3btX1osX5HN8t/45INroWAAAB8Iep0qozBNYlgvnVbwvXWVlZRo7dqzuv/9+HTt2zE4JURmxsbEqLy/XTTfdpL59+6pVq1ZGRwIAOBhmnCrhP09gvbx8n81jOtQ+oVzLRd3/wANq3bq13n77bYWEhCgyMlL/+Mc/NGTIEDsmhi22bt2qAwcO6JdfflFERITRcQAADsjNarVypHUl/OfkcFsOwRzQPkTTB0brhRee1/3333/ph/LWrVs1fvx4rV+/Xt7e3rrnnnv0yiuvqH79+vaOj8uIiYmR2WzW9u3bjY4CAHBQFKdrYLFYNSc9SzNTD1z2XCdbn8AqLS3V1KlTNXv2bJ06dUrt2rXT1KlTdccdd9gzPv5LVlaWWrRoofT0dN18881GxwEAOCiKUxUoLvvtcMst2fnX/ARWWlqannnmGW3cuFG+vr568MEHNXXqVPn6+topPSSpV69eOnjwoDIzK3YiPACgZqE4Oahz587pueee04cffqiCggJ16tRJ//znP9WtWzejo7mc/Px8NWzYUF9++aX69etndBwAgAPjqToHVbduXb3++uvKz89XSkqKzGazEhIS1KBBA02YMEGlpaVGR3QZo0ePVsOGDSlNAICrojg5gd69e2vr1q3Kz8/X4MGD9e6776pOnTq69dZbtXnzZqPjObXS0lItWbJEzz//vNFRAABOgOLkRPz9/fXee+/p7NmzWrRokc6cOaMuXbooODhYL7zwgsrKyoyO6HTGjRsnHx8fPfbYY0ZHAQA4AYqTkxo4cKB27dqlvLw83X777XrllVdUp04d3X777crIyDA6nlOwWCxKTk7WU089ZXQUAICToDg5ucDAQH344Yc6f/685syZo5ycHEVHR6tp06aaMWOGLBaL0REd1ssvvyyr1arnnnvO6CgAACfBU3Uu6NChQxo7dqy++eYbWSwW9enTR6+99hqnYf+Ov7+/hg4dqvfee8/oKAAAJ8GMkwtq1qyZFi1apOLiYs2aNUsZGRlq1aqVWrRooffff59ZKElz5szRuXPn9PrrrxsdBQDgRJhxqiEOHDigsWPHavny5TKZTLrrrrs0ffp0NWvWzOhohmjcuLFiY2P1xRdfGB0FAOBEmHGqISIiIvT111+rtLRUL730kn788UeFhYWpdevWWrBggdHxqtWyZcuUl5fHEh0AoMKYcarBdu3apXHjxmn16tWqXbu2Bg0apOnTpyswMNDoaHbVqlUrNW3aVKtXrzY6CgDAyTDjVINFR0dr5cqVKi4u1jPPPKPvvvtOQUFBio6O1tKlS42OZxdbtmxRZmamZs+ebXQUAIATYsYJ/2PTpk0aP3680tLSVKdOHd13332aNm2a/P39jY5WJTp27Cir1aqffvrJ6CgAACfEjBP+R+fOnbVu3TqdP39ejz/+uL744gsFBASoY8eOWrlypdHxrsmvv/6qbdu2sbcJAFBpzDjhqtasWaNnn31WmzZtkr+/vx566CG98MILqlu3rtHRKqR79+46dOiQDhw4YHQUAICTYsYJV3Xbbbdpw4YNKigo0AMPPKB58+bJz89PsbGxSktLMzqeTU6dOqU1a9bojTfeMDoKAMCJUZxgs3r16umtt97SmTNn9M0336ikpETx8fEKDAzUpEmTVFpaanTEKxo9erQaNWqkvn37Gh0FAODEKE6olDvuuEPbtm3TqVOndNddd2nmzJmqW7euEhIStHXrVqPj/Y+SkhItXbpUL774otFRAABOjuKEaxIQEKAPPvhARUVF+vTTT3XixAl16tRJjRs31rRp02Q2m42OqLFjx6pu3bpKSkoyOgoAwMlRnFBlhgwZooyMDB05ckQJCQl68cUX5e3trT59+mjPnj2GZLJYLJo7d66efvppQz4fAOBaKE6oco0bN9aCBQt07tw5zZ49W7/++quioqIUGhqqt956q1ovGZ46daokafLkydX2mQAA10Vxgt2YTCY9/PDD2r9/v7KyshQTE6NnnnlG3t7euvvuu3Xw4EG7fK7ValW/fv00d+5czZgxQyNGjJDJxH/qAIBrx08TVIuwsDAtXrxYxcXFmjFjhrZv364WLVqoZcuWSk5OrtJZKDc3Ny1btkyjRo3S2bNnFRISonPnzlXZ+wMAai6KE6qVyWTSmDFjlJ2drb1796p169YaNWqU6tSpo2HDhuno0aNV8jnu7u4qKyuT9Nsy3cyZM6vkfQEANRsnh8NwZrNZM2bM0KxZs3T06FG1bt1af//73zV06FCbxheXmZWyO09bc/JVWHJRft4eeu+FcSr6eZ1q13LT448/rldeeUW1atWy8zcBALg6ihMcyrZt2zR+/HitXbtWXl5eGjJkiF599VU1aNBA58+f1x133KHk5GS1bNlSFotVyWkHNWtNpopK/3jsgaX0nPq1rK2Zo+6SyeRmwLcBALgaihMcUllZmV566SW9//77OnnypKKjoxUbG6vZs2crMDBQ27Zv1ytrj2rp9tyrvteA9iF6bVA7yhMA4JpRnODw0tPTNWHChEv34rm5uanFHY/pYpTt16dMSozUyK7N7RURAFBDsDkcDi82NlZvvPGG3N3df/sH7p660LJbhd5jZuoBFZcZf4o5AMC5uRsdALBFbm6uwsLCVL9+fXm0itNhr7oVGl9UatbyjDwN7NDETgkBADUBS3VwOhOW7tJnWw5XeNy9MU31zwHRdkgEAKgpWKqD0yksuVit4wAA+A+KE5yOn7dHtY4DAOA/KE5wOh1DAyo1LiascuMAAPgPihOcTmLbIPl6Vey5BsuF85r+5DBNmDBBhw4dslMyAICrozjB6fh4uuuJ21pWaExB+mfalP6DXn31VR05csROyQAAro7iBKeUFNdcA9qH2PRa84F0FW3+SpLUrl073XLLLfaMBgBwYRQnOCWTyU2vDWqnSYmRV1y28/Vy16TESE3oFiKTyU1t2rTRnj17FBkZqYKCgmpODABwBZzjBKdXXPbb4ZZbsvNVWHJRft4eigkLUGJUsLw9a6m0tFSjRo3Sa6+9pgsXLqhz5846ffq0VqxYofj4eKPjAwCcCMUJNY7FYtHAgQP19ddf67nnntOUKVOMjgQAcBIUJ9RY7733nsaMGaObb75Zqamp8vT0NDoSAMDBUZxQo2VkZKhr164ymUxKS0tTZGSk0ZEAAA6MzeGo0aKionTs2DG1atVKbdu2VXJystGRAAAOjOKEGs/Ly0sbNmzQM888o0cffVSDBw+WxWIxOhYAwAGxVAf8l9TUVN1xxx0KDAzUpk2bFBQUZHQkAIADYcYJ+C8JCQnKzc1V7dq1FRoaqpSUFKMjAQAcCMUJ+J2AgADt379fQ4cOVd++ffXUU08ZHQkA4CBYqgP+xCeffKKHHnpIUVFRSktLU506dYyOBAAwEMUJuIqsrCx16dJFJSUlSk1NVceOHY2OBAAwCEt1wFWEh4crNzdXXbp0UefOnTVjxgyjIwEADEJxAmzg7u6u7777TtOmTdP48ePVu3dvmc1mo2MBAKoZS3VABW3ZskUJCQmqU6eONm7cqLCwMKMjAQCqCTNOQAXFxMTo2LFjCgoKUkREhD7//HOjIwEAqgnFCaiEunXraseOHRo9erSGDh2qESNGGB0JAFANWKoDrtGyZcs0cOBAhYWFacOGDQoICDA6EgDATphxAq5R3759lZ2drZKSEjVp0kRr1641OhIAwE4oTkAVCA4OVnZ2thITE5WQkKDnnnvO6EgAADtgqQ6oYrNnz9bo0aPVuXNnrVmzRp6enkZHAgBUEYoTYAd79uxR165dJUk//PCD2rRpY3AiAEBVYKkOsIM2bdooLy9PN9xwg9q1a6fZs2cbHQkAUAUoToCdeHp6Ki0tTc8++6z+7//+TwMHDpTFYjE6FgDgGrBUB1SDtWvXKjExUQ0aNNDmzZsVFBRkdCQAQCUw4wRUg27duik3N1fe3t4KDQ3VsmXLjI4EAKgEihNQTa677jr98ssvGjZsmPr166e//OUvRkcCAFQQS3WAARYuXKjhw4erTZs2SktLU926dY2OBACwAcUJMEh2dra6dOmi8+fPKzU1VTExMUZHAgBcBUt1gEHCwsJ05MgRxcbGqkuXLpo+fbrRkQAAV8GME+AAZsyYofHjx6t79+5KSUmRu7u70ZEAAJdBcQIcxNatW5WQkCBvb29t3LhR4eHhRkcCAPwOS3WAg+jYsaPy8vIUEhKiVq1a6ZNPPjE6EgDgdyhOgAPx8fHRtm3bNGbMGD3wwAN66KGHjI4EAPgvLNUBDmrZsmUaOHCgQkNDtXHjRgUEBBgdCQBqPGacAAfVt29f5eTk6MKFC2rSpIlSU1ONjgQANR7FCXBgQUFBysrKUmJionr06KFJkyYZHQkAajSW6gAnMXv2bI0ePVqdOnVSamqqvLy8jI4EADUOxQlwInv37lVcXJwsFot++OEHRUVFGR0JAGoUluoAJxIZGaljx46pTZs2uvHGG/Xee+8ZHQkAahSKE+BkPD09lZaWpsmTJ+vxxx/XgAEDZLFYjI4FADUCxQlwUlOmTNHatWu1YsUKhYWF6ejRo7JYLHrllVd05swZo+MBgEviQizAicXHx+vo0aPq0qWLwsPDNWjQIC1cuFCFhYV6+eWXJUnFZWal7M7T1px8FZZclJ+3hzqGBiixbZB8PPktAAAqgs3hgIvo06ePVqxYIUmqU6eOjuTmatHOU5q1JlNFpeY/vN7Xy11P3NZSSXHNZTK5VXdcAHBKFCfABZw8eVItW7bU2bNnJUlubiZ1mzBHB60Nrzp2QPsQvTaoHeUJAGzAHifABVitVvXt21fNmzdXrVq15Btzl02lSZKWbs/VnPQsOycEANfAjBPgYs4UFSv21bUqNtv+v7avl7s2TezOnicAuApmnAAXs3p/foVKkyQVlZq1PCPPTokAwHVQnAAXszUnv1LjtmRXbhwA1CQUJ8DFFJZcrNZxAFCTUJwAF+Pn7VGt4wCgJqE4AS6mY2hApcY197XKbP7jeU8AgP+P4gS4mMS2QfL1qtjTcZYL5/XXgbfKx8dHp0+ftlMyAHB+FCfAxfh4/nYieEUUpH+m0nNn1bt3b9WvX99OyQDA+VGcABeUFNdcA9qH2PTa8swfVbT5K1mtVq1atUqLFi2yczoAcF4UJ8AFmUxuem1QO01KjLzisp2vl7smJUbq7Qe6SLLqo48+0r333qt7771XsbGxKigoqN7QAOAEODkccHHFZb8dbrklO1+FJRfl5+2hmLAAJUYFy9uzlqxWq9LT0xUbGys3Nzft2LFDiYmJOn36tN58802NGjXK6K8AAA6D4gTgssaNG6fXX39dUVFRWrFihYKDg42OBACGY6kOwGVNnz5d+/fvV3FxsZo2baqpU6caHQkADMeME4CrmjZtmp577jk1a9ZMK1asUEREhNGRAMAQzDgBuKqJEyfq8OHDqlevnlq3bq2nnnpKFovF6FgAUO2YcQJQIbNnz9YTTzyhgIAAffvtt+rQoYPRkQCg2jDjBKBCHn30UZ08eVIRERHq2LGjhg8fzuwTgBqD4gSgwurVq6f169dr0aJFWrJkierXr6/Vq1cbHQsA7I7iBKDSBg0apNOnTysuLk49e/bUnXfeqQsXLhgdCwDshuIE4Jp4eXnp3//+t1atWqW0tDTVr19fS5YsMToWANgFxQlAlUhISNDp06c1cOBADR48WPHx8Tp79qzRsQCgSlGcAFQZk8mk+fPna/PmzTpw4IAaNmyo2bNnGx0LAKoMxQlAlevYsaNyc3M1evRojRo1Su3bt1deXp7RsQDgmlGcANiFyWTSG2+8oX379uns2bNq2rSppk2bZnQsALgmHIAJoFpMnTpVU6ZMUXh4uFasWKEWLVoYHQkAKowZJwDVYvLkyTp06JB8fHzUqlUrjRs3zuhIAFBhzDgBqHbvvfee/vrXv6pBgwb69ttvdeONNxodCQBswowTgGo3atQoHT9+XOHh4erQoYMefvhhrm0B4BQoTgAM4e/vr7S0NC1cuFCLFi1Sw4YNtXbtWqNjAcCfojgBMNQ999yj06dPq0uXLkpISFD//v1VVlZmdCwAuCyKEwDDeXl56dtvv9XKlSu1du1aBQQE6KuvvjI6FgD8AcUJgMPo2bOnTp06pf79+2vAgAHq1q2bzp07Z3QsALiE4gTAobi7u2vBggXatGmT9u7dqwYNGmju3LlGxwIASRQnAA4qJiZGx44d06OPPqqRI0fqpptu0okTJ4yOBaCGozgBcFgmk0kzZ87Uzz//rPz8fIWEhGj69OlGxwJQg1GcADi866+/XllZWZo8ebImTpyo66+/XtnZ2UbHAlADUZwAOI1//OMfys7Olqenp1q0aKFnnnnG6EgAahiuXAHglN5++209/fTTCgwMVEpKiqKjo42OBKAGYMYJgFMaM2aMTpw4oWbNmunGG29UUlIS17YAsDuKEwCn5e/vrx9//FGffPKJPv30UwUGBiotLc3oWABcGMUJgNMbOnSoTp06pZiYGMXHx2vgwIFc2wLALihOAFyCj4+Pli9frm+//VarVq1S/fr19c033xgdC4CLoTgBcCl9+vTR6dOn1bdvX/Xv31/du3fn2hYAVYbiBMDluLu7a+HChdqwYYN2796thg0bav78+UbHAuACKE4AXFbnzp2Vl5enESNGaMSIEYqJidGpU6eMjgXAiVGcALg0k8mkd955RxkZGTp58qSCg4P1+uuvGx0LgJOiOAGoESIjI5Wdna2JEydq/PjxioyMVE5OjtGxADgZihOAGuWFF17QwYMHZTKZ1Lx5c02aNMnoSACcCFeuAKix3nrrLY0bN05BQUFKSUlRVFSU0ZEAODiKE4AaLT8/X4mJidq8ebMeffRRvfvuu1q7dq3ee+89LVq0SG5ubpKk4jKzUnbnaWtOvgpLLsrP20MdQwOU2DZIPp7uBn8LANWF4gQAkhYsWKCRI0eqTp06Ki8vV3FxsebPn68hQ+5RctpBzVqTqaJS8x/G+Xq564n/196dR0V532sAf2aYGYZhFRDZdADZorhg0bhgUhCDnViVxeq9nrS2MWmsit4sJ+Z4jlqz2JtoNJqkjcHcpKWkMSwKJmg0AVFI3CPSoggIAoLIJssAAzNz/7CX3jSA7yDwDvB8/kvO+3t5Jn+Eh/f9ze8b5os1oT6QSiUiJCeiocTiRET0T1qtFgEBAaioqAAAODo6YdX+Y0i7MYQG7QAAGhtJREFUeueBa6ODPbArdhrLE9EIx83hRET/VFBQgMrKyu5/7vJ9TFBpAoCUy5U4mHNzsKIRkZmw2L59+3axQxARmYOOjg5YWFjA2dkZcitrSB57DhKZQvD678sbsXquF+QW/JuUaKTiqzoioh4kXazAi0lXTF63e/k0xMzwHIRERGQO+GcREVEPLpTV92vd+dJ6NDY24vvvvx/gRERkDliciIh6cK+ts1/rPj/yBZydnREeHj7AiYjIHLA4ERH1wN5K3q91jTVV0Ov1sLa2RnFx8QCnIiKxsTgREfUgRO3Yr3W6ir8DAGpqauDr6wu5XA5PT09ERkbijTfewPXr1wcyJhENMRYnIqIeaKa4wlZp2ong1gop3PR3IJVKkZCQAJ1Oh7S0NCxduhR1dXV48803ERgYCJlMBnd3d0RERGDHjh3Iz88fpE9BRAON36ojIurFgexivJFxTfD1WzSP4KmZ7njzzTexfv16ODr++KlVV1cXsrKykJ6ejm+//RZFRUVobGyERCKBi4sLAgMDMX/+fERHR2P69OkD+XGIaACwOBER9cJgMOLFpCtIuVz5wGsf5uRwg8GA7OxspKenIzc3F4WFhWhoaIBEIoGzszMCAwMxb948REVF4Sc/+QmkUr4sIBILixMRUR8MBiMO5tzEvm9u9DqrLi7cD0/P8x7QcSsGgwG5ublIS0tDTk4OCgsLUVdXBwBwcnJCQEAA5s2bhyVLlmDOnDksU0RDhMWJiEgAra4LH3x5Dm98eAhQqGBvJcMfXngGT05xh5XCYkgyGAwGXLhwAampqcjJycG1a9dQV1cHo9EIR0dH+Pn5Ye7cuVi6dClCQ0NZpogGAYsTEZFAy5cvR3JyMoxGI6ytrREfH4+VK1eKHQuXLl3C4cOHcfr0aRQUFODu3bswGo1wcHCAn58fZs+ejSVLliAsLIxliughsTgREQnQ2toKe3t7yOVytLe3Qy6XIyQkBLm5uWJH69HVq1eRmpqKU6dOoaCgADU1NdDr9XBwcMDEiRMxZ84caDQaLFy4EDKZad8eJBrNWJyIiAQwGo24dOkSrl27hqeeegq3bt2Ck5MTrKysxI4mWEFBAVJSUpCdnY38/HzU1NSgq6sLdnZ2mDhxIh599FFoNBpERkZCoRA+3JhoNGFxIiIyQVlZGby8vDBS/tdZVFSElJQUZGZmIj8/H9XV1ejq6oKtrS18fHwwc+ZMaDQa/OxnP4NSqRQ7LpHoWJyIiEzQ1tYGlUoFvV4/YvcLlZaWIjk5GZmZmbh69SqqqqrQ2dkJGxsbeHt7IyQkBIsWLcLixYuhUqnEjks0pFiciIhMJJFI0NDQAAcHB7GjDJny8nKkpqbi66+/Rl5eHm7fvg2dTgdra2uo1WqEhIQgMjISS5YsgY2NjdhxiQYNixMRkYmkUimuX78OPz8/saOIqqqqCqmpqTh58iSuXLmCyspKdHR0QKVSYcKECZgxYwYiIyOxdOlS2Nvbix2XaECwOBERmUgmk+H06dOYM2eO2FHMTk1NDQ4fPowTJ07g+++/R0VFBdrb22FlZYXx48cjODgYCxcuRFRUVI8jaYjMHYsTEZGJFAoFUlJSsHjxYrGjDAv19fXdZerSpUsoLy9HW1sblEolPD09MX36dERERCAmJgbOzs5ixyXqE4sTEZGJVCoV/vSnP+GXv/yl2FGGraamJhw+fBjHjx/HpUuXcOvWLWi1WlhaWsLDwwPTpk1DREQEoqOj4erqKnZcom4sTkREJrK3t8eOHTuwceNGsaOMKC0tLUhPT8exY8dw4cIFlJWVobW1FQqFAu7u7pg6dSrCw8MRExMDT0/PQc2i1XXhy6vVuFBWj3ttnbC3kiNE7QjNFFeoFDwwdDRjcSIiMtHYsWOxfv16bNu2TewoI55Wq8XRo0e7y9TNmzfR0tICuVwONzc3TJkyBWFhYYiNjYVarX7on2cwGBF/pgT7M4t6Heq8IcwXa0J9BnSoMw0fLE5ERCby8PDAihUr8Pbbb4sdZVRqb29HRkYGMjIycO7cOZSUlKC5uRkymQyurq4ICgpCWFgYYmJiMHHixF7vk52dDR8fn+6nVwaDES8mXUHK5coHZogO9sCu2GksT6MQixMRkYkmTpyIsLAwxMfHix2F/kmn0+Grr77CF198gXPnzqG4uBj37t2DTCaDi4sLJk+ejMcffxwxMTEIDAwEAEyYMAGNjY1ISUlBREQEDmQX442Ma4J/5hbNI3hmvs9gfSQyUyxOREQmmjx5MiZPnoxDhw6JHYX60NXVha+//hpHjx7Ft99+i+LiYjQ2NsLCwgLOzs6oqamB0WiEpaUlXty8BUcks3p8PdcbW6UMZ19ZwD1Po8zInBdARDSIVCoVWlpaxI5BDyCTyRAZGYn9+/fjwoULaGhoQGdnJ44fP47HHnsMEsn912wdHR3Ym5xlUmkCgOb2LmTkVw9GdDJjLE5ERCZSqVRobW0VOwb1g0wmw4IFCxAaGgqpVAqFQgFfX19MenxJv+53vrR+gBOSuWNxIiIykY2NDYvTMLdo0SIkJCSgsrISN27cwE/mhPbrPvfaOgc4GZk7FiciIhPZ2Nigra1N7Bj0EPz9/bFixYruk8rtreT9uk9u1kmEh4ejqKhoIOORGWNxIiIyka2tLdrb28WOQQMoRN2/uXnF332FM2fOwMHBYYATkblicSIiMpG9vT2L0wijmeIKW6Vp344ztLdAey0HEokE8fHxMBgMg5SOzAmLExGRiezt7dHR0SF2DBpAKsX9E8FN0XouGQoLYPHixdi+fTusra2xZs0aNDU1DVJKMgcsTkREJnJwcEBnJzcFjzRrQn0QHewh6NroYA+8uioMUVFRSE5OhlarxdatW3H48GGMGTMGCxcuxI0bNwY5MYmBB2ASEZkoISEBzz77LLRardhRaIAZDEYczLmJfd/c6HVWXVy4H56e593ruJXU1FS8/PLLKCoqwqRJk7B7925ERkYOdnQaIixOREQm+vLLL7Fs2TLodDqxo9Ag0eruH255vrQe99o6YW8lx0wvR2iC3GClsBB0j7y8PKxbtw45OTkYN24cNm/ejA0bNkAq5cue4YzFiYjIRGfPnsXcuXOh1+vFjkLDQG1tLeLi4pCUlAS5XI7Vq1fjrbfegkqlEjsa9QNrLxGRiZydncG/OUkoZ2dnJCYmQqvV4vnnn0diYiLs7Ozw5JNPoqysTOx4ZCIWJyIiE40dO5bFiUwmk8nw6quvoqGhAZ988gn+8Y9/wNvbG8HBwcjOzhY7HgnE4kREZILy8nLk5+cDAA4dOoRjx46JnIiGo1WrVuHmzZv47rvvoFAo8NOf/hSenp6Ij48XOxo9APc4ERGZwNfXF5WVlWhvb4dSqYSlpSUaGhogkfT8DSsiIaqqqrB+/XqkpaVBqVTiueeew+uvvw6FQiF2NPo3fOJERGSCLVu2wMLi/reqDAYD1q1bx9JED83NzQ3JyclobW3Fc889hw8++ADW1taIiYnB7du3xY5H/w+fOBERmUCv1yMwMBBFRUWQy+WoqKiAi4uL2LFoBPrwww+xY8cOVFZWYubMmXj33Xcxc+ZMsWONenziRERkgg69EU9tfQ+OizZg4q924u0z1Ui6WAGt7seHJRI9jGeeeQbl5eXIysqCTqfDo48+Cm9vb/z1r38VO9qoxidOREQCGAxGxJ8pwf7Mol5PlN4Q5os1oT69nihN9DDKysqwbt06HDt2DLa2tli/fj22bdsGmcy04cT0cFiciIgewGAw4sWkK0i5XPnAa6ODPbArdhrLEw0arVaLl156CR9//DE6OzsRGxuLffv2wdnZWexoowJf1RERPUD8mRJBpQkAUi5X4mDOzUFORKOZSqXCe++9h+bmZuzatQuZmZlwcXHB/PnzkZeXJ3a8EY/FiYioD1pdF/ZnFpm0Zt83N7jniQadVCpFXFwcqqqqkJGRgYaGBkyfPh3+/v5ITU0VO96IxeJERNSHL69W97inqS/N7fcHxBINlcjISOTn5+PatWtQq9WIjY2Fs7Mzdu7cCYPBIHa8EYXFiYioDxfK6vu17nxp/9YRPQx/f3+cOHECDQ0NWLZsGXbs2AGVSoXf/OY3aGpqEjveiMDiRETUh3ttnUO6jmgg2NnZIT4+Hq2trdi+fTvS09MxZswYRERE4Pr162LHG9ZYnIiI+mBvJR/SdUQDSSqVYvPmzbh79y6Sk5NRXl6ORx55BJMnT0ZGRobY8YYlFicioj6EqB37te7A6y9j/PjxOHPmzAAnIuqfZcuW4fr168jLy4OTkxOefPJJuLq6Yu/evdwHZQIWJyKiPmimuMJWadoBg4b2FjTmZaK+vh7+/v6DlIyof4KCgpCdnY3a2lpERETg5Zdfho2NDdatWwetVit2PLPH4kRE1AeV4v6J4Ka4l/sZjF0d6OzsxFdffTVIyYgejqOjIxISEtDa2oqXXnoJiYmJsLOzg0ajQVlZmdjxzBaLExHRA6wJ9UF0sIegaxf42EB76SiCgoLw1FNP4Ve/+hWmTp2KW7duDXJKov6RyWT4/e9/j4aGBvz5z3/GtWvX4O3tjeDgYGRnZ//g2rS0NHR1/fB4Dq2uC0kXK7A5JQ9r/3oRm1PyRvT8Ro5cISISwGAw4mDOTez75kavs+riwv3w9DxvfPppIkJCQhAQEIAbN25gyZIlKCwsxO9+9zu88847kEr5NyuZt/Pnz2PDhg04d+4c3N3dsXXrVkybNg2zZ8/G2rVr8f7774/a+Y0sTkREJtDq7h9ueb60HvfaOmFvJcdML0dogtxgpbDodd2BAwewceNGqFQqJCYmIjIycghTE/VPdXU11q9fjyNHjsBoNEKv18PKygrvvLMPBfYho3J+I4sTEdEQaW9vx8qVK5GWlobQ0FAcOXIEY8aMETsW0QMVFhZi0qRJ0Ov1AAD7R6PhEPYbweu3aB7BM/N9BivekOLzYiKiIaJUKnH48GGcPXsWJSUlcHFxweuvvy52LKIH+vTTTwEA1tbWkFtZw27OL0xaP5LmN/KJExGRSHbs2IFXX30Vrq6uSE1NRUhIiNiRiHrU1NSEW7duwdraGpmlbdjx1U2T77F7+TTEzPAchHRDi0+ciIhEsnXrVlRXV0OtVmPWrFmIiopCe3u72LGIfsTOzg5BQUHw9vZGYUP/nhyNlPmNLE5ERCJycnLCmTNncPToUWRlZcHR0REHDx4UOxZRr0b7/EYWJyIiM6DRaFBXV4fVq1fj2WefxaRJk1BSUiJ2LKIfGe3zG1mciIjMhFQqxfvvv4/i4mJIpVL4+fnht7/9LeeIkVnp7/zGmV79W2duWJyIiMyMl5cX8vPzcfDgQSQkJMDR0RHp6elixyICcH9+o00fZ5b1xFYpgybIbZASDS0WJyIiM7V69Wo0NDRg4cKFWLp0KebOnYva2lqxY9EodvLkSSxe9ARun/zIpHVx4X59HhA7nLA4ERGZMYVCgc8//xwXL15EZWUl3NzcsG3bNrFj0Si1adMmnDp1Cg25SWi5+rWgNdHBHnh6nvcgJxs6LE5ERMNAcHAwysrK8Nprr2Hnzp1wd3fHd999J3YsGmUyMzMhk8kAGNGWeQARTk2wVcp6vNZWKcMWzSMjatwKwAMwiYiGncbGRkRFReHUqVPQaDQ4dOgQVCqV2LFoFAgNDcXly5cRHByMnJwclJeXw9HFtV/zG4crFiciomHq5MmTWLlyJVpaWrBnzx6sXbtW7Eg0gi1fvhxHjhzBlStX4Ofnh2+++QZPPPGE2LGGHIsTEdEwZjAY8MILL2Dfvn3w9fVFeno6/P39xY5FI8zzzz+Pd955B5mZmXjsscfEjiMq7nEiIhrGpFIp9uzZg7KyMqhUKgQGBuLXv/41urpGxkBVEt+ePXuwd+9eJCYmjvrSBLA4ERGNCJ6enrh8+TISEhLw+eefw9HREcnJyWLHomHu0KFDeOGFF/DWW29hxYoVYscxC3xVR0Q0wuh0OqxevRp/+9vfEBISgrS0NLi6uoodi4aZM2fO4PHHH0dcXBz27NkjdhyzwSdOREQjjEKhQGJiIvLy8lBbWwtPT09s3ryZo1tIsIKCAoSHhyMqKoql6d/wiRMR0Qi3Z88ebN68GQ4ODkhKSsL8+fPFjkRmrKamBj4+Ppg6dSpyc3PFjmN2WJyIiEaBlpYWxMTE4MSJE1i4cCGSk5NhY2MjdiwyM1qtFl5eXrCzs0NhYSGkUr6Y+nf8L0JENArY2Njg+PHjyMrKwuXLl+Hk5IS9e/f+4BqdTidSOjIHBoMBU6dOBQDk5eWxNPWCT5yIiEYZg8GAV155Bbt374ZarUZaWhpu376NmJgYFBQUwMPDAwCg1XXhy6vVuFD2rxOhQ9SO0ExxhUrR85gNGr7mzJmDq1evoqSkBC4uLmLHMVssTkREo1R1dTV+/vOf4+LFi7C0tERHRwcWLVqEo0e/QPyZEuzPLEJz+4/Pg7JVyrAhzBdrQn1G1Ayy0Sw6OhpffPEF8vLyEBAQIHYcs8biREQ0ykVHRyM1NRUAoFBYYunOJJy7++BCFB3sMeIGuI5GGzduxLvvvotTp04hNDRU7Dhmjy8wiYhGsdraWqSnp0OhUAAAlNM1gkoTAKRcrsTBnJuDGY8G2a5du7B//3589tlnLE0C8YkTEdEoptfrkZGRgcrKSlRU30Vi6yToLSwFr7dVynD2lQXc8zQMffrpp1i1ahXefvttbNq0Sew4wwaLExERAQCSLlbgxaQrJq/776jJkJSewyeffIKPPvoI48aNG4R0NJCysrKwYMECbNq0Cbt37xY7zrDCPxGIiAgAcKGsvl/rNr62H9qsD9HW1obm5mYWJzP397//HU888QRiY2NZmvqBxYmIiAAA99o6+7VOb2GJ5uZmAMDs2bNhb2+PMWPGYOzYsXB1dYWnpyfUajV8fHzg7+8Pd3d3nhE0xBITE/GXv/wF7733HmbNmoVZs2bhs88+EzvWsMTiREREAAB7K3m/1kk62yCRSCCRSLB27VpUVlaiuroad+/exY0bN9DU1AStVouOjg50dd0/3kAqlcLS0hJWVlawtbXFmDFj4OzsjHHjxsHDwwNqtRre3t7w9/eHWq2GTMZfVw/jwIEDOH36NPz9/eHl5YXs7GyxIw1b3ONEREQA+r/H6b9mO+Lj7b9DVVUVqqurH3h9fX09rl+/juLiYpSWlqKiogJVVVWoqalBfX09mpqa0Nraivb2dnR23n8KJpVKIZfLu4uWg4MDnJycMG7cOLi7u2PChAnw9vaGr68v/Pz8ur8lOJr0dmBpRIATXJ3HoKOjAwDg7u6O3NxcqNVqkRMPTyxOREQE4P4v3kd3ft3joZe9sVXKcO6VCCjlUty5cweurq4Dnqu1tRWFhYUoKirCzZs3UV5ejqqqKty5cwf19fVobGxES0tLd9EyGo2QSCSQy+VQKpWwsbGBvb09HB0d4eLiAnd3d4wfPx5eXl7w9fVFQEDAsJ7bZzAY+zyw1EoGVJ34CPfOpkKlsoJer0dCQgJiY2NFSDv8sTgREVG3A9nFeCPjmuDrt2gewTPzfQYxkel0Oh2KiopQVFSEkpISlJeXo7KyEnfu3EFdXR0aGxvR3NyMtrY2dHZ2wmAwAADkcjksLS1hbW3dXbTGjh0LNzc3eHp6wsvLCxMnTkRAQACcnJxE/pT3GQxGvJh0BSmXKx94raT0LDb/1BP/+Z//AVtb2yFINzKxOBERUTdTfhGPlJPD9Xo9ysrKUFhYiJKSEty6dat7n1ZtbW130dJqtdDpdNDr9QAACwuL7qJlZ2fX54Z4Nze3AdkQ39LSglOnTkGj0UAikYyIojvcsDgREdEPGAxGHMy5iX3f3Oh1Vl1cuB+enuc97EtTfxgMBlRVVXUXrbKyMlRUVHRviG9oaOhxQ7yFhQUUCgVUKlX3Pi1nZ2e4urrCw8MDEyZM6C5aarUaFhYWP/rZaWlpWLp0KWbMmIEPDv4Pfn2kyuRXqzyw9OGwOBERUY+0ui5k5FfjfOm/NhvP9HKEJsgNVoof/1Kn3tXV1aGwsLDPDfEtLS3o6Ojoc0P8/72G1Ov1sJu2EGN+ttHkLLuXT0PMDM+B/oijBosTERGRmWlpaeneEF9aWory8nLcvn0bFy9eRFlZGQDAcdEG2E6PNPneK2eOxx+ipw505FGDz+qIiIjMjI2NDWbMmIEZM2b84N/HxcXhj3/8I5RKJfyDpqGqH/fu70GndB+PbiUiIhomfvGLX+Djjz/G3bt38ficmf26R38POqX7WJyIiIiGidDQUKxatQpKpRIhasd+3WOmV//W0X0sTkRERMOQZoorbJWm7bixVcqgCXIbpESjA4sTERHRMKRSyLAhzNekNXHhfvxG5ENicSIiIhqm1oT6IDrYQ9C10cEeeHqe9yAnGvl4HAEREdEwxgNLhxaLExER0QjAA0uHBosTERERkUDc40REREQkEIsTERERkUAsTkREREQCsTgRERERCcTiRERERCQQixMRERGRQCxORERERAKxOBEREREJxOJEREREJBCLExEREZFALE5EREREArE4EREREQnE4kREREQkEIsTERERkUAsTkREREQCsTgRERERCcTiRERERCQQixMRERGRQCxORERERAKxOBEREREJxOJEREREJBCLExEREZFALE5EREREArE4EREREQnE4kREREQkEIsTERERkUAsTkREREQCsTgRERERCcTiRERERCQQixMRERGRQCxORERERAKxOBEREREJxOJEREREJBCLExEREZFALE5EREREArE4EREREQnE4kREREQkEIsTERERkUAsTkREREQCsTgRERERCcTiRERERCQQixMRERGRQCxORERERAKxOBEREREJxOJEREREJBCLExEREZFALE5EREREArE4EREREQn0v4teNPOdZaNYAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 576x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ac58407"
      },
      "source": [
        "We can also plot the vertices attributes (a.k.a. the digits):"
      ],
      "id": "5ac58407"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "950ebb3f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "outputId": "377d784d-8e08-496d-833e-4262f997ff64"
      },
      "source": [
        "digit = x[5].reshape(28,28)\n",
        "plt.matshow(digit, cmap='gray')"
      ],
      "id": "950ebb3f",
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f1365aa8fd0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAECCAYAAAD+eGJTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOPUlEQVR4nO3df6hXdZ7H8dfLqf5I+8OyFSmtnTB0KNb0IgtJtAw7tNMf6j+2RWWwrAUGE+wfmkH1z6gsU7vzl+BsMQaNS6BWDMFMxIAGEllIqde2YbBUTFf8o1/Ior73j3ucuVP3fs71e77f7zn6fj5A7vee9/d7zttz9XXPj8/383VECEBe09puAEC7CAEgOUIASI4QAJIjBIDkCAEguVZCwPZ9tj+x/Ufb69voocT2Edsf295ve18H+nnZ9inbB8Ytu97227Y/rb7O7Fh/z9s+Xu3D/bZ/2mJ/c23/wfYh2wdt/6xa3ol9WOhvKPvQwx4nYPsHkv5H0j9KOibpfUkPRsShoTZSYPuIpJGION12L5Jk+x5JX0t6JSLuqJb9u6QzEbG5CtKZEbGuQ/09L+nriPhFGz2NZ3uOpDkR8aHt6yR9IGmFpMfUgX1Y6G+VhrAP2zgSWCrpjxHxp4j4P0n/LWl5C31cNiJit6Qz31m8XNK26vE2jf2jacUk/XVGRJyIiA+rx19JGpV0kzqyDwv9DUUbIXCTpKPjvj+mIf6Fpygk/d72B7bXtN3MJGZHxInq8ReSZrfZzCSetP1RdbrQ2unKeLZvlXSXpPfUwX34nf6kIexDLgxObFlELJb0T5LWVoe7nRVj53RdG/+9RdJtkhZJOiHphXbbkWzPkLRD0lMR8eX4Whf24QT9DWUfthECxyXNHff9zdWyzoiI49XXU5J2aewUpmtOVueSF88pT7Xcz1+JiJMRcT4iLkj6lVreh7av1th/sFcjYme1uDP7cKL+hrUP2wiB9yXNt/23tq+R9M+S3myhjwnZnl5dnJHt6ZJ+IulA+VWteFPS6urxaklvtNjL91z8z1VZqRb3oW1LeknSaES8OK7UiX04WX/D2odDvzsgSdWtjv+U9ANJL0fEz4fexCRs/1Bjv/0l6SpJv2m7P9vbJd0raZakk5Kek/S6pNckzZP0maRVEdHKxblJ+rtXY4exIemIpMfHnX8Pu79lkvZI+ljShWrxBo2dd7e+Dwv9Pagh7MNWQgBAd3BhEEiOEACSIwSA5AgBIDlCAEiu1RDo8JBcSfTXVJf763Jv0nD7a/tIoNM/CNFfU13ur8u9SUPsr+0QANCyRoOFbN8n6ZcaG/n3XxGxueb5jEwCWhIRnmh5zyHQy+QghADQnslCoMnpAJODAFeAJiFwOUwOAqDGVYPeQHWro+tXYoG0moTAlCYHiYitkrZKXBMAuqjJ6UCnJwcBMDU9HwlExDnbT0r6nf4yOcjBvnUGYCiGOqkIpwNAewZxixDAFYAQAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJK7qu0G0D8LFiwo1qdPn16snz59utH2Z82aVay///77xfqFCxcabb/OtGnl33k7duwo1jdv3lysj46OFuvffvttsd6WRiFg+4ikrySdl3QuIkb60RSA4enHkcA/RESzXyEAWsM1ASC5piEQkn5v+wPba/rREIDhano6sCwijtv+G0lv2z4cEbvHP6EKBwIC6KhGRwIRcbz6ekrSLklLJ3jO1ogY4aIh0E09h4Dt6bavu/hY0k8kHehXYwCGwxHR2wvtH2rst780dlrxm4j4ec1rettYEnX3+e+5555ifd26dcX6LbfcUqzv3r27WLddrN99993Fet19+rbHCTTd/tKl3zsQ/iv79+9vtP6mImLCH2DP1wQi4k+S/q7njgB0ArcIgeQIASA5QgBIjhAAkiMEgOQIASA55hMYorr79Fu2bCnW6+7DN7Vs2bKBrv+JJ54Y6PrrzJs3r1hfv379kDrpFo4EgOQIASA5QgBIjhAAkiMEgOQIASA5QgBIruf5BHraWPL5BA4ePFisz58/f6Dbbzrv/qZNmxptv+330y9ZsqRY37t3b6P1f/LJJ8X6nXfe2Wj9TU02nwBHAkByhACQHCEAJEcIAMkRAkByhACQHCEAJMd8Apegbj6At956q1hfuHBhsd503vvR0dFive371G2r+9yEunEUV6qcf2sAf0YIAMkRAkByhACQHCEAJEcIAMkRAkByjBO4BHXz0tfNB1A3DqCuvmfPnmL9scceK9azq5s7o+k4jV27djV6fVtqjwRsv2z7lO0D45Zdb/tt259WX2cOtk0AgzKV04FfS7rvO8vWS3onIuZLeqf6HsBlqDYEImK3pDPfWbxc0rbq8TZJK/rcF4Ah6fXC4OyIOFE9/kLS7D71A2DIGl8YjIgoTSBqe42kNU23A2Awej0SOGl7jiRVX09N9sSI2BoRIxEx0uO2AAxQryHwpqTV1ePVkt7oTzsAhq32dMD2dkn3Sppl+5ik5yRtlvSa7X+R9JmkVYNsMou6cQBr164t1o8ePdrPdnCJ6saRPPvss0Pq5NLUhkBEPDhJ6cd97gVACxg2DCRHCADJEQJAcoQAkBwhACRHCADJMZ/AOCtXrizWV6xo9j6pus8FqJsPgHEAGASOBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASC7VOIHp06cX64sXLy7Wb7zxxkbbP3v2bLHOOIDBsl2sT5uW83dizr81gD8jBIDkCAEgOUIASI4QAJIjBIDkCAEguVTjBOrmhV+3bl2x3vTz63fu3Nno9WgmYtJPy5PU/Od7ueJIAEiOEACSIwSA5AgBIDlCAEiOEACSIwSA5Fx377SvG7OHt7EJnDt3rlgf9H3ia665ZqDrR9mSJUuK9b179w50+23//CNiwgkVao8EbL9s+5TtA+OWPW/7uO391Z+f9rNZAMMzldOBX0u6b4Ll/xERi6o/b/W3LQDDUhsCEbFb0pkh9AKgBU0uDD5p+6PqdGFm3zoCMFS9hsAWSbdJWiTphKQXJnui7TW299ne1+O2AAxQTyEQEScj4nxEXJD0K0lLC8/dGhEjETHSa5MABqenELA9Z9y3KyUdmOy5ALqtdj4B29sl3Stplu1jkp6TdK/tRZJC0hFJjw+wx8vGo48+2nYLaNHl+vOvDYGIeHCCxS8NoBcALWDYMJAcIQAkRwgAyRECQHKEAJAcIQAkl2o+gbr5AgY9n8Add9xRrB8+fHig27/SLViwoFg/dOhQsd705z8yUh4Uu3///kbrb6rn+QQAXNkIASA5QgBIjhAAkiMEgOQIASA5QgBIrvatxFeStscJvPLKK8X60qWTTtCEKajbv23//LuKIwEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJJLNU5g48aNxfr69esHuv158+YV61u2bCnWX3311WL93XffveSeLicPP/xwsT537tyBbr9u/x87dmyg2x8UjgSA5AgBIDlCAEiOEACSIwSA5AgBIDlCAEgu1ecOXHvttcX67bffXqxv2LChWF+xYkWxPm1aOXPr3s9+9OjRYv3+++8v1tv+XIO6zwWomw+gbhzADTfcUKzX7f8dO3YU62vXri3WT58+Xay3refPHbA91/YfbB+yfdD2z6rl19t+2/an1deZ/W4awOBN5XTgnKR/i4gfSfp7SWtt/0jSeknvRMR8Se9U3wO4zNSGQESciIgPq8dfSRqVdJOk5ZK2VU/bJql8LAygky7pwqDtWyXdJek9SbMj4kRV+kLS7L52BmAopvwGItszJO2Q9FREfGn/5RpDRMRkF/1sr5G0pmmjAAZjSkcCtq/WWAC8GhE7q8Unbc+p6nMknZrotRGxNSJGIqL8ka0AWjGVuwOW9JKk0Yh4cVzpTUmrq8erJb3R//YADFrtOAHbyyTtkfSxpIs3sjdo7LrAa5LmSfpM0qqIOFOzrlbHCTT1zDPPFOt18xHMmDGjWD9//vwl93Qpmo5TuNy3v3379mL9kUceGej22zbZOIHaawIR8a6kCV8s6cdNmgLQPoYNA8kRAkByhACQHCEAJEcIAMkRAkByqeYTGLSnn366WF+8eHGxvnz58n628z1t36cf9PZ37dpVrD/wwAON1n+563k+AQBXNkIASI4QAJIjBIDkCAEgOUIASI4QAJKb8vRiqLdp06ZifdasWcX6N998U6w/9NBDl9xTl3z++efF+saNGxut//XXX2/0+qw4EgCSIwSA5AgBIDlCAEiOEACSIwSA5AgBIDnmE+iQunEEN998c6P1L1y4sFjftm1bsV53H7/uPv3Zs2eL9cOHDxfraIb5BABMiBAAkiMEgOQIASA5QgBIjhAAkiMEgORqxwnYnivpFUmzJYWkrRHxS9vPS/pXSf9bPXVDRLxVsy7GCQAtmWycwFRCYI6kORHxoe3rJH0gaYWkVZK+johfTLUJQgBoz2QhUDuzUESckHSievyV7VFJN/W3PQBtuaRrArZvlXSXpPeqRU/a/sj2y7Zn9rk3AEMw5RCwPUPSDklPRcSXkrZIuk3SIo0dKbwwyevW2N5ne18f+gXQZ1N6A5HtqyX9VtLvIuLFCeq3SvptRNxRsx6uCQAt6fkNRLYt6SVJo+MDoLpgeNFKSQeaNglg+KZyd2CZpD2SPpZ08bOjN0h6UGOnAiHpiKTHq4uIpXVxJAC0pOdbhP1ECADtYT4BABMiBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgOUIASI4QAJIjBIDkCAEgudrZhvvstKTPxn0/q1rWVfTXTJf763JvUv/7u2WywlAnFfnexu19ETHSWgM16K+ZLvfX5d6k4fbH6QCQHCEAJNd2CGxteft16K+ZLvfX5d6kIfbX6jUBAO1r+0gAQMsIASA5QgBIjhAAkiMEgOT+H/sWSJmDpHHcAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 288x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e878896d"
      },
      "source": [
        "Let's now put the process above into the ```Dataset``` class of PyG. Specifically, we are going to create an ```InMemoryDataset```. From the official PyG documentation we see that some methods need to be override:"
      ],
      "id": "e878896d"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f564647f"
      },
      "source": [
        "    torch_geometric.data.InMemoryDataset.raw_file_names(): A list of files in the raw_dir which needs to be found in order to skip the download.\n",
        "\n",
        "    torch_geometric.data.InMemoryDataset.processed_file_names(): A list of files in the processed_dir which needs to be found in order to skip the processing.\n",
        "\n",
        "    torch_geometric.data.InMemoryDataset.download(): Downloads raw data into raw_dir.\n",
        "\n",
        "    torch_geometric.data.InMemoryDataset.process(): Processes raw data and saves it into the processed_dir.\n"
      ],
      "id": "f564647f"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "654eac90"
      },
      "source": [
        "import torch\n",
        "import os\n",
        "import pandas as pd\n",
        "from torch_geometric.data import InMemoryDataset, Data, download_url, extract_zip\n",
        "from torch_geometric.utils.convert import to_networkx\n",
        "import networkx as nx\n",
        "from tqdm import tqdm\n",
        "\n",
        "class Frankenstein(InMemoryDataset):\n",
        "    \n",
        "    # Base url to download the files\n",
        "    url = 'http://nrvis.com/download/data/labeled/FRANKENSTEIN.zip'\n",
        "    \n",
        "    def __init__(self, root, transform=None, pre_transform=None):\n",
        "        super(Frankenstein, self).__init__(root, transform, pre_transform)\n",
        "        self.data, self.slices = torch.load(self.processed_paths[0])\n",
        "\n",
        "    @property\n",
        "    def raw_file_names(self):\n",
        "        # List of the raw files\n",
        "        return ['FRANKENSTEIN.edges', 'FRANKENSTEIN.graph_idx',\n",
        "                'FRANKENSTEIN.graph_labels', 'FRANKENSTEIN.node_attrs']\n",
        "\n",
        "    @property\n",
        "    def processed_file_names(self):\n",
        "        return 'data.pt'\n",
        "\n",
        "    def download(self):\n",
        "        # Download the file specified in self.url and store\n",
        "        # it in self.raw_dir\n",
        "        path = download_url(self.url, self.raw_dir)\n",
        "        extract_zip(path, self.raw_dir)\n",
        "        # The zip file is removed\n",
        "        os.unlink(path)\n",
        "\n",
        "\n",
        "    def process(self):\n",
        "        # Read the files' content as Pandas DataFrame. Nodes and graphs ids\n",
        "        # are based on the file row-index, we adjust the DataFrames indices\n",
        "        # by starting from 1 instead of 0.\n",
        "        \n",
        "        path = os.path.join(self.raw_dir, 'FRANKENSTEIN.node_attrs')\n",
        "        node_attrs = pd.read_csv(path, sep=',', header=None)\n",
        "        node_attrs.index += 1\n",
        "        \n",
        "        path = os.path.join(self.raw_dir, 'FRANKENSTEIN.edges')\n",
        "        edge_index = pd.read_csv(path, sep=',', names=['source', 'target'])\n",
        "        edge_index.index += 1\n",
        "        \n",
        "        path = os.path.join(self.raw_dir, 'FRANKENSTEIN.graph_idx')\n",
        "        graph_idx = pd.read_csv(path, sep=',', names=['idx'])\n",
        "        graph_idx.index += 1\n",
        "        \n",
        "        path = os.path.join(self.raw_dir, 'FRANKENSTEIN.graph_labels')\n",
        "        graph_labels = pd.read_csv(path, sep=',', names=['label'])\n",
        "        graph_labels.index += 1\n",
        "        \n",
        "        \n",
        "        # In the loop we extract the nodes' embeddings, edges connectivity for \n",
        "        # and label for a graph, process the information and put it in a Data\n",
        "        # object, then we add the object to a list\n",
        "        data_list = []\n",
        "        ids_list = graph_idx['idx'].unique()\n",
        "        for g_idx in tqdm(ids_list):\n",
        "            node_ids = graph_idx.loc[graph_idx['idx']==g_idx].index\n",
        "            \n",
        "            # Node features\n",
        "            attributes = node_attrs.loc[node_ids, :]\n",
        "            \n",
        "            # Edges info\n",
        "            edges = edge_index.loc[edge_index['source'].isin(node_ids)]\n",
        "            edges_ids = edges.index\n",
        "            \n",
        "            # Graph label\n",
        "            label = graph_labels.loc[g_idx]\n",
        "            \n",
        "            # Normalize the edges indices\n",
        "            edge_idx = torch.tensor(edges.to_numpy().transpose(), dtype=torch.long)\n",
        "            map_dict = {v.item():i for i,v in enumerate(torch.unique(edge_idx))}\n",
        "            map_edge = torch.zeros_like(edge_idx)\n",
        "            for k,v in map_dict.items():\n",
        "                map_edge[edge_idx==k] = v\n",
        "            \n",
        "            # Convert the DataFrames into tensors \n",
        "            attrs = torch.tensor(attributes.to_numpy(), dtype=torch.float)\n",
        "            pad = torch.zeros((attrs.shape[0], 4), dtype=torch.float)\n",
        "            x = torch.cat((attrs, pad), dim=-1)\n",
        "\n",
        "            edge_idx = map_edge.long()\n",
        "\n",
        "            np_lab = label.to_numpy()\n",
        "            y = torch.tensor(np_lab if np_lab[0] == 1 else [0], dtype=torch.long)\n",
        "            \n",
        "            graph = Data(x=x, edge_index=edge_idx,  y=y)\n",
        "            \n",
        "            data_list.append(graph)\n",
        "            \n",
        "        # Apply the functions specified in pre_filter and pre_transform\n",
        "        if self.pre_filter is not None:\n",
        "            data_list = [data for data in data_list if self.pre_filter(data)]\n",
        "\n",
        "        if self.pre_transform is not None:\n",
        "            data_list = [self.pre_transform(data) for data in data_list]\n",
        "\n",
        "        # Store the processed data\n",
        "        data, slices = self.collate(data_list)\n",
        "        torch.save((data, slices), self.processed_paths[0])  "
      ],
      "id": "654eac90",
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9b885cf1"
      },
      "source": [
        "import torch_geometric.transforms as T"
      ],
      "id": "9b885cf1",
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "85eb01bf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "37bfbd3e-cbba-4574-bcee-97f8cfe143ac"
      },
      "source": [
        "dataset = Frankenstein(root=raw_dir, pre_transform=T.GCNNorm())"
      ],
      "id": "85eb01bf",
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading http://nrvis.com/download/data/labeled/FRANKENSTEIN.zip\n",
            "Extracting /content/drive/MyDrive/dataset/FRANKENSTEIN/raw/FRANKENSTEIN.zip\n",
            "Processing...\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 4337/4337 [00:22<00:00, 191.66it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Done!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VhPiJx9Kx3Ov",
        "outputId": "192e3ffc-2c5b-424c-d066-4c571adf33d1"
      },
      "source": [
        "dataset"
      ],
      "id": "VhPiJx9Kx3Ov",
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Frankenstein(4337)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "03244bc3"
      },
      "source": [
        "### Open Graph Benchmark datasets"
      ],
      "id": "03244bc3"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eb881451"
      },
      "source": [
        "Open Graph Benchmark is available as a python library, to install it just run\n",
        "\n",
        "```pip install ogb```"
      ],
      "id": "eb881451"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7895e748"
      },
      "source": [
        "OGB allows to load a dataset in three ways: for PyG applications, for DGL (Deep Graph Library, another widely used tool for GNNs in python) and in an 'agnostic' manner. There is a naming convention to load a dataset, depending on the task an the dataset name:\n",
        "\n",
        "    ogbn-[name]: for node tasks\n",
        "    ogbg-[name]: for graph tasks\n",
        "    ogbl-[name]: for link tasks"
      ],
      "id": "7895e748"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d4Yfr_4f4gnq"
      },
      "source": [
        "!pip install ogb"
      ],
      "id": "d4Yfr_4f4gnq",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "439595c0"
      },
      "source": [
        "from ogb.nodeproppred import PygNodePropPredDataset\n",
        "#from ogb.graphproppred import PygGraphPropPredDataset\n",
        "#from ogb.linkproppred import PygLinkPropPredDataset"
      ],
      "id": "439595c0",
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c05b1bf4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "61624a71-a9f0-467d-9613-797a0075fe7a"
      },
      "source": [
        "dataset_name = 'ogbn-arxiv'\n",
        "dataset = PygNodePropPredDataset(name = dataset_name, root='data') \n",
        "\n",
        "split_idx = dataset.get_idx_split()\n",
        "train_idx, valid_idx, test_idx = split_idx[\"train\"], split_idx[\"valid\"], split_idx[\"test\"]\n",
        "graph = dataset[0] "
      ],
      "id": "c05b1bf4",
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\r  0%|          | 0/81 [00:00<?, ?it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading http://snap.stanford.edu/ogb/data/nodeproppred/arxiv.zip\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloaded 0.08 GB: 100%|██████████| 81/81 [00:02<00:00, 34.18it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Extracting data/arxiv.zip\n",
            "Processing...\n",
            "Loading necessary files...\n",
            "This might take a while.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 1/1 [00:00<00:00, 885.62it/s]\n",
            "100%|██████████| 1/1 [00:00<00:00, 2039.04it/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Processing graphs...\n",
            "Converting graphs into PyG objects...\n",
            "Saving...\n",
            "Done!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2e478337",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b4ca242d-aca6-45f4-e340-df35cd3c171e"
      },
      "source": [
        "graph"
      ],
      "id": "2e478337",
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Data(edge_index=[2, 1166243], node_year=[169343, 1], x=[169343, 128], y=[169343, 1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c67e494d"
      },
      "source": [
        "### Benchmarking Graph Neural Networks Datasets"
      ],
      "id": "c67e494d"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a74b8e8a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "beb058ae-a896-422b-a324-f81e3259edfe"
      },
      "source": [
        "import torch_geometric.datasets as datasets\n",
        "\n",
        "datasets.__all__"
      ],
      "id": "a74b8e8a",
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['KarateClub',\n",
              " 'TUDataset',\n",
              " 'GNNBenchmarkDataset',\n",
              " 'Planetoid',\n",
              " 'NELL',\n",
              " 'CitationFull',\n",
              " 'CoraFull',\n",
              " 'Coauthor',\n",
              " 'Amazon',\n",
              " 'PPI',\n",
              " 'Reddit',\n",
              " 'Reddit2',\n",
              " 'Flickr',\n",
              " 'Yelp',\n",
              " 'AmazonProducts',\n",
              " 'QM7b',\n",
              " 'QM9',\n",
              " 'ZINC',\n",
              " 'MoleculeNet',\n",
              " 'Entities',\n",
              " 'GEDDataset',\n",
              " 'MNISTSuperpixels',\n",
              " 'FAUST',\n",
              " 'DynamicFAUST',\n",
              " 'ShapeNet',\n",
              " 'ModelNet',\n",
              " 'CoMA',\n",
              " 'SHREC2016',\n",
              " 'TOSCA',\n",
              " 'PCPNetDataset',\n",
              " 'S3DIS',\n",
              " 'GeometricShapes',\n",
              " 'BitcoinOTC',\n",
              " 'ICEWS18',\n",
              " 'GDELT',\n",
              " 'DBP15K',\n",
              " 'WILLOWObjectClass',\n",
              " 'PascalVOCKeypoints',\n",
              " 'PascalPF',\n",
              " 'SNAPDataset',\n",
              " 'SuiteSparseMatrixCollection',\n",
              " 'TrackMLParticleTrackingDataset',\n",
              " 'AMiner',\n",
              " 'WordNet18',\n",
              " 'WordNet18RR',\n",
              " 'WikiCS',\n",
              " 'WebKB',\n",
              " 'WikipediaNetwork',\n",
              " 'Actor',\n",
              " 'JODIEDataset',\n",
              " 'MixHopSyntheticDataset',\n",
              " 'UPFD',\n",
              " 'GitHub',\n",
              " 'FacebookPagePage',\n",
              " 'LastFMAsia',\n",
              " 'DeezerEurope',\n",
              " 'GemsecDeezer',\n",
              " 'Twitch']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f233a890",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ff3a838a-3107-4c3b-d81f-66f5df630866"
      },
      "source": [
        "datasets.GNNBenchmarkDataset.names"
      ],
      "id": "f233a890",
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['PATTERN', 'CLUSTER', 'MNIST', 'CIFAR10', 'TSP', 'CSL']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "22aeaf4b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "91ded53b-134b-4668-9c91-265e0569c322"
      },
      "source": [
        "dataset = datasets.GNNBenchmarkDataset(name='MNIST', root='data')"
      ],
      "id": "22aeaf4b",
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading https://pytorch-geometric.com/datasets/benchmarking-gnns/MNIST.zip\n",
            "Extracting data/MNIST/raw/MNIST.zip\n",
            "Processing...\n",
            "Done!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "56defb77",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9e91c9f5-c5dc-4377-d9ab-dd925dc8b35b"
      },
      "source": [
        "dataset"
      ],
      "id": "56defb77",
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "MNIST(55000)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    }
  ]
}