examples/notebooks/1_Image_Classifier_with_PyTorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Image Classifier with PyTorch**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 0. Set Up\n",
    "\n",
    "For this tutorial, we will be using ``numpy`` to generate data and all operations, ``matplotlib`` to render and plot images and of course ``nets`` for ``Tensor`` operations and computational graph creation. We will set the ``SEED`` for random numbers from ``numpy``, so each time we reload the notebook the results will be the same as numbers will be generated in the same way. Finally, we set an autoreload with ``%load_ext autoreload`` and ``%autoreload 2`` for debug mode purposes. This enable the notebook to reload the modules each time we run a cell, so it keeps everything up-to-date. More info [on StackOverflow](http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython).\n",
    "\n",
    "Let's load the packages we need to run our tests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random as rd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# This is a bit of magic to make matplotlib figures appear inline in the notebook\n",
    "# rather than in a new window.\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.style.use('seaborn-darkgrid')\n",
    "# Change the font ?\n",
    "font = {'size'   : 15}\n",
    "plt.rc('font', **font)\n",
    "\n",
    "# Some more magic so that the notebook will reload external python modules;\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's fix the seed\n",
    "\n",
    "SEED = 42\n",
    "rd.seed(SEED)\n",
    "np.random.seed(SEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Import the Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to .data\\MNIST\\raw\\train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "25e51451b563449abba66d6ff8e8b516",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting .data\\MNIST\\raw\\train-images-idx3-ubyte.gz to .data\\MNIST\\raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to .data\\MNIST\\raw\\train-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "969bee51206f43638cd4662306efc99e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting .data\\MNIST\\raw\\train-labels-idx1-ubyte.gz to .data\\MNIST\\raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to .data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6d383dde34514907a6a6062372e46c30",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting .data\\MNIST\\raw\\t10k-images-idx3-ubyte.gz to .data\\MNIST\\raw\n",
      "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to .data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "83ef2266e89845448d42347fa0a65524",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting .data\\MNIST\\raw\\t10k-labels-idx1-ubyte.gz to .data\\MNIST\\raw\n",
      "Processing...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import random as rd\n",
    "import torch\n",
    "import torchvision\n",
    "import matplotlib.pyplot as plt\n",
    "from torchvision import datasets, transforms\n",
    "\n",
    "# Normalize the data !\n",
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                              transforms.Normalize((0.5,), (0.5,)),\n",
    "                              ])\n",
    "\n",
    "# Download the dataset on the default 'root' folder, '.data'\n",
    "root = '.data'\n",
    "train_data = datasets.MNIST(root, download=True, train=True, transform=transform)\n",
    "test_data = datasets.MNIST(root, download=True, train=False, transform=transform)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Load the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sanity Check\n",
      "torch.Size([60000, 28, 28])\n",
      "torch.Size([60000])\n"
     ]
    }
   ],
   "source": [
    "print('Sanity Check')\n",
    "print(train_data.data.shape)\n",
    "print(train_data.targets.shape)\n",
    "\n",
    "BATCH_SIZE = 64\n",
    "train_loader = torch.utils.data.DataLoader(train_data, batch_size=BATCH_SIZE, shuffle=True)\n",
    "test_loader = torch.utils.data.DataLoader(test_data, batch_size=BATCH_SIZE, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABGkAAAIuCAYAAAAMmVVFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3xUVf7/8TMzCSGh914CSWiiIIKUFXujo9gAWVekKIJlbVtcu7sWLIAoYll1UZFVRBDFhmUp0gXpndCL9JrMzO+P/f4ey/ueIcMkk+QkvJ7/vc+ce+7Z9XLvnZN7P+MLh8MGAAAAAAAAhctf2BMAAAAAAAAAizQAAAAAAABOYJEGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOCAhJw+vNx/Hb/PXUx9HZrgy8/xOXaKr/w8djhuii/OOcgtzjnIDc45yC3OOcgNjhvkxqmOG56kAQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMABLNIAAAAAAAA4gEUaAAAAAAAAB7BIAwAAAAAA4AAWaQAAAAAAABzAIg0AAAAAAIADWKQBAAAAAABwAIs0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHJBQ2BMoLOuebSd5RZ9XrD4hE5bsNz7Jo/elSp7SrEKcZgcAKI6O9DzfatvaUa8tHdouk/xuvR9j3k/D8YMlp90zO+YxkH+O9mgjOWvQHsmHv6kmueZ3+60xJk95V3KiL6BjhoOSO943xBqj7AccF8VdoFJFq231/Y0k1/ohW3LSF3Ojjrv97vaSF9w/Kua5pX05UHJG/3kxjwHAHce6tLHaNnUPR+iZv9IabLfapjb+VLL3mvnVkUTJz/frbY3hm/lLHGZ3eniSBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHnDE1abw1aL698TnJIZNsbdNx8fWSe9fVd2UHll8jeYppnZcpoog4frX+d97ezv5nVO9vswpqOgActubFtpLX3vBagezXu5+Ghho1heVA77ZW25inX5LcKFHfje9Woqfk1XVqW2M0e/dOyYO7TpM8pMJKyY888bY1xh2/u0Vy+pCfrT4o4iqWt5qW3az1Y/b3OSa518C7JUeqUVNx+QnJS7M0N0nU+g6R3Nv2a8lTDLUdgaLEW1/t+RftGq/nlNDs9zwjEjKhmPebmzG8PbI8pXIuTM6S/MB5pawxqs08renFBU/SAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOCAYlGTJlC+nNW29eZmklf00fdvvTVoWo4Yao1R6xl98ezFkVdJHnzNaMlHu9u/DZ88aU6EGaMoOzZ0r+QVLf5t9ek0rpfk4Mo1Vh8Ubwm1akre1rWe5BLddlnbzDjnoxzHfHhnC8mfTO5g9aEeUsE60vN8yT+9MsbTY1HMYzYcPzjHzzu0XWa1vVvvx5j3g4JRYcAmq81bg+asHwZITh+8TnLageg1hL557zzJr9+g9yyf3/Kstc3yHnpvdM0/tBZOdubmqPtF0VfOX1Jy4n3bJfu+S7K2KTFN6zQOufcuycOH67HVsgR/FwaKuoT6dSV7a9B4688UJcP3nCW55rgVVp9gQU3G8CQNAAAAAACAE1ikAQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMABxaJwsLdIsDHG/PzQy5JDnvWokAlJrjduozVGtienTtSW0DU6xm+N7f87a02ymlDE7B7UTvL8Fq9KHra1tbVNtELBq8bqNhkD5uZydnDFlgfbS+5x00+SP60yOeoYIU/2G5/kx6oulPzXW7VwozHGNC87THLa3dELjuL0eIsEGxOpULDqt7Gj5PXPNrH6pEz8WXKayfm/2Y5IjVtz3AQFqe3ZEvvVjP5vv0FvLTCdm+KEwWWrJNd7RPPQjtdb20zM0JuUdp/rteuDDy6RXPsfEQqTh8OxTBMFbN+5VWPe5svGelx0LX2Z1Sd4/LjklE/0PDbhr/pjGi2r2derm8pqEfT3brlPcoV/Ugi/uMu+tJU2eE8nPmPZ01QLWVdapseid4wtF9qFr32ePjV/0jGSth2U7D2/nqmCW7So+F/WXSN5SuNPCnI6cTV2/u8kZ+yZX0gz+S+epAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAAcWiJs2A2+33vf2e9acdwaOSez56v+SKm2N/79W7j8PpJ2IeA+7rMMB+j/pkM8aeZ7VVNno8BRqlSV7feazkK02LXM4OBeHATW0l7zrXfkl66vXPSW6YkCx5bfYxyV1m3mGNUXmSbrM/Vc8xR+ppXawXL33fGmPFda9I7vbwRZJDB/U9a5y+rR0jvBzv0XD8YMlp92h9mRSjdRtyI1JtHGMWRWhDYTjQIEVyz9I7rT4tX71Lch0zM1/nZIwxwYvtwkXdjNZHm7JF38G//84lkt++ub41xsSmVfI+OeSbAzcUzjn/q3e1nt/T99v3UuX8JSWfKBv9HAt3eGvxVVqm9yh7+9vHXpd6SyU/VvV1yd6aod7vWqfTJ1djDNA+AzMvkrxVbwPPWOEs/a6buae85Ej/X3tdsUzr2GybWSv2iXiLCoVjP3fUe0SvuxmmcGvQePEkDQAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOKBY1aULh6O8aWjVo3oq9Bk20fZRaXSLPY6LwrRqr7+hPq6n1Y9ou6iW58pjox9KRBhXyPjEUmLXjWkqe3XG4ZO979P+lbWmfaW2SJiP3SU5d9kvUeZSJ8vmDT95stXX+wyjJG9+uJ7lOr1+j7heRdWi7zGq7sqbWk0ozs60+8Zb6wPJ83wfOTI0/GiJ52fUjJfcqs8ra5rVh3SVXG5H/9XVwasc76z3My+e8VSjzqD1+neTHbrFr7z1ShVpargo0zbDaDrygNWd+aa73GyGjdUL8xq4TEr2PP8rnp9NHP//8SDlrhKDnu+P5JbVm17xtdSTXNPb1H8ac2K412LzfjSNJeqi05HrzuWZEwpM0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADigWNSkmdLMrvcxxeg7uRVN3mvQbLwtKPl0fgseRU/Xljm/I501qYqnZU3UMQ/WLRb/1Iqt3QPbSV5z8auSg+HkqGN8ezRJcs3v9B3p4DK7lkNepWy339X2vps9suUHkodXuUTntWtX3OdVXO1od6BQ9lttVlnJ79b7MeYxav4Yjt4JcVHxp82Su6/sUUgziV3pDXpfczB0QnIZv11770RZqwmFKDRst+SOJU+couep/W7xdZLLHciMeYzsbdslT37nAqvPI/fp/Vb9a9dKPjoi5t0il9Y9q/dB7/UaZfVpmaT1RkKe70F2PZLoNUO9fbyfN/l+YITZ5qzqZ3o/VmFG9ON3bI2KkmvOXRLzfs8EgUr6/9PvL/ypkGZS/LHKAAAAAAAA4AAWaQAAAAAAABzAIg0AAAAAAIADWKQBAAAAAABwANVMY3BhAy0Q6y1uVW/cRmub7HydEfJq96B2Vtu0mlo0NvXzAZIzxsRehDqxO8VZXZFQr47VNu3h5yV7CwWHTPSiqw8/fpvk8hPyXqw8mpqfb7HaQn/Sub69Q4s1Bvf8lq9zQt4d6Xm+5HfrjcnzmD+9omP0e6Cj1aewCiMXN9mZWjjY98dmVp9zX1kmedeT+Tql01b95ZmSJw1uKLlv2dgLyMJ9a7KOSy71TDnJ4Swt6JsblZZGL2A8tPY3kodX0PNUcO/ePM8D/5V9SSvJ3kLBrZPsHybwFgreETwqefSe9pI/+/B31hiVluk3o5KT5+Q4z4ZmYY6fn47T+i622b6fgs17D/n5Zr2+PVT5l6hjDP/4DclrsyrFPI8HPvi95PqT9P4lPH9pzGO6hidpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHAANWlykFCntuTX63wm2ftuZjbvMxY5HQbMs9raLuolOWPA3IKajlg1trXkivP0n2vlXNTGgTEmZNeXCUXolpOBmRdZbZU+Xyk5GOOYubHp2lpR+xzKStKG0L58mg1yq9qsspLjUYMmmnfr/Wg3btV4wZBBklMm/pyPMyq+Arv3W22NS2+XvMuULKjpxCTg07Ojn7/tOceXoPcGSQmxV0Ps/uEfJad+Xzj3Fx1Lat2aoXc0kVznKa2ZhNx76o3XJbdM0n/r3u84xhjzyj6tUfXlLVrzLjx3ieRahv9exc3mP2ndoUUtRnp6RL9GNE7U+9KMxIMxz6Nz/1ckf9tbxxw89VZ7v4+tlux6jUautgAAAAAAAA5gkQYAAAAAAMABLNIAAAAAAAA4gJo0OdjYu67kkAl7cqyVLFDYAo3SJI+o+W+rT6ux53la1uR5v7Nb2Ps52bA1K6y2zimLJLead3ue5wFjsjM3W20D110j+eO0L3Ic44cVGVZbyTHHJZ84Xk/yWbW14Mcvq+tYY5RbXEJyje/1fVn/zr2SL71pTo7zNMaYXzfXlNzQ7Ii6DfKPt/6MMaeoD5ODfhs7Wm0zZjfNcZsObZfFvM/UB5ZLXm/Ol0yNmtMT6Zwzq1NDT4ubNe2CYf1bXpcV3a0+9cd43vPP1xnB68QlLSRPa5z/Na1yY9MV0b9yrMnS62jNmcfyazpnvNZJPsneGjR+o58bY8yQ8mslj+h3peTUMq0kJ3w3Py9ThIMW3ak1aHL3XViPtXiMcWHyEckvXD3O2uKBwzdLTv2T27U9eZIGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAdQkyYHxytpDRrv+5mNPr5Tcrrh/XzX/fZi9D6Ha2kuc3VryceGal2QaPVmTkfnFPu969TPB0hu8v0uybz3Hz8bPvbUh3hQo/ff/urLx+Z9p2kR2q725Acj9IlK59qwz8LcDIJ8Emv9GWOMaTh+sOS0e2ZbfdKM3XYybyWihi8OtvqsveE1yd659nvAM+bEHHeJHGRvdrMGTTRnV7Dn/UOntpIrvLPL6oP8s/miRMmRaol4XbG8h+TUh+JfmyHQrJHkx7pOsPok+gKSM7PL6RjTF8R9Xviv6HU27b/je/usvGa0fn6Nfv7YTq1RY4wxX4/sILniW27XBYFqM6+P5NnnvRfzGE/vbi75nJRNkq9O0e9ZuRFpjKv7jZDcNvMuyVVHz8zzfuOJJ2kAAAAAAAAcwCINAAAAAACAA1ikAQAAAAAAcACLNAAAAAAAAA6gcPD/SahT22r7R49xkr1FthpMzMrXOaFwrLjtVW24Lf77GLZVixGv6VPP6pOxcq5kCgXnnxpvL9EGT8Fe77/909Hy536ST6wsK7lOa7sA55dNtBKrtwDk6cyj/cKbJFc0q6JuA7dcMGSQ5LSJORcFzo1IxYfNDTlv4y0kfKVpEccZoSh4stocq61lnXaSKxTUZGCMMabGLL07CP0++nVi23S9561jNp2iZ+6tuqWi5OtL77T6ZHmmmmVKxn0eiOzSO26XXP5ePQYmpk2NsJXfk7xFqvXzJ6ouskZ47An9MYNL9+g8kifZ5xi4Y9/OMjFv06mf/lBB0qL1kueUrC/59VLJUcfcdkU1yYPumCT5D+U2RB0jeIWnuPDoyP0KC0/SAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOCAYluTZk9/fUd636VHJVf9TN973d7Bfoe3Wyl9V+3aNZ0lB6YvyMsUUUS1XdRL8o4t9hv4XVvqe7gjamp9mRljz5NceeWsOM0OuRE6eFDy0qwTkpskJsY85i/nvyf55Yw0yWeVzIx5zNOxf0klyRVP0Q+Fo+H4wVZbzR/1+pMy8eeCmo7wzm3tDa8VyjxQeDb9rb3kfmVHenrwtz3XhO/clePn24JHrbaKK+Nf5S7QJF3yY90+inmMFzZcITkhH2rl4L+8tV+Oa0kPc+XF0Qsyruup90bLrx3l6WGfL0ImJDmzq+YMzzzglowB+n2mm2l9ip7/k2DmS47H2afqqrWSR5TvLrn/7d5rl218izcl39NqgOTw/KW5nF18cLUFAAAAAABwAIs0AAAAAAAADmCRBgAAAAAAwAFFsyZNm+YSW435xeoyuNJzkmsE9DfXQxfqO5D+03hvct3UBpJrme3R5wqnlByp9WNa1b3d6lNmU7bkpC/0/ctyZo0n265csyLHeVQeQw0al93XR2tzvPOBvmdd2XM+OR1DK6yOeZsPD1WRfH3pnZKXntBj1Rhj0kdtlGz3QGFKu2d2nsc40vN8qy0edWw6tF2W5zGQPw70bmu13fbwp5IDPr1nCYb1vua5CT2tMRp8uEfy27foe/ze+6DN2cetMcpssmv6oeBUL3Ugx8+XnKhstZX6d97PF+ue1dqPI695S/KlyUdiHnPPZ7UlV6MmTaE5nbqb6dM1X/7tEMnTR0eqa6bnpfGXvir50Sa9JQeXx37vhDNP6tsbJP+h06VWnzfrfS15dZaeGwu7Bo0XT9IAAAAAAAA4gEUaAAAAAAAAB7BIAwAAAAAA4IAiWZNmXa/SkqdWXWT1Sf/4j5JTtgQkP9b/X5J7lNoXYU+6hlVyN+9dF3Xe+jJJcRgz0CjNauucosdk20W9JHvr2sAtvhn636/Lk/dLDpb0WduUX5sleV/DRMlJe/X8Uf49uy7RsS5tJN8+fIJkv9H97gvZtXGyt2y12uC2arPKSn633o859m843q5PkjYx7/OItl8UnA3jz5b8VuvRVp/zkoKSL/zlJsm7Vur79sv6j7DGeOWaRpJbeS6Kv99wheSth+0qbOXfpcZaYVr3doY2PP5VnscMNNPjotZbm60+H9YcLrm0P/Y7qmf2NJNcanvoFD0RC2+9IGOMue7yGZLnt8z73+kT6mgNod89qjXXQsb+3uStc9WyhM7jRPUykgPL8zJD5JWvlf4bda1uy6nM+DXdbvTUpPnXDu+/kz3GJTxJAwAAAAAA4AAWaQAAAAAAABzAIg0AAAAAAIADWKQBAAAAAABwQJEsHOwtfjV6X6rVp/FftdKUr5wWZjT9NZ5Ocas/PThO8oidN0ouOXlOxPmieFt+X4WofbImVfG0UDi4KKn8euyFMatF+TzQxC5qlnzvFsnXldYiZt7z1F2jBltjVDczT2+CKBTeIsHGRC/Y229jR8lp98w+Rc+8zSOahuP1eEszeZ8HInvgbC3+6i0SbIwxv1vYR3L5Z0pJPnGWXeDca0iFlZL7b7pU8s9ztIDsZR1+scb46q3zJFeapUXTK42lsHB+qrjsiOSlJ7Ilt06yi2GW+0+lHMfsXuUbydeX3hmhV2yFgl/b18Bqm9VZf3ihdCbnlHi4u8sUq21guQ2SWw8dKvlAhp5j/njp1Khj+M0Cyd57FO+PHfz/1pNlfDFI8/R5EbZBfgik6ffnlY+Wt/pM/N2rkjOztc93+5tK/jpTrxmRnFisY1RdoMfezpb28kTVhXpe23mu/iDQBVculvxJ7UlR57H4W51rPcfun3mSBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHFImaNNmXtpL8WNXXNe9qYW0TPHBAcrfZ6zWX2iv5kZ0trTE+mN9G8vpOb+gYr+l7ehnd9b1KY4xp+ug2ydmbt1h9ULRVq7U3eiec8QKVKkre9FQJq8/CRh96WvR97vPm9pVc840l1hghqwUF6UjP8yWnPqD10aLVnzHGrkGzo92BU/TM33l4xaMWDk5PwKf/kv0R/qZ2eHZlyRV/0vfpq/+m79u/us+ugzWk/FrJ3uOi4ZqGkn/40r7fMnVOSGze/1fJOz7VmmzBXbvsMc5QO4a1l1xtROw1EXyztE7QsFVaL/Hbs/5tbTMu9Sur7WSJPq33kGWXbYzZZ9vPttr8mZl5HxiWUNg+X3jrbM59aGSOn0c653j7eP/WH+1zY4xp87TWwmk6aaNkrTyC/LT7ZV0GWNHijQi9tMZYs0StgXVF8lzJ/6iuORJ/a723DfWP/QTjrXdk15aN/hxKvUfcqkHjxZM0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADigSNSkWd9Dp+l957Flir7PaIwx4964XvLAcmNyHGPG39paY2RMniP5sotvzXGeGdPnWW28WwnAGGP2jysveWHzd6Nu8/BOrf9Q80atrRU6dizvE0Ncbe2o70n/lIvaL15rXrSvT15rb3jN07Ioz/u9YIjWWUsxP+d5TJyeYDharQdjxt46SnLfmoMlv3rFO5IvTj5kjfHS3saSv+ujtfjq1NT7r/2D7RpsH53zT8k3zrtNct0Dq61t8F+VF8f/HH74k+qStzU5avWpEUiO+3697traQXKJAQGrD/fI+eOdFzpZbd0feU5y3YTSkr11h7x1iSL18dYFeW1fWtR5VH1T64BwDLgj0nXGFv3a5OoYf9vZOhf7KTw8SQMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA4pETRovv2dt6drSB6w+3a7WGjQ7gvpObs9H75dccfKsqPsNTF9wulPEGeThjClR+1QeE/34QvGy7n2tJ7Oi+VueHj7j9e3RJMkLf99UcujYirjMDfmn5o+el/ZviH2Md711bOJQ1+Z0WDVoJlKDprC8+M41ki+6/Vmrz3lJer5Y1f1VyZuy9b5n2JbLrTEyb6kjObRsueSkX7R/jVnlrDF6varHTc/GutGvtepLzl63wRrjTBX4Pv73ld77jU5XDrL6LDw/ek20k+0P2bVz7s7sLHnB53q9Sv1XpuTsjRti2idyr9Kb9j3nwMW3S151c0qOY/grnYi6n/pv6H1MiflrdB4HuPd12e61FbWhReR+RcHC47o20HvyEKtPowe9tfrcruvIkzQAAAAAAAAOYJEGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAcUicLBjf+ihewuSr9O8vfNJ1jbhExIslUo+C2KWSH/tF3US3I5s+YUPVFc/DYlQ/KKllooOGS0oOzvN1xmj3F3bcnhxUviNDsUFG+x3QuMFu1MfUCvZ8ZEKBQcBw3HD5bsLWgcqShwiqFQsCtq/WOm5Ju23W/1OVTHLj5+sjKb9L95+Xcj3fesimlewX37rbbUm7RQ8IILWkrefk2y5JrPb4hpn8ibxO/sYs/m/NjG6Pzn+6y28u/p8VTH6DGbHdsukM/Cc/V+In1u/PcRjP+QyEeNHtJj4rLv77T6bL5Sv09bv3nh+a2EMZf+0xrjwuQjkr1F7S/78p4c5xlJlZm6hFHl+82S0zfOtrYJWS1u40kaAAAAAAAAB7BIAwAAAAAA4AAWaQAAAAAAABxQJGrSBA8ckFz6Ks1dTKuoY1Q01KBB/vjj+3+w2o7X0LexI7wRjiJs+93trbYJzZ/ztGgdhvXZxyRvfSrNGiNpbj68JI5C5a39smOi3edK0yLu+00z9vvYKLoqvGPfw1QohHmcDv9PCyXX/KmQJgJjjDFVR8202rqMin7ffLLy3EMDxU7oiNaKSfnErkuX8UlsYw43zSK05SzD5P3etzjWwOJJGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcUCRq0gAuq/c33tU+0yQeDFttqQklJd+xpYPkjUO0Bg31ZwAAAAB48SQNAAAAAACAA1ikAQAAAAAAcACLNAAAAAAAAA6gJg0AxKjSm3Ydoi5vtvK0HPXkJfk2HwAAAADFA0/SAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA1ikAQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMABvnA4XNhzAAAAAAAAOOPxJA0AAAAAAIADWKQBAAAAAABwAIs0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA1ikAQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMABLNIAAAAAAAA4gEUaAAAAAAAAB7BIAwAAAAAA4ICEnD683H9duKAmgoL1dWiCLz/H59gpvvLz2OG4Kb445yC3OOcgNzjnILc45yA3OG6QG6c6bniSBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA1ikAQAAAAAAcEBCYU8AAAAAQBz4fBID6Q2sLivurBzTkCM7vWO13bfwOslJP5aRXP31+ZLDx4/HtE8AOJPxJA0AAAAAAIADWKQBAAAAAABwAIs0AAAAAAAADigWNWl8Cfb/jEAVfd923QB9Jzdpn/Y/VC9kjXFVx4WSR9X6WXLGj/0kN/j9SmsM3sEt/gKVK1ltg2fNknz3rBslp/dbkK9zQvGU0KC+5E5T5lt9FhyoJ3lz20P5OSUAgEM2P9RO8uI7R+XLfq5q76lT017ja7frtWjqFWdbY2Rv3hLvaQFARFmXtZJ8tGqifl5K63ntaX/CGqN+7d2S+9bWtYGAT9cTHv+2pzVG+pCfrbZIeJIGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAcUiZo0CXVqS179rNYAqVjusLXNT+eMj/s8ssKal17wtuQmb91mbZM+cJXk0GF7rijaVo+oY7V1K/Wt5LsLajIoVgJpqZIvn6R1svqW1fOLMcaMmNhFcn0zy+oDIH4C5ctJXvFEY8n+SnmvTfd+u7FWWyisf2cbu/NCyT+sS4s6btXPSkquMGOz5OxMzXBPQo3qku+4eXLUbY6HsyWv8tzg9pw+JOoYS67QWjfJvhKSB5fbKHlylY72INSkAXAaAtWqSj7SSmte7b7tiOTOqUutMZ6p9qbkYFjrxwR8/hw/P50+3s8/apppjRG0WiLjSRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHMAiDQAAAAAAgAOKROHglXdp4f5yiewAACAASURBVODlHV8ppJnkbPlFb1htHXrdKbnCOxTxLOq2PNhe8q8XvhyhV6LGvSUi9AHUngHtJPe9+wvJQ8qvldxnvRYJNsaY+n/hHJNba15sKzlc8YTkEhu0yKoxxtT/dL9us9AuVneyhFo1rbaD5+k1bvM12Vafk13cyC4YfUe17yT3+kKvPRl3zMlxTORBzWoS3+v8quQ2SZ5fHTDGhIwWG/R7/mYW7fNIfV6r84N+Xmd69DEu1DGmHNYfZnh0qZ5javZcZo2BwhUuW1py46StktO/sX/UosKMJMmVX9frRoaZF3W/57ygP4mw6obRUbdB0ZZ1WSvJV774o+T7Kq6U7C2iaoxdaPX8v2mR6kpvcg9zpvP+YJAxxlSdoPdar9d5TbLf+CSHjH3dfXCHHr+/7K0lefVi/SGYcit1TGOMqTZrv9V2so1dy0uu//qaHPvnhCdpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHBAkahJk/H2XsmNEvX9xabnbLS2WbWtquRqE+xaAtHs7XtI8sLz3415DP8Nu7ThnZiHQCELlC8n+fkBb0o+FrbrR7QaeZfkRq8skaxv5OJMEO7QQnIo0V4jP3jZYcneGjR7Q8ckb3k5zRqjtNmT2ymecU5c1VryrF7DJVfyJ0cdI/u2oOSb1nbKsX+PqnZtmD5lpkTdT3R6OZ/b9UXJly2/X3K1kTPjsE8YY0xwmdYIerzBuZLXPau1powxpsEDBV93IdJ7/ht715V8fs/Fkhe0eU9yq0/7WmPU6LE8DrNDbgVXas2DZxs2l5xuFuTLfn3VjufLuHDD8c6trbY3R+t1JcVTsiP9a733NT67LsiMi0dI7n3PNMlfvVtZcjhLa8Oh+DneSY+1sa+9aPX5+rDe716+9FrJ+6Zovb+a79r1AYP7vPVktkhK8+RIon1/q7PIs8+oI54aT9IAAAAAAAA4gEUaAAAAAAAAB7BIAwAAAAAA4IAiUZMmuHSl5HTPK49ZEbZJNdvyvN/QwAZ5HsOMq+xpyP3vpaNwLB+eLvmqlB8kp04eZm2T8Q+t90ANmjPPhvFnS57R/hXJ5fx2nSy/0Re8vcfNsE1dJZee8HPuJwhzpKpeAk+nBo1XgglIntBw2il6FqwKnv8tdw/5t+QJE+16A9mbo7+PjdgVRv2ZSLIzN1tttZ7Rtg1zW0me88aPkrvUs9/zn8/f+4qdQNmykvd2bmr1Gd9+hKdFz6ebso9I9h+379bzUq8B8ZVQX+tTXfD3n6w+dRP0unL260Mlpz8WvdbZ/TO7SH673reSv0k/T7K35heKPm99tMdGvSY5MzvF2uajJtUlJ5kNkqt5cnE4t3BlBQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMABRaImTUFY815Lq21F8zfyPG7KjkgVc1CUdD57ieSssL7pmDqBijMwJqFBfckjW30g2VuDpsmHQ6wxVt44WvLe0DHJmS9nSC5tZsc6TZyk0rw9ku/a2k5yg+RdktOStltjdE45FPd5fXioiuQdWeWibnNWyUzJlyYfl3xzGZ37+3W89dKM8VGT5oy35cIkyW2SwpI/L8jJoMB4a9Bs6X+W5AX3jYqwVc5fIXo9db/kysvcqM2EyNbcVkvyp5UmWn3On99bcr2n50nWs0XurH1E75Ua/i3N6hNcSX3PoiSQrjVer/p0vuQmJU7o5w/cY41R9gy83+VJGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA86cwsH+gMTsi1pI/qbjyxE2So5pFzdvuNxqK/H9L5LjUVQL+et459aSX645RnLapNslZ3wzJ9/nBPeFDx+R/Fz/vpo9/YN9gyaaJSe0mGPpj868wmn5KbhsleTV+k/frDb6//+3dZpaYzzXupbVlldlftSiiMHde07R83/evm+A5EX3RCr0+T/rrkmx2hpS1/OM1+HKxZJDhsL4xZH/nCaS1/45UfKy3+V8/ohk7P46kqtP2yw5O+YRUZCqtdbi8vOP231qDDsqOTvrhN3pJL6kJKvtnLKZEXr+z6+/e1tyhxdutPpU6JzjEHDMhhurSx5cfp3km9Z1lVxh6nJrjOh3zMUPT9IAAAAAAAA4gEUaAAAAAAAAB7BIAwAAAAAA4IBiW5PGX7Kk5FVj9f3bFZe87tkiev2ZrLC+EXf2jwMlZzypNSmMMSac/VvUceGWjddo5aCAT9cyk7cU2382yIPgjp2S/Z7s9d27/7HaAr7ScZ0T4is7c7PVlhKhLa9y8+511YVaQOBoWGsFJPtKSP7+Bm+VJGNu+/MlksNR6g2g6DvavY3k1+u8Jnnucb3+zR90ToRRlsR7WoijQLNGVttDEz+Q3CEp77WHBpTTWiNzx6dK/m6BHmvGGNN4zEHJoV/sWhQoGH6f3vtuD5az+gQ3b81xjBNXnid5Yyf7fvnzCqM9Lb4cxzz6fRWrrYJZneM2KDx7BrSz2pYM1hpXnx6uIPlob/3OHty3O/4TK4J4kgYAAAAAAMABLNIAAAAAAAA4gEUaAAAAAAAABxSL4hr+lBSrbeVorUGz0qpBE7vm0wdJTrt5oeQz8Tfci6OEPYnxH7NBfckrb68hueo8fRfYGGPKjJ8d93mg8Gy9v73kmglzo25z+wSte5VqZsV1Tig+Er6dL/mbo5Uld005ILlGwL5uGn/OtQFQ/JS/d5PkkNFrUd9Zt0luOEfve+C+7HIlrbZ41KCJ5vU6P2qDNxtjOjXqJtnfWecaOnYs7vNCZBvXa+2Xzs32W33uffNsyRm3/So5Zek2yWf/Odsaw3uOieZE2dj6o2Ad6N1W8jt/ecHqEzJaE+++6Tdoh8f1fJSyqq41RqXleiwlfzonlmkWSTxJAwAAAAAA4AAWaQAAAAAAABzAIg0AAAAAAIADikVNmv1dz7baVl7+Sp7HPXfOzZIb/eOIZGrQFE9VFnjef+0b+xgJ9epI3vZSkuQ1570q+aJzetiDjI99v3BXiY67JfsjrJHvDx2VXPMH+31u4HTc801vyV27vRZ1m3DLRtowe3E8p4RCFihfzmq7vPJyyX6jdYnKzErO1zkh/yXst+u63LDuCsl/qj1VcosS+vWg3SJPDQljzKGjel8zubWeY+onRKh75TG18WeSL71Eaz8mTY1euw3x0WT4b5I3XXXU6rPysrGSB/50keSf1mldm+dr/jvqfldlnZCckaj1S0ruoVaaS7w1aGY9r//ug2E9L0SypssYye8drK5jtLXvj/uX2y653aBrJVe4T89ZwaUro87DdTxJAwAAAAAA4AAWaQAAAAAAABzAIg0AAAAAAIADWKQBAAAAAABwQLEoHLyzdf6MGwzqGlZw6bL82RGcUubD2ZKf+Uu65NG3apGsf1x6tTXGn+tPktyhZM7roVsW1rDaGpgNOW4Dt2Vf2kryxHNelpzoK21tc964eyU3+GJW/CcGnMLGTqUk15t9io4oko6fm2a1DSz/jeS5x/VaVePbXZL5wYSiJ1IBzYMXaP5L6/6Sj1fU4p+V56+3xqiwe4/k2zvcKXldDy06vbJ39B/02HSjHmHpU0/REXEXXLVWcv877rH6VP3TOsnjUr/SDnW+lzjjWKI1xnnPD5VcufNmyV82mSi5yi924WsUHG/B+ccef1NyMBySHDKeH18xxlzwixYePzFZC0xXeTX6ve7wh6+R/MvgkZLPulHPP/Ufjjqk83iSBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHFIuaNA3/fcRuvCnv485r+5bkP89vL3nmCC2GU+Ed6kcUR++/cbnk6+9dKPnLxp9H2Crn9c/UL2+TnPEgx05xs6uFvtNfI6Dv52eFI1R3sF/lRTGTkFrPastev7EQZoIzzfEH9lptfs+16qbvBknOWD4vX+cEN4TnLpFcwvP56dQi8s1YJLns2e3yNikUqqTP51ptB6bp18buleyajOKoXU+mdlm93j0+TGvQtF/YV3LF7/WeG4Xr7+s7SX4j+ZDk7c83tLYp9+kcT8uamPdbOlNvkP3GJzmrrH6+8w79zm6MMTUnb5KcnbnZ6uMSnqQBAAAAAABwAIs0AAAAAAAADmCRBgAAAAAAwAHFoiaNb/4Kq63Rx3dIXnnt6JjHTfQFJD9X/WfJR576SXKbjHutMer/hVojRV31l2dKHvyzHlsH66dY2+zrpe9oLmv/L8mlVnnf+EZxU+by7TFvU3/y0XyYCVyyt00Nq61MPtSkadYkM+5jomib3nyC1RYyIcmlVnNtQu4EMrQWRffBP8Q8RtqYUPROKDTh7GzJwR07Yx5j071nSW5RQr+KHppbWXLF8KqY94H4Ce7bL7nE5Zo1GZNs9uTLPI50OyA55CniuLqXfs9v/uqd1hiu16Dx4kkaAAAAAAAAB7BIAwAAAAAA4AAWaQAAAAAAAByQ7zVp/CVLSg4dOxb3fYSzTlht6XfPk9zlX7dIXntdaclfX/+cNUbthOQc95vi03e3n7jufavP2x9cLTn0q10/B0XM7MUSy8y2u+xq2U7ypjZao6be+C2S9S1fFAcD6v8nx89nHEu02vz/WZRf04EjyoyPcMLIB0uX19GG9ALZLRzmN76IrScruTscoQ8QXdm390r+a+Vfo27TbtENkivOWSaZo7H489YWSaA0X7ETqFZV8u6rtH7VwXp6bWrbaYk1xrS64yQHw7pN2lcDJGc8qfVEiyKepAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAASzSAAAAAAAAOCDfCwdvuudcybX/XkCFfEJBzXO0CFHDOfpx920PWEPU7rpB8qSMyTnusmep36y2F57VkrDlOuU4BIqpg6GA5Oz1GwtpJsgvO+9sL/mSlGc9PbQQ+VO/72eN4TcUDkZ8lNye75d3uK5Nc4khM9/qEjIhyZXenJWvU0L8BSpVlJx5a2PJh5rYP66Rces8qy3HfaQ3sNrSPsiU/ET1zz09kiQ1en+INUajl/ReKDvCD4Gg6PIl2NehhHP25bhNnSm7JQdP0Q8FI1C+nOTlTzeSXD99h+S+tX+2xmiWpNeeVnpqsIrae4tJG2PMx4cqSB41TIuOZ3w519qmqONJGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAckO8vrRdEDRp/qVJW27Zbz5FcbaTOI6FGdcnhCP9PXFZlRd4nBxhj5h+rU9hTQD7rMeB7yTUCWoNm4mGtG5CwcLU1RshqAXIn9QN9T9wMjL5N1y6zJS/+WxwnhAK35eIykr3v/RtjTJPv9cBoaBbm65yQd/v7tpU84omRkluV+Eby98cSrTGeP/s6yaHFer+b+bDWWPuw/wvWGM0SS3hakqw+J0t7/4DVlr1la47boGjze+olGWPMgjbvSe6/6WLJodUb8nNKiNGmt2pLXnP+a5IDPn3eIxiOdCdrX3tyGuPTQ2WtPi8/cKPk5C/nWH2KG56kAQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMAB+V6TpiCserK51Taxp74/+9dePSR3qLhW8r0VP4//xIwxOzbr77qXO0U/FG+PzukmOd0sKKSZoLA8+B+tAZBxeF4hzQTFUtuzJf5x3LiYh1i8t5Zkf5n9kkMHD8Y+LxSaw+knJIdM2OqTuCrZaoPbdl15XHKrEoEc+19UMstqazD5DckHPYUZGyRofapkn7f+jK3TCr3PSeibrR12rYw6BoqXbdelRe0zY1ZTyWlZs0/RE4Whc+pSydZ1xFODJtJ1xmvk3nTJr02+UnLam9utbZLXFP8aNF48SQMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA4pFTRr/Cfv315skJkr+OC1/as6crNXcvvY8HlglOZjvswCQ3wIZDa22c1Km5bhN6WXR3+kHcm32YolD/zlI8rLbR0sOet4jN8aYv6V+Jvnpyj0lU5PGbYGmGZI/uGSM5ESfXbuk5B5PQxtPjb85S+IxNcRRmTmeOkKXxD5G3YSUPM/jw0NVJO+YVFdytW0z87wPFG2hRLvNb+zvbHDXkou1mmrG07drh2T9ZhvYY/9Hr+Ipw1lh6nLJqftmSea78n/xJA0AAAAAAIADWKQBAAAAAABwAIs0AAAAAAAADmCRBgAAAAAAwAHFonBw+tjtVttnPStI7lZqb9z322ZeH8m1+22x+gQPHIj7fgEUrlWDqlhtnVP257hN3Y/1/JAd1xkBKjslLDlSoWCvLdl63cxevzGuc0I+26z3QrOPaoHzVklrrU1m/+llyY/tbCV5fkv+lueaaqO0yGa3T7tJXn5fbcnDLvvSGuPuChske88PLefq/a35Uc8NxhhTc+Q8nVcWhYIRXciEo3eCM4L79N424445eR8zzyOcGbj6AgAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOCAYlGTJrhmvdX2+Oi+kkve+ZbkK5IPRx13YOZFkv/zw1mS0x5ZqPM4dizqmDgzlN7kk7y/RiFNBPkimBK9vkeL2f0k196wLL+mA1i+6fOcpyUl6jbXemq3vdDnJsnlxs3O67SQj7w18F6edZnkuzttsLbJ8pSH+GB+G8kZRuuOwAFh/Y+WnblZcvpdmr8w5a0hvjAtctxFDbM8+jSi9gAA5BZP0gAAAAAAADiARRoAAAAAAAAHsEgDAAAAAADggGJRkyaS6i/NlDzipcaaT2uUg5IamFmSo1elwJmq6iszPbmQJoJ8kXH7HKuty+2tJNc2SwtqOoDlwqn3Sl7T9bWYxwj7fdE7wVlNH90mOevqoNXnlX0Nc9wmO/7TAnCGqPzL8ah9OnecL3lVq2aSw/O5l8KZiSdpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHBAsa1JAwDAmarR0EWS03yDJK/pMsbapvPKrpIrTV4m2a5oApdlb94iuUutVqfoebIt0bsAwGlImL7AasuYpteie8//WvKaA3Ukc93BmYonaQAAAAAAABzAIg0AAAAAAIADWKQBAAAAAABwAIs0AAAAAAAADqBwMAAAxUw464TkjEFzJXcy50bYSovGUrARAJBr4bDVlHHrPMlTTAVPj3X5OCGg6OBJGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAc4AtHeF8QAAAAAAAABYsnaQAAAAAAABzAIg0AAAAAAIADWKQBAAAAAABwAIs0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcwCINAAAAAACAA1ikAQAAAAAAcACLNAAAAAAAAA5gkQYAAAAAAMABLNIAAAAAAAA4gEUaAAAAAAAAByTk9OHl/uvCBTURFKyvQxN8+Tk+x07xlZ/HDsdN8cU5B7nFOQe5wTkHucU5B7nBcYPcONVxw5M0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADiARRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAASzSAAAAAAAAOCChsCcAAAAAAECswh1aSH7yX2MlLzha39pmYtMq+TklxCDz4faSr+gxx+ozvLq2BXz6nMnu4GHJF84ZaI1Rt89ayaFjx2KaZ0HjSRoAAAAAAAAHsEgDAAAAAADgABZpAAAAAAAAHEBNGiBG+25uJ7nRHUutPnO+bia5wYsrJAf37o3/xFCkBCpXstpWvlhX8qpL3pTccm4fybUe9VljhBYti8PsAAAA3Lf65hKSW5UISP7DgousbWob+94dhWPgjVMlDy2/Luo2wXBIcgV/suTFbd+ztrliSg/JCZdtOt0pFgqepAEAAAAAAHAAizQAAAAAAAAOYJEGAAAAAADAAdSkAaLIvqSV5AlPPSe5WiDJ2sbff7rk7u9dpx2oSXPG8ZcqJTllol1PZkWDNySHPJ/Pb/0vyTeMuMoa4+jVup/Q4cMxzBJFQZel9vkjxX9c8sz9aZJnTT5bcp2nZsZ/Yij2As0aWW3Lh5aTvL7b65K9tQP+eaCmNcZHTarHYXYAzgS7B2ptyCVdXvT00Bo1R3aWMnDXV71aSx45yL63feSqf0vuU2ZnzPupX/o3yZtjHqFg8SQNAAAAAACAA1ikAQAAAAAAcACLNAAAAAAAAA6gJk0Ott3bXvKfB30geWtWBclLDtWyxljzbFPJKZ/8HKfZoaAEjmVL3pqdLLlawFs5xLb83sqSM25fl/eJoUhZ9WRzzQ1GW32iH0lqfMMvrbbGTw6RnHbP7BhHhev6l1tttaX4S3j6bJe8auA0yUOf6hD/iaHI8bfQe5SNnctLPrfzMslP137TGqNGQK+JwbDW25pwqJLkd+/vZo1R0syJPlkUmD39tebHH/44xeozsNwGyX6j/91DJpzj58YYc2Xf2yQHpi+IZZoohgJN0iUfeinb6vNB4+clJ/v0HPTl0RTJTUbtt8YI5naCiLvgcr2nqTarktWnUpdDed7P0lFnSS5n3L4/5kkaAAAAAAAAB7BIAwAAAAAA4AAWaQAAAAAAABzAIg0AAAAAAIADztjCwcGLz5W8866jVp+FrUdJ9hZB+/xIluShFexijstf+FryAxv7Sw7PXxp9sihUvpm/SL79haGSZ//p5ahjVKy9L65zgvt8LZtJHtH1n1G3WXhCSwff+MNgycNafyt5SPm11hivdH1b8sgXLpecnbk56jzgloQG9SUn+uYXzkRQpAQapUne1KOq1WfSHc9KrpugBTg/PqRF7y/++D5rjEvaL5H8cA0tUv2n6Xoey5hCkWDXbPu0ieQp5z4n2Vsc2hhjQlape39MnxtjzLqeiZLTp+c8TxQ/O4bpj7QMv2uM5ItK6net/7KPx5MN/fL3ktOX8qMtRUnCrTustquSj8Q0RuNxQ6y2BuPcLhTsxZM0AAAAAAAADmCRBgAAAAAAwAEs0gAAAAAAADig2Nak8SUlSV73mNagmXaTvm9bO8F+v3FDttap6f3w/ZIr/2eb5Bea2+97fzv6VcnHK+t+SlhbwHWJV+8q7CnAQYGmGZJv/EDrMlyRfFjyjOMBa4wn+t0qOf0/WntkWpk6ko/M0POcMcY8WGm55JeqldcO1KQpcjZ3ryk50WcfO9GU9/xJxn92Y6tPaPGKmMeFO452byP5zufGS+5Z6jdrm83Zmnuu7iI566EqksN9tDafMcaMrv2j5IxJ92q+gxo0hSlQvpzVduDDSpIXNh8nOeSp+eE3PmuMi5bcIPn4hGo5zuNYZXuMJuM2Ss62eqC42fh4O8kLbn1JcpIv9q+m3x/T2kaNX9U6kMGYR0Rh2jethtW2trF+J28Y4Xv7ycL26abI4UkaAAAAAAAAB7BIAwAAAAAA4AAWaQAAAAAAABxQbGvSHJus77MtbTZK8iM720ueMK2DNUbaU79KLn9wlmTvu7NbhlaPOq/sUlpLgJo0RZ8/wlqnt2aEvxi8G4n/8da8MsaYWm9vkdynzE7J3ho0T1/XxxrDP39Rzvv16YEU8IWsPgEfa+/FzeHa9n/nWHX/tZ/kstSfKfL2fp4u+d/NX5BcI6Dv7Dd5/05rjIwx2yUH16yXvHpUXckre462xmj20x90TGrQOGXrzc2stp+bvyw55LmPCRk953jrzxhjTLnrd0s+dHFlyb811q8YtZ6ZaY1BDZriJaGW1k/b92ZJq88vzUfoNp6votuCRyTXCKRE3e9t32g9v4ylc6NuA3fVeME+V9zzyU2Slz2sdWDXXPW65JW9X7HGOLvBzZLrDdTassE9dt22wsTdPAAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOKBY1afbc1s5qm9rkeckPbb9A8opra0tusEHrzRhjTKxVAPpe8pPVtilbf9f9WHldF4v+piVc53132xhjssKaKz5pv5eLostfr7bVNrr2R5K9R8Wgd+6QXHe+/c5tNCsfbyp5UsXpVp+/7mwh2ffrGsmeQxMO8tY8uv6S2I8Vr+2ZFSWXNWvzPCYKzoHeba22WS20Psw/D6RJfvOxHpIbfmjf5/jq6LnskiWHJU+t+KrkJu/adW1S/2SPi8JztHsbyfMfGmX18dag8Rutd9bm73dJrjrKPgetflbvvVf00RoQIc/Vpsf73a0xsjM3W20oOkIXtJTc4RU9Th6stNza5lBYKxF1GKXHWoOr10memDbVGmNV1jHd5qO8122D27I3bJKcMUDrQHarq9e7v3+v9+TGGLO47XuSLz1/kOSkqdSkAQAAAAAAgAeLNAAAAAAAAA5gkQYAAAAAAMABLNIAAAAAAAA4oEgWDk6oXUvynfd9bPV59bfWkldco9tkb9QCRPHQs+wCq+3Gx+6XXPEtCuydiQJ7DkkOFtI8EB9rbqkatc/6bC1sV2/qQcmRCvj6U7SU+MrRTSTPuXS4Zwu7IPUH0ztITjs2O+eJwjn+unq9ur78B54eSQZnlgP17b+peQuzvvNQN8llJum/fV/r5tYYHd/SPvdWXCG56dtaKLjBY/OtMShG7pbMrlpE1Xuc/LdN+wzMvERyjW93Sd7Z3/6Bjm9vfM4zZnKO+9jYu641Rq1nKBzsLJ/Patr4qB4H4/q9JLlFCf1aOeFQJWuM55+/UXKK55c23mrg/U6XbLy6ztQfYmj4rX1eQjEX0m9S3sLCPaZoQWpjjFndUwvh++7ZqR3sGtWFiidpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHBAkaxJs/4P9ST3KfOZ1afRVz0lp2/8Oe7z8CVpXYC//K6n1afiFmrQFHXeOiHlSh47RU/gf6784h7JTQ7vk5x5f3trm7/cprVHri39k6eH1qC5riifhgAAIABJREFUYe1V1hgZf10iOWT1gOtCZfUd/CYl8v73lMD+Inm5x/+p88V+q23H4KOS73xuvOSnUvtIHjboE2uMW8puleytQZP6hNbaC2ediD5ZFKqbWs2R7Dd2bRHv32i//7WR5KaHtkv+rYV9JakR0POUvR/dR7cb/2ONMf8Z/lbsikClipK3v13F6rO01ShPi15XPj1cXvKzL2n9GWOMqTJWvxdd9qvW66vg1+NqYGZHa4yMYZmSqfMIr4TKR6P2uaf+15JfMRn5NZ1c4ewIAAAAAADgABZpAAAAAAAAHMAiDQAAAAAAgAOK5EvqJVvvidqn4pK8rz+FO7SQvL6bvifZ8UKt/TB3Qn1rjBrDt1ptKFqOXthM8rTGowtpJnBG6pGoXX7q9ILkg1fpOSkjUevLGGNMyIQle2vO7HqpgeQyP66xxzh8OOrc4LZwQI+VJF9insdM++iQ7iPPI6IghRcutdoumnCf5OU3viK55/0jJe8N2fXUWj+hY6S+OV/3Sw0a97VpLnFwpVclh4zeu/63TWvMNH10m+Rlj9aQvOpq+77HO4b3777ezydvOMsao6ZZZrUhfyRUryZ55X2pkqdeN1xywwT7uDkUPi65xZdDJTd5dq/kKqvsupxrXmyr+62Q8z31vPFnW23Vd8/McRuceQIVKkgecd74U/Q8aRuf21UbeZIGAAAAAADAASzSAAAAAAAAOIBFGgAAAAAAAAcUyZo0+zaW14ZWdh9/j92S955oJ/lgqk/yOZevsMb4MPWfkrPCwRzn1bhFeo6fo3jwn8baZqIvUAAzQUFZ/w89f0xqN9zq4zdaY6ZGIMWTtf97B6tbY7w8qpfkqqP0vesUs0tyzmckFFVbLyyT5zHGHawkOZC5U3J2nveAwlZ2tS96p5Nc9tL9VluN1/QcQ62iomdjZz1f1AhoLRG/sY+TRh/fKbnmG3ptWdV8TNQxvH/ntfvo5+HZnnt35JvD155vtZ33Z6039Vn1Lzw97Bo0Xud+f4dkby2j7M1bJIcuaGmN8Z9rn/e0pFh9TjZk4KdWW497V0u+6ffDJCd8p/9bUfztvLax5MuTv426zfPrr5RcwmyM65zyiidpAAAAAAAAHMAiDQAAAAAAgANYpAEAAAAAAHAAizQAAAAAAAAOKJKFgxs99Kvkpzs2t/rMaPGhNrSIfT+P7dJx/1x5SY79Gz+y12qjOGPxEzKhqH2yqL5YpO28s73k5TePkhwySdY2oRhLbi46XNdqq/mpFi0LV9bir8Hde2LaB4qmQ41P5HmMxydcL7n+9ll5HhNueevBlyRvC2ZJPhjSauUL79XzmDHGdP1Mi5UHV6+L0+xQULLTj0i271Hsv8cOu3Sa5CHl18Y8RrQ+p3OvhPhoMl+/zj1XfbTVJ3Lx59isuvhNyRtm6LF3OKzzqJdgX3dK+3IuFOzVv+xmq23ucf2hhpJrKYx/pvG1bCb5k7895+lhH2fHw3pkhF+o6ulB4WAAAAAAAAB4sEgDAAAAAADgABZpAAAAAAAAHFAka9KEDh+WPPOcElafK68YJHlbO7vPyVJHr7balj9TT/IjVy2TPGxra8nBLdty3AeAoqFNv4U5fr4m67jV1mn6UMlzLxshuZxf36EeXn2ONUboZ61r8+MxPW89sbar5I3rvO/TGpMx2B4XbkuoU1vyzMtf8vQoHfOYF12xSPKGv8Y8BByyakxrqy3FP0PygF63S97eoYzkefeNtMZY/3QpyfV66zknnJX3+kgoWH7P318j1SLx1qDx9pl/XMfoO7u/Nca/2mp9ktZJOY9R65mZp5gx8urPVX+Q7DfJBbLf+gmx1ZfJjW3BI1bbTdPvlpyxcV6+zwNuWTlIr121AtGPxYt+6SO5whdz4zqneONJGgAAAAAAAAewSAMAAAAAAOAAFmkAAAAAAAAcUCRr0pyOxK/0/cS6X0XZoEm61fT3Dh9LXnRc61As+Pu5kksd//n0J4giI3A8JHlrth4HNROSCnI6yAe+ls0k/6PGWE8PrSfT+/n7rDEyRun79t1u/KPktLu0ptWDNaZZYxwO6yl52TGtVTKl6YeSk5olWmO0+VVr41QdRR0A1x3LqCa5jD/vl+Yfp7SUXNdwHLjMl6D/zQ9ec57kH65+3trmhof0PFR2zmzJNRbomP2vv9ga49f270g+9847JVd/kePGdYmrtBZD6MKQp4f999iQybnPH97R60i9/9fefYdHWSVuH38mk0JIAOkQCD0BRJqASnEVCyKigggrKGvBRlPcRdeC+tpWpYh0bIgFwYJiY5EVRJEuAioECL23CAIJgUz5/bHXe633c0ImCZPkJPl+/rtPznPm7DJMZo7z3Cw2e9haux4n4Frj748NllzO0ecnwqfPfdrRsvdy83dI9FHtDKoz96jkfVdUlJywUH+enWCUV/KwWR9Lvjr2lHHNvFP6fH322TslV1p3TLInM8tYI3kTHTQlWoTXGNo8uY3kjddPcs0wr3Grctcfkv153ljh4ps0AAAAAAAAFuCQBgAAAAAAwAIc0gAAAAAAAFigxHbS5FXzD1KNsd7xaZIbzdP7a5Nn00FTGkQuXC35r7/p/bOLW31QmNtBQYjU8+r4iJx7hqovP26MBV253Cy9//6g1sk4w1sNMNbwnPFJ9m/YLPni7XrPbetoc29+KpKKnWON9A8yPqLMWWbmXrT5FIXF0q/X++0XjdX77S/7h9mDVX5Wzh0fQZ++nhzJjA+5j6y4kFNgmTpzT0j+ql9lyT3itOPDcRxn6rFGes2df9E1V2oX0dYPWhlrRLj+O2+Eo50nZQ+dOcuOEW4xc1dJbjA39DXuVqIa63L+eXYiE7U3L7sOGrfHJ92lj/uuPtdy87jInfSbL5bsK6N/RyvN08++/iP6ubeweMuXl7zxhabGnC3XT3FfJcnnapi5/JdbjDXKH96Rr/0VFb5JAwAAAAAAYAEOaQAAAAAAACzAIQ0AAAAAAIAFOKQBAAAAAACwQKktDk6doGVKX1SbaMy5cNXfJDe+/xfJ7qJQlEynr20neUmrqa4Z5llnlEcLrfZ3qSG5Wuq2sOwN4RGRflrydl+m5OQobdPce7mWnDmO4yT8lLfHDKzdYIx5YrT1d8eHLSS3if455LoJi/6QzOuU/arPXC951xMnJdeJDF346hZ5ij95m3mTGkieOvZVyR+d1EJOdxF5Qam9IL1QHgdhtPJXidOuuFTylKTqxiXRq7fowPFfjTl/9v4lbxljAVfF6+rT+l4o+oAWGmutJ0qCHf3r5Pmacrt4JhSWx156R3LX2AzJS57Vv7MPjB1krJHwkb5W+A8eOud97Xqqg+R/9vtEcv9yi/K85jOHtHy//LVb87yGbfgmDQAAAAAAgAU4pAEAAAAAALAAhzQAAAAAAAAWKDWdNN6mSZKnX/ea5Ihszqu8886THMw6E/6Nodhx34ednSxXHURE1yM6MCmMG8I582/YLPnaxUMkb+6s9+M/es+HxhrvT9TXmEBmpjHnz9z9M47jOEf7XCj5t07aleV+5jWdOdhYo+HqwumuQPj4jx+X7H79yI2TAX2+VXlt2blsCQWs7DT9My/r0Z6GGdf9xXXF9jw/xt5H9b7/dclm996NqddJ9q5LlRz6tx1s49uzV7LXlR0ndD/M3n/qc6ddjNmHFnC9b+678D7JySl5LGpDgdr8VlvJVzTbKPmHRc0l17pwv7FGw/L6XnZUjVdcM6IleT3mZ6sGw1MkH/53WcmBDO1NQf49t7m75K4tP5LcMcbVK/Wo+Tvi6COnJPfdfIvk7QeqSC4Ta35W/k9b/cxdzRu6XzGUpw+3lLyuR13XjN3n/BhFjW/SAAAAAAAAWIBDGgAAAAAAAAtwSAMAAAAAAGCBUtNJc6iT3jPXPkbvyL14dT/jmqpTuacf4VFuSoWi3gLyoPELJ3Wgs8Y+8YeMa1YtqS95w1C9XzbymN7Xu/XpMsYa7g4at+0+7R1p9GG6MScfdSYoYkfvaC+5XuS5368Ne/iuaGOMTa+vxWS3bOkt2b8l7x00ex7XHpGlg8ZInvpHsrm34ZUlBzPMHgogkM1vFnc/3/n/T587vgLdEfLq5Us/ltwr7qhO6P9DPlaNzvGnp4NZxlggqN8P8ERH6QQqacKm8r36vrNbwu2Sdz+if6/XtH/bWKNiRKzkeU0+1wlNcrMT7R1ydxX5g/pasjHrtLFCn9f+Ibne21sl+w4U/w4aN75JAwAAAAAAYAEOaQAAAAAAACzAIQ0AAAAAAIAFSmwnjfc87QC5cpD2y7jvk6z2nHlfJd0OcBzH8Z7WeyX3+fReyYTIGOMad3dIXIp2mHCvtt38KamSu/W6Q3LTSRuMa4ZWWSQ5apbmLNcLSr1IvUfXcRzXHf6O8++McpJffnyQ5PhVK4w1UPz4XS8h7vu1UbwFojzGWIxHexiOTawjOc7Rfg9P62bGGhdPXyt5WKXRku/cdqPkUw9UNdYIrv0tmx2jtOvRd7HkKI/XmOP+nebbs7cgt4Rz9M5fLpE8YmwNyfFx+r51WvN3jTVWZ9aVPCn1Msmnl2vHVbXVZ4w1or/5KfRmERbG30lXTrxZf9zl+sHGGmfK6fuRP27SzsaOidqf9nCN+cYaDSO112bRKV1z4Lv63rbu18eNNWr/tFRyafgcxTtBAAAAAAAAC3BIAwAAAAAAYAEOaQAAAAAAACzAIQ0AAAAAAIAFSmxxcMqoZMmfV5sq+YIf75Vcb9UvBb4nFE+RC1dL7vvEcMkNBm4yrtkzKkly7PaV4d8YCs9yfX1IaWNOGXTRQMmpQ7QYdFrH6ZL/tb+DscbSf7eQ3OD1bZLj91MUXBJVnaavMW2695G8us1HIddo/uUDkpMdXnOKE/+AI5LT724g+fvm7xnX7PJlSO7w1iOS649LkRw4ahaeA47jOGkD2ku+t9IoyVlBLf50HMcJGFX3sJnvwEHJ9fsePMvM//qnc3HINas65vtfFF9lvjTfN5Rx5fIfaN7l+vlQp2OeH7eOo6XA/MM9/8U3aQAAAAAAACzAIQ0AAAAAAIAFOKQBAAAAAACwQInopEnvZd43+XPXsZK3+/QOtzoTOJ9C/lR4f7nktPfNObH0QZQ+K3+VmPQ3/fGLjvbNOM5JYwn3fbm+cOwL1gtmnZFc40HNAz7oJDnGaz4zmo4/Ktkfpr3h3JU5kG6MvZzWTPIPLXLuHWq96jZjLOEZfR9TZ42+fvAcQG4du/KU5Jpe7aCJ8niNa+7cdblr5ER4NwUApRgnFQAAAAAAABbgkAYAAAAAAMACHNIAAAAAAABYoER00pT5PcsYSw8GJN+0+j7JtX5cW6B7AgAgP3zbdkjec0lurkotiK0gDALrUoyxxS3KaHba5LhGTcdcI5jNPCA/4peVlbzyEo9kbzYNR7sfS3LN+Tn8GwOAUopv0gAAAAAAAFiAQxoAAAAAAAALcEgDAAAAAABggRLRSeP9zrwPdkCdTpJrOesLazsAAABAsVB9wlLJz064MOQ1dNAAQMHhmzQAAAAAAAAW4JAGAAAAAADAAhzSAAAAAAAAWIBDGgAAAAAAAAtwSAMAAAAAAGABDmkAAAAAAAAswCENAAAAAACABTikAQAAAAAAsIAnGAwW9R4AAAAAAABKPb5JAwAAAAAAYAEOaQAAAAAAACzAIQ0AAAAAAIAFOKQBAAAAAACwAIc0AAAAAAAAFuCQBgAAAAAAwAIc0gAAAAAAAFiAQxoAAAAAAAALcEgDAAAAAABgAQ5pAAAAAAAALMAhDQAAAAAAgAU4pAEAAAAAALAAhzQAAAAAAAAW4JAGAAAAAADAAhzSAAAAAAAAWIBDGgAAAAAAAAtwSAMAAAAAAGABDmkAAAAAAAAswCENAAAAAACABTikAQAAAAAAsACHNAAAAAAAABbgkAYAAAAAAMACHNIAAAAAAABYIDKnH14d0TtYWBtB4fpP4GNPQa7Pc6fkKsjnDs+bkovXHOQXrznID15zkF+85iA/eN4gP872vOGbNAAAAAAAABbgkAYAAAAAAMACHNIAAAAAAABYgEMaAAAAAAAAC3BIAwAAAAAAYAEOaQAAAAAAACzAIQ0AAAAAAIAFOKQBAAAAAACwAIc0AAAAAAAAFuCQBgAAAAAAwAIc0gAAAAAAAFggsqg3AAAAzs22ke0lb7h1ouSWU4dKTnxuaYHvCQAAAHnHN2kAAAAAAAAswCENAAAAAACABTikAQAAAAAAsACdNABQBDxtmhljbd/6RfIzVddJ3u/PkNx91CPGGtUn0DVS0u14vr0x5u6g+TqjguR6UzZJ9od/WwAAAAgDvkkDAAAAAABgAQ5pAAAAAAAALMAhDQAAAAAAgAWKvJMm9Z0LjbFtV0+TvCQzILn/vwdKTnpHexocx3EiUnZIDp45oz+Pj5Ps//2ouUZsrGRPbBljTij+pNqSjz55SvJd9bU/4svu7Yw1fNt25PlxAdglooy+fjz28QxjTseYgDH2ZzW9ZSV/NnykMeeOHX+XXObLlbndImwR4ZV4cPDFkqf10/4Zx3Gc08EsyaMfvVVy3JEVYdocCoO3YkXJG59PNubUWOyRXG7W8rDvw/0e7Y1O7xhzxlzfS7J/w+aw7wMAgNKEb9IAAAAAAABYgEMaAAAAAAAAC3BIAwAAAAAAYIEi76Rxgh5jyB/UXoZLYvTnqT2m6EAPc9n3TtSQPC/tAskz6n0rufGiAcYatzT7SfIzVX80HyjMXn3pCmOsbp8Cf1gABazN8nTJ2fXPPH24peQP53WS/Ebv1yRfWkY7ahzHcQ72196rul/maZuwQGRiguQVj44LeU2Tr4dKTp5NB01xtuXhJpJTeow35oy5VN/XfD8r1phzrhZcro+bEBljzHk+sbzk6A1h3wZKKW/1apJ33tXImBNx0THJFcvq78Ah9RdKfiu5fph2ByAcMrtfJHn/bZnGnPWXvp3jGhe9qO+Bqk1cepaZxQffpAEAAAAAALAAhzQAAAAAAAAW4JAGAAAAAADAAhzSAAAAAAAAWKDIi4NrzIsyB6/O+ZpGc++T3P78LcacLpXXS+5a+TfJ7mLhEW2+NtaIcLTYc3OWFhn9HigjOfWMruk4jjM/rZnk9Z80Neb8WfVUX44/L812PN9ecuKC08Yc73c/F9Z2gBwdGNZB8pNVtfx1fZbfuGZNVy2MTap4RHJi3+OuK8zi4JuT10pe5XhDbRWW2fxi5Rx/PsZVhO84jnP+07sl85vEbhHlyklOfUr/TBf+dZTkjID539TmvKL/0EBFZ1mYdgeEn7sE2HEcZ28/LQJuevNGyb2rLpfcI25eyMcZtLej5PGP3yI5zqFUvaTzltcyc09Zs1T96GVaIH2khxZO+w/qNY2G6XMR2fNERRtjEa6y7qOj9R3KM0laCtw51iwOzgqa75n/7O1/jJU86MiDxpxys4rXnyHfpAEAAAAAALAAhzQAAAAAAAAW4JAGAAAAAADAAkXeSVNx5YGQcw75MyTX/Ux/nnbPUeOamU6CMXau3r38BsnRh9Ml+9dvyuYq3VsNZ2m4t1Vq1BtROPfbR5TVno/fe7WUXOU/2yT7DhzMx4NoT0hEnNktcqifdhT4YzySj7fRezYb37fBWCOQad7XicLxxKAZkiNd3TB3/usB45oqB/Q5vm1YA8n1Is3nCYo/f+cLJf/Qabzkg65bsRf31/mO4ziB/Slh3xcKjqeOvkdZ32+8a0aMpPM/HGqs0Wh6+H8nbvugleSEyFWS/3lAu+Ecx3FiFmgPVjDsu0JRi6ybaIwFysVJPplUQXLFYTslz2r4hbFGjCfnjyFpAe0JGbT3KmPOuvH6Hq3CB/qcjQvQQVOc7XqqgzHW8pqN2cz8nzuqL5F8deyps8z8nx0+/az5/rGLJC91zK4VmHY+0dYYW333q3lcJe9dislR+hkp4o5DxpzA/taSt92ov2dr/aBdtLFzVuZ5H+HEN2kAAAAAAAAswCENAAAAAACABTikAQAAAAAAsECRd9LkxutH9b7AmLmrzjIz/7xNk4yxk8kVJcevPyzZv2V72PeBwhVZu5Yx1n+B9gY1if5O8uMrb5XsOZJmrBFRobzknfc2kdz5ptWSxyV8n83udMzr0TNVf1DvnWxzzxBjheoT6EAqLBEtm0q+PHaJa0aspCqvh+6TaHdpzvddo2TYeY3eF13Jq/nOHV0kB9bRP1PcbX0qJsefrzmtr/dNRprvN3xh2Eewg/Z5zO4w1TVDuwG+WmT2DTT0LQ/DTmCTzZP1ffc7XV835lSO0J6PJlE5P6ez+8ix9ow+i+/5tb/kqs+7ekBW/mqsUcHh+VecnenaTnLw7/pZ6+m6M41rqkYez3HNkTuulTxwV3VjTuRBfW4lT90r2bdjV46Pgf/aOkp7yhb9dWQ2s/T/65Qs/emofV0lr1qVbKzQ+PXfc9zH/Z9/JXn+BbOMOb9N196aC6K0Qa1FULsiG87J8SELHN+kAQAAAAAAsACHNAAAAAAAABbgkAYAAAAAAMACxaKTplP8JsnLW/aRnJv7873JDSVHv3FScv+a3xjX9IrXex5fTtPemjkvXim50vIDxhq+bTtC7g2Fx1teu2L++u0KY07veO2Y6fDoMMm+y3V+rdf1ueQ4jvNZo7mukW8lbfXpvdytVt5jrHHyQLzkxde9Irmmt6zkhAVHjDX8xggKyolG+tyqHKEdNC+naWdNbizbqK9bTr0FIa/ZeMJ977X5vEDRiaybaIw911PvnfYH9T7pTe9qp1UVJ3SfEexx9I72xtjqjmNdI9r9cttngyU3PFAwvRsZCWUkN47ynmXmf5Xf4snx5yiejve9RPJjnb+Q3DFGO/D+Sztoem3RHpC9JypI9nxS2VihytdbJFc9vMmYg5Il4gL9fTZ84nuSl53Uz1pvN66bj0fZIynZlbMTjo6v0mDfwx0kp/SbIDkr6OqRysbdo/RzVbXJ2p/ZKJueqVCfZzICoTqxHKd1tH43JSto96ckvkkDAAAAAABgAQ5pAAAAAAAALMAhDQAAAAAAgAU4pAEAAAAAALBAsSgOTvNrgWrE8QzNDeoZ1xy4qqbkXkMWSv5n5dBlw+4zrOGVtNBs+CjNc9LPM1aYuvsyydG3a/mab8/eXOwD+RVRrpzkkx9rcd2t5Q4Z1+z16/Or1dC1kifXWiJ5fdYZY41WK++SHDtHC/Qqf/KL5IT0DcYau5/Uci53UbD7+RbYutNYA4Xn2K1mgfSffT76Csnn5aL8tclY15pdQ+9j41fJkmtRHGyVrQNqG2M94/V1aMVpLd6r8hpFwcXZqapm2W6UJ+eC3sT5hVNjWS5V/4GEn07rvtrGaLHi0eZm0WLV8G8LBczT9gLJ1z/2neQB5bVoNbv3OY91vEmyb/9ByZUCmrNjd20nzpWnXXNjrPbEbZK7xup77mGzO0luQFF+kdo6Sovv3UXB7t9ls09WMdZwlz9Xc5Yac/Jqy1gtO+8Vv9o1w/wd697rLtc/4lLpV7uK8fkmDQAAAAAAgAU4pAEAAAAAALAAhzQAAAAAAAAWKPJOmuCJdGPsvRM1JPcvd0Dyik92S47wBI01vqj+qeTpxxMkN1tyu+S6o8297btM+0yiLk2TvKrNTMk94o4Za/Ro8rnk/yyKlfzAJ9pd0mj0ZmMN/5E0Ywy5s+UJve964wWTQl5Ty9X9MqDqD5KTvr1PcpMHtxprJBwzO2b+LODKEWXLGnM+uGusayRK0ksv3Sq50mnu2y1KE1vOzPHnlWZrD5H7OeA4juOJiZF8bFRWjmu+8UeiMVZ7zErJ5qsjitKZeqdDzqnh1d+Lex7XfqoON64zrmkWr/1mgaD+N5i3PtBCo9r/Ovd7wlFwjjXUXqLqUdHGnGA2PSF55cnU15j0QIxrhvZFVPw15y4d2CeyVoIxduN72tPo7qBx6/v6342x2vtX6ECAhpmSLrJGdclpV9WXXOf+VMmv1JlirOHuV3T7ru8oyV2OP2LMSXyB318FwVu9mjH2So93JGcF9e+5u9flpUmDjDWq57GDxluxojF2+sIGkl/o9mGO+8qN5ZnalVNxul2fo/gmDQAAAAAAgAU4pAEAAAAAALAAhzQAAAAAAAAWKPJOGv/hw8bYR70668Ds7ySOrPFTyHXdvTYTJ94kue6k0PfHJWi1g+MZq/933ZDYU/KOW2oZazx75/uS3b01Kf21I+XWS68y1jh+bXnJ/uPHs98wnMzuF0n+4pYxrhllQq5xY+p1kgM9MyUnHf1Zcjjuwt4xvJUx1jz6R8lz0s+TXGWW9lJk13GCguFp08wYaxylf17PH9HnoqeOqxcgRe/ddhzH2Te4jeQ1LSbmuI/xM240xhJ93Ktd3ES4/ntJ/Uh9nVo7eELINaI82hXivj976GB9vi24y+wFGHvbX3Vg+S/GHORd4mf7jbFJtzeWPLjiJsnLnxgn+fZbrzHW2Ds6SXLZz1YYc0Lxb9oiec2pepIvi825Xw32Od73Esm3Pfm1MSdUB43bs3e8b4y9OaGl5MCJE3laE8VP5vvaWbWk6WTJJ4Paufbswb8Ya8z54SJj7M8iqut77u/uG2nM6eJoTw0dNfnj7qA571OzB/GqWLNv9c/6Pv2w5OrTz/3P4uSs84yxec3MfqNzNeJ7PRtIdlaF/THOBd+kAQAAAAAAsACHNAAAAAAAABbgkAYAAAAAAMACRd5Jkx3/er03+7Xtl0ru3+JjyTNOmP+u+0c9L5NcLeXc75EL+nySfdt3Sq79ombHcZw35l0v+aXn9H6/5a1nSZ5R71tjjUaj75OcfK9d98wVmQivMdRohN4/nxyVcwdN0x/uNNcYtEuy/+jRfGwub6bcPjXknJf/davkihnLCmo7CCGzutnnUc2rYyOq/CbZu0Cfm5f/1sNYY02znDtokuYM1Mx92MXO5A4zjLGAq1FqxEG9Z3/Fs+1Crnuqkr4ethu4RvLYhMXKaAWYAAAO7UlEQVSSO8eeNNbwv/+R5Ik36/3agbX0k+SHf8t2Y+zbntpDFvO5vje4u8I2ye/U+8ZY48T4LyV/81IdyU9/2Vtyva/OGGt4F/1sjKF4S2vukXx/BfO96ayTVSWnB7RrxN1Z4+5TdBzHeWpIc8m1X+T3UUl35KvakhuvHSS5znz9nBQ13+wQbeQsz/Ex0ntdLLnmZeb7rVOJPmMMeecpGyv5uiqr87xGxen6WSQiLs6Yc7LLBZJ/b6LvV568Y6bkXvHmPrKCed5aSIlfe0JPKkJ8kwYAAAAAAMACHNIAAAAAAABYgEMaAAAAAAAAC3BIAwAAAAAAYAEri4Pdyl+7VXK7gYMl11h42LjGvym1QPeUW8E16yVX6q4/bzV8iORJgyYba2zv/obk62pcI9l34OA57LD4Sn27pTE2N/HNHK+5MfU6ye6SYMcpnKLgwwPbS24bYxapzU6vLrnybC2i1apRFKa4n83nzbxTWm7XNTZDsj+of2ILmn0a8nEe3KfPk6TBK3K7RVjqStfzwnHMv8s/jtbixPJzci5adBzHiXXl7bP0+Xj+C0Mlb+gzwVjjqtgTkh/pdp7k2mtDbgO55E/VYuC53dtITvkwQXL78luMNS4po4WwfeIPae47SXJUP7Nsf056vOTEyN8lRzh6TUZns3B6V7UOkjt31zLiHV31uehP08dAeCV+qwXRSRUGGnOaPO16Pp3R4uqs5frnnl358Onm5msZSrYar4a/HPpMVy3GnzxmnOSvMyoZ1zQdoZ8L/WHfVeng/gdwnvq6tzGnZ5/xOa6x+xMtBS5fNtOYs7CF+X4jZ+bvqtKIb9IAAAAAAABYgEMaAAAAAAAAC3BIAwAAAAAAYIFi0UnjVnXKMsnF+V7EhNF6f+eQ4CBjzpq/T5ScMqKe5KQhpbOTJjkx9P9udwdNoKfeK1kY/TPZuW3wN5JjPdHGnNEv9JNc8cQyYw6KRnY9UJOv1q6olt/PkFzTq70MubF4hvZU1HDCfz84CpY3qYFrZHXIayot3iPZl4/HDWRoX0STV/fqhD75WBQFxrdth+RUrWlwUp1E45opvW6WXGOY9jS8WneO5Oped3OR41xb1v070CMp4GpMWtdxmrGG01FjZlCfsV2veUhy+Q9Cdywh/yIX6mtM0kJzTqj3zeM/1QLF+++cZMzp32yl5KWO+T4GRcTjMYb++Lqh5GENF0h+fn03ydWmmq8XUfN/CsPm1Kkq2j/SLEqfR4/vuMy4xn9kf9j3Acf98u84juNEeXLuh9nQ4X3JWcHsXl10jY5rb5F8IqOM5Dq9fzVWODhUu89WPZrXnhvHmXlCuz7L7tX3ScE8r1iw+CYNAAAAAACABTikAQAAAAAAsACHNAAAAAAAABYolp00gOM4TsRgs+Oj9bVDJNeetl6y/9gfBbqnswm2byn5xnJ6f3ff7Tca11T6QO8rt+1eydIsIi7OGEubFCU5Px00bidaa4dSjXNeEYVtZ287/tT+aJuQ52sqr89PGw4KS9zsFZJPzNaf393mPsm7r6lgrFH7yl2Sv2jy2Tnvq93i+yXXp4Om2MmqEAg9CdZqvMr8eHc6oL1kmUF9z7Lu4vckr21tvv4PfOZByVEZ+s50X5fQvzOiy52R/Frb1yU3XjRA84jfQ66J8Gj8ptnTeeWigZJ/vz1d8ohmcyX7s/n+x4jvb5Lc5KENkium65rZid+vXTfrz+hzLTkqm0IdlzpR+lzyx+rfAdu+uWLbfgAAAAAAAEolDmkAAAAAAAAswCENAAAAAACABeikscypqqGbR2IO5fxv1pcW/pRUY6yma8xvzCga0S8eklwvUvtKjg81eyuCWWkFuifknruDJu0js99jWcsPc1zjkQNtJX/x7cXGnI23aVdRbPzp3G4Rlqr1fYbkiMHZ3Tet/73k4DWJkiu/uSfPj+utWlVy0sN6D3hENv+NpsnHgyU3+pwukeIsuFo72WqvNud4RurbwGuv0P6Bj94aJ7lcRLSxxtWD9HlT/4tVedkmcuCJNN+mZ3ZpLTkrXt8Txn+U97+3kfXqSJ7W7Y2Q13y+s7nkqs6mPD8uCsb4BPPvYP25d0vePVLfx4zrUl3ykMGfGmsse2GSMZZXo39vLPmhMdph1XDyUsk0oxUe/3rz73Cs/hpxan2u+W2nbsh1kx19Puan8SruE+1g69VB+5F+7TM+5BrfnWwqOSpNu3Bs+cz4//FNGgAAAAAAAAtwSAMAAAAAAGABDmkAAAAAAAAswCENAAAAAACABYq8ODiiZVNjLKNOOcnxa/dJ9u3Oe4mirSITa0uecPO0kNfUH68tTrYVHcFxAp1aSX6lvpat9d7aQy/4zSxBhj02jbxAcmqrKSGvabbkdskNH9Dy6PibsyuQVeNbzZI80ml+lpmwlWfJWsnvnTBLwvuW2yv5tof+LXn+D20k+zdvDfm4B3s2kvxZopbqZVfc12hWRjajKMmCPq3ljJr/k+QTAf3HDMpl85/2vJmuZ1Mw9D+AgNzZ+9BFxtjaYRMlt3p1iOT4fDzO5n9VknxpGX1euIvvHcdxqj2jHyH4U7dHs2W3GmMzr3hNcqWrMyUfC2gpeNWI7P7hgrLZjP3PhGMNJE+f2s2Yk/CefoapdmypMQdw8yY3lDy75zjXjNDvqX/qpiXH/r2bz3VbBYpv0gAAAAAAAFiAQxoAAAAAAAALcEgDAAAAAABggULvpAl21K6O4e++Z8y5vEyW5KTPBkqu+1VNydHzVoVpd4UvdXCi5KtjTxlz5qSfJzmY5TPmwC6dJy+T3DAyVnLGQ9UkB7MOF/iekHuR9epI/rTbeNeMKOOa5pO1F6Duiysk+wLaHnW8Yf2Q+5iw5yrXyMGQ18Buz3x/ozHWt/tkyQPP046qcY/o8+D85/T3huM4Tvr52nXz/uNjXDP0Odtk7iBjjcar10mmYwJvHW0v+cmqPxtzdvTSLoDkbwp0S6VKZKffjbFTwTOSK/+WZcz5M09UtDH2x80XSp7fcbRkr0ebbeZua2asUWf1rzk+LopO7V7rjbGnL7lTcmo/fV96Qcudkk+ciTHWODK/lj7O/KM6YZt2hlY/YfbN0KOJ/Ng4pIrk5KjQHTRuvr37Qk+yCN+kAQAAAAAAsACHNAAAAAAAABbgkAYAAAAAAMAChd5Jc6hNWcnu/pnspPacInnvDRmSu0x/2LimwaStkv0HD+V2iwUqskE9yS/eNCPkNU+9fZvk2unmPZ4oOoeGdDDGhlV6VXKvLdfrhHWbCnJLOEe7X42T3Dxa+zySPtWeLMdxnOSXV0r2JmhHyIbnNG/poj0kjuM4+/362nb6gUquGXTSFHdNxx83xsZ3bCL5gYobJW+8Vn8HOtea60a4/ptLIEQHTdNRZteFP+uMMYbSbeZ3HSU/2cfspKmS8IcOeFxdAUHajfKrWvxJYywrGJDc4rm1khfeo/0xN9Q3u2Oeq+b+/aPvzYftbyu5wQNmbx7tiMXM8l8kJi3XH592TTebjBwnwdHemkA2c4CC0KldiuQojzfH+W1HDTXGajjF6/Mz36QBAAAAAACwAIc0AAAAAAAAFuCQBgAAAAAAwAKF3klTfeIKyS3ihhhzyv9Fexd+bPGx5FpevXd2/YBJxhrz+umcNF98jvsaOb2PMVZzySnJEYvX5LhGeq+LjbH9nfTe7NdveEOyu5Nn+vEEY426b6ZK9ue4CxS2XvcuNMZiPPpXK3N4NclBH90iNutZ/5ccfx6RaZ5vb57WQvKXf9HXpSZRMZJ92fxNvv6lRyRXW1e87p9FaP71Zh/Vd5fXkzxlzGWSU656LeS6A3Z1lrz1laaSG8/R31/0zyA3mozbpwPmWyXnx9barXdDu7t0wkqzEwW5kzYz0Rh79N4rJU+utUQn1NT32bnReoy+F689Y4tk/8EDeV4TAPLDW7GiMVY+Ujsbs4I5fxo+Xan4d6HxTRoAAAAAAAALcEgDAAAAAABgAQ5pAAAAAAAALMAhDQAAAAAAgAUKvTjYCWjRT+0XzWJMz+hoyTfU7il5299qSZ74N7NUsWtshmvEndWtQycaY3sH6TWLT9WVPPvghZJn1H/FWKNiRKzkOennSW40v7fkpqNPGmv4D5tFkyg6O59tL/mLyuONOb22dNeBNSkFuSUUsk39zLJyU0yOP+1y/2BjrNqXFAWXRv4jaZKTbtd8g9MuF6sclxTvaHlo8a/QQ1EIHDgk+fYd1xhz3qn3jeTub/8g+atmZgkkcqfym8uMsZ3v6HvkztfdL3lfp9D//TXxW30vXvMbfRx/kFcMAEVj7x1NjbFPar6apzV+vmucMdbzqYvyvaeiwDdpAAAAAAAALMAhDQAAAAAAgAU4pAEAAAAAALBA4XfS5EIw64xk3/adkus8o3n0Kx2MNcZ49fwpo0Oy5F3d8n4+FSyr9/BuueZ114xYx+2in2+RXPWpKMnJa1ZL1keADbwV9X76F26ZIXmP77RxzZn7y0sO+g6Gf2MoMMtb6t/Tbs6FZ5mZf2WclWFfEwDCKZCZKTnlI/O18McHvpfcLX695K+cTuHfWCnmfo8cO0d/lzScU5i7AYDwKnsgYIxtztKerOQoT45rTD3WJKx7Kgp8kwYAAAAAAMACHNIAAAAAAABYgEMaAAAAAAAAC1jZSZNXgRMnQs6JmbtKctLcc3/c3PRUVHE2Sw6eZR7steVhva+xR9wCyR0fGW5cUyFleYHuCQCAwlZj3FJjbOS45kWwEwBASVR+pvkZalDwQckRdxySfGRFDcn1Pz2azcop57y3wsQ3aQAAAAAAACzAIQ0AAAAAAIAFOKQBAAAAAACwQInopAEKUt/rfpD8jwMXSa40f6txjb9AdwQAAAAAJV+5Wa6emlka45xtkgMFvJ/CwDdpAAAAAAAALMAhDQAAAAAAgAU4pAEAAAAAALAAhzQAAAAAAAAWoDgYCGF5yyjXiM+VDxfWVgAAAAAAJRjfpAEAAAAAALAAhzQAAAAAAAAW4JAGAAAAAADAAp5gMFjUewAAAAAAACj1+CYNAAAAAACABTikAQAAAAAAsACHNAAAAAAAABbgkAYAAAAAAMACHNIAAAAAAABYgEMaAAAAAAAAC/wfHSejGkrXLocAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1440x720 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Configure the mosaic to display images\n",
    "fig = plt.figure(figsize = (20, 10))\n",
    "columns = 8\n",
    "rows = 4\n",
    "\n",
    "# Create the mosaic\n",
    "for i in range(1, columns * rows + 1):\n",
    "    \n",
    "    # Choose randomly an image\n",
    "    image_index = rd.randint(0, len(train_data))\n",
    "    image, label = train_data.data[image_index], train_data.targets[image_index]\n",
    "    \n",
    "    # Plot one image in the grid\n",
    "    fig.add_subplot(rows, columns, i)\n",
    "    plt.imshow(image.numpy().reshape(28, 28))\n",
    "    \n",
    "    #Remove axis\n",
    "    plt.axis('off')\n",
    "    \n",
    "# Render\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Build a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Classifier(\n",
       "  (layer1): Linear(in_features=784, out_features=100, bias=True)\n",
       "  (layer2): Linear(in_features=100, out_features=100, bias=True)\n",
       "  (layer3): Linear(in_features=100, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "\n",
    "# Just a simple model\n",
    "class Classifier(nn.Module):\n",
    "\n",
    "    def __init__(self, input_dim, hidden_dim, output_dim):\n",
    "        super().__init__()\n",
    "        self.layer1 = nn.Linear(input_dim, hidden_dim)\n",
    "        self.layer2 = nn.Linear(hidden_dim, hidden_dim)\n",
    "        self.layer3 = nn.Linear(hidden_dim, output_dim)\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        out1 = nn.functional.relu(self.layer1(inputs))\n",
    "        out2 = nn.functional.relu(self.layer2(out1))\n",
    "        return self.layer3(out2)\n",
    "    \n",
    "model = Classifier(784, 100, 10)\n",
    "\n",
    "# Let's check the architecture\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Criterion & Optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.optim import SGD\n",
    "from torch.nn import CrossEntropyLoss\n",
    "from nets.nn.utils import one_hot\n",
    "\n",
    "# How much do we update the parameters\n",
    "LEARNING_RATE = 0.01\n",
    "\n",
    "optimizer = SGD(model.parameters(), lr=LEARNING_RATE)\n",
    "criterion = CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 0/50 | loss: 2.950E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 1/50 | loss: 2.731E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 2/50 | loss: 2.539E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 3/50 | loss: 2.355E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 4/50 | loss: 2.186E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 5/50 | loss: 2.039E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 6/50 | loss: 1.906E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 7/50 | loss: 1.781E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 8/50 | loss: 1.671E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 9/50 | loss: 1.570E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 10/50 | loss: 1.478E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 11/50 | loss: 1.396E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 12/50 | loss: 1.321E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 13/50 | loss: 1.250E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 14/50 | loss: 1.192E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 15/50 | loss: 1.134E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 16/50 | loss: 1.080E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 17/50 | loss: 1.030E-01\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 18/50 | loss: 9.879E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 19/50 | loss: 9.442E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 20/50 | loss: 9.095E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 21/50 | loss: 8.704E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 22/50 | loss: 8.378E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 23/50 | loss: 8.084E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 24/50 | loss: 7.758E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 25/50 | loss: 7.489E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 26/50 | loss: 7.195E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 27/50 | loss: 6.931E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 28/50 | loss: 6.699E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 29/50 | loss: 6.439E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 30/50 | loss: 6.271E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 31/50 | loss: 6.081E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 32/50 | loss: 5.878E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 33/50 | loss: 5.635E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 34/50 | loss: 5.474E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 35/50 | loss: 5.296E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 36/50 | loss: 5.073E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 37/50 | loss: 4.975E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 38/50 | loss: 4.800E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 39/50 | loss: 4.658E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 40/50 | loss: 4.525E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 41/50 | loss: 4.391E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 42/50 | loss: 4.217E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 43/50 | loss: 4.089E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 44/50 | loss: 3.974E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 45/50 | loss: 3.851E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 46/50 | loss: 3.751E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 47/50 | loss: 3.594E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 48/50 | loss: 3.543E-02\n",
      "Training: 100% | [==================================================] | Done !\n",
      "epoch: 49/50 | loss: 3.388E-02\n",
      "\n",
      "time: 1165.0111677646637\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "from nets.utils import progress_bar\n",
    "\n",
    "# Training iterations\n",
    "EPOCHS = 50\n",
    "\n",
    "#Display the time during training\n",
    "start = time.time()\n",
    "# Record the loss\n",
    "history = []\n",
    "\n",
    "# Run the simulation EPOCHS times\n",
    "for epoch in range(EPOCHS):\n",
    "    \n",
    "    # Keep track of the loss at each epoch\n",
    "    epoch_loss = 0.0\n",
    "    epoch_start = time.time()\n",
    "    \n",
    "    # Number of batches\n",
    "    N  = len(train_loader)\n",
    "\n",
    "    # Predict for all batches\n",
    "    for idx, batch in enumerate(train_loader):\n",
    "        \n",
    "        # Gradients accumulates, therefore we need to set them to zero at each iteration\n",
    "        model.zero_grad()\n",
    "        \n",
    "        # Get the images and labels from one batch\n",
    "        image, label = batch\n",
    "        # Let's not forget to reshape the images from (BATCH_SIE, 1, 28, 28) to (BATCH_SIZE, 1 * 28 * 28)\n",
    "        image = image.reshape(-1, 784)\n",
    "        \n",
    "        # Run the forward pass by calling the model on our images\n",
    "        predictions = model(image.float())\n",
    "        # Get our loss ie how far our predictions are from reality\n",
    "        loss = criterion(predictions, label.long())\n",
    "        \n",
    "        # Compute the gradient\n",
    "        loss.backward()\n",
    "        # Update the parameters\n",
    "        optimizer.step()\n",
    "        \n",
    "        # Record the loss for plotting\n",
    "        epoch_loss += loss.item()\n",
    "        # Let's plot a progress bar in the console\n",
    "        progress_bar(idx, N-1, \"Training:\")\n",
    "    \n",
    "    # Update the history of all previous loss\n",
    "    history.append(epoch_loss / N)\n",
    "    \n",
    "    # Check the mean loss for this loop\n",
    "    print(f\"epoch: {epoch}/{EPOCHS} | loss: {epoch_loss / N:1.3E}\")\n",
    "\n",
    "# Duration of the total training loop\n",
    "print('\\ntime:', time.time() - start)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x21e7bd0b988>]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXyV5Z338c8v+75vkIWEVQKyBtwBrRVcKrZqRWtr1Rbb6sz05fTpY5enztixU3WmtQu12ta2jlNxV8aNoqKiAhL2JSxhzQIkJAQCCWS7nj9ycGIMciDLnZzzfb9e55Vzbye/S+I3V677uu/bnHOIiEjgCvG6ABER6V0KehGRAKegFxEJcAp6EZEAp6AXEQlwYV4X0FlaWprLz8/3ugwRkQFl5cqVB5xz6V1t63dBn5+fT3FxsddliIgMKGa2+2TbNHQjIhLg/Ap6M5tlZlvMrNTM7uli+7fMbL2ZrTGz982ssMO2H/iO22JmM3uyeBERObVTBr2ZhQLzgMuBQuDGjkHu8zfn3NnOuQnAg8AvfMcWAnOAMcAs4He+zxMRkT7iT49+KlDqnNvhnGsC5gOzO+7gnDvcYTEWOHFfhdnAfOfccefcTqDU93kiItJH/DkZmw2UdVguB87pvJOZ3QncDUQAl3Q4dlmnY7O7OHYuMBcgLy/Pn7pFRMRP/vTorYt1n7oTmnNunnNuGPB/gR+f5rGPOeeKnHNF6eldzg4SEZEz5E/QlwO5HZZzgMrP2H8+cM0ZHisiIj3Mn6BfAYwwswIzi6D95OqCjjuY2YgOi1cC23zvFwBzzCzSzAqAEcBH3S/70+oamnj4za1s2VffGx8vIjJgnXKM3jnXYmZ3AQuBUOBx59xGM7sPKHbOLQDuMrNLgWbgIHCL79iNZvYMsAloAe50zrX2Ulv43eLtHGps5t4vjOmtbyEiMuBYf3vwSFFRkTvTK2Pv/O9VLN1Rw7IffI6IMF0LJiLBw8xWOueKutoWUGl43eQcao82sXhLldeliIj0GwEV9BeNSCM9PpLnVpZ7XYqISL8RUEEfFhrClyZms3hzFQeOHPe6HBGRfiGggh7g2sk5tLQ5Xl6jWZwiIhCAQT8yM57xOYkavhER8Qm4oIf2k7Ilew+zsfKQ16WIiHguIIP+C+MHExEaol69iAgBGvRJMRF8vjCTl9dU0tTS5nU5IiKeCsigB82pFxE5IWCDXnPqRUTaBWzQa069iEi7gA160Jx6EREI8KDXnHoRkQAPetCcehGRgA/6E3Pqny1Wr15EglPAB31STASXn53F8yvLOXq8xetyRET6XMAHPcDXz8+n/ngLL6xSr15Egk9QBP3EvGTG5yTylw930d+eqCUi0tuCIugBbjk/n+3VR3m/9IDXpYiI9KmgCforxw0iLS6Cv364y+tSRET6VNAEfWRYKDdNzeOtzVXsqWnwuhwRkT4TNEEP8JVzhxBqxhNLd3ldiohInwmqoM9MiGLW2CyeLi7TVEsRCRpBFfQAt16QT/2xFl5cXeF1KSIifSLogn5SXjJjsxN4YqmmWopIcAi6oDczbjkvn637j7B0e43X5YiI9LqgC3pov/9NSmwEf9ZUSxEJAkEZ9FHhodw4NZe3SvZTVqupliIS2IIy6AFuPncIZsaTy3Z7XYqISK/yK+jNbJaZbTGzUjO7p4vtd5vZJjNbZ2ZvmdmQDttazWyN77WgJ4vvjkGJ0cwak8VTH+3RVEsRCWinDHozCwXmAZcDhcCNZlbYabfVQJFzbhzwHPBgh22NzrkJvtfVPVR3j7jtwgIOH2vhed3VUkQCmD89+qlAqXNuh3OuCZgPzO64g3NusXPuxGD3MiCnZ8vsHZOHJDMxL4nH399Ja5umWopIYPIn6LOBsg7L5b51J3M78HqH5SgzKzazZWZ2TVcHmNlc3z7F1dXVfpTUc75x4VB21TTwVsn+Pv2+IiJ9xZ+gty7Wddn9NbObgSLgoQ6r85xzRcBNwMNmNuxTH+bcY865IudcUXp6uh8l9ZyZYzLJTormj+/v7NPvKyLSV/wJ+nIgt8NyDlDZeSczuxT4EXC1c+74ifXOuUrf1x3AO8DEbtTb48JCQ7j1gnw+2lnLuvI6r8sREelx/gT9CmCEmRWYWQQwB/jE7Bkzmwg8SnvIV3VYn2xmkb73acAFwKaeKr6n3DAll7jIMP6kXr2IBKBTBr1zrgW4C1gIlADPOOc2mtl9ZnZiFs1DQBzwbKdplKOBYjNbCywGfu6c63dBHx8Vzpwpuby6bi+VdY1elyMi0qOsv93Yq6ioyBUXF/f59y0/2MC0BxfzzWlD+cHlo/v8+4uIdIeZrfSdD/2UoL0ytrOc5BguP3sQf1uuC6hEJLAo6Du4/cIC6o+18Gxx2al3FhEZIBT0HUzKS2ZSXhKPf7BLF1CJSMBQ0HfyjYuGsqe2gUWbdAGViAQGBX0nlxVmkpMczR+X7PC6FBGRHqGg7yQsNITbLiigePdBinfVel2OiEi3Kei7MGdqLimxEfz67VKvSxER6TYFfRdiIsL4xkUFvLe1mrVlui2CiAxsCvqT+Oq5Q0iMDuc36tWLyACnoD+J+Khwbr0gnzdL9rOp8rDX5YiInDEF/We49fwC4iLDmLdYvXoRGbgU9J8hMSacr503hNc27KW0qt7rckREzoiC/hRuv7CAqLBQ5i3e7nUpIiJnREF/CqlxkXzlnDxeXlPB7pqjXpcjInLaFPR+mDttKGGhIfxOvXoRGYAU9H7ISIhizpRcnl9VToUeTCIiA4yC3k93TB+GGfz+HfXqRWRgUdD7KTspmmsn5fD0ijL2HTrmdTkiIn5T0J+GOy8ejsPxy0VbvS5FRMRvCvrTkJsSw9fOy+eZlWWU7NXVsiIyMCjoT9M/XDKchKhwfvZaCf3tweoiIl1R0J+mpJgI/vFzI1iy7QDvbK32uhwRkVNS0J+Br547hPzUGH72agktrW1elyMi8pkU9GcgIiyEey4/i21VR3i6uMzrckREPpOC/gzNHJPF1PwUfrloK/XHmr0uR0TkpBT0Z8jM+NGVozlwpInfv6uLqESk/1LQd8P43CRmTxjMH5fspFK3RhCRfkpB303/Z+YoHPAfC7d4XYqISJcU9N2UkxzD7RcW8MLqCtaV60HiItL/+BX0ZjbLzLaYWamZ3dPF9rvNbJOZrTOzt8xsSIdtt5jZNt/rlp4svr/4zoxhpMVF8JOXN9LWpouoRKR/OWXQm1koMA+4HCgEbjSzwk67rQaKnHPjgOeAB33HpgD3AucAU4F7zSy558rvH+KjwvnxlYWsKavjqRV7vC5HROQT/OnRTwVKnXM7nHNNwHxgdscdnHOLnXMNvsVlQI7v/UxgkXOu1jl3EFgEzOqZ0vuX2RMGc/6wVB54fTPV9ce9LkdE5GP+BH020PGqoHLfupO5HXj9dI41s7lmVmxmxdXVA/O2AmbGT68Zy7HmNn72WonX5YiIfMyfoLcu1nU5EG1mNwNFwEOnc6xz7jHnXJFzrig9Pd2PkvqnYelx3DF9KC+uruDD7Qe8LkdEBPAv6MuB3A7LOUBl553M7FLgR8DVzrnjp3NsILnz4uHkpcTw45c2cLyl1etyRET8CvoVwAgzKzCzCGAOsKDjDmY2EXiU9pCv6rBpIXCZmSX7TsJe5lsXsKLCQ7lv9hh2VB/lD+/t8LocEZFTB71zrgW4i/aALgGecc5tNLP7zOxq324PAXHAs2a2xswW+I6tBX5K+y+LFcB9vnUBbcaoDK48exC/ebuU3TVHvS5HRIKc9beHZxQVFbni4mKvy+i2fYeO8bn/fIei/BT+cusUzLo6XSEi0jPMbKVzrqirbboytpdkJUbxz5eN4t2t1by2fp/X5YhIEFPQ96KvnTeEMYMTuHfBRuoamrwuR0SClIK+F4WFhvDgdeOoa2jivlc2eV2OiAQpBX0vGzM4ke/MGMYLqyp4e/N+r8sRkSCkoO8Dd14ynJGZcfzwhQ0c1tOoRKSPKej7QGRYKA9dN56q+mPc/4pujyAifUtB30fG5yYxd9owni4uY8m2gXk/HxEZmBT0fei7l45gaHos9zy/niPHW7wuR0SChIK+D0WFh/LQdeOoPNTIz1/XEI6I9A0FfR+bPCSF2y4o4Mlle1i6vcbrckQkCCjoPfC9y0YxJDWG7z+/VrNwRKTXKeg9EB0Ryi++PIHKumPc8/w6+tv9hkQksCjoPTJ5SDL/Z+YoXlu/jyeX7fa6HBEJYAp6D829aCgzRqXz01dK2FBxyOtyRCRAKeg9FBJi/OLLE0iJjeCuv63SlEsR6RUKeo+lxEbw6xsnUnawkR++sF7j9SLS4xT0/cDUghTu/vxIFqytZP6KMq/LEZEAo6DvJ749fRgXjUjjXxZspGTvYa/LEZEAoqDvJ0JCjF/eMIHE6HDu1Hi9iPQgBX0/khYXya/mTGR3TQN3P72GtjaN14tI9yno+5nzhqXyoytG8/dN+/n129u8LkdEAoCCvh+69YJ8rp2Uw8NvbmPhRj1YXES6R0HfD5kZ939xLONzErn76TVs3V/vdUkiMoAp6PupqPBQfv/VyURHhDH3iWIONejmZyJyZhT0/digxGge/eokKuoaueupVbTq5KyInAEFfT83eUgK980ey5JtB3jwjc1elyMiA1CY1wXIqd04NY+NlYd49L0dDM+I4/qiXK9LEpEBREE/QPzkqjHsOtDAPS+sJy0+kotHZXhdkogMEBq6GSAiwkJ45OZJnJUVz3eeXMWasjqvSxKRAcKvoDezWWa2xcxKzeyeLrZPM7NVZtZiZtd12tZqZmt8rwU9VXgwio8K58+3TiEtPoLb/rKCnQeOel2SiAwApwx6MwsF5gGXA4XAjWZW2Gm3PcDXgb918RGNzrkJvtfV3aw36GXER/HEbecA8LXHl1NVf8zjikSkv/OnRz8VKHXO7XDONQHzgdkdd3DO7XLOrQPaeqFG6aQgLZY/f30KB+qbuPXPK6jXA8ZF5DP4E/TZQMebpJf71vkrysyKzWyZmV3T1Q5mNte3T3F1dfVpfHTwGp+bxCM3T2LLvnq+/eQqmlr0O1ZEuuZP0FsX607nyp0851wRcBPwsJkN+9SHOfeYc67IOVeUnp5+Gh8d3GaMyuCBa8fxfukB/mn+appbFfYi8mn+BH050HHidg5Q6e83cM5V+r7uAN4BJp5GfXIK107O4SdXFfL6hn18d/4aWhT2ItKJP/PoVwAjzKwAqADm0N47PyUzSwYanHPHzSwNuAB48EyLla7ddmEBbc7xb6+WYAYP3zCBsFDNnBWRdqcMeudci5ndBSwEQoHHnXMbzew+oNg5t8DMpgAvAsnAF8zsX51zY4DRwKNm1kb7Xw8/d85t6rXWBLFvXDQU5+D+10owM3755fEKexEB/Lwy1jn3GvBap3U/6fB+Be1DOp2P+xA4u5s1ip++OW0obc7x769vxoBfKOxFBN0CIeDcMX0YbQ4eeGMzIQb/+eUJhIZ0dT5dRIKFgj4AfXvGMNqc46GFW3DAf1w/nnD17EWCloI+QN158XBCzHjgjc3UNTTzyM2TiInQP7dIMFI3L4B9e8Ywfv6ls1myrZqb/rCcg0ebvC5JRDygoA9wc6bm8cjNk9m09zDXP7qUyrpGr0sSkT6moA8CM8dk8cRtU9l/6BjXPvIhpVV62LhIMFHQB4lzh6Yy/45zaW51XPf7pazec9DrkkSkjyjog8iYwYm88O3zSYwO58Y/LOPvG/d5XZKI9AEFfZDJS43huW+dz6isBO54ciV/en8nzp3OPepEZKBR0Aeh9PhI5n/zXGYWZvHTVzZx74KNuhmaSABT0Aep6IhQfveVSdwxbShPLN3NN58o5sjxFq/LEpFeoKAPYiEhxg+uGM39XxzLe9sOcP3vl7L3kKZfigQaBb3wlXOG8PjXp1BW28A18z5g2Y4ar0sSkR6koBcApo9M57lvn0d0eCg3/mEZD7yxWY8nFAkQCnr52FlZCbz6jxfx5cm5PPLOdr70yAeUVh3xuiwR6SYFvXxCbGQYD1w3jt/fPJmKg41c9ZslPLlst6ZgigxgCnrp0qyxWbzx3WlMyU/hxy9t4Bt/LebAkeNelyUiZ0BBLyeVmRDFX2+dyr1fKGRJ6QFmPfweizdXeV2WiJwmBb18ppAQ49YLCvifuy4kLS6SW/+ygp+8vIFjza1elyYiflLQi19GZcXz0p0XcPuFBTyxdDdX/eZ9NlYe8rosEfGDgl78FhUeyv+7qpD/un0qhxubuWbeBzz23nba2nSiVqQ/U9DLabtoRDoLvzuNz52Vyc9e28ycPyxj14GjXpclIiehoJczkhwbwSM3T+Kh68ZRsvcws371Hn9csoNW9e5F+h0FvZwxM+P6olzevHs6Fw5P499eLeHaRz5k2349wUqkP1HQS7dlJkTxh68V8as5E9hdc5Qrf/0+8xaX0qxbH4v0Cwp66RFmxuwJ2Sy6ezqXjcnkoYVbuPq3H7Bytx5ZKOI1Bb30qLS4SH570yQe/epk6hqauPaRD/nBC+uoa2jyujSRoKWgl14xc0wWi+6ezjcvKuCZ4nIu+c93eaa4TFMxRTygoJdeExcZxo+uLOSVf7iQgrRYvv/cOm54bCmb9x32ujSRoOJX0JvZLDPbYmalZnZPF9unmdkqM2sxs+s6bbvFzLb5Xrf0VOEycIwelMCzd5zHg9eOo7TqCFf8agn//Mxa9tQ0eF2aSFCwU91+1sxCga3A54FyYAVwo3NuU4d98oEE4HvAAufcc771KUAxUAQ4YCUw2Tl30jN0RUVFrri4+MxbJP1a7dEmHnmnlCeW7qa1zXF9US7/cMlwBidFe12ayIBmZiudc0VdbfOnRz8VKHXO7XDONQHzgdkdd3DO7XLOrQM6z6ebCSxyztX6wn0RMOu0WyABIyU2gh9dWch737+Ym87J47mVZcx46B3+ZcFGquqPeV2eSEDyJ+izgbIOy+W+df7w61gzm2tmxWZWXF1d7edHy0CWmRDFfbPHsvh7M/jSpGz+a9lupj24mAff2MyhxmavyxMJKP4EvXWxzt+pE34d65x7zDlX5JwrSk9P9/OjJRDkJMfw82vH8dbd05k5JovfvbOdaQ8u5tF3t+tWyCI9xJ+gLwdyOyznAJV+fn53jpUgkp8Wy6/mTOTVf7yQiXlJ/Pvrm5nx0DvM/2gPLbrCVqRb/An6FcAIMyswswhgDrDAz89fCFxmZslmlgxc5lsn0qUxgxP5y61TmT/3XAYlRXHPC+u57OH3eKa4TD18kTN0ylk3AGZ2BfAwEAo87py738zuA4qdcwvMbArwIpAMHAP2OefG+I69Dfih76Pud879+bO+l2bdyAnOORZt2s8vFm1l8756kmLCuWFKLjefM4TclBivyxPpVz5r1o1fQd+XFPTSmXOO5Ttr+euHu/j7pv20OcfnzsrklvOHcOHwNMy6OhUkElw+K+jD+roYkdNlZpw7NJVzh6ZSWdfI35bv4amP9vBmyX5GZcbznYuHceXZgwgL1YXeIl1Rj14GpOMtrbyydi+PvredrfuPkJcSw7emD+PaydlEhoV6XZ5In9PQjQSstjbHmyX7mbe4lLXlh8hMiOSbFw3lxql5xEbqD1YJHgp6CXjOOT4orWHe4lKW7qghPiqML07MZs6UPAoHJ3hdnkivU9BLUFm15yBPfLiL1zbso6mljfG5Sdw0NZerxg1WL18CloJegtLBo028sLqC+R/tYVvVEWIjQrl6QjbXTspm8pBkzdaRgKKgl6DmnGPl7oM89VEZr66v5FhzGznJ0VwzIZtrJg5meEa81yWKdJuCXsTnyPEW/r5xHy+uruCD0gO0ORibncA1E7L54sRsUuMivS5R5Iwo6EW6UFV/jP9Zu5eXVlewvuIQ4aHGzDFZ3Dg1j/OGphISoqEdGTgU9CKnsG1/PU99VMbzq8o51NjMkNQY5kzJ47rJOaTHq5cv/Z+CXsRPx5pbeWPDPp76aA/Ld9YSFmJcNCKNy88exGWFmSTFRHhdokiXFPQiZ6C06gjPFpfx6vq9lB9sJCzEOG9YKlf4Ql/j+dKfKOhFusE5x4aKw7y2YS+vrd/L7poGQgzOH5bGNROzmTU2izjNzxePKehFeohzjpK99by2fi8L1layp7aBqPAQLh2dyRcnZjNtZDrhurmaeEBBL9ILnHOs2lPHS6sreGVdJQcbmkmJjeCKs7O4fOwgzilI0R01pc8o6EV6WVNLG+9trebFNRW8VbKfY81tJMWE8/nRmcwam8UFw9OICtddNaX3KOhF+lBjUyvvbq3mjQ17eaukivrjLcRFhjFjVDoXj8pg2sh0TdmUHqcHj4j0oeiIUGaNzWLW2CyaWtr4cPsBFm7cx6JN+3ll3V4AxgxOYMaodKaPzGBiXpLG9aVXqUcv0kfa2hyb9h7m3a3VvLulmpV7DtLa5oiPCuPiURnMGpvF9JHpusOmnBEN3Yj0Q4ePNfNh6QHe3lzFmyVV1B5tIiIshGkj0pg5JotLR2eSHKsLtMQ/GroR6YcSosKZNXYQs8YOoqW1jeLdB1m4cR8LN+zjzZIqQkOMwkEJnJ2TyLjsRM7OSWRkZryGeeS0qUcv0s+cuEDr75v2sWrPQdaVH6L+WAsAEWEhFA5KYEJuEkX5yRQNSSErMcrjiqU/0NCNyADmnGN3TQPrKg6xvryOteWHWF9+iMbmVgBykqMpGpJMUX4KU/JTGJERpztvBiEN3YgMYGZGflos+WmxXD1+MADNrW1sqjxM8e6DrNxdywfba3hpTSUAyTHhTC1I4ZyCVM4ZmsLorAQFf5BTj14kADjnKKttZPnOGpbvrGX5zhrKahsBSIxuD/4Zo9K55KwMBiVGe1yt9Ab16EUCnJmRlxpDXmoM1xflAlBR18jyHTUs31HL+6UHWLRpPwCFgxL43OgMLjkrg/E5SertBwH16EWCgHOObVVHeHtzFW+XVFG8u5Y2B6mxEUzMS2ZsdgJnZycyNjuRzASd3B2I1KMXCXJmxsjMeEZmxvOt6cOoa2ji3a3VvLOlmrXldby1eT8n+nzp8ZGMHZzA2TlJTMxLYkJOkubzD3AKepEglBQTwewJ2cyekA20PzS9ZO9hNlQcYkNF+9d3t26jzRf++akxTMhNYmJeMpOHJFM4SCd4BxK/gt7MZgG/AkKBPzrnft5peyTwBDAZqAFucM7tMrN8oATY4tt1mXPuWz1Tuoj0lLjIMKb4pmeecPR4C+vKD7G67CBr9tR9YmZPamwEFwxP46IRaVw0Il1z+fu5Uwa9mYUC84DPA+XACjNb4Jzb1GG324GDzrnhZjYHeAC4wbdtu3NuQg/XLSK9LDYyjPOGpXLesFSgfZy//QRvLUu2VfN+6QEWrG0P/pGZcZxTkEpeSgzZydFkJ0UzOCmatLgIzNTz95o/PfqpQKlzbgeAmc0HZgMdg3428C++988BvzX964oEFDMjJzmGnMkxXDs5h7Y2x+Z99SzZVs2SbQd4YVU5R5taP3FMZFgIOcnRjM1OZHxOEuNzkxgzOEH35u9j/gR9NlDWYbkcOOdk+zjnWszsEJDq21ZgZquBw8CPnXNLOn8DM5sLzAXIy8s7rQaIiDdCQozCwQkUDk7gjunDcM5xuLGFirrG9tfBBioPHWPXgaN8tLOWl33DPmEhxuhBCYzPTaRwUCKjsuIZmRlHfFS4xy0KXP4EfVc9885zMk+2z14gzzlXY2aTgZfMbIxz7vAndnTuMeAxaJ9e6UdNItLPmBmJMeEkxoRTODjhU9v3Hz7GmrI61pbVsaasjpdWV/Lksj0fb89OivaFfnvwj8iIZ1hGLDERmjPSXf78FywHcjss5wCVJ9mn3MzCgESg1rVP0j8O4JxbaWbbgZGAJsqLBJnMhChmjsli5pgsoP3+/BV1jWzZV8+W/fVs2VfP1v3tQ0HNrf/b38tOimZ4RhzDM+IYkRHHiMw4hmfEkxitvwD85U/QrwBGmFkBUAHMAW7qtM8C4BZgKXAd8LZzzplZOu2B32pmQ4ERwI4eq15EBqyQECM3JYbclBguLcz8eH1zaxu7a45SWnWEbfuPUFrd/nX5zhqONbd9vF9GfCQjfD3/kZnxjMtpHwbSbZw/7ZRB7xtzvwtYSPv0ysedcxvN7D6g2Dm3APgT8F9mVgrU0v7LAGAacJ+ZtQCtwLecc7W90RARCQzhoSEMz4hneEY8s8b+7/q2Nkf5wUZKq+vZtv8I26raX88Wl318EjgqPISxgxMZn9t+4nd8TiK5yTFBP+dft0AQkQHtxA3d1pS3j/+vLatjfcUhjre09/6jwkMoSItjWHosw9LjGJbR/j4/NTagHtuoWyCISMDqeEO3jrdx3rKvnvUVhyitOsL26iOsKz/Eq+v30rFvmx4fSX5qDENSYzt8jaUgPZa4APolEDgtERHxCQ8NYazvJm0dHWtuZVfNUbZXHWVXzVF21xxlV00DS7ZV89zK45/YNyM+koK0WIamxzHU9zyA7KRospOjB9yJYAW9iASNqPBQzspK4KysT0//bGxqZXftUXYdOMr26qPsPND+emPDXg42NH9i3/jIMLKTo8lJjiYnOYbCQe3XE4zMjCcirP+dDFbQi4gA0REn/yVw8GgTu2sbqDjYSEXdia+NlB9s5MPtNTT4TgaHhxojMuIZ47uQLD0+koSocOKjwkiIDichKpyE6DAiw/r2ymAFvYjIKSTHRpAcG8GE3KRPbWtrc+yqOcrGysO+1yHe3lzFsyvLT/p5mQmRjBmc+PFfAoWDEshL6b3ZQQp6EZFuCAmx9nH89Di+4DsZ7JzjwJEmDjY0cbixmcPHmjnc2EL9sWbqGprZceAomyoP8+7Walp994KOiwxjxqh0fnvTpB6vUUEvItLDzIz0+EjS4yM/c79jza1s3V/PpsrDbNp7mPio3olkBb2IiEeiwkMZl5PEuJxPDwn1pP53elhERHqUgl5EJMAp6EVEApyCXkQkwCnoRUQCnIJeRCTAKehFRAKcgl5EJMD1uwePmFk1sLsbH5EGHOihcgYStTu4qN3BxZ92D3HOpXe1od8FfXeZWfHJnrISyNTu4KJ2B5futltDNyIiAU5BLyIS4AK14hYAAANmSURBVAIx6B/zugCPqN3BRe0OLt1qd8CN0YuIyCcFYo9eREQ6UNCLiAS4gAl6M5tlZlvMrNTM7vG6nt5kZo+bWZWZbeiwLsXMFpnZNt/XZC9r7Glmlmtmi82sxMw2mtk/+dYHerujzOwjM1vra/e/+tYXmNlyX7ufNrMIr2vtDWYWamarzewV33KwtHuXma03szVmVuxbd8Y/6wER9GYWCswDLgcKgRvNrNDbqnrVX4BZndbdA7zlnBsBvOVbDiQtwD8750YD5wJ3+v6NA73dx4FLnHPjgQnALDM7F3gA+KWv3QeB2z2ssTf9E1DSYTlY2g1wsXNuQof582f8sx4QQQ9MBUqdczucc03AfGC2xzX1Gufce0Btp9Wzgb/63v8VuKZPi+plzrm9zrlVvvf1tP/Pn03gt9s55474FsN9LwdcAjznWx9w7QYwsxzgSuCPvmUjCNr9Gc74Zz1Qgj4bKOuwXO5bF0wynXN7oT0UgQyP6+k1ZpYPTASWEwTt9g1frAGqgEXAdqDOOdfi2yVQf94fBr4PtPmWUwmOdkP7L/O/m9lKM5vrW3fGP+uB8nBw62Kd5o0GIDOLA54HvuucO9zeyQtszrlWYIKZJQEvAqO72q1vq+pdZnYVUOWcW2lmM06s7mLXgGp3Bxc45yrNLANYZGabu/NhgdKjLwdyOyznAJUe1eKV/WY2CMD3tcrjenqcmYXTHvL/7Zx7wbc64Nt9gnOuDniH9nMUSWZ2oqMWiD/vFwBXm9ku2odiL6G9hx/o7QbAOVfp+1pF+y/3qXTjZz1Qgn4FMMJ3Rj4CmAMs8LimvrYAuMX3/hbgZQ9r6XG+8dk/ASXOuV902BTo7U739eQxs2jgUtrPTywGrvPtFnDtds79wDmX45zLp/3/57edc18hwNsNYGaxZhZ/4j1wGbCBbvysB8yVsWZ2Be2/8UOBx51z93tcUq8xs6eAGbTfunQ/cC/wEvAMkAfsAa53znU+YTtgmdmFwBJgPf87ZvtD2sfpA7nd42g/8RZKe8fsGefcfWY2lPaebgqwGrjZOXfcu0p7j2/o5nvOuauCod2+Nr7oWwwD/uacu9/MUjnDn/WACXoREelaoAzdiIjISSjoRUQCnIJeRCTAKehFRAKcgl5EJMAp6EVEApyCXkQkwP1/f++7AqH4cwAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}