pytorch-lightning_ipynb/ordinal/CORN-light_cement.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a5f4cdf9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch            : 1.10.1\n",
      "pytorch_lightning: 1.6.0.dev0\n",
      "torchmetrics     : 0.6.2\n",
      "matplotlib       : 3.3.4\n",
      "coral_pytorch    : 1.2.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -p torch,pytorch_lightning,torchmetrics,matplotlib,coral_pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e83c61eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext pycodestyle_magic\n",
    "%flake8_on --ignore W291,W293,E703"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "705ad02a",
   "metadata": {},
   "source": [
    "<a href=\"https://pytorch.org\"><img src=\"https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/pytorch-logo-dark.svg\" width=\"90\"/></a> &nbsp; &nbsp;&nbsp;&nbsp;<a href=\"https://www.pytorchlightning.ai\"><img src=\"https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/docs/source/_static/images/logo.svg\" width=\"150\"/></a>\n",
    "\n",
    "# CORN MLP for ordinal regression and deep learning -- cement strength dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c030661",
   "metadata": {},
   "source": [
    "This tutorial explains how to train a deep neural network (here: multilayer perceptron) with the CORN loss function for ordinal regression. \n",
    "\n",
    "**CORN reference:**\n",
    "\n",
    "- Xintong Shi, Wenzhi Cao, and Sebastian Raschka \n",
    "[Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities](https://arxiv.org/abs/2111.08851)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3dc673f",
   "metadata": {},
   "source": [
    "**Note:**\n",
    "    \n",
    "To keep the notation lean and minimal, this notebook only contains \"Squared-error reformulation\"-specific comments. For more comments on the PyTorch Lightning use, please see the cross-entropy baseline notebook [baseline-light_cement.ipynb](./baseline-light_cement.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1c21dd6",
   "metadata": {},
   "source": [
    "## General settings and hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "853b3b04",
   "metadata": {},
   "outputs": [],
   "source": [
    "BATCH_SIZE = 128\n",
    "NUM_EPOCHS = 200\n",
    "LEARNING_RATE = 0.005\n",
    "NUM_WORKERS = 0\n",
    "\n",
    "DATA_BASEPATH = \".\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f24df342",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Converting a regular classifier into a CORN ordinal regression model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "302d4c8e",
   "metadata": {},
   "source": [
    "Changing a classifier to a CORN model for ordinal regression is actually really simple and only requires a few changes:\n",
    "\n",
    "**1)**\n",
    "\n",
    "Consider the following output layer used by a neural network classifier:\n",
    "\n",
    "```python\n",
    "output_layer = torch.nn.Linear(hidden_units[-1], num_classes)\n",
    "```\n",
    "\n",
    "In CORN we reduce the number of classes by 1:\n",
    "\n",
    "```python\n",
    "output_layer = torch.nn.Linear(hidden_units[-1], num_classes-1)\n",
    "```\n",
    "\n",
    "**2)** \n",
    "\n",
    "We swap the cross entropy loss from PyTorch,\n",
    "\n",
    "```python\n",
    "torch.nn.functional.cross_entropy(logits, true_labels)\n",
    "```\n",
    "\n",
    "with the CORN loss (also provided via `coral_pytorch`):\n",
    "\n",
    "```python\n",
    "loss = corn_loss(logits, true_labels,\n",
    "                 num_classes=num_classes)\n",
    "```\n",
    "\n",
    "Note that we pass `num_classes` instead of `num_classes-1` \n",
    "to the `corn_loss` as it takes care of the rest internally.\n",
    "\n",
    "\n",
    "**3)**\n",
    "\n",
    "In a regular classifier, we usually obtain the predicted class labels as follows:\n",
    "\n",
    "```python\n",
    "predicted_labels = torch.argmax(logits, dim=1)\n",
    "```\n",
    "\n",
    "In CORN, w replace this with the following code to convert the predicted probabilities into the predicted labels:\n",
    "\n",
    "```python\n",
    "predicted_labels = corn_label_from_logits(logits)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e062c5b6",
   "metadata": {},
   "source": [
    "## Implementing a `MultiLayerPerceptron` using PyTorch Lightning's `LightningModule`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3dc5a901",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "\n",
    "# Regular PyTorch Module\n",
    "class MultiLayerPerceptron(torch.nn.Module):\n",
    "    def __init__(self, input_size, hidden_units, num_classes):\n",
    "        super().__init__()\n",
    "\n",
    "        # num_classes is used by the corn loss function\n",
    "        self.num_classes = num_classes\n",
    "        \n",
    "        # Initialize MLP layers\n",
    "        all_layers = []\n",
    "        for hidden_unit in hidden_units:\n",
    "            layer = torch.nn.Linear(input_size, hidden_unit)\n",
    "            all_layers.append(layer)\n",
    "            all_layers.append(torch.nn.ReLU())\n",
    "            input_size = hidden_unit\n",
    "\n",
    "        # CORN output layer -------------------------------------------\n",
    "        # Regular classifier would use num_classes instead of \n",
    "        # num_classes-1 below\n",
    "        output_layer = torch.nn.Linear(hidden_units[-1], num_classes-1)\n",
    "        # -------------------------------------------------------------\n",
    "        \n",
    "        all_layers.append(output_layer)\n",
    "        self.model = torch.nn.Sequential(*all_layers)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.model(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0af9e429",
   "metadata": {},
   "source": [
    "- In our `LightningModule` we use loggers to track mean absolute errors for both the training and validation set during training; this allows us to select the best model based on validation set performance later.\n",
    "- Given a multilayer perceptron classifier with cross-entropy loss, it is very easy to change this classifier into a ordinal regression model using CORN. In essence, it only requires three changes:\n",
    "    1. Instead of using `num_classes` in the output layer, use `num_classes-1` as shown above\n",
    "    2. Change the loss from   \n",
    "    `loss = torch.nn.functional.cross_entropy(logits, y)` to  \n",
    "    `loss = corn_loss(logits, y, num_classes=self.num_classes)`\n",
    "    3. To obtain the class/rank labels from the logits, change  \n",
    "    `predicted_labels = torch.argmax(logits, dim=1)` to  \n",
    "    `predicted_labels = corn_label_from_logits(logits)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "734379e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install coral-pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "70caed83",
   "metadata": {},
   "outputs": [],
   "source": [
    "from coral_pytorch.losses import corn_loss\n",
    "from coral_pytorch.dataset import corn_label_from_logits\n",
    "\n",
    "import pytorch_lightning as pl\n",
    "import torchmetrics\n",
    "\n",
    "\n",
    "# LightningModule that receives a PyTorch model as input\n",
    "class LightningMLP(pl.LightningModule):\n",
    "    def __init__(self, model, learning_rate):\n",
    "        super().__init__()\n",
    "\n",
    "        self.learning_rate = learning_rate\n",
    "        self.model = model\n",
    "\n",
    "        self.save_hyperparameters(ignore=['model'])\n",
    "\n",
    "        self.train_mae = torchmetrics.MeanAbsoluteError()\n",
    "        self.valid_mae = torchmetrics.MeanAbsoluteError()\n",
    "        self.test_mae = torchmetrics.MeanAbsoluteError()\n",
    "        \n",
    "    def forward(self, x):\n",
    "        return self.model(x)\n",
    "        \n",
    "    # A common forward step to compute the loss and labels\n",
    "    # this is used for training, validation, and testing below\n",
    "    def _shared_step(self, batch):\n",
    "        features, true_labels = batch\n",
    "        logits = self(features)\n",
    "\n",
    "        # Use CORN loss --------------------------------------\n",
    "        # A regular classifier uses:\n",
    "        # loss = torch.nn.functional.cross_entropy(logits, y)\n",
    "        loss = corn_loss(logits, true_labels,\n",
    "                         num_classes=self.model.num_classes)\n",
    "        # ----------------------------------------------------\n",
    "        \n",
    "        # CORN logits to labels ------------------------------\n",
    "        # A regular classifier uses:\n",
    "        # predicted_labels = torch.argmax(logits, dim=1)\n",
    "        predicted_labels = corn_label_from_logits(logits)\n",
    "        # ----------------------------------------------------\n",
    "        \n",
    "        return loss, true_labels, predicted_labels\n",
    "\n",
    "    def training_step(self, batch, batch_idx):\n",
    "        loss, true_labels, predicted_labels = self._shared_step(batch)\n",
    "        self.log(\"train_loss\", loss)\n",
    "        self.train_mae(predicted_labels, true_labels)\n",
    "        self.log(\"train_mae\", self.train_mae, on_epoch=True, on_step=False)\n",
    "        return loss\n",
    "\n",
    "    def validation_step(self, batch, batch_idx):\n",
    "        loss, true_labels, predicted_labels = self._shared_step(batch)\n",
    "        self.log(\"valid_loss\", loss)\n",
    "        self.valid_mae(predicted_labels, true_labels)\n",
    "        self.log(\"valid_mae\", self.valid_mae,\n",
    "                 on_epoch=True, on_step=False, prog_bar=True)\n",
    "\n",
    "    def test_step(self, batch, batch_idx):\n",
    "        loss, true_labels, predicted_labels = self._shared_step(batch)\n",
    "        self.test_mae(predicted_labels, true_labels)\n",
    "        self.log(\"test_mae\", self.test_mae, on_epoch=True, on_step=False)\n",
    "\n",
    "    def configure_optimizers(self):\n",
    "        optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n",
    "        return optimizer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84ed8c2c",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# Note: There Are No Changes Compared To The Baseline Below\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23803478",
   "metadata": {},
   "source": [
    "## Setting up the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94b55c58",
   "metadata": {},
   "source": [
    "### Inspecting the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "74df9ef0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>response</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>540.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1040.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>540.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1055.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>332.5</td>\n",
       "      <td>142.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>228.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>932.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>270</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>332.5</td>\n",
       "      <td>142.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>228.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>932.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>198.6</td>\n",
       "      <td>132.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>192.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>978.4</td>\n",
       "      <td>825.5</td>\n",
       "      <td>360</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   response     V1     V2   V3     V4   V5      V6     V7   V8\n",
       "0         4  540.0    0.0  0.0  162.0  2.5  1040.0  676.0   28\n",
       "1         4  540.0    0.0  0.0  162.0  2.5  1055.0  676.0   28\n",
       "2         2  332.5  142.5  0.0  228.0  0.0   932.0  594.0  270\n",
       "3         2  332.5  142.5  0.0  228.0  0.0   932.0  594.0  365\n",
       "4         2  198.6  132.4  0.0  192.0  0.0   978.4  825.5  360"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "data_df = pd.read_csv(\"https://raw.githubusercontent.com/gagolews/\"\n",
    "                      \"ordinal_regression_data/master/cement_strength.csv\")\n",
    "data_df[\"response\"] = data_df[\"response\"]-1\n",
    "\n",
    "data_labels = data_df[\"response\"]\n",
    "data_features = data_df.loc[:, [\n",
    "    \"V1\", \"V2\", \"V3\", \"V4\", \"V5\", \"V6\", \"V7\", \"V8\"]]\n",
    "\n",
    "data_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "83b32cbb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of features: 8\n",
      "Number of examples: 998\n",
      "Labels: [0 1 2 3 4]\n",
      "Label distribution: [196 310 244 152  96]\n"
     ]
    }
   ],
   "source": [
    "print('Number of features:', data_features.shape[1])\n",
    "print('Number of examples:', data_features.shape[0])\n",
    "print('Labels:', np.unique(data_labels.values))\n",
    "print('Label distribution:', np.bincount(data_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e928da3",
   "metadata": {},
   "source": [
    "### Performance baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5262fcd7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline MAE: 1.03\n"
     ]
    }
   ],
   "source": [
    "avg_prediction = np.median(data_labels.values)  # median minimizes MAE\n",
    "baseline_mae = np.mean(np.abs(data_labels.values - avg_prediction))\n",
    "print(f'Baseline MAE: {baseline_mae:.2f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1cb0a07",
   "metadata": {},
   "source": [
    "### Creating a `Dataset` class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b36c978c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset\n",
    "\n",
    "\n",
    "class MyDataset(Dataset):\n",
    "\n",
    "    def __init__(self, feature_array, label_array, dtype=np.float32):\n",
    "        self.features = feature_array.astype(dtype)\n",
    "        self.labels = label_array\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        inputs = self.features[index]\n",
    "        label = self.labels[index]\n",
    "        return inputs, label\n",
    "\n",
    "    def __len__(self):\n",
    "        return self.features.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "719a608c",
   "metadata": {},
   "source": [
    "### Setting up a `DataModule`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e6730811",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "\n",
    "class DataModule(pl.LightningDataModule):\n",
    "    def __init__(self, data_path='./'):\n",
    "        super().__init__()\n",
    "        self.data_path = data_path\n",
    "        \n",
    "    def prepare_data(self):\n",
    "        data_df = pd.read_csv(\n",
    "            'https://raw.githubusercontent.com/gagolews/'\n",
    "            'ordinal_regression_data/master/cement_strength.csv')\n",
    "        data_df.to_csv(\n",
    "            os.path.join(self.data_path, 'cement_strength.csv'), index=None)\n",
    "        return\n",
    "\n",
    "    def setup(self, stage=None):\n",
    "        data_df = pd.read_csv(\n",
    "            os.path.join(self.data_path, 'cement_strength.csv'))\n",
    "        data_df[\"response\"] = data_df[\"response\"]-1  # labels should start at 0\n",
    "        self.data_labels = data_df[\"response\"]\n",
    "        self.data_features = data_df.loc[:, [\n",
    "            \"V1\", \"V2\", \"V3\", \"V4\", \"V5\", \"V6\", \"V7\", \"V8\"]]\n",
    "        \n",
    "        # Split into\n",
    "        # 70% train, 10% validation, 20% testing\n",
    "        \n",
    "        X_temp, X_test, y_temp, y_test = train_test_split(\n",
    "            self.data_features.values,\n",
    "            self.data_labels.values,\n",
    "            test_size=0.2,\n",
    "            random_state=1,\n",
    "            stratify=self.data_labels.values)\n",
    "\n",
    "        X_train, X_valid, y_train, y_valid = train_test_split(\n",
    "            X_temp,\n",
    "            y_temp,\n",
    "            test_size=0.1,\n",
    "            random_state=1,\n",
    "            stratify=y_temp)\n",
    "        \n",
    "        # Standardize features\n",
    "        sc = StandardScaler()\n",
    "        X_train_std = sc.fit_transform(X_train)\n",
    "        X_valid_std = sc.transform(X_valid)\n",
    "        X_test_std = sc.transform(X_test)\n",
    "\n",
    "        self.train = MyDataset(X_train_std, y_train)\n",
    "        self.valid = MyDataset(X_valid_std, y_valid)\n",
    "        self.test = MyDataset(X_test_std, y_test)\n",
    "\n",
    "    def train_dataloader(self):\n",
    "        return DataLoader(self.train, batch_size=BATCH_SIZE,\n",
    "                          num_workers=NUM_WORKERS,\n",
    "                          drop_last=True)\n",
    "\n",
    "    def val_dataloader(self):\n",
    "        return DataLoader(self.valid, batch_size=BATCH_SIZE,\n",
    "                          num_workers=NUM_WORKERS)\n",
    "\n",
    "    def test_dataloader(self):\n",
    "        return DataLoader(self.test, batch_size=BATCH_SIZE,\n",
    "                          num_workers=NUM_WORKERS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "353aea97",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(1) \n",
    "data_module = DataModule(data_path=DATA_BASEPATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b61d3c",
   "metadata": {},
   "source": [
    "## Training the model using the PyTorch Lightning Trainer class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "dfd42d86",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pytorch_lightning.callbacks import ModelCheckpoint\n",
    "from pytorch_lightning.loggers import CSVLogger\n",
    "\n",
    "\n",
    "pytorch_model = MultiLayerPerceptron(\n",
    "    input_size=data_features.shape[1],\n",
    "    hidden_units=(40, 20),\n",
    "    num_classes=np.bincount(data_labels).shape[0])\n",
    "\n",
    "lightning_model = LightningMLP(\n",
    "    model=pytorch_model,\n",
    "    learning_rate=LEARNING_RATE)\n",
    "\n",
    "\n",
    "callbacks = [ModelCheckpoint(\n",
    "    save_top_k=1, mode=\"min\", monitor=\"valid_mae\")]  # save top 1 model \n",
    "logger = CSVLogger(save_dir=\"logs/\", name=\"mlp-corn-cement\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "35ddeed1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:90: LightningDeprecationWarning: Setting `Trainer(progress_bar_refresh_rate=50)` is deprecated in v1.5 and will be removed in v1.7. Please pass `pytorch_lightning.callbacks.progress.TQDMProgressBar` with `refresh_rate` directly to the Trainer's `callbacks` argument instead. Or, to disable the progress bar pass `enable_progress_bar = False` to the Trainer.\n",
      "  rank_zero_deprecation(\n",
      "GPU available: True, used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name      | Type                 | Params\n",
      "---------------------------------------------------\n",
      "0 | model     | MultiLayerPerceptron | 1.3 K \n",
      "1 | train_mae | MeanAbsoluteError    | 0     \n",
      "2 | valid_mae | MeanAbsoluteError    | 0     \n",
      "3 | test_mae  | MeanAbsoluteError    | 0     \n",
      "---------------------------------------------------\n",
      "1.3 K     Trainable params\n",
      "0         Non-trainable params\n",
      "1.3 K     Total params\n",
      "0.005     Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation sanity check: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:258: PossibleUserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n",
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:258: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n",
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1850: PossibleUserWarning: The number of training samples (5) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.\n",
      "  rank_zero_warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "13306318e6de400fa02aa1687d6bc6a0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training took 0.32 min in total.\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "\n",
    "trainer = pl.Trainer(\n",
    "    max_epochs=NUM_EPOCHS,\n",
    "    callbacks=callbacks,\n",
    "    progress_bar_refresh_rate=50,  # recommended for notebooks\n",
    "    accelerator=\"auto\",  # Uses GPUs or TPUs if available\n",
    "    devices=\"auto\",  # Uses all available GPUs/TPUs if applicable\n",
    "    logger=logger,\n",
    "    deterministic=True,\n",
    "    log_every_n_steps=10)\n",
    "\n",
    "start_time = time.time()\n",
    "trainer.fit(model=lightning_model, datamodule=data_module)\n",
    "\n",
    "runtime = (time.time() - start_time)/60\n",
    "print(f\"Training took {runtime:.2f} min in total.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aabf616",
   "metadata": {},
   "source": [
    "## Evaluating the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cf02efe3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='Epoch', ylabel='MAE'>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAnFElEQVR4nO3deXhV5bn38e9NEhIgCaOEMAhYREVEMAi0KoojonUeT1uF1lI7nOpr9ZVOp9rhuuw5rfb01GrrqcPxaFGxqB2sWt5QtCpKKGCYxAFkknlImBPu949nBQNJIAnZe22yfp/r2lf2fvZae/1Y2dxZ+1nPfpa5OyIikhxt4g4gIiLppcIvIpIwKvwiIgmjwi8ikjAq/CIiCZMdd4DG6Natm/fr169Z627bto0OHTq0bKAWkKm5IHOzKVfTZGouyNxsrS1XWVnZenc/qs4T7p7xt5KSEm+u0tLSZq+bSpmayz1zsylX02RqLvfMzdbacgGzvJ6aqq4eEZGEUeEXEUkYFX4RkYQ5Ik7uikjrYmZ8+OGH7Ny5M+4o++nYsSMLFy6MO0Ydh8qVl5dH7969ycnJadTrqfCLSNp16NCBgoIC+vXrh5nFHWefiooKCgoK4o5Rx8FyuTsbNmxgxYoV9O/fv1Gvp64eEUm7rKwsunbtmlFF/0hlZnTt2rVJn55U+EUkFir6Laep+1KFX0QkYVT4RUQSRoVfRBJn8+bN/PrXv27yeuPGjWPz5s1NXm/8+PFMmTKlyeuligq/iCROQ4W/urr6oOv95S9/oVOnTilKlT4azikisbr7j/NZsGpri77moJ6F/OCzJzb4/KRJk3j//fcZOnQoOTk55OfnU1xczOzZs1m0aBGXXXYZy5cvZ+fOndxyyy1MnDgRgH79+jFr1iwqKyu58MILOf3003n99dfp1asXzz//PO3atTtktmnTpnH77bdTVVXFqaeeygMPPEBubi6TJk3ihRdeIDs7m/PPP5+f/exnPPPMM9x9992YGZ07d2bGjBktsn9U+EUkce655x7Ky8uZM2cO06dP56KLLqK8vJxu3boB8PDDD9OlSxd27NjBqaeeypVXXknXrl33e40lS5bw+9//noceeohrrrmGZ599ls9//vMH3e7OnTsZP34806ZNY+DAgdxwww088MAD3HDDDUydOpVFixZhZvu6k374wx/y0ksvUVhYeMhPI02hwi8isTrYkXm6jBgxgv79+1NRUQHAL3/5S6ZOnQrA8uXLWbJkSZ3C379/f4YOHQpASUkJS5cuPeR2Fi9eTP/+/Rk4cCAAN954I/fffz/f+MY3yMvL46abbuKiiy7i4osvBuC0005j/PjxXHLJJfzLv/xLC/1r1ccvIrLfXPfTp0/nb3/7G2+88QZz585l2LBh9X45Kjc3d9/9rKwsqqqqDrmdMFNyXdnZ2bz11ltceeWVPPfcc4wdOxaABx98kB//+MesWLGCoUOHsmHDhqb+0+rfXou8iojIEaSgoGDf0f2BtmzZQufOnWnfvj2LFi3izTffbLHtHn/88SxdupT33nuPAQMG8Pjjj3PmmWdSWVnJ9u3bGTduHKNGjWLAgAEAvP/++4wcOZJBgwbx8ssvs3z58jqfPJpDhV9EEqdr166cdtppDB48mHbt2lFUVLTvubFjx/Lggw8yZMgQjjvuOEaNGtVi283Ly+ORRx7h6quv3ndy9+abb2bjxo1ceuml7Ny5E3fnvvvuA+COO+5gyZIlVFdXc95553HyySe3SA4VfhFJpCeffLLe9tzcXF588cV6n6vpx+/WrRvl5eX72m+//faDbuvRRx/dd/+cc87hn//8537PFxcX89Zbb9VZ7w9/+APQ8pPHqY9fRCRhdMQvItJCvv71r/OPf/xjv7ZbbrmFCRMmxJSofir8IiIt5P777487QqOkrKvHzPLM7C0zm2tm883s7qj9LjNbaWZzotu4VGUQEZG6UnnEvws4290rzSwHeM3Mas6Y3OfuP0vhtkVEpAEpK/wevqlQGT3MiW71f3tBRETSxhr6JlmLvLhZFlAGDADud/c7zewuYDywFZgFfMvdN9Wz7kRgIkBRUVHJ5MmTm5WhsrKS/Pz8Zq2bSpmaCzI3m3I1TabmAigsLOTYY4+NO0Yd1dXVZGVlxR2jjsbkeu+999iyZct+bWPGjClz9+F1Fnb3lN+ATkApMBgoArII5xd+Ajx8qPVLSkq8uUpLS5u9biplai73zM2mXE2Tqbnc3WfPnh13hHpt3bq1wec6dOjg7u4rV670K6+8st5lzjzzTH/77bcbfI2+ffv6unXrWjRXjQULFtRpA2Z5PTU1LeP43X0zMB0Y6+5r3L3a3fcCDwEj0pFBRKQl9OzZM6MuqtIcKevjN7OjgD3uvtnM2gHnAj81s2J3Xx0tdjlQ3uCLiEjr9+Ik+Pidln3NHifBhfccdJE777yTvn378rWvfQ2Au+66i927dzNz5kw2bdrEnj17+PGPf8yll16633pLly7l4osvpry8nB07djBhwgQWLFjACSecwI4dOxod8d577+Xhhx8G4KabbuLWW29l27ZtXHPNNaxYsYLq6mq+//3vc+211/KDH/yAv/71r/vN1X84Ujmqpxh4LOrnbwM87e5/MrPHzWwo4UTvUuArKcwgIlKv6667jltvvXVf4X/66aeZMmUKkyZNorCwkPXr1zNq1CguueQSzKze13jggQdo37498+bNY968eZxyyimN2nZZWRmPPPIIM2fOxN0ZOXIkZ555Jh988AE9e/bkz3/+MxAmjNu4cSN//OMfeffdd/ebq/9wpHJUzzxgWD3tX0jVNkXkCHSII/NUGTZsGGvXrmXVqlWsW7eOzp0706NHD77zne8wY8YM2rRpw8qVK1mzZg09evSo9zVmzJjBN7/5TQCGDBnCkCFDGrXt1157jcsvv3zfdNBXXHEFr776KmPHjuX222/nzjvv5OKLL+aMM86gqqqq3rn6D4fm6hGRxLrqqquYMmUKTz31FNdddx1PP/0069ato6ysjDlz5lBUVFTvXPy1NfRp4GC8gdGUAwcOpKysjJNOOolvf/vb/PCHPyQ7O5vS0tI6c/UfDhV+EUms6667jsmTJzNlyhSuuuoqtmzZQvfu3cnJyaG0tJRly5YddP3Ro0fzxBNPAFBeXs68efMatd3Ro0fz3HPPsX37drZt28bUqVM544wzWLVqFe3bt+fzn/88t99+O7Nnz6ayspKtW7cybtw4fvGLXzBnzpzD/Wdrrh4RSa4TTzyRiooKevXqRXFxMddeey3XX389w4cPZ+jQoRx//PEHXf+rX/0qEyZMYMiQIQwdOpQRIxo3SPGUU05h/Pjx+5a/6aabGDZsGC+99BJ33HEHbdq0IScnhwceeICKigquvvpq9uzZs99c/YdDhV9EEu2ddz4ZUdS1a1feeOONeperrAwTEfTr12/fXPzt2rWjKV8urX1d3ttuu43bbrttv+cvuOACLrjggjrrTZ8+XfPxi4hI8+mIX0SkhY0cOZJdu3bt1/b4449z0kknxZRofyr8IhILd2/WiJgjwcyZM9O6vYZGCTVEXT0iknbV1dVs2LChyQVL6nJ3NmzYQF5eXqPX0RG/iKTdtm3bqKioYN26dXFH2c/OnTubVEDT5VC58vLy6N27d6NfT4VfRNLO3enfv3/cMeqYPn06w4bVmXAgdi2dS109IiIJo8IvIpIwKvwiIgmjwi8ikjAq/CIiCaPCLyKSMCr8IiIJo8IvIpIwKvwiIgmjwi8ikjApK/xmlmdmb5nZXDObb2Z3R+1dzOwVM1sS/eycqgwiIlJXKo/4dwFnu/vJwFBgrJmNAiYB09z9WGBa9FhERNIkZYXfg8roYU50c+BS4LGo/THgslRlEBGRuiyV82GbWRZQBgwA7nf3O81ss7t3qrXMJnev091jZhOBiQBFRUUlTbmuZW2VlZXk5+c3a91UytRckLnZlKtpMjUXZG621pZrzJgxZe4+vM4T7p7yG9AJKAUGA5sPeG7TodYvKSnx5iotLW32uqmUqbncMzebcjVNpuZyz9xsrS0XMMvrqalpGdXj7puB6cBYYI2ZFQNEP9emI4OIiASpHNVzlJl1iu63A84FFgEvADdGi90IPJ+qDCIiUlcqr8BVDDwW9fO3AZ529z+Z2RvA02b2JeAj4OoUZhARkQOkrPC7+zygzrXC3H0DcE6qtisiIgenb+6KiCSMCr+ISMKo8IuIJIwKv4hIwqjwi4gkjAq/iEjCqPCLiCSMCr+ISMKo8IuIJIwKv4hIwqjwi4gkjAq/iEjCqPCLiCSMCr+ISMKo8IuIJIwKv4hIwqjwi4gkjAq/iEjCqPCLiCSMCr+ISMKkrPCbWR8zKzWzhWY238xuidrvMrOVZjYnuo1LVQYREakrO4WvXQV8y91nm1kBUGZmr0TP3efuP0vhtkVEpAEpO+J399XuPju6XwEsBHqlanv1mvEfDP3nd9O6SRGRTGfunvqNmPUDZgCDgduA8cBWYBbhU8GmetaZCEwEKCoqKpk8eXKTt9vvwyfou+wZZox+Bm+T0+z8qVBZWUl+fn7cMeqVqdmUq2kyNRdkbrbWlmvMmDFl7j68zhPuntIbkA+UAVdEj4uALMKnjZ8ADx/qNUpKSrxZ5j7l/oNC9zULm7d+CpWWlsYdoUGZmk25miZTc7lnbrbWlguY5fXU1JSO6jGzHOBZ4Al3/0P0h2aNu1e7+17gIWBEygJ0HRB+bliSsk2IiBxpUjmqx4DfAQvd/d5a7cW1FrscKE9Vhn2Ff70Kv4hIjVSO6jkN+ALwjpnNidq+A1xvZkMBB5YCX0lZgrxCdrXtTO6G91K2CRGRI03KCr+7vwZYPU/9JVXbrM/29r3I1RG/iMg+rf6buzva9VIfv4hILa2+8G9v3wt2bIJtG+KOIiKSEZJR+EFH/SIikeQU/vXvxhtERCRDtPrCvzOvO+QWwqo5cUcREckIrb7wY1nQezgsnxl3EhGRjND6Cz9An1GwZj7s3BJ3EhGR2CWk8I8AHFa8HXcSEZHYJaPw9x4O1gaWvxV3EhGR2CWj8OcWQNFg+OjNuJOIiMQuGYUfoM9IWDELqqviTiIiEqtGFX4z62BmbaL7A83skmjK5SPH0aNgzzZYk7rJQEVEjgSNPeKfAeSZWS9gGjABeDRVoVKiz8jwU/38IpJwjS385u7bgSuA/3L3y4FBqYuVAp36QGEvWK5+fhFJtkYXfjP7NPA54M9RWyrn8k+NPiPgI32RS0SSrbGF/1bg28BUd59vZscApSlLlSp9RsHWFbBlRdxJRERi06ijdnf/O/B3gOgk73p3/2Yqg6XE0TX9/DOhY+94s4iIxKSxo3qeNLNCM+sALAAWm9kdqY2WAkUnQU57dfeISKI1tqtnkLtvBS4jXDrxaML1dI8sWdnQq0QneEUk0Rpb+HOicfuXAc+7+x7CxdKPPEePgo/LYVdl3ElERGLR2ML/G2Ap0AGYYWZ9ga0HW8HM+phZqZktNLP5ZnZL1N7FzF4xsyXRz86H8w9osj6jwKthZVlaNysikikaVfjd/Zfu3svdx3mwDBhziNWqgG+5+wnAKODrZjYImARMc/djCV8Gm3QY+Zuu93DAND+/iCRWY0/udjSze81sVnT7OeHov0HuvtrdZ0f3K4CFQC/gUuCxaLHHCN1H6dOuE3Q/QRO2iUhimfuhu+rN7FmgnE8K9heAk939ikZtxKwfYdqHwcBH7t6p1nOb3L1Od4+ZTQQmAhQVFZVMnjy5MZuqo7Kykvz8/P3aBi7+Nd3Xvso/TvtfvE1Ws173cNWXK1NkajblappMzQWZm6215RozZkyZuw+v84S7H/IGzGlMWwPr5gNlwBXR480HPL/pUK9RUlLizVVaWlq3cf5z7j8odH/v/zX7dQ9XvbkyRKZmU66mydRc7pmbrbXlAmZ5PTW1sSd3d5jZ6TUPzOw0YMehVopGAj0LPOHuf4ia15hZcfR8MbC2kRlazrHnhwuwvzMl7ZsWEYlbYwv/zcD9ZrbUzJYCvwK+crAVzMyA3wEL3f3eWk+9ANwY3b8ReL5JiVtCTjs44bOw8AXYszPtmxcRiVNjR/XMdfeTgSHAEHcfBpx9iNVOI5wLONvM5kS3ccA9wHlmtgQ4L3qcfiddBbu2wpKXY9m8iEhcmjTDpodv79a4DfjFQZZ9DbAGnj6nKdtNiX6joaAYZv0OBl0SdxoRkbQ5nEsvNlTUjwxZ2TDqa/DBdFihL3OJSHIcTuE/MqdsqG34BMjrBK/+PO4kIiJpc9DCb2YVZra1nlsF0DNNGVMntwBG3gyL/wzrl8SdRkQkLQ5a+N29wN0L67kVuPuRdwWu+pz6JWiTA2//d9xJRETS4nC6elqH/O5w4mUw50nN2CkiiaDCD3Dql8PQznlPxZ1ERCTlVPghXIS9x0mhu6cRcxeJiBzJVPgBzGDERFi7AJa9HncaEZGUUuGvMfiqMLTz7YfiTiIiklIq/DXatodhn4eFf4SKj+NOIyKSMir8tQ3/IuytgrJH404iIpIyKvy1df0UDDgPZj0C1XviTiMikhIq/Aca8WWo/Dh0+YiItEIq/AcacC506qtv8opIq6XCf6A2WWEah2X/gDXz404jItLiVPjrM+wLkJ2no34RaZVU+OvTvksY1z/3KdixOe40IiItSoW/ISO/Anu2wZu/jjuJiEiLUuFvSPEQGHQpvHE/bFsfdxoRkRajwn8wY74He7bDjJ/FnUREpMWkrPCb2cNmttbMymu13WVmK81sTnQbl6rtt4ijBsIpN8Bbv4XV8+JOIyKtmXs4p1h7ypiKNSmZMTiVR/yPAmPrab/P3YdGt7+kcPst49y7oH1X+OM3YW913GlEpDX4YDo8Mx42vB8eV+2CRy+Gn/aFnx8H/3slPDMB7hsES19r8c2n7PKJ7j7DzPql6vXTpl1nuPAemPLFcOQ/6qtxJxKRI8EH0yG/B3Q//pO26j1hVoCpN0P1LljyNzj7e7BuISx7Dc74FmTlwlu/geqqcE3wzv1g2fstGi2O6+Z+w8xuAGYB33L3TTFkaJoTr4A5v4dpP4LjL4ZOfeJOJCKZYvNH8Pefhm6az/4ndOgG70yBZ78Uni/sHYaIV++GLStgdyUUD4VL/gtevBP+emdY7tPfgHP+Ldw/47bQxZPdNtpIyxZ+8xRecSo64v+Tuw+OHhcB6wEHfgQUu/sXG1h3IjARoKioqGTy5MnNylBZWUl+fn6z1q0tb8caTn37X9nUeQjlg78bLt5yGFoqVypkajblappMzQWZm62hXJ02zWPguw+woeuprO1+GjvzitiT05Geq/7KgPd+t2+53W07srb7aHqt/AuV+f1ZU3QGHbcsJrtqG3vb5LC7bWc2dT6JjV1K2JuVC0DB1iUUbn2XVT3Px9vkNCnXoYwZM6bM3Ycf2J7Wwt/Y5w40fPhwnzVrVrMyTJ8+nbPOOqtZ69bx+q/g5e/C1Y+FC7QfhhbN1cIyNZtyNU2m5oLMzbbgqbsZtGU6FPaEEy+HwVfCxg/gobOhTTbs2AQenevLyg3dNQPODUf629bBC98MU70U9oQvvQKFxS2Sq7n7y8zqLfxp7eoxs2J3Xx09vBwoP9jyGWfkzfDO0/Di/4VjzoJ2neJOJCJNtWY+/Pl2+HgelIyH/qND+wfTGbTw19DtuPDcoj/BjP+ATcsgJw9uegWy28Gqf8KW5aGLp8sxUDIB2rSBjr3h5lehanfoEciq/+g9E6Ss8JvZ74GzgG5mtgL4AXCWmQ0ldPUsBb6Squ2nRFZ2+Mv+0Nkw7W64+L64E4mIO6x4Gz5+J4yOOeYsKBoEu7eHT+gry+Doz0DPoWFY9swHIa9jWO7NX8Mbv9r3Uh8XnU2PL08OR/ezHoZ5T4ch3SXjQ5GHQx/F7+uXz1ypHNVzfT3Nv6un7cjScxiM+lp4s5x0DfT9dNyJRFo/d9jwHpQ/C1lt4ehPQ9VOWD4T5k+FdYv2X76wN+CwdRX0PhXKHoGZOwELRfycfwsnXCvWhBOue6vgqONYNHMOPbJD3zsjvhxurVAco3qOfGd9GxY8D3+6Fb7y6hHxF17kiLFpWehSrdoVivvSf4Siv2srWBvwvbUWNjh6VBghM+Dc0LTwT+Eof8fGqP2cMIxy4wfhSL7rpz5ZvaAo3BJGhb85cvPhonvhyavhH/8JZ94RdyKRI8/yt+Hl70HVDsgthLYdQtGvOXo3C4W+96lw8vXQ7Vg44bOAweq50LZ96I/PP2r/1x05se62snLgqONS/k86UqjwN9fA88P4/hn/DoMu0ZtKpMbe6tA10yYLKtfA2oXhZOm7L8Hm5dC2A6fsrIK/fwAFxVB0IuzcErpcOvcNI2lOvg4Ke4URNPWdJC04P/3/rlZEhf9wjL0HPvw7PH0D3PQ3yC2IO5FIetUU+azsMP1A2aMw50nYvh5y2odJDmsUnQR9PwN7tlP18XIYPiH0ted1PMgGNI9kKqjwH46CIrj6Ufify8JXsK95PAzrEmlNdm+HTUth29pQ6CvXhD70BS+ENgzyu4d2y4Ljx0H3E2FXBXTpHz4NH3V8WCYyL0PH8SeFCv/h6j8azv8RvPQdePXn6u+XI0flujAMMjcfOnQPfeYfzoBFfwmfZAt6hE+xq+d98qWlGtl5MPCCUOC9OnTTdP0UDP1cWE8ymgp/Sxj1NVg1B0p/Ei7gMvCCuBOJYHuroXJt+CLSuy+FfvQ22ZCdG758tP7d+lcs7B362betDyNpTv8/0P2EUNAtCzocFearqhn2KEccFf6WYBa+2LVuETx7E3y5FLoNiDuVJEF1VXjfbVkejrrXLgzT+G5dyZm7K2FGtFyXT4VpBPZWh4J+1HHheyj9R4fJwyrXhOkI+oyAHkMOey4qyWwq/C2lbXu47gn4zZnw1Od0slda1vaNYWTMmgVhtMzeqlDkF78YTqTWaJsfvtw04Bw+/HgT/Y8fCr2HQ68SFXPZR4W/JXU6OpzsffxyneyVpnMPE31t/DB0wyx7Pfys+Bi2rqi7fF5H+NTZcNxFYTqBjr1DN0z0nls2fTr9R52V3n+DHBFU+FvaMWd+crL35e/BBT/RkZZ8YldFKObbN8Gad2Dd4tBXvmkZrJgFuys+WbZdF+hxEvQ7LfSxFw8Nj7HwnmrXWe8taRYV/lQY9bUw/O3N+8N/5It/ET6eS3Ls3Qsb3w9j23dXhvfDgufg43LCHIWRwl6hjz2/CE6+FroNhM79wxF8l2P0iVFSQoU/Fczgwn8PX0N/9Wewcytc8VuNgmhNqvfA2gWwaSm9l0+HF18M30rdXQFtC2DFW6HbprY+I8M8Tz0GQ/tuYfhjh25xpJeEU+FPFTM45/thzv6Xvxc+3p/3wzCRlD6eHxmqq6I+9/dhycuwfUMo6h/Pg5WzwxwzwACA5QVhiGNuQZjxsf/o0P/e7bjQF9+u035fYBKJkwp/qn3mX8NH9pe+A09cFeYAP+Nbdb8QI+m1dy/s3Ay7t4U/xCveDhfYyGkfRsssnxmGONbMBNkmB9p3DWPhiwaF6QZ6D4euA3itfDmnn3uR/qDLEUOFPx2OvwgGnAezfhcuyvzYZ/lMTifYdiUMuSZ0AahoNN/u7eGofNkbULE6dKnt2ByO1rdvCF9aMgtDIndsCtP17tjMfn3tEJbbWxUmDut/ZhilVVAUvtDU9zOQV1jv5qsWb9LvT44oKvzpkt0WRn01XM1nyctsnv5bus95Mvwx6HYc9D8Deg0PR5FdPqWTejV2VcD6JbBrK93WvQFzVodvk65bFI7QN38UinuNmuKdWxiO0Nt3CUftvjeMkul0dGhr1yWMimnbIboIx/FhXnff+8kfCpFWSoU/3dp2gBMvZ8G6znT/9PBwRaHyKTB3Mrz932GZvI7QewT0Oz10E2XnhS/p5HUMVx/asiKMBKlt97ZwoYldW8OJx6pdYZm91WFEUVbbML1tm+xP7mflhC6MnHbhRGN+EeyqpPuaJbBoW2jPaR9+ts2HPTvCaxb2DG1tcsIX19xD8WzMNUb37oVdW8LRd8Vq2Loatq4M93duDdc23bkljF2vWB3GtEdH5oMB5kevk1sYroZ2wiVh/HqnvtDnVOjc75N/c7No9JW0fir8ccrNh5Ibw21vdRjTvXJWGM/90Rvwt1ea9noduoej2awcyMoNBb7mW541RXtvVfhZvSfc9u4JXSV7tu17mUEACxu5zbYF4XzFnu3hfn73cKS9tyraxq5P/ghV7QxFfb8rKNV6nbyOYZm8QsjvAcUnw5BroWgwtOvMrHcWM/y0MeEEarvODRd3DZ0VOSgV/kzRJiucNCwaFLqDIBwVb1kOVbtDQd+1Ndzv2DsccdeW1Tb8IWkO9/ApYsdGyC1g5puvM3LYSeGPxZ7t4Q/D7sroKD87XMe0enco6JVrQltex5C3ck14nTbZ4Y9Pdtvoj1BO+OTSrlPoZmnfJXzCKOwVLl7diOktKpfuCdP8ishhSVnhN7OHgYuBte4+OGrrAjwF9AOWAte4+6ZUZTjitY8KZKqZhaGInfoAsKP9R+FoW0RapVSeQXwUGHtA2yRgmrsfC0yLHouISBqlrPC7+wxg4wHNlwKPRfcfAy5L1fZFRKR+6R4zWOTuqwGin/oqo4hImpm7H3qp5r64WT/gT7X6+De7e6daz29y984NrDsRmAhQVFRUMnny5GZlqKysJD+/mSc9UyhTc0HmZlOupsnUXJC52VpbrjFjxpS5+/A6T7h7ym6Ek7jltR4vBoqj+8XA4sa8TklJiTdXaWlps9dNpUzN5Z652ZSraTI1l3vmZmttuYBZXk9NTXdXzwvAjdH9G4Hn07x9EZHES1nhN7PfA28Ax5nZCjP7EnAPcJ6ZLQHOix6LiEgapWwcv7tf38BT56RqmyIicmiaCUxEJGFU+EVEEkaFX0QkYVT4RUQSRoVfRCRhVPhFRBJGhV9EJGFU+EVEEkaFX0QkYVT4RUQSRoVfRCRhVPhFRBJGhV9EJGFU+EVEEkaFX0QkYVT4RUQSRoVfRCRhVPhFRBJGhV9EJGFU+EVEEkaFX0QkYbLj2KiZLQUqgGqgyt2Hx5FDRCSJYin8kTHuvj7G7YuIJJK6ekREEsbcPf0bNfsQ2AQ48Bt3/209y0wEJgIUFRWVTJ48uVnbqqysJD8//zDSpkam5oLMzaZcTZOpuSBzs7W2XGPGjCmrtyvd3dN+A3pGP7sDc4HRB1u+pKTEm6u0tLTZ66ZSpuZyz9xsytU0mZrLPXOztbZcwCyvp6bG0tXj7quin2uBqcCIOHKIiCRR2gu/mXUws4Ka+8D5QHm6c4iIJFUco3qKgKlmVrP9J939rzHkEBFJpLQXfnf/ADg53dsVEZFAwzlFRBJGhV9EJGFU+EVEEkaFX0QkYVT4RUQSRoVfRCRhVPhFRBJGhV9EJGFU+EVEEkaFX0QkYVT4RUQSJpYLsTSVma0DljVz9W5AJl7iMVNzQeZmU66mydRckLnZWluuvu5+1IGNR0ThPxxmNssz8GLumZoLMjebcjVNpuaCzM2WlFzq6hERSRgVfhGRhElC4a9zIfcMkam5IHOzKVfTZGouyNxsicjV6vv4RURkf0k44hcRkVpU+EVEEqZVF34zG2tmi83sPTObFGOOPmZWamYLzWy+md0Std9lZivNbE50GxdDtqVm9k60/VlRWxcze8XMlkQ/O6c503G19skcM9tqZrfGtb/M7GEzW2tm5bXaGtxHZvbt6D232MwuSHOu/zCzRWY2z8ymmlmnqL2fme2ote8eTHOuBn93Me+vp2plWmpmc6L2dO6vhupD6t5j7t4qb0AW8D5wDNAWmAsMiilLMXBKdL8AeBcYBNwF3B7zfloKdDug7d+BSdH9ScBPY/49fgz0jWt/AaOBU4DyQ+2j6Pc6F8gF+kfvwaw05jofyI7u/7RWrn61l4thf9X7u4t7fx3w/M+Bf4thfzVUH1L2HmvNR/wjgPfc/QN33w1MBi6NI4i7r3b32dH9CmAh0CuOLI10KfBYdP8x4LL4onAO8L67N/eb24fN3WcAGw9obmgfXQpMdvdd7v4h8B7hvZiWXO7+srtXRQ/fBHqnYttNzXUQse6vGmZmwDXA71Ox7YM5SH1I2XusNRf+XsDyWo9XkAHF1sz6AcOAmVHTN6KP5Q+nu0sl4sDLZlZmZhOjtiJ3Xw3hTQl0jyFXjevY/z9j3PurRkP7KJPed18EXqz1uL+Z/dPM/m5mZ8SQp77fXabsrzOANe6+pFZb2vfXAfUhZe+x1lz4rZ62WMeumlk+8Cxwq7tvBR4APgUMBVYTPmqm22nufgpwIfB1MxsdQ4Z6mVlb4BLgmagpE/bXoWTE+87MvgtUAU9ETauBo919GHAb8KSZFaYxUkO/u4zYX8D17H+Akfb9VU99aHDRetqatM9ac+FfAfSp9bg3sCqmLJhZDuGX+oS7/wHA3de4e7W77wUeIkUfcQ/G3VdFP9cCU6MMa8ysOMpdDKxNd67IhcBsd18TZYx9f9XS0D6K/X1nZjcCFwOf86hTOOoW2BDdLyP0Cw9MV6aD/O4yYX9lA1cAT9W0pXt/1VcfSOF7rDUX/reBY82sf3TkeB3wQhxBov7D3wEL3f3eWu3FtRa7HCg/cN0U5+pgZgU19wknBssJ++nGaLEbgefTmauW/Y7C4t5fB2hoH70AXGdmuWbWHzgWeCtdocxsLHAncIm7b6/VfpSZZUX3j4lyfZDGXA397mLdX5FzgUXuvqKmIZ37q6H6QCrfY+k4ax3XDRhHOEP+PvDdGHOcTvgoNg+YE93GAY8D70TtLwDFac51DGF0wFxgfs0+AroC04Al0c8uMeyz9sAGoGOttlj2F+GPz2pgD+Fo60sH20fAd6P33GLgwjTneo/Q/1vzPnswWvbK6Hc8F5gNfDbNuRr83cW5v6L2R4GbD1g2nfurofqQsveYpmwQEUmY1tzVIyIi9VDhFxFJGBV+EZGEUeEXEUkYFX4RkYRR4RcBzKza9p8RtMVmc41meozzOwci+8mOO4BIhtjh7kPjDiGSDjriFzmIaI72n5rZW9FtQNTe18ymRZOOTTOzo6P2Igvz4M+Nbp+JXirLzB6K5lt/2czaxfaPksRT4RcJ2h3Q1XNtree2uvsI4FfAL6K2XwH/4+5DCBOh/TJq/yXwd3c/mTD3+/yo/Vjgfnc/EdhM+GaoSCz0zV0RwMwq3T2/nvalwNnu/kE0kdbH7t7VzNYTph3YE7WvdvduZrYO6O3uu2q9Rj/gFXc/Nnp8J5Dj7j9Owz9NpA4d8Yscmjdwv6Fl6rOr1v1qdH5NYqTCL3Jo19b6+UZ0/3XCjK8AnwNei+5PA74KYGZZaZ7zXqRRdNQhErSz6ELbkb+6e82Qzlwzm0k4ULo+avsm8LCZ3QGsAyZE7bcAvzWzLxGO7L9KmBFSJGOoj1/kIKI+/uHuvj7uLCItRV09IiIJoyN+EZGE0RG/iEjCqPCLiCSMCr+ISMKo8IuIJIwKv4hIwvx/Xg5VnuRORNcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA9uElEQVR4nO3deXhU5dn48e8zM5kkk8m+EUggIYR9J2yKGFAQUEFaVNza19ZaF1r7qq222qpv1dr6q1UrimjdcMG9brgLBAFZwg6yryFsCZBkErI/vz+eSUhCEpKYmYmZ+3Ndc2XmnDPn3HMYzj3PepTWGiGEEP7L4usAhBBC+JYkAiGE8HOSCIQQws9JIhBCCD8niUAIIfyczdcBtFRMTIxOTk5u1XuLiooICQlp24DaQHuNC9pvbBJXy0hcLdMR48rKysrVWsc2uFJr/aN6DBs2TLfWwoULW/1eT2qvcWndfmOTuFpG4mqZjhgXsFo3cl2VqiEhhPBzkgiEEMLPSSIQQgg/96NrLBZCdAzl5eU4nU6+//57X4dyhvDw8B9tXEFBQSQmJhIQENDs/UoiEEL4RHZ2NvHx8SQmJqKU8nU4dRQWFhIaGurrMM5wtri01uTl5ZGdnU1KSkqz9ytVQ0IInygpKSE8PLzdJYEfM6UU0dHRlJSUtOh9kgiEED4jSaDtteac+l0imPfdPj5Yd9DXYQghRLvhd4ng7dUHeG+NJAIhhKjmd4kgOsROXlGpr8MQQvjYyZMnefrpp1v8vilTpnDy5Mm2D8iHPJYIlFIvKKWOKqU2NbJeKaWeVErtVEptUEoN9VQstUU7AznuKvPGoYQQ7VhjiaCysrLJ9y1YsICIiAgPReUbnuw++hLwFPBKI+snA2nux0jgGfdfj4oOsZNbVIbWWhqqhGgnHvhoM1tyCtp0n307h3Hfpf0aXX/33Xeza9cuBg8eTEBAAE6nk4SEBNatW8eKFSu47LLLOHDgACUlJdx2223ceOONACQnJ7N69WpcLheTJ09mzJgxLFu2jC5duvDBBx8QHBzc4PEyMjIYMmQIWVlZHDt2jFdeeYW//e1vbNy4kSuvvJIHH3wQoNHjfvHFF9x7771UVFSQmprKiy++iNPpbJNz5bESgdY6EzjexCbTgFfc8yF9B0QopRI8FQ87v2bY6t/RzZpLWUUVrtIKjx1KCNH+PfLII6SmprJu3ToeffRRVq5cyUMPPcSWLVsAeOGFF8jKymL16tU8+eST5OXlnbGPHTt2cOutt7J582YiIiJ49913mzym3W4nMzOTm266iWnTpjF79mw2bdrESy+9VLP/ho6bm5vLgw8+yIcffsiaNWtIT0/nsccea7Nz4csBZV2AA7VeZ7uXHaq/oVLqRuBGgPj4eBYtWtTig0XnrmaAaw+nsrcAiSz4egnxIe2jicTlcrXqM3lDe41N4mqZ9hhXeHg4lZWVFBYWcntGV48co7CwsNF1LpeLqqoqCgsLKS4uZtiwYcTExFBYWEhlZSWPPvooH3/8MQAHDhxg3bp1jBgxAq01LpcLl8tFt27dSE1NpbCwkP79+7Nt27ZGj1lZWcmFF15IYWEhqamp9O7dG6fTSVlZGd26dWPr1q0MHDiwwePm5eWxefNmJkyYgFKKsrIyRowY0eixSkpKWvTv7ctE0FC9jG5oQ631XGAuQHp6us7IyGj50fZYYRMMS0uAHdCj/xCGdYts+X48YNGiRbTqM3lBe41N4mqZ9hjX999/j9Vq9dkIXqfTicViITQ0FIfDQVhYWE0sCxYsYMmSJaxYsQKHw0FGRkZNrEqpmiqZ4ODgmvc4HA5cLlejn8dqtRIZGUloaChOpxOHw1GzbUBAAIGBgWRlZTV43ODgYCZOnMjcuXObdb6CgoIYMmRIs8+FL38SZwNJtV4nAjkeO1qgOXmRVtNjKM8lPYeE8GehoaGN/qIuKCggMjISh8PB1q1b+e6777wSU35+foPHHTVqFEuXLmXXrl0AFBcXs3379jY7ri9LBB8Cs5RS8zGNxPla6zOqhdpMUBgAYZZTgIO8Iuk5JIQ/i46O5txzz6V///4EBwcTHx9fs+7CCy/k5ZdfZuDAgfTq1YtRo0Z5JaZJkyYxZ86cM44bGxvLSy+9xC9+8QsqKkz75oMPPkjPnj3b5LgeSwRKqTeADCBGKZUN3AcEAGit5wALgCnATqAYuN5TsQAQaBKBk1OAlAiEEPD66683uDwwMJBPP/20wXV79+4FICYmhk2bTveOv/POO5s8Vu06+4yMjDpVdbXXNXbc8ePHs3jxYo9UpXksEWitrzrLeg3c6qnjn8FdNRRQ4SI00EaujCUQQgjAn6ahtgVSpWxYSguJdto5LlVDQggPuPXWW1m6dGmdZbfddhvXX+/ZSo8fwn8SAVBhc2AvLSTaGSjTTAghPGL27Nm+DqHF2kdHei+ptDqgtJCoEDt5UjUkhBCAnyWCClswlBYS47RLG4EQQrj5VSKoLhFEhwRyoriMqqoGx68JIYRf8atEUGFzQEk+0U47lVWa/FPlvg5JCCF8zq8SQaXVVA1FOwMBpMFYCNFs1dNK5OTkMGPGjAa3ycjIYPXq1d4Mq034VSKosIVAaSERwQEAnCyWEoEQomU6d+7MO++84+sw2pRfdR+tLhGEBFoBKCpr+gYUQggv+fRuOLyxbffZaQBMfqTR1XfddRfdunXjlltuAeD+++9HKUVmZiZ5eXlUVlby4IMPMm3atDrv27t3L5dccgmbNm3i1KlTXH/99WzZsoU+ffpw6tSpJkNyOp3ceuutfPXVV0RGRvLwww/zhz/8gf379/P4448zdepU9u7dy3XXXUdRUREATz31FOeccw4ATzzxBB988AGlpaVMnz6dBx544IecoRp+ViJwQGUpTptJAMVyTwIh/NbMmTN58803a16/9dZbXH/99bz//vssWbKEhQsXcscdd2AmQWjYM888g8PhYMOGDdxzzz1kZWU1ecyioiIyMjLIysoiNDSUe++9ly+//JL333+fv/zlLwDExcXx5ZdfsmbNGt58801++9vfAubGNLt27WLlypWsW7eOrKwsMjMz2+BM+F2JwAGAkxJASgRCtBtN/HL3lCFDhnD06FFycnI4duwYkZGRJCQk8L//+78sWrQIm83GwYMHOXLkCJ06dWpwH5mZmTUX6oEDBzJw4MAmj2m325k0aRIAAwYMIDAwkICAAAYMGFAzh1F5eTmzZs1i3bp1WK3WmllGv/jiC7755pua6aVdLhc7duxg7NixP/hc+FUiqLCZW8g5dDEAxWVSIhDCn82YMYN33nmHw4cPM3PmTF577TWOHTtGZmYmUVFRJCcnU1JS0uQ+WnLL24CAgJrtLRYLgYGBNc+rZxX917/+RXx8POvXr6eqqoqgoCAAtNbcfvvt3Hbbba35qE3yq6qh6hKBQ5t6PLldpRD+bebMmcyfP5933nmHGTNmkJ+fT1xcHAEBASxcuJB9+/Y1+f6xY8fy2muvAbBp0yY2bNjwg2PKz88nISEBi8XCvHnzqKw0NRcXXXQR8+bNw+VyAXDw4EGOHj36g48HflciMInAXunCoqC4VKqGhPBn/fr1o7CwkC5dupCQkMA111zDpZdeyvnnn8/QoUPp3bt3k++/+eabuf766xk4cCCDBw9mxIgRPzimW265hZ/+9Ke8/fbbjBs3jpCQEAAmTpzI2rVrGT16NGAanl999VXi4uJ+8DH9KhFUWk3VkCotJMRuo0iqhoTwexs3nu6tFBMTw/LlyyksLDxj3v/qX+LJyck19yEIDg5m/vz5zT5W9T7A9FJqaF1aWlqdksXf/va3mue33HILd911V7OP11x+VTVUXSIwXUhtUiIQQgj8rkRQnQgKcATGSYlACOERI0eOpLS07swF8+bNY8CAAT6KqGl+lQjqlAjsNoql+6gQPtVUH/0fsxUrVvjs2K05p35VNVRlsYPFBqWFOOxWiqTXkBA+ExQURH5+fodNBr6gtSYvL6+my2lz+VWJAKXMvYvdbQRHC5vuHyyE8JzExETWr19fpwG1vSgpKWnxxdQbmhNXUFAQiYmJLdqvfyUCqEkEDrtVGouF8KGAgABcLhfp6em+DuUMixYtqhnB2554Ki6/qhoCIDAMSgqk+6gQQrj5XyKwO6FMuo8KIUQ1/0sEgU4odRESaKWorEIaqoQQfs//EoE9BMpcOOw2qjSUVlT5OiIhhPApP0wEoVBWVHNzGpl4Tgjh7/wvEbirhhx202FK2gmEEP7O/xJBdWNxgPno0nNICOHv/C8RBDpBV+G0mQQgN6cRQvg7/0sEdicAYRZzc5oiqRoSQvg5/0sEgWaO8RD3fYulRCCE8HceTQRKqUlKqW1KqZ1KqbsbWB+ulPpIKbVeKbVZKXW9J+MBakoE1YnAJSUCIYSf81giUEpZgdnAZKAvcJVSqm+9zW4FtmitBwEZwD+VUnZPxQSYNgIg2H3fYikRCCH8nSdLBCOAnVrr3VrrMmA+MK3eNhoIVUopwAkcBzx7ZbabqqHqRCBtBEIIf6c8NcWCUmoGMElrfYP79XXASK31rFrbhAIfAr2BUOBKrfUnDezrRuBGgPj4+GEtuUdobS6Xizh1ghGrZrG5zx1csnYYl6QG8NM0zxZCmhOX0+n0aQyNaa+xSVwtI3G1TEeMa9y4cVla6wanevXkNNSqgWX1s85FwDpgPJAKfKmUWqK1LqjzJq3nAnMB0tPTdUZGRqsCWrRoESOGDIFV0K9HV0K22IjtlEhGRv0aK+9atGgRrf1MntZeY5O4Wkbiahl/i8uTVUPZQFKt14lATr1trgfe08ZOYA+mdOA59hDz1z3xnLQRCCH8nScTwSogTSmV4m4AnompBqptP3ABgFIqHugF7PZgTDW9hihzEWK3yVxDQgi/57GqIa11hVJqFvA5YAVe0FpvVkrd5F4/B/gr8JJSaiOmKukurXWup2ICwGoDW7C5S1mgVW5gL4Twex69VaXWegGwoN6yObWe5wATPRlDgwKdUOYi0mEnz1Xq9cMLIUR74n8ji8FUD5W6SIx0cODEKV9HI4QQPuWficBdIuga5eB4UZm0Ewgh/Jp/JgJ7KJS6SIoKBuDA8WIfBySEEL7jn4kg0NyToGuUA5BEIITwb/6ZCNxtBEmRJhHsl0QghPBj/pkI3G0EEY4AQgNt5Bwv9HVEQgjhM/6ZCNxtBEopzg/L4Y9rL4CdX/k6KiGE8An/TASBTigvgqoqLrUuJ4By+OROKC/xdWRCCOF1/pkIqucbKi9iaMkKcnQ0nNgDS5/wbVxCCOEDfpoI3PMNHdpAbMle5lZcTFnSGNjyX5+GJYQQvuCficB932LWvw7AV1VDOeFMhZMHwEP3ZxBCiPbKPxNBUIT5u/ZVyqJ7k63jOEQslBVCyUlfRiaEEF7n0Unn2q3u58OlT0J5Mdau52KffZDd5VEMBlMqCI70cYBCCOE9/pkIbIEw7OeAmR87JeYkW4rD+AlA/gFIGOjL6IQQwqv8s2qontS4ELJOutsNTh7wbTBCCOFlkgiA1FgnG04GoG3BpkQghBB+RBIBJhFUVkG5szOc3O/rcIQQwqskEWASAUBBYCcpEQgh/I4kAqB7rBlpfMQSJ20EQgi/I4kACAm0kRAexL6KaCjOhTKZlloI4T8kEbj1iHOy0eXuOZSf7dtghBDCiyQRuI1Ni2VVdRfSwxt8G4wQQniRJAK3C/vGs1734GRICnzzVyg/5euQhBDCKyQRuKXEhJAUG87TwTfBib3w7b98HZIQQniFJIJaJvSJ54WcrpSnTYGVc6GywtchCSGEx0kiqGV87zgqqjSboifCqROQvdLXIQkhhMdJIqhlUFIENotiUcUAsATAtk99HZIQQnicJIJaggKs9EkIY9WhCkgeI4lACOEXJBHUMygpnA3Z+VT1nAx5OyB3p69DEkIIj5JEUM+gxAhcpRXsjx1rFmyXUoEQomOTRFDP4KQIAFafDIX4/lI9JITo8DyaCJRSk5RS25RSO5VSdzeyTYZSap1SarNSarEn42mO7rFOnIE21h84Cb0mw/7voPi4r8MSQgiP8VgiUEpZgdnAZKAvcJVSqm+9bSKAp4GpWut+wOWeiqe5rBbFgC7hfLc7D91zEuhK2PGlr8MSQgiP8WSJYASwU2u9W2tdBswHptXb5mrgPa31fgCt9VEPxtNs04d2YcdRF4tcieCMl3YCIUSHprTWntmxUjOASVrrG9yvrwNGaq1n1drmcSAA6AeEAk9orV9pYF83AjcCxMfHD5s/f36rYnK5XDidzrNuV1GluSvzFJFBinkRzxGTu5Kl584DpVp13LaKyxfaa2wSV8tIXC3TEeMaN25cltY6vcGVWmuPPDDVPM/Xen0d8O962zwFfAeEADHADqBnU/sdNmyYbq2FCxc2e9tXlu/V3e76WO9e8ITW94VpfXxPq4/blnF5W3uNTeJqGYmrZTpiXMBq3ch11ZNVQ9lAUq3XiUBOA9t8prUu0lrnApnAIA/G1GwzhiZit1pYWNjFLMhZ59N4hBDCUzyZCFYBaUqpFKWUHZgJfFhvmw+A85RSNqWUAxgJfO/BmJot2G5lWLdI3j8YYaabyFnr65CEEMIjPJYItNYVwCzgc8zF/S2t9Wal1E1KqZvc23wPfAZsAFZiqpI2eSqmlhqTFsPGIyWUx/aVRCCE6LBsnty51noBsKDesjn1Xj8KPOrJOFrrvLQYHv18G9lBPUk59AVo7bEGYyGE8JUmSwRKqbAm1nVt+3Dal36dw4l0BLCqLBlK8uHEHl+HJIQQbe5sVUOLqp8opb6ut+6/bR1Me2O1KEZ1j+bzEwlmgVQPCSE6oLMlgtr1IFFNrOuwUmOdLC2IRVvtkgiEEB3S2RKBbuR5Q687pKSoYEqqrJTF9JUupEKIDulsjcVxSqnbMb/+q5/jfh3r0cjaiaRIBwAnwvvRad9HUFUFFpm0VQjRcZztivYcZuoHZ63n1a+f92xo7UOiOxEcCOoFpQVwfLePIxJCiLbVZIlAa/1AY+uUUsPbPpz2JyEiCIuC71V3hgMcWgcxPXwclRBCtJ0W1XEopfoqpf5PKbUDeMZDMbUrAVYLCeHBrC9NAFuQNBgLITqcsw4oU0p1A65yPyqAbkC61nqvZ0NrPxIjg9l3ogw6DZBEIITocM42oGwZZmRwADBDaz0MKPSnJACQFOUg+8Qp6DwEDq03DcZCCNFBnK1q6BimcTie072E/KLbaG1JkQ6OFJZQHj8QylyQt9PXIQkhRJtpMhForacBA4A1wANKqT1ApFJqhDeCay8SI4PRGg6H9DELpHpICNGBnLWxWGudr7V+QWs9ARgF3Ac8rpQ64PHo2omkKNOFdA9dIMAhiUAI0aG0qNeQ1vqI1vpJrfU5wBgPxdTuJEUFA3Ag391gfGidbwMSQog21GSvIaVU/RvJ1De1DWNpt+JCgwiwKg4cdzcYr3kFqirBYvV1aEII8YOdrfvoaOAA8AawAj+ZaK4+q0XRJSKY7BPF0G8IrJgDudshro+vQxNCiB/sbFVDnYA/Af2BJ4AJQK7WerHWerGng2tPEiMdHKjuQgrw0sXwxlVQUQYb3oZ5PzGlBCGE+JE5W6+hSq31Z1rrn2MaincCi5RSv/FKdO1IUlQw2ceLIaYnjP0DpIyFbQvg6wdgwR2w62s4mOXrMIUQosWaM7I4ELgYM7I4GXgSeM+zYbU/iZEO8orKKC6vxDH+HrOwogyWP2Vubq+ssO1TSPKrnrVCiA7gbCOLXwaWAUOBB7TWw7XWf9VaH/RKdO1IYqTpOZR94tTphZMfgaAIGHsndDvHJAIhhPiROVsbwXVAT+A2YJlSqsD9KFRKFXg+vPajeizBgePFpxdGdIU7tkHG3dBrChz7Ho7LfY2FED8uZ2sjsGitQ92PsFqPUK11oze274iqb1BTp0QAEBBk/vaaZP5u+cCLUQkhxA8nt9pqphinnaAAS90SQW1R3SHlfPj2MXAd825wQgjxA0giaCallLsLaSOJAGDKo1BWDB/dZrqUFuV5L0AhhGglSQQtkBQZfGbVUG2xveC8O2DbJ/DeDbDwIe8FJ4QQrSSJoAV6xDnZccRFnqu08Y3G/RFu2wBdz5FxBUKIHwVJBC1w5fCulFVW8dqK/U1vGNnNjCc4shkqmkgaQgjRDkgiaIEecU4yesXyyvJ9lFacZTqJzkOgqtwkAyGEaMckEbTQL8ekkOsq5eP1h5resHpOIpmyWgjRzkkiaKExPWLoHhvCqyv2Nb1hRFcIjpSb2Agh2j1JBC2klOLqEV1Zu/8kW3KaGFytlCkVSCIQQrRzHk0ESqlJSqltSqmdSqm7m9huuFKqUik1w5PxtJUZwxKx2yy8vvIspYLOQ0wbwdOjzbgCIYRohzyWCJRSVmA2MBnoC1yllOrbyHZ/Bz73VCxtLcJh5+IBCXywNofyyqrGNxx0FfS9DAoPwaZ3vBafEEK0hCdLBCOAnVrr3VrrMmA+MK2B7X4DvAsc9WAsbW5i33gKSytYd+Bk4xvFpMHlL5p7F+Tu8FpsQgjREkpr7Zkdm2qeSVrrG9yvrwNGaq1n1dqmC/A6MB74D/Cx1vqMn85KqRuBGwHi4+OHzZ8/v1UxuVwunE5nq95bX1G5ZtbXxUxNDWB6mr3JbVN2v0rX/e+SOfYttCXAo3G1tfYam8TVMhJXy3TEuMaNG5eltU5vcKXW2iMP4HLg+VqvrwP+XW+bt4FR7ucvATPOtt9hw4bp1lq4cGGr39uQqU99q6fP/vbsG657Q+v7wrQ+us0rcbWl9hqbxNUyElfLdMS4gNW6keuqJ6uGsoGkWq8TgZx626QD85VSe4EZwNNKqcs8GFObOq9HDOuz8ykoKW96w+g08zdvp+eDEkKIFvJkIlgFpCmlUpRSdmAm8GHtDbTWKVrrZK11MvAOcIvW+r8ejKlNjUmLobJKs3zXWWYZjU41f/OknUAI0f54LBForSuAWZjeQN8Db2mtNyulblJK3eSp43rT0K6RRIXYeXnZ3uqqroYFR0BIrDQYCyHapbPevP6H0FovABbUWzankW3/x5OxeILdZmHWuB7838dbWLIjl7E9YxvfODoN8nZ5LzghhGgmGVn8A10zqitJUcHc/9Fmluw41njJIKaHVA0JIdolSQQ/UKDNyoOXDeBkcTnX/WclT3zdyMU+Og2KjkFRrncDFEKIs5BE0AbO7xnL8j+OZ2LfeJ5fsof84gZ6EfW4AJQFFv/d+wEKIUQTJBG0kUCblf+d0BNXaQUvLdt75gbx/WD4DbDqechZ5+3whBCiUZII2lCfhDAm9I3nP9/uZvcx15kbjLsHHDHwyR1Q1cQcRUII4UWSCNrYn6b0IcBq4ernVnDgeHHdlcERMPGvcHA1rH3FJ/EJIUR9kgjaWEpMCK/9aiRFZRU8+MmWMzcYeCV0Oxe+uh+KzjIQTQghvEASgQf07hTG1SO68tX3RzlSUFJ3pVJw8T+htBC+vt8n8QkhRG2SCDzkqhFdqazSvLXqwJkr4/rAqJthzStwYKX3gxNCiFokEXhIckwIY3rE8NqK/axv6J4F598NoZ3hk9tRVZVej08IIapJIvCgW8f1oKCknGmzl3LO377m9jfXUVg9U2mgEyb9DQ5vpHPOp74NVAjh1zw615C/G50azYo/XcCH63P4bvdxPlifQ2FpBc9eOwyLRUHfaZA6nu6758FLWxvfUY8LYczvvBa3EMK/SInAw0KDArhmZDf+fdUQ7pnShy+3HOGvn2yhqkqbhuNL/sWJyAFQVdnwoygXvroP9mT6+qMIITooKRF40fXnJrP/eDEvLt3LgePF/PuqoQRHJrNpwL1kZGQ0/KbyUzB7JHxyJ1z6OKDqrrfZofNQk1SEEKIVJBF4kVKK+y7tS/fYEO77cDM3zlvNL8ek8M72MtJHV+AMbOCfIyAYpjwKr18BL05ueMfn/NYMVBNCiFaQROBlSil+NjqZ4AArv39nA0t2mNlI47/czp8v6dvwm3peBDcvB9eRM9etnQfLZ8OgmWY+IyGEaCFJBD5yeXoSEQ47p8oreWfJRl5atpcZwxLpkxDW8Bvi+5pHfQmDYNdCeH2muedBfVGppneSNaBtP4AQosOQxmIfmtA3nqmDOnNFTzvhwQHc+toa9uQWtWwnjiiY/iyEdTajlWs/Tp2AVc/Bd8945gMIIToEKRG0A0674plrhvLrV7O4bPZSXv7FCAYnRTR/Bz0nmkdDXp8Jix4xJQe7s8WxhRZsh+zQugvtDjM6ulplBRxeDw3dnC08EULj6y7TGo5sgoqyuss7DTCN37WVFcHRM7vWhhZsh5IhEBQOleVweEPDxz9bLPUV5UJIjHl+dKs5fn1B4Q2XvoRoKxVlUFpw+rvoYZII2omR3aP58NYxXPufFfzsPyuYc+0whqdEEWD9gYW2yY/A7FHwytRWvX0YwJoGVlz0Nxh9i3n+9s9h68cN7yDAATcvhajup5ctfBgy/3Hmtsnnwc8/Ot0DqqIUnhsPx85MBMMAdv0bblkOb10Hu745+4dpKJbavn0cvv4/+J+P4chmWHBn4/uaPhcGXXn2YwrRGkv+ae5dcucOsHi+4kYSQTvSNdrB678ayZXPfsfVz6/AZlEEWC0M6BLOk1cNoVN4UMt3GpkMt34Hx7a3KqYNGzcwcMDAugtXPAMLH4J+l8Gh9SYJjLoVumfU3a6yFN6/GT69C65+y1zgj22Hb/8FfS6FIT87vW32KpMcNrxpGr4Blv3bJIFJfz/j4r1t9SJ6bZ8Nr0wz03qPuR26jm78g1THsuAPcM3bZ3a3PbHPlJx0JXx0GxQcgpSxMPo3Z+5r8SPwxT2mET84oqnTJ0Tr7FsKxblQkA0RXT1+OEkE7UxipINPfjuGzB25bDtcQHFZJW+tOsAl//6WgYnhAFgUdI91MjAxnEGJESRFOZreaWSyebTC8Rw79MyouzC2pxnb8MplcOo4xPaGCQ803CB9cj98/idzwbYFQe5288v84sfAGXd6ux4Xml/1n94Fm94zy/ZkmoQx6qYzdnsox04vRz6sexUSBsP4e8FibfrD1I+lzgfdbZLD5Efh09+D1Q6XPA7RqWfuJzQe5mbAS5eYtplaBuTlwcGnG4/BGgDn3QFdhjYdq/BfVVWn72KYt1MSgb+KcNiZOqgzDDIXmZnDu/LXj7dwrLAUgPLKKjK351JWae5y9qvzUvjTlD4UlFQQFmRDeXpwWWSyuUiumAOOHnDRw433Shrxazi8CY5uAfIhONKMeaidBMAUf6c9ZQbOVXeT7XaOKQ00ZsIDUHHKlAbOlgSqYzmy2TzIr7su0AlT/w39fwoFByG6R8NJAEx7y6RHYN3rZ3TptZcVgqui8RhO7jPn4ublENCKEp7o+I7vgrJC8zx3J6SO9/ghJRH8CPTqFMqrN4yss6ysoortRwp5bcU+nluyh+W789iSU8AlAzvzrysHY7V4OBkMvso8zsZqg+nN7LUU1weu/6T5MYTEwIwXmr+91QaXNfFrvdqEB86+zchfm0c9WYsWNT5KHEypZ950U7U2YMbZj9OQoAiI7Na69wrvy9sFZS6I6Na8qsTa9zTP2+GpqOqQRPAjZbdZ6N8lnIenDyDQZuXTTYeY2LcTH67PoaS8kulDupDRK45gezN+KQvvSR0P/X4Cy540j1ZRMPM16H1xm4YmPODgGnhunHneZRj8qhmdGnLWmqrL6DTIlUQgmkEpxf1T+3H/VDOqePbCnTz+1Xa+2HKEXvGh3De1L5nbczlWWEqM086s8T0IDZLBZT41fY5pEK9qogqpKd88aNpSumeAPaRNQxNtbP935u+AK2DjW5CfbboxNyVnLXQaaEp91e/3MEkEHcyt43rwyzEpZG4/xp1vr+fq51YQYFXEhQZxuKCEr7ce5f5L+zEoKVwSgq/YAk2Po9YKjjTzTr18KYQmNOst/XKPweHnWn/Mlug5CYZeZ56XumDx32H4L+t2WKgsh4UPE1KSXPe9WsPif0CPCyAx/YfFsfEd2Py+Od8TH4Iw97na9pnpHFDd/bmqCj67CwpyTMeHC/6Ms3A3fLMUxv2p8Qkd83bBimdNm5ctsOFtctaaG1CN/b1JBNs/g+E3nF6//XMTy6ibzevqMTGDrwZHDGx8G8qKzdgdD5JE0AEFBViZ2K8TH3cKY/nuXMb3jic2NJBlu3K55bU1XPsfkxwm9U/g2pFdGZES5fkGZtF2up1jeklt/i+c2NustwSfcsGJFo5ab42SfNi2ABIGmkb1xX83VWA5a+uOEVkxB759jD4hyTD5GtN+A6b78KKHYe2rpttza0s8WsNX95sBgaeOQ1xfGHsnuI7B+zea5YOvNnX2B1fDyrngiDZdoYdcS9f978CxpdBvesNTu2gNH/7GdPNMHQe9GpkQMmctdB4CMWlmupdtn55OBEW58N6NUFFillkDzHkpc0HaRDOgDEyi6NS/deehmSQRdGBdox10jT7d9eyc1Bgy/zCOtftPsnjbMd7JOsBH63NIjQ1hZPdoxqbFcmGfOGw/dBCb8LyxvzePZlp9tkbstnLqJDyVDp/cYX5Nf/c0RKbA3iXmF/rAyyH/oBmzEZmC88QeM3iqz6VQWQZf3GtKDif2Qub/gwvvM/vN22XWx/Ss20OspACCws58fmQT5B8wPcFWv2h+iY+909zbo8TdY2znV6bBftsCsNjg2ndNt+DvPyLq+FqzzbYFphNDaeHpfeftMu/dt/T0Nr0mm7EoZUXmom8NMPHk7YSBV5gE2GuySTilhRAYCl/eByUnzT6ObTWJaNEjkHaR6U59eKNZtycTlKXuaP42Jv/j/UxYUADn94zlL5f2ZcWfLuQfMwbSOSKYj9fncNOrWYx+5BtufW0NH67PQeuzzdkgRD3BETDhr2aA4LzpZlqTX35hfhV/cY+5CH/+J9M+8rP/khc11JQAnhkNc8+H4jy4/GUYdJV7QOF2WPIY/HsoPD3KjGKvtuYV+EcK7F5kksw/UswvbjDVP2Auqr0mQ/Zq2PIBrHvNTNvuiKm17aemlNV5CMT0gm8fw1ZZbBpst39menj9s1fdWD79g2n87TvNVO9kvQxPDDSf4+VLTXXT4Q2ANvsF6DXFJLMNb5m6/3WvmgQIpqfQksdM1dDkv5vEEZ0KlgD4/I9mv7rKY/9sUiLwY8F2K1ekJ3FFehIVlVUs3HaM/647yNp9J/hk4yHeWLGfqZ099+UTHdSgmWYQVNExcxF0xpkBhM+NhzeuMr+kx90Dkcls7nc3YzuXmbvxgbn4dRoAE/4Pti6A939txl2kTTQli5XPmgt34nD44s8moXxyh/n1XVVhRo6nnG9+pXdJN4P/ek02F/N3b4CwRMi42yScrR+bXjnHtsKw/zHH7zUZlj5OpcWOddQtZhR8zlqz7+pYekwwbSDdM0zC2fKBmY4kaZQZjZ75DzM9fHXVTsJg87fbOdD1HPjmr+DsZGK57BnYvRhy1ph99ZwIUSlme3sI/OJzU7IBUyrwEI8mAqXUJOAJwAo8r7V+pN76a4C73C9dwM1a6/WejEk0zGa1MKFvPBP6xlNVpXl95X7+/tlWVu2p4FDgdm7JSCUoQLqiimZQCpLPrbusy1BI/wWs/o+pKz/ntwBUWQOhTwMN5844uODP5gJrd8KlT0BILOxZbKYACe1k6tInPmRKGspi5r/6/I9m5HjOGhj/Z7Ov+P4QnmQuqJMfMRfYXpNN6eBNd6N2z0nmb68psPRxTkQOIqb/T+Dbx8zxR90Mi/4GASEmlvAuZvu0CaCs5tf6pU9AbC/Y+y18+WcIjjIXe2fs6fNy8T9hzhgzM/CVr5oqooRBZjR9yUno9ee65yFxmHl4mMcSgVLKCswGJgDZwCql1Ida6y21NtsDnK+1PqGUmgzMBUaeuTfhTRaL4tpR3bioXyd+88JCnvx6B++tyaZ7rJPEyGCuGt6V/l3CpIFZtMwFf4bCw3Dub5s3qjr9F6Yffur401N5THsaPrkdyovN6O4RvzIlD0e06QVUWgCb3jUX1wGXm/coBefeBse2Qe9LzLLU8dBtDBQdhYFXnv4VnpgOfadx0DaUmPj+ZsxHn0tNFdDJ/dB93OkkAGYa+FE3m/jieptllz4BH9xiqsEGX1P3M8X3NXEf3306ls6DTRsKypR8fMCTJYIRwE6t9W4ApdR8YBpQkwi01stqbf8dcJYOtsKbYkMDuWlQELOm9OeZxTvJP1XOqj3HeX3FfiIdAQxIjGBwYjjXjU4mNrSR7nNCVAuOhKteb/72FuuZo9ITh8GvF9ddVnskeMbd5lHfiF/VfW0PaXgUu8UKV7zCiUWLTAK5/MXT6xoblX7RQ3Vfx/aEG75qeFuAkTfWfV3dhpA00mvTTtfnyUTQBThQ63U2Tf/a/yXwqQfjEa00Ji2GMWnmC5p/qpxPNx5i7f6TbDiYz+xFu3g7K5uHpw8gMsTcSyAuNJDOEcG+DFmIH48u7qofH44UV57qGaKUuhy4SGt9g/v1dcAIrfUZ8/oqpcYBTwNjtNZ5Day/EbgRID4+ftj8+fNbFZPL5cLpbPnNWTytvcYFZ49tX0ElT6wp5XjJ6e+RRcEFXW0UlGqOFGtGJdg4P8lGsK3tqpLa6zmTuFpG4jLC8rdSGJqKtjQ9yPOHxDVu3LgsrXWDo/Q8WSLIBpJqvU4EcupvpJQaCDwPTG4oCQBoredi2g9IT0/Xre0PvchbfalbqL3GBc2L7bILy1lz4ARo0Gi+3HKEN1YeIDTIRkqMk/nb8ll9ws7N56fybOYuhnWL4v6pfQm0tb7xub2eM4mrZSSuas07lqfi8mQiWAWkKaVSgIPATODq2hsopboC7wHXaa1bd+cU4XPhjgDG9To9rfT43vH89oI0woMDcNhtfLsjl1lvrOGOt9fTKSyIN1buJ2vfcZIiHYxJi+G6Ud1kEJsQPuSxRKC1rlBKzQI+x3QffUFrvVkpdZN7/RzgL0A08LS7B0pFY0UX8eOSEH66jWBMWgwfzRrDsl25XDakC19tOcpzS3azN6+Ir7ce5a3V2fzqvBRinIEcLSzl4gEJMmuqEF7k0XEEWusFwIJ6y+bUen4DcEP994mOJynKwZVRZrqLiwcmcPHABLTWLNh4mH9+sY3b3zo9fOTxr7bzyE8G1jRQCyE8S0YWC59RSnHxwASmDOjEyj3HqajSVGnNAx9t4WcvrOCXY1LYdsRFWJCNuyf3JjHSszMwCuGvJBEIn1NKMbJ7dM3rD2edyx1vree5JXvoHB7EieJyvthyhMFJEQxKDCctPpTM7WWsr9jB5emJ0lVViB9IEoFodxx2G7OvHsr+48UkRTk4XFDCc5m7WXvgJC8v20dZZRVWBZ/s2c4TX2/nZ6OTuX1iT8Lk/gpCtIokAtEuWSyK5BgzF32XiOCaO7CVVVRx4EQxuzasos+QkTybuYuXl+/lm61HefPXo+o0UgshmkcSgfhRsdsspMY6OWBVJEU5ePCyAUwb3IXrX1zFzLnf0btTKIUlFQzoEs7MEV1JiZFbOQpxNtJ5W/zoDU+O4qXrh3OqrJJthwtxlVbw4tK9XPR4JrMX7pT7KghxFlIiEB1CenIUK++5sOb10YISHvhoC49+vo2CknLuntRbZksVohGSCESHFBcWxFNXDyHqAzvPLt7NC9/uIS40iD9N6cOUAZ0kKQhRiyQC0WEppXhgaj96J4SSfeIUmduPcevra4gLDWR0ajT3XdqPSEcApRVVctMd4dckEYgOzWJRXDOyGwB3TOjJ+2sPsmxXHp9sPETWvhOEBQWwJ7eI+TeOYlBShG+DFcJHJBEIv2GzWrg8PYnL05P4+TnJ/G7+Wqq0JtIRwK/nZTFtcGfWHjjJRf06MaxbJNWVR92iHUQ47D6NXQhPkkQg/NLgpAgW/X4cAFtyCvjpM8t4/ts9pMSE8NePt9TZ1m6zMKV/J0anRjMwMYLusSHszS3mWGFpzTbdoh0kRckUGOLHSRKB8Ht9O4fx6W3nEWy3Eh8WxLbDhRw8WQxAZRVkbj/Gh+tz+O+6M26nUSPAqrhqRFdWbD3Fs9u/4x8zBkpiED8akgiEgJpRzAC9OoXSq1NozesJfeN5YGo/9uYVsSE7n93HXKTEhtAlwoFSUFWleWt1Nq8s30ecQ3EoJ5+pT33L0K6RgLn1bXJ0CJEhdrYfKWRAl3BmDEuU6ibRbkgiEKIZLBZF91gn3WMbvk3gyO7R3HNxH9atXEpy/+Hc9+FmjhSWAFBRqcnckUtZRRWxoYF8sC6Hf3+zk9duGEn2iWI+3nCI316QRveYEE6eKic6xE6VhqKyCpk/SXiFJAIh2khUiB2LMglj3i9H1llXXllFcVkl4cEBbDqYz6/nZXHls8spKqtEKfhs02FsVkVJeRXRIXaKyyoprajkZ6OTmTEskdAgG92iZboM4RmSCITwggCrhfBgM6NL/y7hvP6rkVz/4ipGpETxuwt78vyS3VRp6Bxh2ihCAm2UVlTx8vK9vLRsL2Bu6HN+WiyH8ku4YniiTLAn2owkAiF8oFt0CN/cmVHz+t5L+ja43S/OTWZ3bhGbcwqYs3gXn2w4BMDczF1c1K8TjkArPxudTHJ0CJtz8unXORy7TaYQEy0jiUCIdiwtPpS0+FAu6teJa0d2paCkggCr4qFPvue73XmcKC5n/soDhAcHkFdURo84JzeMSQFg6+FCLEpxxfDEOvvUWlOlwWqRaTaEIYlAiB+JuLAg4sLM87k/Swcgz1XKY19u50RxGaO7R/Ns5m7ufm8jAMEBViq15oWle4gMVPTavpyqKthxtJDySs1Phnbh2lHd6Bkfiqu0AkeAFYskB78kiUCIH7FoZyAPTR9Q8/rK4V05lH8Ki1J0jgim4FQ5H23IYcHKrVRWaRSKCX3jKa/UzF91gFeW7yM+LJAjBaUkRQVzfs9Ydh8rIiwogHN7RHPNyG5YLIpTZZV8vvkwqbFOBiSG+/ATC0+QRCBEB2K3Wer0LooMsfOz0cl0Ld1LRsY5dbb98yV9eXv1AbYcKqB7jJNlu3J5e3U2vTqZSfo+23yYvKIyUmJC+MsHm8k/VY7DbuWVX4wgPTnK2x9NeJAkAiH8VFSInV+fn1rz+rYL09Bao5RCa80db6/n8a92AJDeLZJfn5/Kwwu+59r/rGBU92gSwoMIsFroGR9Kl4hgAm0WhiVHcrSglNdX7qdTWBDnpEaTFh/aWAiinZBEIISoUX2fBqUUD08fQMGpchIjHfxpSh/sNgv9u4TxxFc7WLv/JJtzCigpq6SwtKLm/dVjIEoqKqm+MdzAxHBOFpdTXFbJgC5hNfM1VVZpvt5Wxjs5a7AoxZQBCVzUL54qDY9+vo2PN+Rw+4SeTB/S5Yz7R1RVaZSizvKqKi1tHK0kiUAI0aCgACvP/3x4nWUJ4cE88tOBNa+11mSfOEWuq5TjRWW8uyabQJuVOy/qhQIWbDzExxsO0a9zGA67jY0HT7J4+zGq3EnCqiApKp+isko+XJ/DoKQIyiuq2HKogC4Rwdz+1noeXvA9vTqFYrOYbrFFpRVsOVSA3WahT6cwAmwWso8XszeviJSYEAYlRtC3cxjBdnOPiZPF5by7Jpui0gp+OjSRLpHBHCssZdvhQhLCgxmREsUFfeLYdczFq9/t47NNh+kbUcXoMZUE2qzsyS3i882H2XgwH1dJBdEhdu6e3Bu7zcIXm49QXlVFereommlJSsorWbDxEKfKK+nfOfyM6c13HCnkaGEpo7pH1+m5VVFZhcaMOWlIQUk5rjLP3HZVEoEQotWUUiRFnZ559YI+8XXW33Bed244r3udZUWlFRzKL0Ep2LVhFRMvGEd5ZRUvL9vLJxsPERhg4eHpA5g5PIkP1h/k2x157M51UaUrAQi0Wrh8WCKlFVVsP1JIcXklqXFOJvSNZ9cxF0t25vLe2oN1jjmsWyTdohw8s3gXWpv5n7pGOVi47SgvLN1DaJCNwpIK7DYLI1OiyNyRy4THMgkNsrE5pwCAxMhgop2BrNxznKW7cimv1BwvKgNMV9zrRnXj/J6xPP7VdtZn59cc+ydDu9DTXT22/3gx81fup0pDl4hgrhnVlbjQIN7NymbdgZMEWBV/mNSbc1KjcQbaiAqx8/Lyfby+Yh+7jhVxaWoAl7Thv181SQRCCK8KCbTRI87M2XTAan4RB1gtDSaN6UMSmT4k8Yx9NEVrzYnicioqqwBzH4qoEDPBX/6pckrLK3EG2XDYbVRUVpG54xgfrT9En4RQLh+WRGSInf83/ys2FodQpTW3T+jJFelJdAoPAsy05Te9mkVSZCAv/M9wokPszFm8q2YUeIjdytPXDGVwUgQvL9vLC0v3UF5pfslbFFw1oisju0fzxor9/OOzbQB0jwnhyuFJbD1cwL3/3VTzWRx2K8VllYxIieKywV0Ice1v6eluFkkEQogORSlVc+GvLzw4AIJPT+Rns1oY3zue8b3rlmTSO9m4M2NEg/vo2zmMRXdm1GmPeGj6AH5/US82HSwgOcZBYqQpIf1xSh/umNiLSnddmMUCgTZTZTV1UGd2HnWRf6qMoV0jaxrpv92Zy/GiMo4XlbH9SCFj02KZ1N/cZ3vRooNnBtQGJBEIIUQLNdQoHeGwMyYt5ozlTU35UV0yqqaU4ry02B8eYAvJpCRCCOHnJBEIIYSfk0QghBB+zqOJQCk1SSm1TSm1Uyl1dwPrlVLqSff6DUqpoZ6MRwghxJk8lgiUUlZgNjAZ6AtcpZSqP+n6ZCDN/bgReMZT8QghhGiYJ0sEI4CdWuvdWusyYD4wrd4204BXtPEdEKGUSvBgTEIIIepRWntmyLJSagYwSWt9g/v1dcBIrfWsWtt8DDyitf7W/fpr4C6t9ep6+7oRU2IgPj5+2Pz581sVk8vlwuls+ObjvtRe44L2G5vE1TISV8t0xLjGjRuXpbVOb2idJ8cRNDT7U/2s05xt0FrPBeYCpKen64yMjFYFtGjRIlr7Xk9qr3FB+41N4moZiatl/C0uTyaCbCCp1utEIKcV29SRlZWVq5Ta18qYYoDcVr7Xk9prXNB+Y5O4WkbiapmOGFe3xlZ4MhGsAtKUUinAQWAmcHW9bT4EZiml5gMjgXyt9aGmdqq1bvWwO6XU6saKRr7UXuOC9hubxNUyElfL+FtcHksEWusKpdQs4HPACrygtd6slLrJvX4OsACYAuwEioHrPRWPEEKIhnl0riGt9QLMxb72sjm1nmvgVk/GIIQQomn+NrJ4rq8DaER7jQvab2wSV8tIXC3jV3F5rPuoEEKIHwd/KxEIIYSoRxKBEEL4Ob9JBGebAM+LcSQppRYqpb5XSm1WSt3mXn6/UuqgUmqd+zHFB7HtVUptdB9/tXtZlFLqS6XUDvffSC/H1KvWOVmnlCpQSv3OF+dLKfWCUuqoUmpTrWWNnh+l1B/d37dtSqmLvBzXo0qpre7JHN9XSkW4lycrpU7VOm9zGt2xZ+Jq9N/Nx+frzVox7VVKrXMv9+b5auza4PnvmNa6wz8w3Vd3Ad0BO7Ae6OujWBKAoe7nocB2zKR89wN3+vg87QVi6i37B3C3+/ndwN99/O94GDMwxuvnCxgLDAU2ne38uP9N1wOBQIr7+2f1YlwTAZv7+d9rxZVcezsfnK8G/918fb7qrf8n8BcfnK/Grg0e/475S4mgORPgeYXW+pDWeo37eSHwPdDFF7E00zTgZffzl4HLfBcKFwC7tNatHVn+g2itM4Hj9RY3dn6mAfO11qVa6z2YsTIN3wTXA3Fprb/QWle4X36HGbXvVY2cr8b49HxVU0op4ArgDU8cuylNXBs8/h3zl0TQBThQ63U27eDiq5RKBoYAK9yLZrmL8i94uwrGTQNfKKWy3BP9AcRr92hv9984H8RVbSZ1/4P6+nxB4+enPX3nfgF8Wut1ilJqrVJqsVLqPB/E09C/W3s5X+cBR7TWO2ot8/r5qndt8Ph3zF8SQbMmt/MmpZQTeBf4nda6AHMvhlRgMHAIUzz1tnO11kMx94m4VSk11gcxNEgpZQemAm+7F7WH89WUdvGdU0rdA1QAr7kXHQK6aq2HALcDryulwrwYUmP/bu3ifAFXUffHhtfPVwPXhkY3bWBZq86ZvySCFk9u50lKqQDMP/RrWuv3ALTWR7TWlVrrKuA5PFQsborWOsf99yjwvjuGI8p9jwj336PejsttMrBGa33EHaPPz5dbY+fH5985pdTPgUuAa7S7UtldjZDnfp6FqVfu6a2Ymvh3aw/nywb8BHizepm3z1dD1wa88B3zl0RQMwGe+5flTMyEd17nroP8D/C91vqxWstr35BnOrCp/ns9HFeIUiq0+jmmsXET5jz93L3Zz4EPvBlXLXV+qfn6fNXS2Pn5EJiplApUZuLFNGClt4JSSk0C7gKmaq2Lay2PVebugSilurvj2u3FuBr7d/Pp+XK7ENiqtc6uXuDN89XYtQFvfMe80RreHh6Yye22YzL6PT6MYwym+LYBWOd+TAHmARvdyz8EErwcV3dMD4T1wObqcwREA18DO9x/o3xwzhxAHhBea5nXzxcmER0CyjG/xn7Z1PkB7nF/37YBk70c105M/XH1d2yOe9ufuv991wNrgEu9HFej/26+PF/u5S8BN9Xb1pvnq7Frg8e/YzLFhBBC+Dl/qRoSQgjRCEkEQgjh5yQRCCGEn5NEIIQQfk4SgRBC+DlJBELUo5SqVHVnPG2z2Wrds1n6asyDEA3y6D2LhfiROqW1HuzrIITwFikRCNFM7nnq/66UWul+9HAv76aU+to9kdrXSqmu7uXxytwLYL37cY57V1al1HPuOee/UEoF++xDCYEkAiEaElyvaujKWusKtNYjgKeAx93LngJe0VoPxEzu9qR7+ZPAYq31IMz895vdy9OA2VrrfsBJzOhVIXxGRhYLUY9SyqW1djawfC8wXmu92z052GGtdbRSKhczVUK5e/khrXWMUuoYkKi1Lq21j2TgS611mvv1XUCA1vpBL3w0IRokJQIhWkY38ryxbRpSWut5JdJWJ3xMEoEQLXNlrb/L3c+XYWa0BbgG+Nb9/GvgZgCllNXL8/4L0WzyS0SIMwUr983L3T7TWld3IQ1USq3A/Ii6yr3st8ALSqnfA8eA693LbwPmKqV+ifnlfzNm1ksh2hVpIxCimdxtBOla61xfxyJEW5KqISGE8HNSIhBCCD8nJQIhhPBzkgiEEMLPSSIQQgg/J4lACCH8nCQCIYTwc/8fUtyhVesZRm8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "metrics = pd.read_csv(f\"{trainer.logger.log_dir}/metrics.csv\")\n",
    "\n",
    "aggreg_metrics = []\n",
    "agg_col = \"epoch\"\n",
    "for i, dfg in metrics.groupby(agg_col):\n",
    "    agg = dict(dfg.mean())\n",
    "    agg[agg_col] = i\n",
    "    aggreg_metrics.append(agg)\n",
    "\n",
    "df_metrics = pd.DataFrame(aggreg_metrics)\n",
    "df_metrics[[\"train_loss\", \"valid_loss\"]].plot(\n",
    "    grid=True, legend=True, xlabel='Epoch', ylabel='Loss')\n",
    "df_metrics[[\"train_mae\", \"valid_mae\"]].plot(\n",
    "    grid=True, legend=True, xlabel='Epoch', ylabel='MAE')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "4e41d562",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Restoring states from the checkpoint path at logs/mlp-corn-cement/version_3/checkpoints/epoch=194-step=974.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from checkpoint at logs/mlp-corn-cement/version_3/checkpoints/epoch=194-step=974.ckpt\n",
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:258: PossibleUserWarning: The dataloader, test_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b2791a16b2994d48aac25c9a1727c9db",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">        Test metric        </span>┃<span style=\"font-weight: bold\">       DataLoader 0        </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test_mae          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">           0.25            </span>│\n",
       "└───────────────────────────┴───────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test_mae         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m          0.25           \u001b[0m\u001b[35m \u001b[0m│\n",
       "└───────────────────────────┴───────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'test_mae': 0.25}]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.test(model=lightning_model, datamodule=data_module, ckpt_path='best')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}