tutorials/usage/tutorial04_prediction.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# User Guide Tutorial 04: Prediction\n",
    "\n",
    "This tutorial shows how to use TemporAI `prediction` plugins."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Skip the below cell if you are not on Google Colab / already have TemporAI installed:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install temporai\n",
    "\n",
    "# Or from the repo, for the latest version:\n",
    "# %pip install git+https://github.com/vanderschaarlab/temporai.git"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## All `prediction` plugins\n",
    "\n",
    "To see all the relevant plugins:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">{</span>\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'one_off'</span>: <span style=\"font-weight: bold\">{</span>\n",
       "        <span style=\"color: #008000; text-decoration-color: #008000\">'classification'</span>: <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'cde_classifier'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'ode_classifier'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'nn_classifier'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'laplace_ode_classifier'</span><span style=\"font-weight: bold\">]</span>,\n",
       "        <span style=\"color: #008000; text-decoration-color: #008000\">'regression'</span>: <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'laplace_ode_regressor'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'nn_regressor'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'ode_regressor'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'cde_regressor'</span><span style=\"font-weight: bold\">]</span>\n",
       "    <span style=\"font-weight: bold\">}</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'temporal'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'classification'</span>: <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'seq2seq_classifier'</span><span style=\"font-weight: bold\">]</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'regression'</span>: <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'seq2seq_regressor'</span><span style=\"font-weight: bold\">]}</span>\n",
       "<span style=\"font-weight: bold\">}</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m{\u001b[0m\n",
       "    \u001b[32m'one_off'\u001b[0m: \u001b[1m{\u001b[0m\n",
       "        \u001b[32m'classification'\u001b[0m: \u001b[1m[\u001b[0m\u001b[32m'cde_classifier'\u001b[0m, \u001b[32m'ode_classifier'\u001b[0m, \u001b[32m'nn_classifier'\u001b[0m, \u001b[32m'laplace_ode_classifier'\u001b[0m\u001b[1m]\u001b[0m,\n",
       "        \u001b[32m'regression'\u001b[0m: \u001b[1m[\u001b[0m\u001b[32m'laplace_ode_regressor'\u001b[0m, \u001b[32m'nn_regressor'\u001b[0m, \u001b[32m'ode_regressor'\u001b[0m, \u001b[32m'cde_regressor'\u001b[0m\u001b[1m]\u001b[0m\n",
       "    \u001b[1m}\u001b[0m,\n",
       "    \u001b[32m'temporal'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'classification'\u001b[0m: \u001b[1m[\u001b[0m\u001b[32m'seq2seq_classifier'\u001b[0m\u001b[1m]\u001b[0m, \u001b[32m'regression'\u001b[0m: \u001b[1m[\u001b[0m\u001b[32m'seq2seq_regressor'\u001b[0m\u001b[1m]\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[1m}\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tempor import plugin_loader\n",
    "from rich.pretty import pprint\n",
    "\n",
    "all_prediction_plugins = plugin_loader.list()[\"prediction\"]\n",
    "\n",
    "pprint(all_prediction_plugins, indent_guides=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now also load data source(s) we will use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SineDataSource = plugin_loader.get_class(\"prediction.one_off.sine\", plugin_type=\"datasource\")\n",
    "DummyTemporalPredictionDataSource = plugin_loader.get_class(\n",
    "    \"prediction.temporal.dummy_prediction\", plugin_type=\"datasource\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a one-off prediction plugin\n",
    "\n",
    "One-off prediction is the task of predicting a single value for each sample (may be classification or regression)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OneOffPredictionDataset(\n",
      "    time_series=TimeSeriesSamples([100, *, 5]),\n",
      "    static=StaticSamples([100, 4]),\n",
      "    predictive=OneOffPredictionTaskData(targets=StaticSamples([100, 1]))\n",
      ")\n",
      "NeuralNetClassifier(\n",
      "    name='nn_classifier',\n",
      "    category='prediction.one_off.classification',\n",
      "    plugin_type='method',\n",
      "    params={\n",
      "        'n_static_units_hidden': 100,\n",
      "        'n_static_layers_hidden': 2,\n",
      "        'n_temporal_units_hidden': 102,\n",
      "        'n_temporal_layers_hidden': 2,\n",
      "        'n_iter': 50,\n",
      "        'mode': 'RNN',\n",
      "        'n_iter_print': 10,\n",
      "        'batch_size': 100,\n",
      "        'lr': 0.001,\n",
      "        'weight_decay': 0.001,\n",
      "        'window_size': 1,\n",
      "        'device': None,\n",
      "        'dataloader_sampler': None,\n",
      "        'dropout': 0,\n",
      "        'nonlin': 'relu',\n",
      "        'random_state': 0,\n",
      "        'clipping_value': 1,\n",
      "        'patience': 20,\n",
      "        'train_ratio': 0.8\n",
      "    }\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "from tempor import plugin_loader\n",
    "\n",
    "dataset = SineDataSource(random_state=42).load()\n",
    "print(dataset)\n",
    "\n",
    "model = plugin_loader.get(\"prediction.one_off.classification.nn_classifier\", n_iter=50)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p><span style=\"font-family: monospace;\">StaticSamples</span> with data:</p><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sample_idx</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "StaticSamples with data:\n",
       "            0\n",
       "sample_idx   \n",
       "0           0\n",
       "1           1\n",
       "2           0\n",
       "3           0\n",
       "4           1\n",
       "...        ..\n",
       "95          1\n",
       "96          1\n",
       "97          1\n",
       "98          0\n",
       "99          0\n",
       "\n",
       "[100 rows x 1 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Targets:\n",
    "dataset.predictive.targets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-16 22:18:45 | INFO     | tempor.models.ts_model:_train:388 | Epoch:0| train loss: 0.68824702501297, validation loss: 0.6883962154388428\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-16 22:18:45 | INFO     | tempor.models.ts_model:_train:388 | Epoch:10| train loss: 0.6999298930168152, validation loss: 0.6879696249961853\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-16 22:18:45 | INFO     | tempor.models.ts_model:_train:388 | Epoch:20| train loss: 0.6818333268165588, validation loss: 0.6867876052856445\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-16 22:18:45 | INFO     | tempor.models.ts_model:_train:388 | Epoch:30| train loss: 0.694374680519104, validation loss: 0.6851812601089478\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-16 22:18:45 | INFO     | tempor.models.ts_model:_train:388 | Epoch:40| train loss: 0.6282318234443665, validation loss: 0.6936542391777039\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "NeuralNetClassifier(\n",
       "    name='nn_classifier',\n",
       "    category='prediction.one_off.classification',\n",
       "    plugin_type='method',\n",
       "    params={\n",
       "        'n_static_units_hidden': 100,\n",
       "        'n_static_layers_hidden': 2,\n",
       "        'n_temporal_units_hidden': 102,\n",
       "        'n_temporal_layers_hidden': 2,\n",
       "        'n_iter': 50,\n",
       "        'mode': 'RNN',\n",
       "        'n_iter_print': 10,\n",
       "        'batch_size': 100,\n",
       "        'lr': 0.001,\n",
       "        'weight_decay': 0.001,\n",
       "        'window_size': 1,\n",
       "        'device': None,\n",
       "        'dataloader_sampler': None,\n",
       "        'dropout': 0,\n",
       "        'nonlin': 'relu',\n",
       "        'random_state': 0,\n",
       "        'clipping_value': 1,\n",
       "        'patience': 20,\n",
       "        'train_ratio': 0.8\n",
       "    }\n",
       ")"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train.\n",
    "model.fit(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p><span style=\"font-family: monospace;\">StaticSamples</span> with data:</p><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feat_0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sample_idx</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "StaticSamples with data:\n",
       "            feat_0\n",
       "sample_idx        \n",
       "0              0.0\n",
       "1              1.0\n",
       "2              1.0\n",
       "3              0.0\n",
       "4              0.0\n",
       "...            ...\n",
       "95             1.0\n",
       "96             0.0\n",
       "97             1.0\n",
       "98             0.0\n",
       "99             1.0\n",
       "\n",
       "[100 rows x 1 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict:\n",
    "\n",
    "model.predict(dataset)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a temporal prediction plugin\n",
    "\n",
    "Temporal prediction is the task of predicting a time series for each sample (may be classification or regression)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TemporalPredictionDataset(\n",
      "    time_series=TimeSeriesSamples([100, *, 5]),\n",
      "    static=StaticSamples([100, 3]),\n",
      "    predictive=TemporalPredictionTaskData(\n",
      "        targets=TimeSeriesSamples([100, *, 2])\n",
      "    )\n",
      ")\n",
      "Seq2seqRegressor(\n",
      "    name='seq2seq_regressor',\n",
      "    category='prediction.temporal.regression',\n",
      "    plugin_type='method',\n",
      "    params={\n",
      "        'encoder_rnn_type': 'LSTM',\n",
      "        'encoder_hidden_size': 100,\n",
      "        'encoder_num_layers': 1,\n",
      "        'encoder_bias': True,\n",
      "        'encoder_dropout': 0.0,\n",
      "        'encoder_bidirectional': False,\n",
      "        'encoder_nonlinearity': None,\n",
      "        'encoder_proj_size': None,\n",
      "        'decoder_rnn_type': 'LSTM',\n",
      "        'decoder_hidden_size': 100,\n",
      "        'decoder_num_layers': 1,\n",
      "        'decoder_bias': True,\n",
      "        'decoder_dropout': 0.0,\n",
      "        'decoder_bidirectional': False,\n",
      "        'decoder_nonlinearity': None,\n",
      "        'decoder_proj_size': None,\n",
      "        'adapter_hidden_dims': [50],\n",
      "        'adapter_out_activation': 'Tanh',\n",
      "        'predictor_hidden_dims': [],\n",
      "        'predictor_out_activation': None,\n",
      "        'max_len': None,\n",
      "        'optimizer_str': 'Adam',\n",
      "        'optimizer_kwargs': {'lr': 0.01, 'weight_decay': 1e-05},\n",
      "        'batch_size': 32,\n",
      "        'epochs': 10,\n",
      "        'padding_indicator': -999.0\n",
      "    }\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "from tempor import plugin_loader\n",
    "\n",
    "dataset = DummyTemporalPredictionDataSource(random_state=42, temporal_covariates_missing_prob=0.0).load()\n",
    "print(dataset)\n",
    "\n",
    "model = plugin_loader.get(\"prediction.temporal.regression.seq2seq_regressor\", epochs=10)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p><span style=\"font-family: monospace;\">TimeSeriesSamples</span> with data:</p><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sample_idx</th>\n",
       "      <th>time_idx</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>-3.110475</td>\n",
       "      <td>-3.566948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.528495</td>\n",
       "      <td>-0.653673</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.275307</td>\n",
       "      <td>-0.695371</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.844060</td>\n",
       "      <td>3.469371</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.420301</td>\n",
       "      <td>5.147500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">99</th>\n",
       "      <th>7</th>\n",
       "      <td>5.994185</td>\n",
       "      <td>6.225290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10.913662</td>\n",
       "      <td>5.346697</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9.558824</td>\n",
       "      <td>7.585175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10.194430</td>\n",
       "      <td>5.795619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>13.774189</td>\n",
       "      <td>8.457336</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1573 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "TimeSeriesSamples with data:\n",
       "                             0         1\n",
       "sample_idx time_idx                     \n",
       "0          0         -3.110475 -3.566948\n",
       "           1          1.528495 -0.653673\n",
       "           2          2.275307 -0.695371\n",
       "           3          4.844060  3.469371\n",
       "           4          4.420301  5.147500\n",
       "...                        ...       ...\n",
       "99         7          5.994185  6.225290\n",
       "           8         10.913662  5.346697\n",
       "           9          9.558824  7.585175\n",
       "           10        10.194430  5.795619\n",
       "           11        13.774189  8.457336\n",
       "\n",
       "[1573 rows x 2 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Targets:\n",
    "dataset.predictive.targets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing data for decoder training...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing data for decoder training DONE.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Training stage: 1. Train encoder ===\n",
      "Epoch: 0, Loss: 68.64124755859375\n",
      "Epoch: 1, Loss: 35.376345291137696\n",
      "Epoch: 2, Loss: 21.269977149963378\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 3, Loss: 17.418056030273437\n",
      "Epoch: 4, Loss: 13.851958045959472\n",
      "Epoch: 5, Loss: 8.335217628479004\n",
      "Epoch: 6, Loss: 6.385376873016358\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 7, Loss: 5.1125048828125\n",
      "Epoch: 8, Loss: 4.690582141876221\n",
      "Epoch: 9, Loss: 4.15880220413208\n",
      "=== Training stage: 2. Train decoder ===\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0, Loss: 29.110622051453316\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 1, Loss: 4.4235033275153635\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 2, Loss: 3.954428958957566\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 3, Loss: 3.8989672662446226\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 4, Loss: 3.8317033024119103\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 5, Loss: 3.8341090269664964\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 6, Loss: 3.717077842797864\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 7, Loss: 3.7477998285170564\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 8, Loss: 3.859872080239\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 9, Loss: 3.8217773019903305\n"
     ]
    }
   ],
   "source": [
    "# Train.\n",
    "model.fit(dataset);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p><span style=\"font-family: monospace;\">TimeSeriesSamples</span> with data:</p><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sample_idx</th>\n",
       "      <th>time_idx</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">0</th>\n",
       "      <th>11</th>\n",
       "      <td>10.618200</td>\n",
       "      <td>8.973257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10.728621</td>\n",
       "      <td>9.108867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10.735380</td>\n",
       "      <td>9.118745</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>10.728010</td>\n",
       "      <td>9.093669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>10.738855</td>\n",
       "      <td>9.097306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">99</th>\n",
       "      <th>17</th>\n",
       "      <td>12.021331</td>\n",
       "      <td>10.417010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>12.091549</td>\n",
       "      <td>10.484824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>12.144796</td>\n",
       "      <td>10.523143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>12.133350</td>\n",
       "      <td>10.510198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>12.097664</td>\n",
       "      <td>10.484853</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "TimeSeriesSamples with data:\n",
       "                             0          1\n",
       "sample_idx time_idx                      \n",
       "0          11        10.618200   8.973257\n",
       "           12        10.728621   9.108867\n",
       "           13        10.735380   9.118745\n",
       "           14        10.728010   9.093669\n",
       "           15        10.738855   9.097306\n",
       "...                        ...        ...\n",
       "99         17        12.021331  10.417010\n",
       "           18        12.091549  10.484824\n",
       "           19        12.144796  10.523143\n",
       "           20        12.133350  10.510198\n",
       "           21        12.097664  10.484853\n",
       "\n",
       "[1000 rows x 2 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict:\n",
    "\n",
    "model.predict(dataset, n_future_steps=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🎉 Congratulations!\n",
    "\n",
    "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the movement towards *Machine learning and AI for Medicine*, you can do so in the following ways!\n",
    "\n",
    "\n",
    "\n",
    "### ⭐ Star [TemporAI](https://github.com/vanderschaarlab/temporai) on GitHub\n",
    "\n",
    "- The easiest way to help our community is by just starring the repos! This helps raise awareness of the tools we're building.\n",
    "\n",
    "\n",
    "\n",
    "### Check out other projects from [vanderschaarlab](https://github.com/vanderschaarlab)\n",
    "- 📝 [HyperImpute](https://github.com/vanderschaarlab/hyperimpute)\n",
    "- 📊 [AutoPrognosis](https://github.com/vanderschaarlab/autoprognosis)\n",
    "- 🤖 [SynthCity](https://github.com/vanderschaarlab/synthcity)\n",
    " "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "RE_py38_temporai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}