tutorials/data/tutorial04_data_splitting.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Tutorial 04: Data splitting\n",
    "\n",
    "This tutorial shows how a `Dataset` can be split or KFold-ed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Skip the below cell if you are not on Google Colab / already have TemporAI installed:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install temporai\n",
    "\n",
    "# Or from the repo, for the latest version:\n",
    "# %pip install git+https://github.com/vanderschaarlab/temporai.git"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Splitting data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OneOffPredictionDataset(\n",
       "    time_series=TimeSeriesSamples([100, *, 5]),\n",
       "    static=StaticSamples([100, 4]),\n",
       "    predictive=OneOffPredictionTaskData(targets=StaticSamples([100, 1]))\n",
       ")"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get a dataset.\n",
    "\n",
    "from tempor import plugin_loader\n",
    "\n",
    "sine_dataset = plugin_loader.get(\"prediction.one_off.sine\", plugin_type=\"datasource\").load()\n",
    "\n",
    "sine_dataset"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The method `train_test_split` (same API as [sklearn.model_selection.train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)) can be used to split the dataset into train and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set:\n",
      "OneOffPredictionDataset(\n",
      "    time_series=TimeSeriesSamples([80, *, 5]),\n",
      "    static=StaticSamples([80, 4]),\n",
      "    predictive=OneOffPredictionTaskData(targets=StaticSamples([80, 1]))\n",
      ")\n",
      "Test set:\n",
      "OneOffPredictionDataset(\n",
      "    time_series=TimeSeriesSamples([20, *, 5]),\n",
      "    static=StaticSamples([20, 4]),\n",
      "    predictive=OneOffPredictionTaskData(targets=StaticSamples([20, 1]))\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "sine_dataset_train, sine_dataset_test = sine_dataset.train_test_split(test_size=0.2)\n",
    "\n",
    "print(\"Training set:\")\n",
    "print(sine_dataset_train)\n",
    "print(\"Test set:\")\n",
    "print(sine_dataset_test)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Using `KFold` and other `sklearn` splitter classes\n",
    "\n",
    "Any of the [sklearn splitter classes](https://scikit-learn.org/stable/modules/classes.html#splitter-classes) can be used with `Dataset` `split` method.\n",
    "\n",
    "The following example illustrates this with the [`KFold` splitter](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html#sklearn.model_selection.KFold):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "\n",
    "kfold = KFold(n_splits=5, random_state=123, shuffle=True)\n",
    "\n",
    "splits = list(sine_dataset.split(splitter=kfold))\n",
    "\n",
    "print(len(splits))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0th split training set:\n",
      "OneOffPredictionDataset(\n",
      "    time_series=TimeSeriesSamples([80, *, 5]),\n",
      "    static=StaticSamples([80, 4]),\n",
      "    predictive=OneOffPredictionTaskData(targets=StaticSamples([80, 1]))\n",
      ")\n",
      "0th split test set:\n",
      "OneOffPredictionDataset(\n",
      "    time_series=TimeSeriesSamples([20, *, 5]),\n",
      "    static=StaticSamples([20, 4]),\n",
      "    predictive=OneOffPredictionTaskData(targets=StaticSamples([20, 1]))\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "train_set_0, test_set_0 = splits[0]\n",
    "\n",
    "print(\"0th split training set:\")\n",
    "print(sine_dataset_train)\n",
    "print(\"0th split test set:\")\n",
    "print(sine_dataset_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🎉 Congratulations!\n",
    "\n",
    "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the movement towards *Machine learning and AI for Medicine*, you can do so in the following ways!\n",
    "\n",
    "\n",
    "\n",
    "### ⭐ Star [TemporAI](https://github.com/vanderschaarlab/temporai) on GitHub\n",
    "\n",
    "- The easiest way to help our community is by just starring the repos! This helps raise awareness of the tools we're building.\n",
    "\n",
    "\n",
    "\n",
    "### Check out other projects from [vanderschaarlab](https://github.com/vanderschaarlab)\n",
    "- 📝 [HyperImpute](https://github.com/vanderschaarlab/hyperimpute)\n",
    "- 📊 [AutoPrognosis](https://github.com/vanderschaarlab/autoprognosis)\n",
    "- 🤖 [SynthCity](https://github.com/vanderschaarlab/synthcity)\n",
    " "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "RE_py38_temporai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}