06_Handling_Imbalance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "partial-ebony",
   "metadata": {},
   "source": [
    "# Flight Delay Prediction\n",
    "## Handling Imbalance in ArrDel15\n",
    "- <strong>Environment Setup:</strong> import libraries and load the dataset\n",
    "- <strong>Imbalanced Data Analysis:</strong> understanding the classifier performance metrics and the accuracy paradox\n",
    "- <strong>Strategies:</strong> to handle the imbalanced dataset\n",
    "    - <strong>Random Undersampling:</strong> 07_Random_Under_Sampling.ipynb\n",
    "    - <strong>Random Oversampling:</strong> 08_Random_Under_Sampling.ipynb\n",
    "    - <strong>SMOTE:</strong> 09_SMOTE.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proof-luxembourg",
   "metadata": {},
   "source": [
    "## Environment Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "downtown-tunisia",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/nive/anaconda3/lib/python3.8/site-packages/numpy/lib/arraysetops.py:583: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n",
      "  mask |= (ar1 == a)\n"
     ]
    }
   ],
   "source": [
    "# Pre-requisites\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Data Partitioning\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "df = pd.read_csv(\"Data/flight_and_weather_encoded.csv\", index_col=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "proprietary-platinum",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "table {float:left}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<style>\n",
    "table {float:left}\n",
    "</style>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adequate-subscription",
   "metadata": {},
   "source": [
    "## Imbalanced Data Analysis\n",
    "\n",
    "An **imbalanced dataset** is one in which one class has disproportionate observations compared to the other classes. Each class in the dataset does not have an equal representation and imbalance causes the **skewed class distribution**.\n",
    "\n",
    "### Accuracy Paradox\n",
    "When we have a disproportionate representation of classes in a dataset, a model is likely to predict the majority class value for all predictions and achieve a high classification accuracy, also referred to as **Accuracy Paradox**.<br>\n",
    "\n",
    "To ensure that the model works great in the real world, false positives and false negatives need to be kept to a minimum. Precision helps calculate the costs of false positives, and Recall helps understand the cost of false negatives. Hence precision, recall, and F1-score are better measures of a model’s performance.<br>\n",
    "\n",
    "From the results, we have obtained, we can see that the classifier performance for the **Delayed (1.0)** observations is significantly worse. The Accuracy is unreliable since **79.04%** of the observations belong to the **Not Delayed (0.0)** class."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "strategic-colombia",
   "metadata": {},
   "source": [
    "## Dataset Without Handling Imbalance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "controlling-federation",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAEfCAYAAAB1ZXBPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAcDUlEQVR4nO3de5hddX3v8fc3kpwWAhIaEhMColwCJEAOIFS8DWIq4JFbBOXSKhcDpaiFWkSpAoIo9dKAAhaVEqTGarUeWy5CIdNQSkoSCKBwiJEghICBEC4h1oD5nj/WmjAMM5k9yezfzB7er+fZz+y9LrM+M8+s+ezf2muvHZmJJEnNNmygA0iSXhssHElSERaOJKkIC0eSVISFI0kqwsKRJBVh4UiDTER8JCIyIj4y0Fmk/mThqKVFxDn1P+eMiIlN3E52uf0uIp6MiLsi4tsRcXBEvK5Z268z7BsRX4yIGyLiiTrH0l7Webib7B23J5qZV+pqk4EOIG2oiAjgJCCBAD4KfLLJmz2//vo6YEtgEvCndY75EXFcZi5q0raPBT4BvAg8AIxtcL1ngRndTF/VP7GkxoRXGlCrioj3AjcCVwMHU5XANpm5pgnbSoDMjG7mjQW+DhwFPArsk5nLN2JbHwH+ATghM6/uNH0KVbH+IjPX1Jkey8wJ6/leD9e5t9/QPFJ/8ZCaWtlH66/fAv4RGA0c0XWhiDivPoTUFhHHRsR/R8Sqjn/Gvc3vTWb+BvgQ0A5sC3ymmwxb1YfDHoiI30bEsxFxS0T8SaM/bGYuzMy7m1GoUgkeUlNLqkcVhwKLMvO/IuI54ExgOvBPPaz2V8BU4F+B2cDr+zi/R5m5NiIuBNqAYyLijKwPH0TEG6nKaHvgNqpR2WbA/wFujIhTMvNbjW5rA/yviDge2A54AbgXmJOZv2/iNqVXsXDUqk4AhlMdTiMzfx4RdwEHRMSOmbm4m3XeDbw1M+/u4Xv2Nr83/wm8BIyhKpcl9fSZwBuBYzLz+x0LR8SWVEV0aUT8tB4pNcMbgO92mbYkIk7IzP9o0jalV/GQmlpOfbLAycBa4JpOs66meo3j5B5WvbKXMult/npl5u+AFfXDreusewLvAn7UuWzq5Z8BzgX+AJi2odvtxT8AB1KVzmbA7sDfUxXiDXU+qQhHOGpF7wZ2AH6WmY91mv494CvARyLis5n5Ypf17uzl+/Y2vxEdJxV0nI3z1vrr6yPivG6W37r+ums/bPtVMvP8LpN+DpwaEauoDiGeRzeve0nNYOGoFU2vv17deWJmroiIf6UaLRwG/HOX9Xp738lGvS8lIv4A2Kp++GT99Y/qr1PrW09Gbsy2N8A3qQrnnYW3q9cwD6mppUTE1sDh9cNZXd/MyMuHpqZ3s3pv7wHY2PcIvJ3qSdxvMvPhetqz9ddPZGas53bCRm67rzpO296s8Hb1GuYIR63mw8AIYAGwsIdlDgXeExFvyswlPSzTryJiGHBO/fB7nWbNrb++A7i0RJYGdRzqe2hAU+g1xcJRq+k4IeC0zOz2NZeIuAD4m3rZc7pbpj9FxBjgG1SnRD8CXNQxLzPnR8RtwJERcWJmXtXN+rtTjYo2+M2iPeSaBDyemU93mf7GOi/Atf25TWl9LBy1jIhoAyYC9/VUNrXvUBXNCRFxbj9nOK++O4yXL23zdqpR153AcZn5VJfVjgVuBb4TER8H/ht4BpgA7AFMphpxrLdwImIX4Owuk0dFxNWdHn+y0/aPAs6OiNlUp2g/T3Wyxfuozoy7nuokC6kIC0etpOPKAt9e30KZ+XBE/DvVi/Tv7+cMHQW2huof+K+pTs3+EXBTZq7tJs/SiNgb+BjVa0zHUV2G5wngfqrL4tzXwLbfQHVIsbNNu0w7D+gonNlUBf2/qQptM6qi+0+q9+V8t+PNqVIJXktNklSEZ6lJkoqwcCRJRVg4kqQiLBxJUhEtf5balltumTvuuONAx2jYCy+8wGabtcabu1spK5i3mVopK7RW3oHKumDBgqcyc+vel+w/LV84Y8eOZf78+QMdo2Ht7e20tbUNdIyGtFJWMG8ztVJWaK28A5U1In5depseUpMkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKiMwc6AwbJcZHcspAp5CksvLcjfvfHRELMnOfforTEEc4kqQiLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoooWjgRcVBEPBgRiyPi7G7mR0RcWs+/NyL2KplPktQ8xQonIl4HXAYcDOwGHBMRu3VZ7GBgp/o2HbiiVD5JajUPPvggU6ZMWXfbYostmDFjBvfccw9vfetb2X333Xn/+9/Pc8891+36DQwCPhkRGRGj68dvqwcD8yJix3ralhHxs4iI3vKWHOHsCyzOzIcycw3wfeCwLsscBlyTlbnAlhExrmBGSWoZEydOZOHChSxcuJAFCxaw6aabcsQRR3DyySfzpS99ifvuu48jjjiCL3/5yz19ix4HARGxLTAVeKTT8n8FTAM+A/x5Pe2zwEWZmb3l3aTPP+GG2wZ4tNPjpcB+DSyzDfB454UiYjrVCAisI0mvQe3t7a94PG/ePEaPHs2SJUu4//77Wbt2Le3t7YwcOZLvfve7HHjggV2/xWbUgwCAiOgYBNxfz/874Czg/3Za50XgD4FNgRcjYgdgm8z8j0Yylyyc7oZbXRuxkWXIzCuBKwFifPTaqpI01LS1tb3i8TXXXMOpp55KW1sbe+65J8899xyHHXYYX/va11ixYsWrlgdG0MMgICIOBR7LzHu6HCn7ItX/3t8Cfwp8hWqE05CSh9SWAtt2ejwBWLYBy0iSOlmzZg0//elPOeqoowC46qqruOyyy9h77715/vnnGTFiRKPfKiNiU+Ac4HOvmpm5MDP/ODMPAN5M9f85IuKfIuLaiBi7vm9ecoQzD9gpIt4EPAZ8CDi2yzI/BU6vh3b7Ac9m5uNIknp0ww03sNdeezF2bPX/fpddduGmm24CYNGiRVx33XXdrbaG7p/g7wC8CegY3UwA7oqIfTPzCagaBvgb4IPAN4Bzge2Bj1OVVbeKFU5mvhQRpwM/A14HXJWZv4iIU+v53wSuBw4BFgOrgRNK5ZOkVjVr1iyOOeaYdY+XL1/OmDFjWLt2LRdeeCGnnnpqd6u9QDeDgMz8BTCmY6GIeBjYJzOf6rTuh4HrMnNlPSJaW982XV/Oou/DyczrM3PnzNwhM79QT/tmXTbUZ6f9RT1/98ycXzKfJLWa1atXc/PNN3PkkUeumzZr1ix23nlndtllF8aPH88JJ1TP3ZctW8YhhxzSefWOQcADwA/qslmvumA+DFxeT/oa8COq13fW+1aWaOBMtkEtxkdyykCnkKSy8tyN+98dEQsyc59+itMQL20jSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVsMtABNtbOm+/Mg+c+ONAxGtbe3k5bW9tAx2hIK2UF8zZTK2WF1srbSlk3liMcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqYiGCycihkXEsE6P3xARJ0fE25oTTZI0lPRlhHMd8DGAiBgJzAe+DLRHxJ81IZskaQjpS+HsDdxa3z8SeA4YA3wU+GQ/55IkDTF9KZzNgWfq+38C/EtmvkhVQjv0cy5J0hDTl8J5BHhbRGwGvBe4uZ6+FbC6v4NJkoaWvnwezteA7wKrgF8Dc+rp7wTu6+dckqQhpuHCycy/j4gFwLbAzZm5tp71K+CzzQgnSRo6+vSJn5k5n+rsNAAiYnhmXtfvqSRJQ05f3ofz8YiY1unxd4DfRsSDETGxKekkSUNGX04a+DjwJEBEvBM4GjgWWAh8td+TSZKGlL4cUtsGeLi+/37gh5n5g4i4D7itv4NJkoaWvoxwngO2ru9PBW6p778I/EF/hpIkDT19GeHcBHwrIu4GdgRuqKdPApb0dzBJ0tDSlxHOXwC3A6OBD2Tm0/X0vYBZ/R1MkjS09OV9OM9RX7yzy/Rz+zWRJGlI6tP7cDpExBuAEZ2nZeYj/ZJIkjQkNVw4EfF64FKq06FHdLPI6/orlCRp6OnLazhfAfYEDgf+h+o9OH8NLAU+2O/JJElDSl8OqR0MHJOZt0XE74EFmflPEfE4cArwz01JKEkaEvoywtmS6irRAM8Cf1TfvwPYvx8zSZKGoL4Uzq+AN9f3HwA+FBFB9emfT/e4liRJ9K1wrgb2qO9/ieow2hrgy8DF/RtLkjTU9OV9OH/X6f6tEbELsA/wy8z0A9gkSeu1Qe/DgXXvu/G9N5Kkhqy3cCLizEa/UWZ+bePjSJKGqt5GOK+6lE0PErBwJEk9Wm/hZOabSgWRJA1tvZ6lFhEHR8TD9aVtus57fT3vT5oTT5I0VDRyWvTHgC9n5rNdZ9TTLgY+0d/BJElDSyOFszvw7+uZfyvVNdYkSepRI4WzNbB2PfOTly9zI0lStxopnKW8fIWB7uwBPNY/cSRJQ1UjhXMdcEFE/GHXGRGxKfD5ehlJknrUyJUGvgB8APhlRHwd+H/19F2B04EALmpOPEnSUNFr4WTm8ojYH7iCqliiYxbwM+C0zPxN8yJKkoaChq6llpm/Bg6JiFHAjlSl88vMXNnMcJKkoaNPF++sC2Zek7JIkoawvnwejiRJG8zCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhF9Oi16MNp80SKI6H3BQaJtoAM0InOgE0gaghzhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUhIUjSSqiWOFExFURsTwift7D/IiISyNicUTcGxF7lcomSWq+kiOcq4GD1jP/YGCn+jYduKJAJklSIcUKJzPnAE+vZ5HDgGuyMhfYMiLGlUmnrh599FHOOOMMdt11VyZNmsQll1wCwNNPP83UqVPZaaedmDp1KitXrux2/UsuuYTJkyczadIkZsyY8Yp5X//615k4cSKTJk3irLPOAuD2229njz324C1veQuLFy8G4JlnnuG9730vmdm8H1RSMVFyZ46I7YF/y8zJ3cz7N+BLmfmf9eNbgE9l5vxulp1ONQpib9j7VQtoo7TPns2KFSt49NFHmTJlCqtXr+aUU07hggsu4MYbb2SLLbbg2GOP5Xvf+x7PP/88p5xyyivWX7JkCZ///Oe54oorGD58OGeddRZnnHEGEyZM4O677+baa6/li1/8IiNGjGDlypWMGjWKz33uc0yfPp0nnniCO++8k9NOO43LL7+c/fffnylTpjSUe9WqVYwcObIJv5HmaKW8rZQVWivvQGU94IADFmTmPiW3uUnJjfUiupnWbRtm5pXAlQD7RPj0t5+1tbUB0N7evu7+XnvtxTbbbMPdd99Ne3s748aNY+LEibS1ta1bpsOTTz7JgQceyEEHVUdQDz/8cJYtW8bxxx/P5ZdfzsUXX8x73vOeV6wzbtw49thjDzbffHOeeuoptt12W4YNG8Zf/uVfNpy7c95W0Ep5WykrtFbeVsq6sQbTWWpLgW07PZ4ALBugLOrk4Ycf5u6772a//fbjN7/5DePGVUc6x40bx/Lly1+1/OTJk5kzZw4rVqxg9erVXH/99Tz66KMALFq0iNtuu4399tuPd73rXcybNw+AT3/600yfPp0ZM2Zw+umnc84553DBBReU+yElNd1gGuH8FDg9Ir4P7Ac8m5mPD3Cm17xVq1Yxbdo0ZsyYwRZbbNHQOrvuuiuf+tSnmDp1KiNHjmTPPfdkk02qP7WXXnqJlStXMnfuXObNm8fRRx/NQw89xJQpU5g7dy4Ac+bMYfz48WQmH/zgBxk+fDhf/epXGTt2bNN+TknNV/K06FnAHcDEiFgaESdFxKkRcWq9yPXAQ8Bi4FvAaaWyqXsvvfQS06ZN47jjjuPII48EYOzYsTz+ePU84PHHH2fMmDHdrnvSSSdx1113MWfOHLbaait22mknACZMmMCRRx5JRLDvvvsybNgwnnrqqXXrZSYXXnghn/3sZzn//PM5//zzOf7447n00kub/NNKarZiI5zMPKaX+Qn8RaE46kVm8rd/+7fstttunHnmmeumH3roocycOZOzzz6bmTNncthhh3W7/vLlyxkzZgyPPPIIP/7xj7njjjuA6vWcW2+9lba2NhYtWsSaNWsYPXr0uvVmzpzJ+973PkaNGsXq1asZNmwYw4YNY/Xq1c39gSU13WA6pKZB5Pbbb+fmm2/miSeeWHeW2EUXXcTZZ5/N0UcfzXe+8x222247fvjDHwKwbNkyTj75ZK6//noApk2bxooVKxg+fDiXXXYZo0aNAuDEE0/kxBNPZPLkyYwYMYKZM2cSUZ0vsnr1ambOnMlNN90EwJlnnsm0adMYMWIEs2bNKvwbkNTfLBx16+1vfzuzZ8/u9uyZW2655VXTxo8fv65sAG677bZuv++IESO49tpru5236aabMnv27HWP3/GOd3Dffff1MbmkwWownaUmSRrCLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklREyxfO8zvvDJktc2ufPXvAM/R6k6QmaPnCkSS1BgtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklSEhSNJKsLCkSQVYeFIkoqwcCRJRVg4kqQiLBxJUhEWjiSpCAtHklREZOZAZ9goEfE88OBA5+iD0cBTAx2iQa2UFczbTK2UFVor70BlfWNmbl1yg5uU3FiTPJiZ+wx0iEZFxPxWydtKWcG8zdRKWaG18rZS1o3lITVJUhEWjiSpiKFQOFcOdIA+aqW8rZQVzNtMrZQVWitvK2XdKC1/0oAkqTUMhRGOJKkFWDiSpCJapnAi4qCIeDAiFkfE2d3Mj4i4tJ5/b0TsNRA56yy9ZT2uznhvRPxXROw5EDk75Vlv3k7LvSUifh8RHyiZr5scveaNiLaIWBgRv4iI/yidsVOO3v4WXh8R/xoR99RZTxiInHWWqyJieUT8vIf5g2Yfq/P0lnfQ7Ge9Ze203KDYx5omMwf9DXgd8CvgzcAI4B5gty7LHALcAATwx8B/D+Ks+wOj6vsHD1TWRvN2Wu5W4HrgA4M5L7AlcD+wXf14zCDO+hng4vr+1sDTwIgByvtOYC/g5z3MHxT7WB/yDqb9bL1ZO/29DPg+1sxbq4xw9gUWZ+ZDmbkG+D5wWJdlDgOuycpcYMuIGFc6KA1kzcz/ysyV9cO5wITCGTtr5HcL8DHgR8DykuG60UjeY4EfZ+YjAJk5UJkbyZrA5hERwEiqwnmpbMw6SOacevs9GSz7GNB73sG0nzXwu4XBs481TasUzjbAo50eL62n9XWZEvqa4ySqZ40Dpde8EbENcATwzYK5etLI73dnYFREtEfEgoj4s2LpXqmRrN8AdgWWAfcBn8jMtWXi9dlg2cc2xEDvZ+s1yPaxpmmVS9tEN9O6ns/dyDIlNJwjIg6g2hHe3tRE69dI3hnApzLz99UT8QHVSN5NgL2BA4E/BO6IiLmZuajZ4bpoJOt7gYXAu4EdgJsj4rbMfK7J2TbEYNnH+mSQ7Ge9mcHg2ceaplUKZymwbafHE6ieEfZ1mRIayhERewDfBg7OzBWFsnWnkbz7AN+vd4TRwCER8VJm/qRIwldq9G/hqcx8AXghIuYAewKlC6eRrCcAX8rqIP7iiFgC7ALcWSZinwyWfaxhg2g/681g2seaZ6BfRGrkRlWMDwFv4uUXXyd1WeZ9vPIFzTsHcdbtgMXA/q3wu+2y/NUM7EkDjfx+dwVuqZfdFPg5MHmQZr0COK++PxZ4DBg9gL/f7en5RfhBsY/1Ie+g2c96y9pluQHdx5p5a4kRTma+FBGnAz+jOpPjqsz8RUScWs//JtWZHYdQ/YGtpnrmOFizfg74I+Dy+hnNSzlAV4ttMO+g0UjezHwgIm4E7gXWAt/OzPWejjpQWYELgKsj4j6qf+SfyswBuax+RMwC2oDREbEUOBcY3inroNjHOjSQd9DsZw1kfU3w0jaSpCJa5Sw1SVKLs3AkSUVYOJKkIiwcSVIRFo4ktZBGLwTaafmjI+L++uKw32t2vvVm8Sw1SWodEfFOYBXVde0m97LsTsAPgHdn5sqIGJMDd21BRzhSdyJibERcEhG/iojfRcRjEXFDRBxSOMfVEfFvJbepwS27uRBoROwQETfW1w68LSJ2qWd9FLgs64uYDmTZQOtc2kYqJiK2B24Hngc+TXWFgGFU12b7JtU72KXB5Erg1Mz8ZUTsB1xOdX2+nQEi4naqNx+fl5k3DlRIRzjSq11O9a7/fTLzB5n5YGY+kJnfoLomGxGxXUT8S0Q8X99+HBHrLn8fEed1PcYeER+JiFVdl4mID9Ujqecj4icRMbpjPvBh4H0RkfWtrdk/vFpLRIyk+uyfH0bEQuDvgY6PjdgE2InqKgfHAN+OiC3Lp3w5jKRaRGwFHAT8TWau6jq/Pg4ewE+A/6F6FplUHzPwk4h4S/bthdHtgQ9SXZp+M6rPzPkCcArwFarrwm0F/Gm9fG+fqaLXnmHAM5k5pZt5S4G5mfkisCQiHqQqoHkF863jCEd6pR2pRjcPrGeZ91CNdI7NzHmZOZ/qQ9/2ojrs1hebAB/JzHsz8w6qQyMHAtSF91vgd5n5RH1b08fvryEuq4+yWBIRR8G6jwLv+DjtnwAH1NNHUx1ie2ggcoKFI3XVyIeR7Aosy8yHOyZk5kNUl+rfrY/b+3VmPtvp8TJgTB+/h15D6guB3gFMjIilEXEScBxwUkTcA/yClz9Z9mfAioi4H5gN/HUO4Mc0eEhNeqVfUh0i2xX4lx6WCXr+4LGO6Wt5dXkN72b5F7tZ3yeC6lFmHtPDrIO6WTaBM+vbgPMPW+okM5+melZ4ev1i7CvUL7jeD2xTn83WMf3NwPh6HsCTwNh45cc3TtmASGuozi6SWp6FI73aaVSjk/kRcVRETIyIXSLiz6k+Y+ffqU6V/seI2Dsi9gH+EbgLuLX+Hu1UL/Z/pn6PxEnABzYgy8PA5DrD6IjobpQktQQLR+oiM5dQnQBwM3AxVcncChwKnFIfpjicahTTTnVs/Ang8I4z1DLzAeDPgen1+lOBizYgzreoTmCYX2/vbRv4Y0kDzkvbSJKKcIQjSSrCwpEkFWHhSJKKsHAkSUVYOJKkIiwcSVIRFo4kqQgLR5JUxP8H/pE+ixrWSK8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0    1463378\n",
      "1.0     388058\n",
      "Name: ArrDel15, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Plotting the percentage of observations that fall under each class\n",
    "ax = df[\"ArrDel15\"].value_counts().sort_values().plot(kind=\"barh\", color=[\"r\", \"g\"])\n",
    "ax.set_axisbelow(True)\n",
    "ax.grid()\n",
    "totals= []\n",
    "for i in ax.patches:\n",
    "    totals.append(i.get_width())\n",
    "total = sum(totals)\n",
    "for i in ax.patches:\n",
    "     ax.text(i.get_width()+.3, i.get_y()+.20, \n",
    "     str(round((i.get_width()/total)*100, 2))+'%', \n",
    "     fontsize=10, color='black')\n",
    "plt.title(\"ArrDel15\", fontsize=20)\n",
    "plt.xlabel(\"Count\", fontsize=14)\n",
    "plt.ylabel(\"Class\", fontsize=14)\n",
    "plt.show()\n",
    "print(df[\"ArrDel15\"].value_counts())\n",
    "fig = ax.get_figure()\n",
    "fig.savefig(\"Img/06_01_Dataset_ArrDel15_Class_Imbalance.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "optional-visiting",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Dataset shape: (1851436, 24)\n",
      "features_train shape: (1481148, 23) | features_test shape: (370288, 23)\n",
      "labels_train shape: (1481148,) | labels_test shape: (370288,)\n"
     ]
    }
   ],
   "source": [
    "features = df.loc[:, df.columns != \"ArrDel15\"]\n",
    "labels = np.asarray(df[\"ArrDel15\"])\n",
    "# Number of samples/observations/rows is greater than 100,000\n",
    "print(f\"\\nDataset shape: {df.shape}\")\n",
    "features_train, features_test, labels_train, labels_test = train_test_split(features, labels, test_size=0.20, random_state=42)\n",
    "print(f\"features_train shape: {features_train.shape} | features_test shape: {features_test.shape}\")\n",
    "print(f\"labels_train shape: {labels_train.shape} | labels_test shape: {labels_test.shape}\")\n",
    "# print(f\"{features_train.shape[1]} Features: {features_train.columns.to_list()}\")\n",
    "del features\n",
    "del labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sunset-relevance",
   "metadata": {},
   "source": [
    "### Performance Summary of the Different Classification Models\n",
    "<table>\n",
    "    <thead>\n",
    "        <tr>\n",
    "            <th rowspan=\"3\">Classification Model</th>\n",
    "            <th colspan=\"8\">Performance Metric</th>\n",
    "        </tr>\n",
    "        <tr>\n",
    "            <th colspan=\"2\">Precision</th>\n",
    "            <th colspan=\"2\">Recall</th>\n",
    "            <th colspan=\"2\">F1 Score</th>\n",
    "            <th rowspan=\"2\" colspan=\"2\">Accuracy</th>\n",
    "        </tr>\n",
    "        <tr>\n",
    "            <th>0.0</th>\n",
    "            <th>1.0</th>\n",
    "            <th>0.0</th>\n",
    "            <th>1.0</th>\n",
    "            <th>0.0</th>\n",
    "            <th>1.0</th>\n",
    "        </tr>\n",
    "    </thead>\n",
    "    <tbody>\n",
    "        <tr>\n",
    "            <th>Logistic Regression</th>\n",
    "            <td>0.92</td>\n",
    "            <td>0.89</td>\n",
    "            <td>0.98</td>\n",
    "            <td>0.68</td>\n",
    "            <td>0.95</td>\n",
    "            <td>0.77</td>\n",
    "            <td>0.92</td>\n",
    "        </tr>\n",
    "        <tr>\n",
    "            <th>Decision Tree Classifier</th>\n",
    "            <td>0.92</td>\n",
    "            <td>0.68</td>\n",
    "            <td>0.91</td>\n",
    "            <td>0.71</td>\n",
    "            <td>0.92</td>\n",
    "            <td>0.69</td>\n",
    "            <td>0.87</td>\n",
    "        </tr>\n",
    "        <tr>\n",
    "            <th>Extra Trees Classifier</th>\n",
    "            <td>0.93</td>\n",
    "            <td>0.81</td>\n",
    "            <td>0.96</td>\n",
    "            <td>0.73</td>\n",
    "            <td>0.94</td>\n",
    "            <td>0.77</td>\n",
    "            <td>0.91</td>\n",
    "        </tr>\n",
    "        <tr>\n",
    "            <th>Gradient Boosting Classifier</th>\n",
    "            <td>0.92</td>\n",
    "            <td>0.90</td>\n",
    "            <td>0.98</td>\n",
    "            <td>0.68</td>\n",
    "            <td>0.95</td>\n",
    "            <td>0.78</td>\n",
    "            <td>0.92</td>\n",
    "        </tr>\n",
    "        <tr>\n",
    "            <th>Random Forest Classifier</th>\n",
    "            <td>0.93</td>\n",
    "            <td>0.88</td>\n",
    "            <td>0.97</td>\n",
    "            <td>0.70</td>\n",
    "            <td>0.95</td>\n",
    "            <td>0.78</td>\n",
    "            <td>0.92</td>\n",
    "        </tr>\n",
    "    </tbody>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "patent-anchor",
   "metadata": {},
   "source": [
    "### Metrics  for Imbalanced Data\n",
    "|Performance Metric|Inference|\n",
    "|:-|:-|\n",
    "|**Accuracy of the model**|Is basically the total number of correct predictions divided by total number of predictions|\n",
    "|**Precision of a class**|Defines how trustable the result is when the model answers that a point belongs to that class|\n",
    "|**Recall of a class**|Expresses how well the model is able to detect that class|\n",
    "|**F1 of a class**|Is given by the harmonic mean of precision and recall and combines them into one metric|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "automotive-continent",
   "metadata": {},
   "source": [
    "### Understanding these Metrics\n",
    "||High Recall|Low Recall|\n",
    "|:-|:-|:-|\n",
    "|**High Precision**|The class is perfectly handled by the model|The model can’t detect the class well but is highly trustable when it does|\n",
    "|**Low Precision**|The class is well detected but the model also include points of other classes in it|The class is poorly handled by the model|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "simplified-belle",
   "metadata": {},
   "source": [
    "## Strategies\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "|Strategy|What happens?|\n",
    "|:-|:-|\n",
    "|**Under-sampling**|Under-sampling or down-sampling the majority class where the observations of the majority class are randomly removed to reduce its influence on the machine learning algorithm.|\n",
    "|**Over-sampling**|Over-sampling or up-sampling, the minority class where the observations for the minority class is randomly duplicated to increase the influence on the machine learning algorithm.|\n",
    "|**SMOTE**|Creating synthetic data for the minority class using SMOTE(Synthetic Minority over-sampling Technique). SMOTE takes random samples from the minority class, finds its nearest k neighbors, and then selects a point between the randomly selected data point and its nearest k neighbors to generate synthetic data.|"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}