09.04.Classification_hyperparameter_tuning_code_challenge.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div align=\"center\" style=\" font-size: 80%; text-align: center; margin: 0 auto\">\n",
    "<img src=\"https://raw.githubusercontent.com/Explore-AI/Pictures/master/Python-Notebook-Banners/Code_challenge.png\"  style=\"display: block; margin-left: auto; margin-right: auto;\";/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f662d169",
   "metadata": {},
   "source": [
    "# Code challenge: Classification - hyperparameter tuning\n",
    "© ExploreAI Academy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26af890c",
   "metadata": {},
   "source": [
    "In this train, we'll tackle a classification problem by tuning hyperparameters, using techniques like grid search to optimise model performance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d230d14",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Learning objectives\n",
    "\n",
    "By the end of this train, you should be able to:\n",
    "- Apply hyperparameter tuning to improve a classification model.\n",
    "- Evaluate model performance with tuned hyperparameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0c49012",
   "metadata": {},
   "source": [
    "## Instructions to students\n",
    "\n",
    "- **Do not add or remove cells in this notebook. Do not edit or remove the `### START FUNCTION` or `### END FUNCTION` comments. Do not add any code outside of the functions you are required to edit. Doing any of this will lead to a mark of 0%!**\n",
    "- Answer the questions according to the specifications provided.\n",
    "- Use the given cell in each question to see if your function matches the expected outputs.\n",
    "- Do not hard-code answers to the questions.\n",
    "- The use of Stack Overflow, Google, and other online tools are permitted. However, copying fellow student's code is not permissible and is considered a breach of the Honour code below. Doing this will result in a mark of 0%.\n",
    "- Good luck, and may the force be with you!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63e6293e",
   "metadata": {},
   "source": [
    "## Honour Code\n",
    "\n",
    "I **YOUR NAME**, **YOUR SURNAME**, confirm - by submitting this document - that the solutions in this notebook are a result of my own work and that I abide by the EDSA honour code (https://drive.google.com/file/d/1QDCjGZJ8-FmJE3bZdIQNwnJyQKPhHZBn/view?usp=sharing).\n",
    "\n",
    "Non-compliance with the honour code constitutes a material breach of contract."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4f17e44",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "Hyperparameters have a direct impact on the performance and predictions made by machine learning models. Within this coding challenge, we will strengthen our ability to produce appropriate classification solutions by extending a base model with tuned hyperparameters. \n",
    "\n",
    "<br></br>\n",
    "\n",
    "<div align=\"center\" style=\"width: 600px; font-size: 80%; text-align: center; margin: 0 auto\">\n",
    "<img src=\"https://raw.githubusercontent.com/Explore-AI/Pictures/master/wine.jpg\"\n",
    "     alt=\"Some fine wine for your fine model\"\n",
    "     style=\"float: center; padding-bottom=0.5em\"\n",
    "     width=600px/>\n",
    "Some fine wine for your fine modelling process. \n",
    "Photo by <a href=\"https://unsplash.com/@hermez777?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText\"> Hermes Rivera</a> on Unsplash\n",
    "</div>\n",
    "\n",
    "The structure of this notebook is as follows:\n",
    "\n",
    " - First, we'll load our data to get a view of the predictor and response variables we will be modelling. \n",
    " - We'll then preprocess our data, binarising the target variable and splitting up the data into train and test sets. \n",
    " - We then model our data using a Support Vector Classifier.\n",
    " - Following this modelling, we define a custom metric as the log-loss in order to evaluate our produced model.\n",
    " - Using this metric, we then take several steps to improve our base model's performance by optimising the hyperparameters of the SVC through a grid search strategy. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5e543f7",
   "metadata": {},
   "source": [
    "## Imports\n",
    "\n",
    "Let's go ahead and load the usual suspects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0bf461fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn import preprocessing\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "219a8d5a",
   "metadata": {},
   "source": [
    "## The dataset \n",
    "\n",
    "For this coding challenge we'll be using the [Wine Quality dataset](https://archive.ics.uci.edu/ml/datasets/wine+quality) from the UCI Machine Learning Repository. The constituents of this dataset are red and white variants of the Portuguese \"Vinho Verde\" wine. \n",
    "\n",
    "This dataset consists of the following variables: \n",
    "\n",
    " - fixed acidity\n",
    " - volatile acidity\n",
    " - citric acid\n",
    " - residual sugar\n",
    " - chlorides\n",
    " - free sulfur dioxide\n",
    " - total sulfur dioxide\n",
    " - density\n",
    " - pH\n",
    " - sulphates\n",
    " - alcohol\n",
    " - quality (score between 0 and 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59fa0bf2",
   "metadata": {},
   "source": [
    "### Reading in the data\n",
    "\n",
    "\n",
    "**Note** the feature we will be predicting is quality, i.e. the label is 'quality' using classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d80ad1b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>type</th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.36</td>\n",
       "      <td>20.7</td>\n",
       "      <td>0.045</td>\n",
       "      <td>45.0</td>\n",
       "      <td>170.0</td>\n",
       "      <td>1.0010</td>\n",
       "      <td>3.00</td>\n",
       "      <td>0.45</td>\n",
       "      <td>8.8</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>6.3</td>\n",
       "      <td>0.30</td>\n",
       "      <td>0.34</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.049</td>\n",
       "      <td>14.0</td>\n",
       "      <td>132.0</td>\n",
       "      <td>0.9940</td>\n",
       "      <td>3.30</td>\n",
       "      <td>0.49</td>\n",
       "      <td>9.5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>8.1</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.40</td>\n",
       "      <td>6.9</td>\n",
       "      <td>0.050</td>\n",
       "      <td>30.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>0.9951</td>\n",
       "      <td>3.26</td>\n",
       "      <td>0.44</td>\n",
       "      <td>10.1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>7.2</td>\n",
       "      <td>0.23</td>\n",
       "      <td>0.32</td>\n",
       "      <td>8.5</td>\n",
       "      <td>0.058</td>\n",
       "      <td>47.0</td>\n",
       "      <td>186.0</td>\n",
       "      <td>0.9956</td>\n",
       "      <td>3.19</td>\n",
       "      <td>0.40</td>\n",
       "      <td>9.9</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>7.2</td>\n",
       "      <td>0.23</td>\n",
       "      <td>0.32</td>\n",
       "      <td>8.5</td>\n",
       "      <td>0.058</td>\n",
       "      <td>47.0</td>\n",
       "      <td>186.0</td>\n",
       "      <td>0.9956</td>\n",
       "      <td>3.19</td>\n",
       "      <td>0.40</td>\n",
       "      <td>9.9</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   type  fixed acidity  volatile acidity  citric acid  residual sugar  \\\n",
       "0     0            7.0              0.27         0.36            20.7   \n",
       "1     0            6.3              0.30         0.34             1.6   \n",
       "2     0            8.1              0.28         0.40             6.9   \n",
       "3     0            7.2              0.23         0.32             8.5   \n",
       "4     0            7.2              0.23         0.32             8.5   \n",
       "\n",
       "   chlorides  free sulfur dioxide  total sulfur dioxide  density    pH  \\\n",
       "0      0.045                 45.0                 170.0   1.0010  3.00   \n",
       "1      0.049                 14.0                 132.0   0.9940  3.30   \n",
       "2      0.050                 30.0                  97.0   0.9951  3.26   \n",
       "3      0.058                 47.0                 186.0   0.9956  3.19   \n",
       "4      0.058                 47.0                 186.0   0.9956  3.19   \n",
       "\n",
       "   sulphates  alcohol  quality  \n",
       "0       0.45      8.8        6  \n",
       "1       0.49      9.5        6  \n",
       "2       0.44     10.1        6  \n",
       "3       0.40      9.9        6  \n",
       "4       0.40      9.9        6  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Explore-AI/Public-Data/master/Data/classification_sprint/winequality.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2c4bfe2",
   "metadata": {},
   "source": [
    "## Question 1 - Data preprocessing\n",
    "\n",
    "We would like to classify the wine according to it's quality using binary classification.\n",
    "Write a function to preprocess the data so we can run it through the classifier. The function should:\n",
    "\n",
    "* Convert the quality for lower quality wines (quality less than or equal to 4) to 0\n",
    "* Convert the quality for higher quality wines (quality greater than or equal to 5) to 1\n",
    "* Split the data into 75% training and 25% testing data\n",
    "* Set random_state to equal 42 for this internal method. \n",
    "\n",
    "_**Function specifications:**_\n",
    "* Should take a dataframe\n",
    "* Standardise the features using sklearn's ```StandardScaler```\n",
    "* Convert the quality labels into binary labels\n",
    "* Should fill nan values with zeros\n",
    "* Should return two `tuples` of the form `(X_train, y_train), (X_test, y_test)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "93ab4297",
   "metadata": {},
   "outputs": [],
   "source": [
    "### START FUNCTION\n",
    "def data_preprocess(df):\n",
    "  df2 = df.fillna(0)\n",
    "  indexes_low = df2[df2['quality'] <= 4]['quality'].index\n",
    "  for i in indexes_low:\n",
    "    df2.loc[i, 'quality'] = 0\n",
    "  indexes_high = df2[df2['quality'] >= 5]['quality'].index\n",
    "  for i in indexes_high:\n",
    "    df2.loc[i, 'quality'] = 1\n",
    "    y = df2['quality']\n",
    "    X = df2.drop('quality', axis=1)\n",
    "    scaler = preprocessing.StandardScaler()\n",
    "    X_scaled = scaler.fit_transform(X)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.25, random_state=42)\n",
    "  return (X_train, y_train), (X_test, y_test)\n",
    "### END FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7de1e047",
   "metadata": {},
   "outputs": [],
   "source": [
    "(X_train, y_train), (X_test, y_test) = data_preprocess(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4390b67d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.57136659  0.07127869 -0.48054096  1.17914161 -0.09303318 -0.79974133\n",
      "   0.0830898  -0.15472329 -0.36573452  0.13010447  0.06101473  0.25842195]\n",
      " [-0.57136659  1.50396711 -0.72301571  0.56008035 -0.63948302 -0.05776881\n",
      "  -0.70572997  0.62379657  0.16787589 -0.86828773 -0.47467813 -0.99931317]]\n"
     ]
    }
   ],
   "source": [
    "print(X_train[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbb6b66b",
   "metadata": {},
   "source": [
    "_**Expected outputs:**_\n",
    "```python\n",
    "(X_train, y_train), (X_test, y_test)= data_preprocess(df)\n",
    "print(X_train[:2])\n",
    "print(y_train[:2])\n",
    "print(X_test[:2])\n",
    "print(y_test[:2])\n",
    "\n",
    "\n",
    "[[-0.57136659  0.07127869 -0.48054096  1.17914161 -0.09303318 -0.79974133\n",
    "   0.0830898  -0.15472329 -0.36573452  0.13010447  0.06101473  0.25842195]\n",
    " [-0.57136659  1.50396711 -0.72301571  0.56008035 -0.63948302 -0.05776881\n",
    "  -0.70572997  0.62379657  0.16787589 -0.86828773 -0.47467813 -0.99931317]]\n",
    "\n",
    "[1 0]\n",
    "\n",
    "[[-0.57136659 -0.15493527 -0.54115965  0.90400327 -0.66050032 -0.31460545\n",
    "   0.53384396  0.03990667 -1.35291379 -0.26925241 -0.34075491  1.18076103]\n",
    " [-0.57136659  0.29749266 -1.20796522  2.8987562  -0.80762143 -0.45729248\n",
    "  -0.19863155 -0.22549783 -1.03274754 -0.7185289  -0.87644778  0.25842195]]\n",
    "\n",
    "[1 1] \n",
    "``` "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b52dee68",
   "metadata": {},
   "source": [
    "## Question 2 - Model training\n",
    "\n",
    "Now that you have processed your data, let's jump straight into model fitting. Write a function that should:\n",
    "* Instantiate a `SVC` model.\n",
    "* Train the `SVC` model with default parameters.\n",
    "* Return the trained SVC model. \n",
    "\n",
    "_**Function specifications:**_\n",
    "* Should take two numpy `arrays` as input in the form `(X_train, y_train)`.\n",
    "* Should return an sklearn `SVC` model which has a random state of 40 and gamma set to 'auto'.\n",
    "* The returned model should be fitted to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f1235b40",
   "metadata": {},
   "outputs": [],
   "source": [
    "### START FUNCTION\n",
    "def train_SVC_model(X_train,y_train):\n",
    "  svm = SVC(gamma='auto', random_state=40)\n",
    "  svm.fit(X_train,y_train)\n",
    "  return svm\n",
    "\n",
    "### END FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9d8427bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1], dtype=int64)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc = train_SVC_model(X_train,y_train)\n",
    "svc.classes_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7035ff0d",
   "metadata": {},
   "source": [
    "\n",
    "_**Expected outputs:**_\n",
    "\n",
    "```python\n",
    "svc.classes_\n",
    "```\n",
    "```\n",
    "array([0, 1], dtype=int64)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18b2e62e",
   "metadata": {},
   "source": [
    "## Question 3 - Model testing\n",
    "\n",
    "Now that you've trained your model. It's time to test its accuracy, however, we'll be using a custom scoring function for this. Create a function that implements the log loss function:\n",
    "\n",
    "$$\\Large  H(p,q)=  -\\frac{1}{N}\\sum_{i=1}^{N} ylog(\\hat{y}_{i}) + (1- y)log(1 - \\hat{y}_{i})$$\n",
    "\n",
    "_**Function specifications:**_\n",
    "* Should take two numpy `arrays` as input in the form `y_true` and `y_predicted`.\n",
    "* Should return a `float64` for the log loss value rounded to 7 decimal places.\n",
    "\n",
    "_**Hint:**_ the numpy subtract function can be used to perform a calculation across an array of values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ecd1016e",
   "metadata": {},
   "outputs": [],
   "source": [
    "### START FUNCTION\n",
    "def custom_scoring_function(y_true, y_pred):\n",
    "    epsilon = 1e-15\n",
    "    y_pred = np.maximum(epsilon, y_pred)\n",
    "    y_pred = np.minimum(1-epsilon, y_pred)\n",
    "    log_loss = -np.mean(y_true*np.log(y_pred) + (1-y_true)*np.log(1-y_pred))\n",
    "    return round(log_loss, 7)\n",
    "\n",
    "\n",
    "### END FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7c1ca05f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log Loss value:  1.2540518\n",
      "Accuracy:  0.9637\n"
     ]
    }
   ],
   "source": [
    "y_pred = svc.predict(X_test)\n",
    "print('Log Loss value: ', custom_scoring_function(y_test, y_pred))\n",
    "print('Accuracy: ',round(accuracy_score(y_test,y_pred),4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "102ba577",
   "metadata": {},
   "source": [
    "_**Expected outputs:**_\n",
    "```python\n",
    "print('Log Loss value: ',custom_scoring_function(y_test,y_pred))\n",
    "print('Accuracy: ',round(accuracy_score(y_test,y_pred),4))\n",
    "```\n",
    "\n",
    "> ```\n",
    "Log Loss value:  1.2540518\n",
    "Accuracy:  0.9637\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c302d72a",
   "metadata": {},
   "source": [
    "## Hyperparameter optimisation\n",
    "\n",
    "### Question 4.1 - Getting model parameters\n",
    "In order to improve the accuracy of our classifier, we have to search for the best possible model (`SVC` in this case) parameters. However, we first have to find out what parameters can be tuned for the given model. Write a function that returns a list of available hyperparameters for a given model. \n",
    "\n",
    "_**Function specifications:**_\n",
    "* Should take in an sklearn model (estimator) object.\n",
    "* Should return a list of parameters for the given model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "97e6db43",
   "metadata": {},
   "outputs": [],
   "source": [
    "### START FUNCTION\n",
    "def get_model_hyperparams(model):\n",
    "    return [key for key in model.get_params()]\n",
    "\n",
    "### END FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "61609a14",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['C',\n",
       " 'break_ties',\n",
       " 'cache_size',\n",
       " 'class_weight',\n",
       " 'coef0',\n",
       " 'decision_function_shape',\n",
       " 'degree',\n",
       " 'gamma',\n",
       " 'kernel',\n",
       " 'max_iter',\n",
       " 'probability',\n",
       " 'random_state',\n",
       " 'shrinking',\n",
       " 'tol',\n",
       " 'verbose']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_model_hyperparams(svc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d36b731a",
   "metadata": {},
   "source": [
    "_**Expected outputs:**_\n",
    "\n",
    "```python\n",
    "get_model_hyperparams(SVC)\n",
    "```\n",
    "\n",
    "> ```\n",
    "['C',\n",
    " 'cache_size',\n",
    " 'class_weight',\n",
    " 'coef0',\n",
    " .\n",
    " .\n",
    " .\n",
    " 'shrinking',\n",
    " 'tol',\n",
    " 'verbose']\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55616b10",
   "metadata": {},
   "source": [
    "### Question 4.2 - Hyperparameter search\n",
    "The next step is define a set of `SVC` hyperparameters to search over. Write a function that searches for optimal parameters using the given dictionary of hyperparameters:\n",
    "\n",
    "- C_list = [0.1, 1, 10]\n",
    "- {C: 0.1, 1, 10}\n",
    "- gamma_list = [0.01, 0.1, 1]\n",
    "- {gamma: 0.01, 0.1, 1}\n",
    "- D = {'C':[0.1, 1, 10], 'gamma': [0.01, 0.1, 1]}\n",
    "\n",
    "and using `custom_scoring_function` from **Question 3** above as a custom scoring function (_**Hint**_: Have a look at at the `make_scorer` object in sklearn `metrics`).\n",
    "\n",
    "_**Function specifications:**_\n",
    "* Should define a parameter grid using the given list of `SVC` hyperparameters\n",
    "* Should return an sklearn `GridSearchCV` object with a cross validation of 5.\n",
    "* Should return a value rounded to 4 decimal places."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "624df1be",
   "metadata": {},
   "outputs": [],
   "source": [
    "### START FUNCTION\n",
    "from sklearn.metrics import make_scorer\n",
    "def tune_SVC_model(X_train, y_train):\n",
    "    D = {'C':[0.1, 1, 10], 'gamma':[0.01, 0.1, 1]}\n",
    "    scorer = make_scorer(custom_scoring_function, greater_is_better=False)\n",
    "    model = GridSearchCV(SVC(), param_grid=D, scoring=scorer, cv=5)\n",
    "    model.fit(X_train, y_train)\n",
    "    return model\n",
    "### END FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7b70eadc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log Loss value:  1.2115421\n",
      "Accuracy:  0.9649\n"
     ]
    }
   ],
   "source": [
    "svc_tuned = tune_SVC_model(X_train, y_train)\n",
    "y_pred = svc_tuned.predict(X_test)\n",
    "print('Log Loss value: ',custom_scoring_function(y_test,y_pred))\n",
    "print('Accuracy: ',round(accuracy_score(y_test,y_pred),4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28a40a4e",
   "metadata": {},
   "source": [
    "_**Expected outputs:**_\n",
    "```python\n",
    "print('Log Loss value: ',custom_scoring_function(y_test,y_pred))\n",
    "print('Accuracy: ',round(accuracy_score(y_test,y_pred),4))\n",
    "```\n",
    "\n",
    "> ```\n",
    "Log Loss value:  1.2115421\n",
    "Accuracy:  0.9649\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be137501",
   "metadata": {},
   "source": [
    "### Question 4.3 - Optimal model parameters\n",
    "Write a function that returns the best hyperperameters for a given model (i.e. the `GridSearchCV`). \n",
    "\n",
    "_**Function specifications:**_\n",
    "* Should take in an sklearn GridSearchCV object.\n",
    "* Should return a dictionary of optimal parameters for the given model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "7d32de7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "### START FUNCTION\n",
    "# function that returns best params\n",
    "def get_best_params(model):\n",
    "    return model.best_params_\n",
    "### END FUNCTION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "701e0d61",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1, 'gamma': 1}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_best_params(svc_tuned)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9fc18a4",
   "metadata": {},
   "source": [
    "_**Expected outputs:**_\n",
    "```python\n",
    "get_best_params(svc_tuned)\n",
    "```\n",
    "\n",
    "> ```\n",
    "{'C': 1, 'gamma': 1}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26fd094b-0fee-46f1-a4b8-73766813c42b",
   "metadata": {
    "tags": []
   },
   "source": [
    "#  \n",
    "\n",
    "<div align=\"center\" style=\" font-size: 80%; text-align: center; margin: 0 auto\">\n",
    "<img src=\"https://raw.githubusercontent.com/Explore-AI/Pictures/master/ExploreAI_logos/EAI_Blue_Dark.png\"  style=\"width:200px\";/>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}