1. Statistical Learning/External Lab - Residency 2_hypothesis_testing_questions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xx5CxeVubU3W"
   },
   "source": [
    "# Hypothesis Testing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6OR23eWKbU3X"
   },
   "source": [
    "The purpose of the test is to tell if there is any significant difference between two data sets.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "604OoWnTbU3Z"
   },
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o-il1iBSbU3b"
   },
   "source": [
    "This module covers,\n",
    "\n",
    "1) One sample and Two sample t-tests\n",
    "\n",
    "2) ANOVA\n",
    "\n",
    "3) Type I and Type II errors\n",
    "\n",
    "4) Chi-Squared Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "G2yYl5cvbU3c"
   },
   "source": [
    "## Question 1 \n",
    "\n",
    "*A student is trying to decide between two GPUs. He want to use the GPU for his research to run Deep learning algorithms, so the only thing he is concerned with is speed.*\n",
    "\n",
    "*He picks a Deep Learning algorithm on a large data set and runs it on both GPUs 15 times, timing each run in hours. Results are given in the below lists GPU1 and GPU2.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ohh5XJ4ZbU3d"
   },
   "outputs": [],
   "source": [
    "from scipy import stats \n",
    "import numpy as np\n",
    "import scipy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6yGkYI6EbU3i"
   },
   "outputs": [],
   "source": [
    "GPU1 = np.array([11,9,10,11,10,12,9,11,12,9,11,12,9,10,9])\n",
    "GPU2 = np.array([11,13,10,13,12,9,11,12,12,11,12,12,10,11,13])\n",
    "\n",
    "#Assumption: Both the datasets (GPU1 & GPU 2) are random, independent, parametric & normally distributed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "L55jqmIXbU3m"
   },
   "source": [
    "Hint: You can import ttest function from scipy to perform t tests "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "v8fsIhPFbU3n"
   },
   "source": [
    "**First T test**\n",
    "\n",
    "*One sample t-test*\n",
    "\n",
    "Check if the mean of the GPU1 is equal to zero.\n",
    "- Null Hypothesis is that mean is equal to zero.\n",
    "- Alternate hypothesis is that it is not equal to zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wgMSWwApbU3o"
   },
   "outputs": [],
   "source": [
    "from scipy.stats import ttest_1samp\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ttest_1sampResult(statistic=34.056241516158195, pvalue=7.228892044970457e-15)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_GPU1=ttest_1samp(GPU1,popmean=0)\n",
    "mean_GPU1\n",
    "## as the probability value i.e. the pvalue is pvalue=7.228892044970457e-15 much lesser than 0.05 we have to reject our null hypothesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "byu8iw46bU3v"
   },
   "source": [
    "## Question 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E65pzWcJbU3w"
   },
   "source": [
    "Given,\n",
    "\n",
    "Null Hypothesis : There is no significant difference between data sets\n",
    "\n",
    "Alternate Hypothesis : There is a significant difference\n",
    "\n",
    "*Do two-sample testing and check whether to reject Null Hypothesis or not.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "N-zpWvyXbU32"
   },
   "source": [
    "https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ttest_ind.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UDDkuOtObU3x"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ttest_indResult(statistic=-2.627629513471839, pvalue=0.013794282041452725)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "scipy.stats.ttest_ind(GPU1, GPU2)\n",
    "## as the probability value i.e. the pvalue is pvalue=0.013794282041452725 is much lesser than 0.05 we have to reject our null hypothesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MbXv5aZvbU33"
   },
   "source": [
    "## Question 3\n",
    "\n",
    "He is trying a third GPU - GPU3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kkh_sQl4bU34"
   },
   "outputs": [],
   "source": [
    "GPU3 = np.array([9,10,9,11,10,13,12,9,12,12,13,12,13,10,11])\n",
    "\n",
    "\n",
    "#Assumption: Both the datasets (GPU1 & GPU 3) are random, independent, parametric & normally distributed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WoYNz3g7bU37"
   },
   "source": [
    "*Do two-sample testing and check whether there is significant differene between speeds of two GPUs GPU1 and GPU3.*\n",
    "\n",
    "#### Answer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X4N11XArbU38"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ttest_indResult(statistic=-1.4988943759093303, pvalue=0.14509210993138993)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scipy.stats.ttest_ind(GPU1, GPU3)\n",
    "## as the probability value i.e. the pvalue is pvalue=0.14509210993138993 is much larger than 0.05 we have to accept our null hypothesis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oyDFS4WZbU4A"
   },
   "source": [
    "## ANOVA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wknPzstFbU4B"
   },
   "source": [
    "## Question 4 \n",
    "\n",
    "If you need to compare more than two data sets at a time, an ANOVA is your best bet. \n",
    "\n",
    "*The results from three experiments with overlapping 95% confidence intervals are given below, and we want to confirm that the results for all three experiments are not significantly different.*\n",
    "\n",
    "But before conducting ANOVA, test equality of variances (using Levene's test) is satisfied or not. If not, then mention that we cannot depend on the result of ANOVA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kGb0GeK8bU4C"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "e1 = np.array([1.595440,1.419730,0.000000,0.000000])\n",
    "e2 = np.array([1.433800,2.079700,0.892139,2.384740])\n",
    "e3 = np.array([0.036930,0.938018,0.995956,1.006970])\n",
    "\n",
    "#Assumption: All the 3 datasets (e1,e2 & e3) are random, independent, parametric & normally distributed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_xiYN6gVbU4G"
   },
   "source": [
    "Perform levene test on the data\n",
    "\n",
    "The Levene test tests the null hypothesis that all input samples are from populations with equal variances. Levene’s test is an alternative to Bartlett’s test bartlett in the case where there are significant deviations from normality.\n",
    "\n",
    "source: scipy.org"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "f2MlJTXgbU4H"
   },
   "source": [
    "#### Answer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VUJP_GGQbU4R"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LeveneResult(statistic=2.6741725711150446, pvalue=0.12259792666001798)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.levene(e1,e2,e3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FigxGCQtbU4Y"
   },
   "source": [
    "## Question 5\n",
    "\n",
    "The one-way ANOVA tests the null hypothesis that two or more groups have the same population mean. The test is applied to samples from two or more groups, possibly with differing sizes.\n",
    "\n",
    "use stats.f_oneway() module to perform one-way ANOVA test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "P3bPYPCbbU4Z"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "F_onewayResult(statistic=2.51357622845924, pvalue=0.13574644501798466)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.f_oneway(e1,e2,e3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yrMK1qb7bU4j"
   },
   "source": [
    "## Question 6\n",
    "\n",
    "*In one or two sentences explain about **TypeI** and **TypeII** errors.*\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#A type I error takes place when the null hypothesis is true, but is rejected.\n",
    "#A type II error takes place when the null hypothesis is false, but is not rejected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hm7v3pcIbU4m"
   },
   "source": [
    "## Question 7 \n",
    "\n",
    "You are a manager of a chinese restaurant. You want to determine whether the waiting time to place an order has changed in the past month from its previous population mean value of 4.5 minutes. \n",
    "State the null and alternative hypothesis.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Null Hypothesis: mean=4.5 i.e. population mean is 4.5 minutes\n",
    "#Alternate hypothesis: mean!=4.5 i.e. population mean is not 4.5 minutes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1SGpq-dKbU4r"
   },
   "source": [
    "## Chi square test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WlxmIu_rdgpc"
   },
   "source": [
    "## Question 8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8J3V015PbU4s"
   },
   "source": [
    "Let's create a small dataset for dice rolls of four players"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xrO5BbIEbU4t"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "d1 = [5, 8, 3, 8]\n",
    "d2 = [9, 6, 8, 5]\n",
    "d3 = [8, 12, 7, 2]\n",
    "d4 = [4, 16, 7, 3]\n",
    "d5 = [3, 9, 6, 5]\n",
    "d6 = [7, 2, 5, 7]\n",
    "\n",
    "dice = np.array([d1, d2, d3, d4, d5, d6])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uF7GRMChbU4x"
   },
   "source": [
    "run the test using SciPy Stats library\n",
    "\n",
    "Depending on the test, we are generally looking for a threshold at either 0.05 or 0.01. Our test is significant (i.e. we reject the null hypothesis) if we get a p-value below our threshold.\n",
    "\n",
    "For our purposes, we’ll use 0.01 as the threshold.\n",
    "\n",
    "use stats.chi2_contingency() module \n",
    "\n",
    "This function computes the chi-square statistic and p-value for the hypothesis test of independence of the observed frequencies in the contingency table\n",
    "\n",
    "Print the following:\n",
    "\n",
    "- chi2 stat\n",
    "- p-value\n",
    "- degree of freedom\n",
    "- contingency\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vqaTIKmgbU4y"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chi2 stat 23.315671914716496 \n",
      " p-value 0.07766367301496693 \n",
      " degree of freedom 15 \n",
      " contingency [[ 5.57419355  8.20645161  5.57419355  4.64516129]\n",
      " [ 6.50322581  9.57419355  6.50322581  5.41935484]\n",
      " [ 6.73548387  9.91612903  6.73548387  5.61290323]\n",
      " [ 6.96774194 10.25806452  6.96774194  5.80645161]\n",
      " [ 5.34193548  7.86451613  5.34193548  4.4516129 ]\n",
      " [ 4.87741935  7.18064516  4.87741935  4.06451613]]\n"
     ]
    }
   ],
   "source": [
    "chi2_stat,p_value,degree_of_freedom,contingency_table=stats.chi2_contingency(dice)\n",
    "print('chi2 stat',chi2_stat,'\\n','p-value',p_value,'\\n','degree of freedom',degree_of_freedom,'\\n','contingency',contingency_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VzyfaNoabU42"
   },
   "source": [
    "## Question 9\n",
    "\n",
    "### Z-test\n",
    "\n",
    "Get zscore on the above dice data using stats.zscore module from scipy. Convert zscore values to p-value and take mean of the array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Sd5xPCuRbU43"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "zscore= [[-0.46291005 -0.18884739 -1.83711731  1.44115338]\n",
      " [ 1.38873015 -0.64208114  1.22474487  0.        ]\n",
      " [ 0.9258201   0.7176201   0.61237244 -1.44115338]\n",
      " [-0.9258201   1.62408759  0.61237244 -0.96076892]\n",
      " [-1.38873015  0.03776948  0.          0.        ]\n",
      " [ 0.46291005 -1.54854863 -0.61237244  0.96076892]]\n"
     ]
    }
   ],
   "source": [
    "zs=stats.zscore(dice)\n",
    "print('zscore=',zs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "prob values [[0.32171442 0.42510621 0.03309629 0.07477068]\n",
      " [0.08245741 0.26041025 0.11033568 0.5       ]\n",
      " [0.17726974 0.23649578 0.27014569 0.07477068]\n",
      " [0.17726974 0.05217856 0.27014569 0.16833418]\n",
      " [0.08245741 0.48493574 0.5        0.5       ]\n",
      " [0.32171442 0.06074513 0.27014569 0.16833418]]\n"
     ]
    }
   ],
   "source": [
    "p_values = scipy.stats.norm.sf(abs(zs))\n",
    "print('prob values',p_values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean 0.23428473233691496\n"
     ]
    }
   ],
   "source": [
    "x=p_values.mean()\n",
    "print('mean',x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OyoFUxf5bU47"
   },
   "source": [
    "## Question 10\n",
    "\n",
    "A Paired sample t-test compares means from the same group at different times.\n",
    "\n",
    "The basic two sample t-test is designed for testing differences between independent groups. \n",
    "In some cases, you might be interested in testing differences between samples of the same group at different points in time. \n",
    "We can conduct a paired t-test using the scipy function stats.ttest_rel(). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "SwVmQ1gRbU48"
   },
   "outputs": [],
   "source": [
    "before= stats.norm.rvs(scale=30, loc=100, size=500) ## Creates a normal distribution with a mean value of 100 and std of 30\n",
    "after = before + stats.norm.rvs(scale=5, loc=-1.25, size=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rB_os5FjbU4_"
   },
   "source": [
    "Test whether a weight-loss drug works by checking the weights of the same group patients before and after treatment using above data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Fq2wyEqlbU5A"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ttest_relResult(statistic=5.255902324156215, pvalue=2.1872166611130706e-07)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.ttest_rel(before,after)\n",
    "# as p_value is less than 0.05 so the weight loss drug doesn't work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "hypothesis_testing_questions.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}