week_2_house_prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logo price](./imgs/logo.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Instructors\n",
    "* Nguyễn Hoàng Phương  - Data Scientist từ Torus-Actions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Phần I - Tuần 2:    Bài toán Regression -  xây dựng mô hình dự đoán giá nhà \n",
    "![home price](./imgs/home_prices.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outline\n",
    "- <a href='#1'>1. Xử lý dữ liệu  </a>\n",
    "    - <a id='#1-1'>1.1. Xử lý dữ liệu thiếu (missing data) </a> \n",
    "    - <a id='#1-1'>1.2. Xử lý dữ liệu số  </a> \n",
    "    - <a id='#1-1'>1.3. Xử lý dữ liệu phân loại  </a>\n",
    "\n",
    "- <a href='#2'>2. Thuật toán lightgbm và tối ưu bayes (bayesian optimization)  </a>  \n",
    "    - <a id='#2-1'>2.1. Thuật toán lightgbm và các tham số quan trọng </a> \n",
    "    - <a id='#2-1'>2.2. Tối ưu Bayes (Bayesian optimization) </a> \n",
    "    - <a id='#2-1'>3.1. Sử dụng tối ưu bayes để tối ưu tham số thuật toán lightgbm </a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nhập các gói thư viện "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import packages\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from scipy import stats\n",
    "import warnings\n",
    "import sys\n",
    "import os\n",
    "warnings.filterwarnings('ignore')\n",
    "%matplotlib inline\n",
    "from sklearn.preprocessing import RobustScaler\n",
    "from sklearn.model_selection import KFold, cross_val_score\n",
    "from sklearn.metrics import mean_squared_error\n",
    "import lightgbm as lgb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Đọc dữ liệu "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_csv('./data/house-prices/train.csv')\n",
    "test  = pd.read_csv('./data/house-prices/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>...</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>208500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>181500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>223500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>70</td>\n",
       "      <td>RL</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "      <td>140000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>50</td>\n",
       "      <td>RL</td>\n",
       "      <td>85.0</td>\n",
       "      <td>14115</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MnPrv</td>\n",
       "      <td>Shed</td>\n",
       "      <td>700</td>\n",
       "      <td>10</td>\n",
       "      <td>2009</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>143000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>75.0</td>\n",
       "      <td>10084</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>307000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10382</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Shed</td>\n",
       "      <td>350</td>\n",
       "      <td>11</td>\n",
       "      <td>2009</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>50</td>\n",
       "      <td>RM</td>\n",
       "      <td>51.0</td>\n",
       "      <td>6120</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "      <td>129900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>190</td>\n",
       "      <td>RL</td>\n",
       "      <td>50.0</td>\n",
       "      <td>7420</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>118000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 81 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "0   1          60       RL         65.0     8450   Pave   NaN      Reg   \n",
       "1   2          20       RL         80.0     9600   Pave   NaN      Reg   \n",
       "2   3          60       RL         68.0    11250   Pave   NaN      IR1   \n",
       "3   4          70       RL         60.0     9550   Pave   NaN      IR1   \n",
       "4   5          60       RL         84.0    14260   Pave   NaN      IR1   \n",
       "5   6          50       RL         85.0    14115   Pave   NaN      IR1   \n",
       "6   7          20       RL         75.0    10084   Pave   NaN      Reg   \n",
       "7   8          60       RL          NaN    10382   Pave   NaN      IR1   \n",
       "8   9          50       RM         51.0     6120   Pave   NaN      Reg   \n",
       "9  10         190       RL         50.0     7420   Pave   NaN      Reg   \n",
       "\n",
       "  LandContour Utilities  ... PoolArea PoolQC  Fence MiscFeature MiscVal  \\\n",
       "0         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "1         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "2         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "3         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "4         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "5         Lvl    AllPub  ...        0    NaN  MnPrv        Shed     700   \n",
       "6         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "7         Lvl    AllPub  ...        0    NaN    NaN        Shed     350   \n",
       "8         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "9         Lvl    AllPub  ...        0    NaN    NaN         NaN       0   \n",
       "\n",
       "  MoSold YrSold  SaleType  SaleCondition  SalePrice  \n",
       "0      2   2008        WD         Normal     208500  \n",
       "1      5   2007        WD         Normal     181500  \n",
       "2      9   2008        WD         Normal     223500  \n",
       "3      2   2006        WD        Abnorml     140000  \n",
       "4     12   2008        WD         Normal     250000  \n",
       "5     10   2009        WD         Normal     143000  \n",
       "6      8   2007        WD         Normal     307000  \n",
       "7     11   2009        WD         Normal     200000  \n",
       "8      4   2008        WD        Abnorml     129900  \n",
       "9      1   2008        WD         Normal     118000  \n",
       "\n",
       "[10 rows x 81 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='1'>1. Feature engineering </a>\n",
    "\n",
    "### Feature engineeing tuần 1: \n",
    "   +  Xóa các hàng dữ liệu bất thường (là các hàng dữ liệu có diện tích lớn hơn 4000 và giá nhà bé hơn 200.000)\n",
    "   +  Xử lý dữ liệu thiếu: \n",
    "        + Thay giá trị NaN ở các trường có tỷ lệ thấp bằng giá trị phân loại có tần suất xuất hiện cao nhất\n",
    "        + Điền NaN ở trường numeric bằng giá trị  0, điền NaN ở các trường phân loại bằng giá trị \"None\"\n",
    "   +  Xử lý dữ liệu số : \n",
    "        + Log transform các cột dữ liệu có giá trị lớn\n",
    "        + Xử lý các cột dữ liệu thời gian (cột năm xây dựng => cột thời gian sử dụng, tính tới năm 2018)\n",
    "   +  Xử lý dữ liệu phân loại: \n",
    "        + One hot encoding tất cả trường phân loại "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ==>> Kết quả RMSLE trên cross-validation đạt: 0.121"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature engineering tuần 2: \n",
    "Ở tuần thứ hai, chúng ta tìm hiểu sâu hơn data, từ đó đề xuất những phương pháp xử lý data phù hợp hơn cho mỗi trường dữ liệu. \n",
    "### Tổng quan các bước xử lý data: \n",
    "   +  Xóa các hàng dữ liệu bất thường (là các hàng dữ liệu có diện tích lớn hơn 4.000 và giá nhà thấp hơn 200.000)\n",
    "   +  Xử lý dữ liệu thiếu: \n",
    "        +  Tạo cột dữ liệu mới tương ứng với NaN hoặc không Nan tương ứng với các cột dữ liệu thiếu \n",
    "        +  Qua tìm hiểu ý nghĩa của giá trị thiếu ở mỗi trường dữ liệu, điền NaN bằng các số liệu thích hợp hơn.\n",
    "   +  Xử lý dữ liệu dạng số : \n",
    "        +  Giới thiệu phép biến đổi mới (box-cox) để chuẩn hóa những dữ liệu skewed. \n",
    "        +  Xử lý các cột dữ liệu thời gian (cột năm xây dựng => cột thời gian sử dụng, tính tới năm 2018)\n",
    "        +  Tạo những cộ dữ liệu mới. \n",
    "   +  Xử lý dữ liệu phân loại: \n",
    "        + Lable encoding những trường dữ liệu mang tính chất thứ tự. Ví dụ: ở trường dữ liệu \"ExterCond\" có các giá trị phân loại \n",
    "          + Ex\tExcellent       -->5\n",
    "          + Gd\tGood            -->4\n",
    "          + TA\tAverage/Typical -->3\n",
    "          + Fa\tFair            -->2\n",
    "          + Po\tPoor            -->1\n",
    "        + One hot encoding các trường dữ liệu phân loại còn lại "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='1.1*'>1.1*. Xóa các cột dữ liệu bất thường </a>\n",
    "\n",
    "Như đã nhận xét ở tuần trước,  các hàng dữ liệu bất thường gây ra hiện tượng underfitting, việc xóa các hàng này sẽ giúp cho việc xây dựng mô hình bằng phương pháp cross validation có độ tin tưởng cao hơn. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='GrLivArea', ylabel='SalePrice'>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEGCAYAAACpXNjrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABXLUlEQVR4nO29e3xdZ3Xn/V17n5uulnyRrzKxEweTBEISx0laJrjhlgBNaBvA7syQAiWZKeXSznQaWgrT0PYNM5+3FDoMxC+lJNPWJoRC0paEJgQT6MT4ljtxYkdJLDm2ZVt3nfve6/1j73N0JB9JR9I5ki2tLx9xznnO3vvZW5H32utZa/2WqCqGYRiGUU2cuT4BwzAMY/5hxsUwDMOoOmZcDMMwjKpjxsUwDMOoOmZcDMMwjKoTmesTOFtYunSpnnfeeXN9GoZhGOcU+/fvP6Wqy8aOm3EJOe+889i3b99cn4ZhGMY5hYi8Wm7clsUMwzCMqmPGxTAMw6g6ZlwMwzCMqmPGxTAMw6g6ZlwMwzCMqmPZYoZhLAh2Hezmrsc66OxN0t5az23XrmfLxra5Pq15i3kuhmHMe3Yd7OZzDzxH92Calroo3YNpPvfAc+w62D3XpzZvMeNiGMa8567HOoi6Qn0sgkjwGnWFux7rmOtTm7fYsphhGPOezt4kLXXRUWN1UZeu3uQZ29ryWXUwz8UwjHlPe2s9qZw3aiyV81jTWj9qzJbPqocZF8Mw5j23XbuenKcks3lUg9ecp9x27fpR29nyWfUw42IYxrxny8Y27rjxYtqaEvSncrQ1JbjjxovPWO7q7E1SF3VHjY23fGZMjMVcDMNYEGzZ2DZp7KS9tZ7uwTT1sZFbY7nlM2NyzHMxDMMIqXT5zJgcMy6GYRghlS6fGZNjy2KGYRglVLJ8ZkyOeS6GYRhG1THjYhiGYVQdMy6GYRhG1THjYhiGYVSdmhkXEXm9iDxZ8jMgIp8WkcUi8rCIHApfW8PtRUS+IiKHReRpEbm85Fi3hNsfEpFbSsavEJFnwn2+IiISjpedwzAMw5gdamZcVPUFVX2zqr4ZuAJIAt8Dbgd+pKobgB+FnwFuADaEP7cCX4PAUACfB64CNgOfLzEWXwM+VrLf9eH4eHMYhmEYs8BsLYu9DXhJVV8FbgLuDsfvBt4Xvr8JuEcDdgMtIrISeBfwsKr2qGov8DBwffhds6ruVlUF7hlzrHJzGIZhGLPAbBmXrcCO8P1yVT0Wvj8OLA/frwY6S/bpCscmGu8qMz7RHKMQkVtFZJ+I7Dt58uSUL8owDMMoT82Ni4jEgBuB74z9LvQ4tJbzTzSHqm5X1U2qumnZsmW1PA3DMIwFxWx4LjcAB1T1RPj5RLikRfhaaJRwFGgv2W9NODbR+Joy4xPNYRiGYcwCs2FctjGyJAbwAFDI+LoFuL9k/ENh1tjVQH+4tPVD4J0i0hoG8t8J/DD8bkBErg6zxD405ljl5jAMwzBmgZpqi4lIA/AO4LaS4TuBe0Xko8CrwAfC8R8A7wYOE2SWfRhAVXtE5AvA3nC7O1S1J3z/O8C3gDrgwfBnojkMwzCMWUCCkISxadMm3bdv31yfhmEYxjmFiOxX1U1jx00V2TAMw5gy6ZxH1vPH/d6Mi2EYhlExmbxHXzLHcCZPY2J8E2LGxTAMw5iUnOfTO5xlKJOvaHszLoZhGMa45D2f3mSOoUzQ+rlSzLgYhmEYZ+D5Sl8yy0B6akalgBkXwzAMo4jvK/2pHP2pHP4MsonNuBiGYRioKgOpPH2pLJ4/8xIVMy6GYRgLGFVlIJ2nP5kj74+fWjxVzLgYhmEsUAbTOfqSOXIT1KtMFzMuhmEYC4zhTJ6e4WxNjEoBMy6GYRgLhFTWoyeZJZPzaj6XGRfDMIx5Tjrn0ZvMksrW3qgUMONiGMa02XWwm7se66CzN0l7az23XbueLRvb5vq0jJBM3qN3OEcyW1lVfTWZrTbHhmHMM3Yd7OZzDzxH92Calroo3YNpPvfAc+w6aL355pps3qd7IM3R3tScGBYwz8UwjGly12MdRF2hPhbcRupjEZLZPHc91mHeyxwxXamWqaKqPH20n/v2dY27jRkXwzCmRWdvkpa66KixuqhLV29yjs5o4TJTqZZK8VV5/KXT7NhzhF8cG5xwWzMuxoLGYgbTp721nu7BdNFzAUjlPNa01s/hWS0sfF/pS+UYmKFUy2TkPJ8fPd/Nt/d28mrPyMPDJauaeXWcfcy4GAuWQswg6sqomMEdYAamAm67dj2fe+A5ktk8dVGXVM4j5ym3Xbt+rk9t3qM6ov9VDamW8Uhm8/zL08e4b/9RTg5liuOb1y1m25Xt/NIFS1jxqfL7mnExFiwWM5gZWza2cQfB77GrN8ka8/xqTq2kWsbSl8zyj08c5f4nX2MwHSQEOAK/8vo2tl7ZzvltjQCIyLjHqKlxEZEW4BvAJYACHwFeAL4NnAe8AnxAVXslOMsvA+8GksBvqeqB8Di3AJ8ND/tnqnp3OH4F8C2gDvgB8ClVVRFZXG6OWl6rce5hMYOZs2VjmxmTWaKWUi0FjvWnuHdfFw89e5xMPpgnHnG44ZIVfGBTOysWJSo+Vq09ly8DD6nqzSISA+qBPwJ+pKp3isjtwO3AHwI3ABvCn6uArwFXhYbi88AmAgO1X0QeCI3F14CPAT8nMC7XAw+Gxyw3h2EUsZiBcS4wlMnTW2Oplpe6h9ixt5NdL3RTWGVrSkR435tX8WuXraalPjblY9bMuIjIIuBa4LcAVDULZEXkJmBLuNndwC6CG/9NwD0apDrsFpEWEVkZbvuwqvaEx30YuF5EdgHNqro7HL8HeB+BcRlvDsMoYjED42wmmQ30v7L52hgVVeXprn527DnCnldGFnaWNca5edMa3vvGldTF3Gkfv5aeyzrgJPC3InIpsB/4FLBcVY+F2xwHlofvVwOdJft3hWMTjXeVGWeCOUYhIrcCtwKsXbt2ipdnnOtYzMA4G0nnPHqGs6RrpP/lq/Jvh0+zc+8Rni9JJ37d4nq2bm7nuo1tRN2Z19fX0rhEgMuBT6jqz0XkywTLU0XC+EjtUh0mmUNVtwPbATZt2lTT8zDOTuYiZmDpz0Y5ai3VkvN8HvnFCXbu7aSzN1Ucv2hlM9s2t3PN+UtwJgjQT5VaGpcuoEtVfx5+vo/AuJwQkZWqeixc9ipoRRwF2kv2XxOOHWVkiaswviscX1NmeyaYwzDmFEt/NsaSzfv0JbMMZWpjVJLZPP/01DHuO9DF6aFscfyqdYvZtrmdN65eNGHW13SpmXFR1eMi0ikir1fVF4C3Ab8If24B7gxf7w93eQD4XRHZSRDQ7w+Nww+BvxCR1nC7dwKfUdUeERkQkasJAvofAv665Fjl5jCMOcXSn40COc+nN5llKF0bo9IznOV7YTpxwXA5AtdtDNKJ1y9rrMm8BWqdLfYJ4O/DTLEO4MMEYpn3ishHgVeBD4Tb/oAgDfkwQSryhwFCI/IFYG+43R2F4D7wO4ykIj8Y/kBgVMrNYRhziqU/G56v9CazDNZIquW1vjCd+LnjxWSAeMTh3W9cyfuvWDOldOKZUFPjoqpPEqQQj+VtZbZV4OPjHOebwDfLjO8jqKEZO3663ByGMddY+vPCxfNHquprYVQOdw+xY88RfvLiyWI6cXMiwvsuW82vvXk1i+qjEx+gyliFvmHMIpb+vPDwfWUgLICstv6XqvJkZx879nSy79WRdOK2pjjv37SGd79xJXXR6acTzwQzLoYxi1j688JBVRlI5elLZauu/+X5yr+9dIodezp54fhIOvF5S+rZemWQThypQjrxTDDjYhizjEmmzH8G0jn6hquv/5XN+zz8ixN8e18nXSXpxJesambb5rVctX5xVdOJZ4IZF8Ng7mpPrOZlflErqZbhTJ5/evoY393fxenhkXTia9YvYdvmdi5Zvaiq81UDMy7Ggmeuak+s5mX+UCuplp7hLN890MUDT73GcCao2Hcd4W0b2/jgle2sW9pQ1fmmijBHqsiGcS4wV7UnVvNy7lMrqZajvSnu3dfJQ88dJ+cF8ZpExOHdbwrSiZc3z0468XjUxVyaE1HqJ9AeM+NiLHjmqvbEal7OXdI5j95kllS2ukblxROD7NzTyWOHRqcT/9plq3nfZatZVDe76cSluI7QlIjSlIhUpD1mxsVY8MxV7YnVvJx7ZPNBVf1wFaVaVJUnjvSxY28n+0vSiZc3x3n/Fe3c8MYVc5ZODIGX0pSI0hBzpyQTY8bFWPDMVe3JbM5riQMzoxZSLZ6v/OzwKXbu6eSFEyPpxOuWNrD1ynZ+5fXL5iyd2HWExniEpkSUWGR652DGxVjwzFXtyWzNa4kD0yfv+fSlclWVasnmff71Fye4d0w68RtXL2Lb5nauWre4JkKSlZCIujTXTd1LKYfUQobgXGTTpk26b9++uT4Nw6g627bvPmP5LZnN09aUYMetV8/hmZ29eL7Sl8wyUEWjMpTJ809PvcZ3DxylpySd+JfOX8LWK+cunXimXoqI7FfVM2S+zHMxjHmOJQ5Ujl+i/1UtqZbTQxm+e+Ao//TUawxnR9KJ3/6GIJ34vCVzk06ciLo0JSI0xiPnluS+YRhnB5Y4MDm1kGo52pvi2/s6+WFpOnHU4b1vWsnNl6+hbQ7SiR0RGhMRmmcQS6kUMy6GMc8xsczxUVUGM/mqSrW8eGKQHXs6+WlJOvGiuii/ftlqbnrzKprnIJ04HnVprqGXUg4zLoYxzzGxzPJUU6pFVTlwpI8de45w4EhfcXx5c5wPbGrnhktWkJjldOKCl9KUiBCPzH4qsxkXw1gAmFjmCMOZPL3J6ki1eL7y00Mn2bGnk0PdQ8Xx9Usb2Lq5nS0Xzn46cbwQS4lFcJy5E7E042IYxoIglfXoSWbJVEGqJZv3+eFzx7l3XxdH+0bSiS9ds4itm9vZfN7sphM7IjTEIzTXzY2XUg4zLoZhzGuqKdUylMnzwJOv8d0DXfQmc8XxX75gCduuXMtFq5pnPMdUiEUcmuuic+6llKOmxkVEXgEGAQ/Iq+omEVkMfBs4D3gF+ICq9kpg5r8MvBtIAr+lqgfC49wCfDY87J+p6t3h+BXAt4A64AfAp1RVx5ujltdqnJ1YZfrCJZP36B3OkczOvKq+kE78wFOvkQyNVMQR3vaGNrZe2c7rZjGduOClNCUisx7HmQqz4bn8iqqeKvl8O/AjVb1TRG4PP/8hcAOwIfy5CvgacFVoKD4PbAIU2C8iD4TG4mvAx4CfExiX64EHJ5jDWEBYZfrCJJv36UtmGaqC/ldnT5Jv7+vk4V+cGJVO/KtvWsXNV6xhWVN8xnNUSiziBMKR8bPPSynHXCyL3QRsCd/fDewiuPHfBNyjQTnsbhFpEZGV4bYPq2oPgIg8DFwvIruAZlXdHY7fA7yPwLiMN4exgDBJ+4VF3vPpTeYYTOcm33gSDh4fYOeeTn566BSFqpeWuii/dvlqbrp09tKJRYSGeCBvfzZ7KeWotXFR4F9FRIG7VHU7sFxVj4XfHweWh+9XA50l+3aFYxONd5UZZ4I5RiEitwK3Aqxdu3bKF2ec3Vhl+sKgWlItqsq+V3vZubeTJ0rSiVcuSvCBTWu4/uIVxGfpBn+ueSnlqLVxeYuqHhWRNuBhETlY+mUYH6mpuNlEc4TGbjsE2mK1PA9j9lkolekLNa7k+0pfKsfADKVaPF/5yYsn2bm3k8Ml6cTnL2tg2+a1vPXCZbizcIM/l72UctTUuKjq0fC1W0S+B2wGTojISlU9Fi57dYebHwXaS3ZfE44dZWSJqzC+KxxfU2Z7JpjDWEAshMr0hRhXUh3R/5qJVEsm5/HQc4E68bH+dHH8ze2L2HrlWq48r3VW0omjrkNzIkpjIjIrRmy2qJlxEZEGwFHVwfD9O4E7gAeAW4A7w9f7w10eAH5XRHYSBPT7Q+PwQ+AvRKQ13O6dwGdUtUdEBkTkaoKA/oeAvy45Vrk5jAXEQqhMX0hxJVVlIJ2nPzkzqZahdJ77nzrKPx44WkwnFuCXLljCb25eyxtW1j6dWERoiAXy9vPBSylHLT2X5cD3QssfAf5BVR8Skb3AvSLyUeBV4APh9j8gSEM+TJCK/GGA0Ih8AdgbbndHIbgP/A4jqcgPhj8QGJVycxgLjPlemT5XcaXZXoobTOfoS+ZmJNVyaijDffu7+Oenj41KJ37HRcv54KZ21i6p/XLpfPVSylEz46KqHcClZcZPA28rM67Ax8c51jeBb5YZ3wdcUukchjHfmIu40mwuxQ1n8vTMUP/rSE+Se/d28q+/OEE+XEari7r86qUr+Y3La59OXPBSmhJR6mLz00sph1XoG8YMmcuA+lzElWZjKS6ZzdObzM1IquX5YwPs2NPJvx0eSSdurY/y65ev5sZLV9GUqG068ULyUsphxsU4JzlbMqTmOqA+F3GlWi7FpXMePcNZ0tM0KqrK3ld62bn3CE929hfHVy5K8MEr23nXRctrmk68UL2UcphxMc455vqGXsrZEFCf7bhSLZbiZirV4vnKrhdOsnPvEV46OVwcv6CtkW1XtnNtjdOJo65DUyJoFbwQvZRyVGxcROR1wAZVfURE6oCIqg7W7tQMozxnww29wEIs1KzmUtxMpVqCdOJAnbg0nfiytS1su7KdK15Xu3RiEaE+FsjblxpaI6Ci34iIfIygkn0xcD5BTcnXsaC5MQfM5IZe7eW0hVKoWUo1luJmKtUymM7x/Sdf43sHjtKXGkkn/ncblrJ1czsbV9QunTjiODTXBV0dZ7tXy7lEpeb24wQFkD8HUNVDYdW9Ycw6072h12I5bSEUapZjuktxM5VqOTk4kk6cCuMyUXcknbh9ce2MekGJ2LyUyqj0t5RR1WzBvRSRCGByKcacMN0bei2W08Z7igfYtn33nCccnC3MVKrlyOkkO/d28sjzI+nE9TGXGy9dxa9fvpqljbVJJ466Do2hUTEvZWpUalx+IiJ/BNSJyDsIihf/qXanZRjjM91lmVrFR8Y+xZ9NCQdzzUylWn7x2gA79h7h3w6fLo611kf5jcvXcOOlq2hMVN+LsFhKdaj0N3c78FHgGeA2gmr6b9TqpAwDJo6PTGdZZrbiIwUPyfOVl08Nk/V8XBG++NDBis75bEmzngkzkWpRVfa80sPOPZ081TWSTryqJcHWK9t550UriEWq70UUMr4sllIdKjUudcA3VfX/AxARNxybvykxxpxyLsdHOnuTuALH+jOIgOsIvq+82D3EroPdE57/fPB6pivV4vnKj1/oZufeTjpK0ok3tDWybfNa/t2GpVVP8zUvpXZU+tv8EfB2oKBHXQf8K/BLtTgpw5jN+Ei1b9rtrfU8caQXkaAlLQSZTFGXSc+/Wtc9F97PUCZP7zSkWtI5jwefPc69+zo5MZApjl++toWtNUonNi+l9lRqXBKqWmx0oKpDIjJ/cy2NOWe24iO14LZr1/PRe/biiqCAKvgoq5oSk55/Na57tr2fZDbQ/8rmp2ZUBlI57n/yNf7xiaP0l6QTX3vhMrZe2c7rVzRV9TzNS5ldKv0ND4vI5ap6AEBErgBStTstY6FTSXxkJk/ntXyy37KxjQ3LGnmlJ4nnKzHXYWljgogrtDUlJpy7GnGh2Soyna5US/dAmvsOBOnE6VxgkKKu8K6LV/CBTWuqHgMzL2VuqNS4fBr4joi8RvBwsQL4YK1OyjAmi4/M5Ol8Np7sb7/hDcU5Ss//mvWLJ5y7GnGhWqsGTFeq5ZXTw3x7byePPN9dzBxriLn86qWr+I3LV7OkyunEVpcyt1T0W1fVvSKyEXh9OPSCqk6vtNYwKmCy+Ejp0/lgOsfJwQzpvMcndz7BV7ZeNitxjemc/2RzVyMuVKusuOlKtTx7tJ+dezv5vy+NpBMvbohx8+Wree+lq2iMV+/mH3EKGl/mpcw1E/5XFZHrVPVREfn1MV9dKCKo6j/W8NyMBc5E8ZHC0/lgOsdrfWlEgsZPyaw3qRdSqyf7cstdO269etQ2n73/2UnnnmlcqNpZcTnPpzeZZShduVFRVX7+cg879hzhmaMDxfHVLXV88Mp23nnR8qqmE9fHIjTXmZdyNjHZf4m3Ao8Cv1rmOwXMuBhzQuHp/ORgppiV5fsQjwhRVyb0QmrxZF/pUtts1NoUvJ87H3yeQ91BHs76pQ1TPk7e8+lL5RicglRL3vP58Qsn+fbeTjpOjaQTX7g8SCd+ywXVSyc2L+XsZkLjoqqfFxEHeFBV752lczKMSSk8nafzHhEnMCw+ytLGxKReSC3qXSpdaptNLbJkzmdNa11xnkrjSp4/UlVfqVFJ5zx+8EyQTtw9OJJOfMXrWtm2uZ3L2luqlk5sXsq5waT/dVTVF5H/BphxMeaMcktOd9x4MZ/c+QTJrEc8IixtTNBcFyWZzU/oCUwW15hOJlklS22F4w5ncuQ8JRZx2NDWVJMalOnElXxfGQgLICvV/+pP5fj+E0f53hNHGQiXzRyBazcsY+vmdi5cXp10YvNSzj0qNf2PiMh/Bb4NFH1dVe2ZbMewmn8fcFRV3ysi64CdwBJgP/AfQ1HMOHAPcAVwGvigqr4SHuMzBPIzHvBJVf1hOH498GXABb6hqneG42XnqPBajbOMcZecbryYr2y9bFRWVjKbr8gTGC+uMd1MssmWu0qPu3JR3SiPpRa1J1OJK6kqA6k8falsxfpfJwbSfGd/Fz94+hjp/Eg68fUXr+ADm9pZ3Vo384sgMIpBxpdbs74sRm2o1LgU0o4/XjKmQCW+/KeA54FCg4UvAl9S1Z0i8nUCo/G18LVXVS8Qka3hdh8UkYuArcDFwCoCQ3dheKyvAu8AuoC9IvKAqv5igjmMs4xyXsLTXX1842cvM5TJIyKor8SiDsubEkhMRj2F77j16qpW3U83k2yy5a7ZbnBWSWxHVRnM5Okbrlz/6+VTQTrxjw6WpBPHXW66dBW/fvkaFjfEZnzuriM0JaI0JSJEzUs5Z6k0FXnddA4uImuA9wB/Dvy+BI8e1wG/GW5yN/DfCW78N4XvAe4D/le4/U3ATlXNAC+LyGGC3jIAh1W1I5xrJ3CTiDw/wRzGJMxmYWI5L+ETOw4wnPUQAb9Q3g5kcj6v9Qd1u8110VFP4dWsup9uJtlkS22z3bFyMmM3VamWZ4/2s2NPJ493jKQTL2mI8RtXrOFX37SShiqkE9eFvecbzEuZF0yWinwVsJ2g++QzwEdU9fkpHP+vgP8GFBZelwB9qlrIaewCVofvVwOdAKqaF5H+cPvVwO6SY5bu0zlm/KpJ5hh7fbcSdNhk7dq1U7is+clsFyaWe5ofznr4Cg5BtS4S2BcFHIRTQxma66JVz7AqGMaTgxlODWZYsShBUyIwBpXONZGRm+2OleMZuyvXLaarN1mRVIuvys87eti5d3Q68ZrWOrZe2c7b3zDzdGLXERrjEZrroualzDMme9z4KvBfgceAGwmMxbsqObCIvBfoVtX9IrJl+qdYO1R1O4HxZNOmTQu++dlMlm6ms++h7kGSmTy5UCJlWVOcwpK/KqF1GUFRsp6SzObpT+WIuQ5v+eKjM5ZvKTWMK5rjHO1L09WbYnWLEnGdKWdzlfPg5qJjZamxS2U9epJZTgykJ9krSCd+9GCgTvzK6RHP6vUrmti2uZ1fPn/m6cSJqEtznXkp85nJjIujqg+H778TBtYr5ZeBG0Xk3UCCIObyZaBFRCKhZ7EGOBpufxRoB7rCTpeLCAL7hfECpfuUGz89wRzGBMxk6Waq++462M1gOo+viusIeV+DYkgCL0VktIFxwloWBaKOIEDW86si3zLWMIJwYjDN8YEMl69trWh5r2BMmuIRTg5lWFQXPSP54I4bL665IvNY0jmP3mSWVHZy/a9UzuNrP36JH/7iODlv5FlrU5hO/OYZphMXvJSmRLQm/ViMs4vJjEvLmOr8UZ8nqtBX1c8AnwEIPZf/qqr/XkS+A9xMkM11C3B/uMsD4efHw+8fVVUVkQeAfxCRvyQI6G8A9hDcdjaEmWFHCYL+vxnu8+Nx5jAmYCZLNxPtW+5J/q7HOmitj3J6OIv6gTHx0WLv7GLSUviaiDh4Ck2JCK/1p6mPuVULjo81jM11QTC5P5U7o8J+LGOXAw93D5H3lYZYpGzywWz1ZJmK/ld/Msf3njzKffu6SJaIUNZHXerjLjdfvobL1rZO+1wSUbcoHGleysJhMuPyE0ZX55d+nm6F/h8CO0Xkz4AngL8Jx/8G+D9hwL6HwFigqs+JyL3AL4A88HFV9QBE5HeBHxKkIn9TVZ+bZA5jAmaydFNu34FUjkw2z0fv2UvUcVjeHOeV00Pc9nf7yeZ96mMuLXVRklmPrOfjiCAoi+oiDKTzRQMTdcDTQORwMJUj4wVdDk8PZVjdWk9TIjqj4PhMjOpYr8dTxRGKsSGobeB+LFPR/zo+kOY7+7p48JmRdGIBmusitNbHiLkOqZzHzr2dbF6/eErn4YjQGNalxCPudC7FOMeZrEL/w9WYRFV3AbvC9x2MZHuVbpMG3j/O/n9OkHE2dvwHBC2Xx46XncOYmJmIJo7dtzEeQYFTyRyuBMteR3pSRc+ksKyVTfmsWlRHc12UQ92DxCMO7YtHpEqS2TwnBzM0xF1ODGQoLcPI+dDVm2JNa7DkMt3g+EyM6livJ+Y65DyfbEkWVi0D9wWmov/VcXKInXs7efRgd/H32RiPoKq0NceJOiNLVomow/GByrtrxKMuzealGFSYiiwiy4G/AFap6g1h7ck1qmoewTxjJmm9pfte/6WfMJTJF/t1jEUJKsIdJ8gAi7hCzlPWtCRGbVcXdRnOemRyHuXq+/K+crw/TVtzYtrB8ZkY1bFez7KmOF29KSKuoKo1D9xPRf/rma5+duw9wu6OkdrnJY0xbr58Db966Uo++73nOD2cIVoSDknnfFY0T1wQ6YjQEA8kWcxLMQpUmpz+LeBvgT8OP79IUK1vxsU4g10Huzl0cghXBEcoaxSCtr8OjkA679PWlCDmOqOe+CF46m+IuUVpkXIocMeNF88onjEVozo2gF/oolgXdXEdobU+ypKGGP2pXM0C956v9CWzDExiVHxVHn/pNDv3dvLcayPpxO1hOvHbStKJt17ZzpcfPUQq55GIOqRzPnlf2Xple9ljxyIOzXVRGmMRnCr3tjfOfSo1LktV9d5CtlhYhzK19nPGguGuxzqIOg4KuEJZnaqIE8QnVrfW09aUYMetVxeD46eG0vQnc2Q8n4jj8O5LlvO9J4+NO9/la1tnLVA+NoCfynmBoXSkaEz+5D0X1ex8/BJRyXK/1z0dPezc28lr/UkSkQjpvDdKSHLjiia2bm7nLRcsxRmzbLV5/WI+xQZ27u3k+ECKFc2BASqNt4gUMr4iJKLmpRjjM5U2x0sIc3dE5Gqgv2ZnZZzTdPYmWd4c51h/JniiLXFdBHCd4CblODJqyWjLxjZu7urjq7tewvOVRCTIMtp/pJ+lDVFODZ/Zny7qSk2WnMZTGyhXzwPQ2hDnod+bOLNsJqiOGJXx9L/2dPTwpR+9SCbnM5TJk/dH5PQ2n9fK1s1ruXTNogljIZvXLy4bvI9FnECSJW5eilEZlRqX3ydIFT5fRP4NWEaQ6msYZ1CIQ6xqSXByMIMX1rHEXGHlogQnBjLkfJ/1ixu4/YY3jHrKf7yjhzWtdaMyt5LZPEsb44gIfckseT94yom6wnvfuIK7Huvgs/c/e0Yx5XSlbCZSG5htGRfVIDOuPzmx/ldfMsuXHnmRk0Ojkx7qoy7trfXc+RtvmvLcIkJD3KU5ETUvxZgylWqLHRCRtxK0ORaszbExAYXsq6grrFvaUExLXtIQYzjrcdkEhYnj3bz7Uzn+582Xjgq6X7N+MfcdOFrWCABTkqMpNUQDqRz1MZdFdUFyQWmtymzKuAyG8vcT6X8d709z775OHnz2OJlR6cRRFtdHibhCf3pqguBRN4ilmJdizITJtMXGtjcuYG2OjXEpl31VaRxiopv32KD7tu27x5WcASb8rtSjGWukjvWnSOU84hH3jFqVL9x0Sdm40E2Xriqe10zEPwGGM3l6k9kJ9b9eOjnEzj2d/PiFkXRiN/Q0ljbGiITpxKmcN2m2F5iXYlSfyTyXcu2NC1ibY2NcppvSXGnNya6D3Rw40ovn+8QjLsua4qOKKRXKekCHugfP8Gi+uuslFjdEccTl5VPDeD54KK/2JBEg5gqtDTHWLW0cNy5034GjvGlNCzA1j6mUgv5XJlc+V0ZVebqrnx17O9nz8kg68dLGGO+/Yg0rmhJ8/acd5DzFdXTSbC8IvZRElMZEpGrthw0DZqmI0liYTOcJvpKak0JMREK9sYIu2aqW0cWU5TygbN5nUd2IR5P3lEze51h/BuEMrUwUyHjKiYEMv7k5UM4ujQsNpHKcGsqQyWf55M4nWLUoMWUBz8n0v3xV/u/h0+zce4RfHBssjq9dXB+mE7cVFYXjUXfCbC8IvZRQ3r4uZl6KURuk0h7ZIvIegoZdxSo3Vb1j/D3OLTZt2qT79u2b69OYN5QGxUs9kInqUSo1Rtu276Z7ME3eU17rT+EgKIojQltzgpsvX82Dzx7nxe4hoq6wvCleVDcezuRYuagOEWEgleO1/hR5L9A0K4hmjiXIcBPqY8Ey2cnBDCua4ySzHqeGskWhTRQcB9a01NFcN9I0q5Dp9dM/vG7UcSfT/8p5Po88382393ZypGckYeCilc1s29zONecvOSOdeCKibqFVcNS8FKNqiMh+Vd00drzSCv2vA/XArwDfIMgU21PVMzTmFXc91kHO8zg9lCfr+cRch+a6yLhP8FPpB1MI+kssuEGeGsqQDQ3EzZevLsZP1rQEmWlHelLUhSKMOU85NZRhWVOCU0MZHATX0WIGWimF22/EEXK+MpzNs3ZxPacGM3T2pCiNiKgG2zvAicHMKOMyNuA/mf5XMpvnX54+xnf2d3FqaCQYf9W6xWzb3M4bV0+cTjzqGsxLMeaISlORf0lV3yQiT6vqn4rI/ws8WMsTM849Sj2PEwNpPE+JRpyipP6pwSw5b7Ds9gOpHA3x8hlaY41LadC/uS5Kc12UZDZPzHX4xs9eZjibJxHGYZY3Jzjal8JXZUVzgtPDGboHgxt21vNDAzKezxJ4JLkwYu77QZvfRXXRUYWJBcIiMHJhz5lSj+2a9YvZetfjvNqTZHlTouxyVW8yy/eeOMr3n3itaHgcges2trH1ynbWL2us+L+FeSnGXFOpcSko1yVFZBWBavHK2pyScS4y1vM42huIVGbywQ1cJPgpZECN3f54f5pUNsjQKnSAHK9+ZDwFZgWSWY9ISX+YoA9MoAYgIixtDIzXcMYL+sOEy8IRR/B8HWVigu6XI0Td4Li9ydxIv5mQgnkq6KO1NSWKMaOr1rXy7X1dOAKNcZfTwxm+/OghPsUGNq9fzLH+FPfu7eLB544Xfz/xiMO737gyCNQvGq23Nh7mpRhnE5Ual38WkRbgfwD7w7Fv1OSMjHOSMxtujaCEN2KFgXSeXQe7ufPB5+keTOP5iisjN/ZXTieJOEI8EiyjnbfkzKf1ckH/aLh0lcp65L1AENMnCNZHXSFW0kJ3SUOciJPjCzddwm1/tx8hOAegeB4FY+GNcmgER4LjUrKNlHwQoCkRZcetV+P7Sl8qx0e+tRdHAmMJFA3i3/7bK/zr8yfYVZJO3JSI8L43r+LXLltNS32MSiiX8TXTdGjDmCmT1blcCXSq6hfCz43AM8BB4Eu1Pz3jXGBsWnDUdcZZZAr4xI4DDGa8kaf9MVvnfUXzHulBn6vXxdi2ffcZN8nSVOddB7u57e/246viAHmFiA9IYCh8DdSKC5TWzTTGXdI5n3TeR1URCWTzM3m/6AEVyHo+UZWi1yISaKflfYqraq4EWWp9ySw/fPY4O/Z08vTRPmKusKQxTn1oWHqSOTqzKV7oDpYJ25ri3HzFGt7zxpUVeR0T1aVMJX5lGLViMs/lLuDtACJyLXAn8AngzQS9500CZoFTLi14ODuxeMNgJki5ncgAaejufO/JY0QdcByH4/1pDhzp5eNbzueTb79w9PwEoo55DeX8RRGgLurQGA+e6MtJ4F+4vJmXTw2RHc7iiIOEKs3hWZyh6pzzg7HVi+I0JqJ0nBoGAqPkAggMpvN8+eEXeegXJ4peWDbvc7w/XdRTK/C6JUE68XUbR9KJJ6ISja9y+mcz6dRpGNNhsr9mV1UL1VofBLar6ndV9U+AC2p7asa5QOFGtrwpERiLyjLbx6VQtOjryKFy/kjsxlflq7teYtfB7lHzNycieCX7BJ6F8J/fej7/8+ZLaWtK0J/K0daUGJUOfdu16+lNBsZQnNGnn/fLtwtwRMh4yrvfuBIRCZbdHCmmBTcnIty7v6toWGJhi2ZPKRqWqCt86OrX8Te3bOJdF6+Y0LBI2NVxVUsda1rrWVQXnVCWpbM3WVyCKzCb3TANAyowLiJS8G7eBjxa8l2l8RpjHlO4kTXXRVm1qI6IO/PMpJwXeAJj7+s5X/F8Je/5RRmXwvzDYSC/cM8VoK0pxuMljbHK2b0tG9toSkSIhgH9sYWUY6/GEVjTGnTOfPDZ4+Q9n5ynZDxFBdqaErQ2xEhmPVI5j5dPJ+lPjU45vmBZI1+48RJ+65fPm7BOJeo6LGmIs3Zx0JagUlmW9tZ6UmOq/GejG6ZhlDKZcdkB/ERE7ifIGPspgIhcwCSS+yKSEJE9IvKUiDwnIn8ajq8TkZ+LyGER+baIxMLxePj5cPj9eSXH+kw4/oKIvKtk/Ppw7LCI3F4yXnYOo/qU3sia66JTSpctRxj3x5Uzb+wQeBKewqEwVlGYP+v5oafgEo841MdcljTEOXRigM898Bzdg+lR8YeC5wOwoa2JlS11bFzRjOvIqNTdUoMUcYS1i+tpSkTJ5j1eODGI6whRJ+jnohoYv+MDmaDF81C2KI8vQEtdhPVL6mmMR/jSj17k97/9FHtKjB+M9lLaF9ezqH7qqcS3Xbu+mA6tGrzWshumYZRjQuMS9q7/LwSdKN+iI+X8DkHsZSIywHWqeilBjOb6sA/MF4EvqeoFQC/w0XD7jwK94fiXwu0IWypvJVAHuB743yLiiogLfBW4AbgI2BZuywRzGFWmcCM7OZim4+QQzx8bmHSfie6V9VGXqCNEXIemePkndYeRlObC/K4j+Kr4qmgYwA+MjhbjDyLBa9SVoudTeoxkNk8m7+FIUJEfcUYbuDWtdTTEI2TzPicGgtbMQaJAoBDg+0EB5WCZrpkK9KXydA9lOD2coTkRKaYk7+nombaXUo4tG9u448aLx10KNIzZYNKlLVXdXWbsxQr2U2Ao/BgNfxS4DvjNcPxu4L8DXwNuCt8D3Af8LwnKkG8CdqpqBnhZRA4Dm8PtDqtqB4CI7ARuEpHnJ5jDqDKlQo55zy8boyjFkcAD8FXxSqriY65w/rJGHvz0tcUgffdguuwxNNy+MP8dwJ0PPs8LJ4YQCepZjg+kaYxHiEWcSeMPo1ObU4jAyqZEURH55GCanuEsqko27xUFIVc0x3FFiLhCKjdy4aVLc6WxIwiMYjElOeaSyXn84xNH2XrV2ol/cVNkusKhpVg6szETaho3Cb2L/QTB/68CLwF9qlp4tOsCVofvVwOdUGyj3A8sCcdLDVzpPp1jxq8K9xlvjrHndytwK8DatdX9x72QeLyjh8UNUU4OZgEdN6jvOtCSiNKTDJKPHYG2xjhtzYmi/haM3Oxv+7v9xX0LHkRBw2vD8ubid4Ub3h/c9xSD6XzQVEsFAZY0xEjlPDxfOTmYIev5uBL0mSmlcDMuGLaIK8UlJccJpGQ6TqeK2ydc6BnOlWSWBZL3K5oT5HyP00NZXFeIhGeuqmS9YNlMJFh6c8K052P9Kc42LJ3ZmCk1NS6q6gFvDgswvwdsrOV8U0VVtxOkVLNp06YZ5jktLHYd7OaLDx2k49RwsUnVeDgCyxrjJKIOXX0p4lEHP7zJ9iZz1MXcUWrGENzALl/byiunhzg9lCtW+Ht+IFA5Nn5w12MdNNdFWbFopHdJMptHRDg5kGIw4xULHj2Uk0OZURlnpU/nd9x4MV/7yUt09iRpa0rQjM+TXaOX+9IeEKqLxVwpekmfuO4Cdu7tpC+ZK9bCwIj2WMQRYpGR1ehTQ2mGMx6b/uxhsmHB54XLm+fcS7B0ZmOmTJ5YXwVUtQ/4MXAN0FKSgbYGOBq+Pwq0A4TfLwJOl46P2We88dMTzGFMgV0Hu9m2fTdv+eKjbNu+u3gz3nWwmz+47ykOdQ8FhYeTHKe9tZ625gSv9aXQMB035ymeF1QfHu9Plw04X7N+MT3DOTzfJ+f5ZHLBzXxZY4zP3v/sqHMaL/325FCGVN4f5UyJI0Qc4c4Hnz8j2P8n9z/LQCrHnb/xJj553QaAMwxLKeuWNNBaH2Xt4gY+/bYL2bx+MVuvbKchHsH3Fd/3gx9VGuNuUQdNVTk1lKZ7MIvrQH8yF8jYpPO8fGrojKSD2cbSmY2ZUjPPRUSWATlV7ROROuAdBIH2HxMUX+4EbgHuD3d5IPz8ePj9o6qqIvIA8A8i8pfAKmADgSKzABtEZB2B8dgK/Ga4z3hzGBVSuiziCjxxpJeP3L03KAgMl3eAEu2u8Y8V9DvxyPkQcYKn9zxBJX6wo9IQVqUX1vlfPDHAUMajIeaSzkHG83EdIRFx8FTpH85yrD9VLKpsikc43D1EzldUA+8m4kggA6NKIuogoRn0fWUwnac3mSv2ZVFVYm5Q7PjNf3uFrVe286UfvUh6nMZdBf7mt85QGueq85fw+fdexNd+8hKvnA5uxucvqef2G94AjMjWDGc8ljXGGEwHS2+OSPHcViwaX0F6NpjNds7G/KTifi5TPrDImwiC6S6Bh3Svqt4hIusJbvqLgSeA/6CqGRFJAP8HuIxAGHNrSbD+j4GPAHng06r6YDj+buCvwjm+GWa3Md4cE52v9XMZTaFnihcKQPqqo6RQZkqpLldd1GFlSx2nhjIMZ/JAYBCCPioOq1oC4cZXTyeL+7hOoFI88YLcyFyuA1E3MGAaXovrCBcsa8AnWG4LwkVKXzIXLKcNZZjsn8eb17QUFY6n2tXxLV98lJa6aDGlWQjTmVV5/fKmsj1gZovp9OMxFiYz6ucyHVT1aQJDMXa8g5Fsr9LxNPD+cY7158Cflxn/AfCDSucwKqfQM+XlU8NVNyxQUkkPtDUn8HylP5VHgHhESOeDLTzP59WeJM6YfSYJ85wxV7C9R8QNPB9H4HWL6xnMBPL8EGRynRzKkMx6xbmk5BhjEeD0cIa//vFhPtfwBt51ydSEwttb63nl9BCeHywTOqGETjzizLmXUElHUMOYCKuyN8pSWBZJ5338KhuWUhwJVIQ7TgZZ64EhGD2fKky8OFUZng+qPp5CzIUjPUkyeR/XhYjjkM6NtlgFQ5fzgtqZsfbMdWBRXZRTQxn+4L6n+cK/PD+llN1r1i9mzys9RQPmayBv0xTOOddFj9VIZzYWLrMS0DfOPQqFhWN7nNSKQuMu4UzjUi2UIHurKe6S8wKBSgXyHkXD0hh3aU5EWFIfwXWCc/HCtbhCUWU8IsRdh3jEZTCd5/RwluFsflwFgPF4vKOHZY0xElG3mJrsEBhBW34yznXMczHKUiiO/MtHDtXk+E4YdIlFHAZS2WIf+8kYv2dkZfuKwHDJstdYsnmPvB8062pORMMamaA4tBB8DzpTKksbg1bJAFHH4eVTw8U6mi8+dHBS49DZm2RpY5xlTSPNwAr1PmZYjHMdMy7GuDze0UMiEvQ2qaYv4Qq0NcfD9FufV3sqLyJUOKPPylT2nST5i6wHUTfwWAbTeVa2JGhORDnen2L9siYOdQ8ymM7TWh+lKRHhaF8K1UC0UglkY3xfebF7iF0Hu0cZibEV742xIFBuGVnGfMSMywKlEmmPzt4ky5vjU7r5V4IjgitSXJaaKsU0aCrLFpsqokEKswqcHsoSdR02LG9mx61XAyO/u67eJPUxl3TOQ2REcl8IDFRpKnG5ivdCa2ZgVEbWXMdaDKMamHFZgIy90b18aojb/m4/TYkIG9qaioamENRPRJxRMifToTT1eMWiRFHqZSbH88PXprjLUNbDD6vgG2LB5+miYadJREnnffpTOWKuw1u++GjREJcamo/esxf8ka6XAiyti44qOCxt6xxzHZY2xmmuixJzHVrqY5aRZcw7zLgsMHYd7OaTO59gOBuk4DbEXHrDG30ykx+lIXXN+sV8dddLk8q7VEKph7KmtZ6jfacnFbmc6FiFpTEFBjIezQmXoYxHfdjbZSY4YR9jzw+0v4Qg4aCcxtaWjW2sbIrT1R/EXoKaGqE3meeCZUFr5V0Huzl0cgg39NjynvJaf4pVixJk8z4PfvraGZ2vYZyNmHFZQBQ8lmTYWCvvKyeHsjhSiEcEUizNdRG++NBBhrMeddGgar1aMRfXCTLR9rzSw6QVihMwNuYykPaIujIjjwWCeJCq4iu01EdZ1hgn6/kTamw11cWIDGaDQsjQ6ykIVEKo0+U4YbZasA2hPP9l7a3jnoupEhvnMpaKvIAoiBHGIw5oECNQAgUW1SCDK+8rpwazHO4eIuoG9RYxN5Ctn2LPqrII8NF79hXjJtWktDe9jHmthPqoQyLqFj2jZY1xBjP5CTW2vvLIixw8PkjeVzJ5n2w+aFq2uiXBUCYQ5i7ErjSsYyn8b6L4SmnbgammOBvG2YB5LguEXQe7OXCkF88PUmU95YxoeMRxAn0rCSry66Iu2bDpSi4/ea+WSgjKSWpfOTPRDAU5mLGrfcmcT1tjjPWLGospwRNpbH3lkRf58qOHRx3XV2iMR4i4Dm1hinHhGKtaEqNk/89f1jCuJ1KJKrF5NsbZjHkuC4DCU7CE8iJI0C8+54/cXYXAcylU47sS3EQdEXK+VsWwzAXlTrtUPibmOjglLZVPDWeBEQMyUcvgb/zs5WLzs9K5ugczo7yS0m6Z65Y2BN0mmxP84fXjd6CYTJXYPBvjbMeMywKg8BS8vCkRLIN5fjGzqYBPELSOuEJDzCXmurxyOlmVYP7ZwETLYxFn5J+Br4wyIBO1DB7OBi2RI65DNKywh8DIlFbYT6ftcHtrPakxRTmlNTClns147ZsNYy6xZbEFQEGEUmLB3e/VnpEU2dJaEV8h6gqDGY+2phiep3QPZWf/hGvAeI5X3veJRwLh7qznIwJtTYlRS0zjaWw1hEWQBQMTCY9XF3XP2H6qOl23Xbs+TL7Il62BKfw3LcX6rRhnE+a5LABKn4Kbx9yQxvol/ak8zQmXpY2JGaf0ngsEYpE+SrBs9Xtv28COW6+uyBD89lvW4WtgUHz1w9dgfKZM5u1M5tkYxlxjnssCoPQpOO+Nv8wlgONAMuPRcXLonDYuk2mQlXpseT/wQn77Lev45NsvrHiOwrbf+NnLDGe9aR1jIibydibzbAxjrqlZs7BzjfneLKyQWXTgSO+4dStRVxBVsv709bvOBiT8v4n+tCNhTcqiRIR9f/LO2Tq1qlIqQ2PV/cZcMevNwoyzgzPEEuMujc1xOntTZ2SA5TwtrpOeq4YFQo9lktP3VWlrjLNuaeNsnFJNsH4rxtmMGZd5zFceeZGv7noJz1fiEYe85zOU8YiG6bfl7Mf8yA0LKFTLj+0mGaRdC7GIa8tIhlEjahbQF5F2EfmxiPxCRJ4TkU+F44tF5GERORS+tobjIiJfEZHDIvK0iFxecqxbwu0PicgtJeNXiMgz4T5fkVBvY7w5FhK7Dnbz1V0v4asGS1ye0j2YwfOV4wOZSdsEV6Maf65Y1hilvbWOv73lSqJuoEIw1o7Wx9yKG3LtOtjNtu27ecsXH2Xb9t1zXktytp2PYZSjltlieeC/qOpFwNXAx0XkIuB24EequgH4UfgZ4AZgQ/hzK/A1CAwF8HngKmAz8PkSY/E14GMl+10fjo83x4Lhrsc6yPtBd8d03ifj+WFmU2XLXefwqhinh3Osaa1ny8Y2PvErFxCPOGF/+uAPPhZx+O23rKvYsExWrDibN/tdB7v5g/ue4okjvRzvT/HEkV7+4L6nZjSnGSujFtTMuKjqMVU9EL4fBJ4HVgM3AXeHm90NvC98fxNwjwbsBlpEZCXwLuBhVe1R1V7gYeD68LtmVd2tQVbCPWOOVW6OBUNnbzJU8z2HrcQYohW6U74Gis4QZHR9fMv5RF0HEUhEHRY3RLnvwNGKbqKTFSvOdqX8Fx86SG8y6AMTcQMxzN5kji8+dHBax7NKf6NWzErMRUTOAy4Dfg4sV9Vj4VfHgeXh+9VAZ8luXeHYRONdZcaZYI55TyGAf6I/PenS17lGrkJ3SoD7DhzlTWta2LKxjcc7eljTWjdKH2ysTtd4jC1WHEjlODWU4ZXTSbZt301nzzCnhrJ4Ggh8LmuKF41PLYLtHaeGQy8sbEwmoKJ0nBqe1vEq0TAzjOlQ8yJKEWkEvgt8WlUHSr8LPY6aPlpPNIeI3Coi+0Rk38mTJ2t5GrPCVx55kdv+bj97Xjld8Y14PrK0MTbKu5hMp2siSosVB1I5XutPkfV84q7wwvEBuvrSpPM+nq9k8h6v9aXJe/45Uyk/k9+NYUxETY2LiEQJDMvfq+o/hsMnwiUtwteC/30UaC/ZfU04NtH4mjLjE80xClXdrqqbVHXTsmXLpneRZwmlAfyou7CFF+pjkVE3yJlUs5cKV54aKjQEE5oSUXqTI900g3hWUO1/YiBTs0r5dUvqA1UBX4O+M6Go6Lol05vPKv2rg8WtzqSW2WIC/A3wvKr+ZclXDwCFjK9bgPtLxj8UZo1dDfSHS1s/BN4pIq1hIP+dwA/D7wZE5Opwrg+NOVa5Oc5JKvnDveuxDnLhE3Q6N8/Ww6ZAxBG6+lI8f2yArt4Ur//sg3SeHmIglSurbDwZpTIs6bxP1BFWhb1aSn3DQjTI8yHn+zVLcb79hjfQUh9FHPBUESdoanb7DW+Y1vEmUn02KsPiVuWpWYW+iLwF+CnwDCPlE39EEHe5F1gLvAp8QFV7QgPxvwgyvpLAh1V1X3isj4T7Avy5qv5tOL4J+BZQBzwIfEJVVUSWlJtjovM9Wyv0C3+42bzHYDpPJu/jOsLHt5w/SmZkwx/9CwvYphSJuVJMYnAlaDnsK9RFHda01jOUyU+7mn3b9t3F3i4Hjw/ghV6DEMQ+CiuRG5c38tDvvbXKVzZCtSvzrdJ/ZpT+XRRIZvO0NSXYcevVc3hms8OsV+ir6s8YX+n8bWW2V+Dj4xzrm8A3y4zvAy4pM3663BznInc91kE273F6OIuDEHEET5Wv7nqpGLD+vZ0HFpxhEWBZY4zeVC5sfqZBj5qS7LhYJIgliK/kfKWlPjajfvWlel5RR/BVQYPGYxHXwfMVR2TaXkSlVLsy3yr9Z4YpVJfHKvTPcjp7kwym8zgIjiPkPR9PlZwqv/P3B1i7uI6DJ4bm+jRnDSH4h9sQd2mqizKc9QKpfIS1i+s42pcKZGxKHmtEgp72pY22JurgON73Wza2cQeBwe9P5fDSeZoTDplcUEcUcRw+vuV8u1EvMCbqVrqQWdiR33OA9tZ6Mvmgz0je80d1hUzmPA51LxzDUsBT5T9e/TramhLURYPWzEsaozTGI7hhim6pcVENlsfWtNZPuj4+2fdbNrax49ar2ffZd3DXf7iCjSsWsaQxzubzlnDXf7iiaorIxrmDxa3KY57LWc416xez++XT5PMjyz2la43zqEayIpSROpaCfEtpzGDd0gaO9iZJ5ny8sI2zr9ASj3LbtesnreuYSt2HLScZwCiP1uJWI5hxOYvZdbCb+w4cZVEiQn9qJDvJcQTP13NaFn8mrFiUwHVG6ljGLmFBUMleKCy8YGk9t9/wBrZsbOOz9z877vr4roPdHDjSixd2p1zWFKcpET2jd/1ES2rGwmQhPmgU/i1El533xnLfWz+XkLnKFht7s7pm/WIe7+ihszfJQCpHfcwlHnE53p8is9DclDI4AhevWoSqcnwgXZRjKW2YVU6QsrSfjQgsb0oUu3Ims3lirsNw1qN7MI3vKyKCKqxqCQxZofXx5x54rqL5DGM+U1g+jrrCT++8JZk9+WrD2G3Mc5lDSv8DtdRFeeH4AI93nA6D1g6pnM9AOj/Xp3nW4AokwmryVM4jm/dZVDf5Elbp73lFc5yjfWmO9qUAJeI65Dwl6ihRV1jelOC1/hQSNoU53p+mrTnBNesX88mdT5DMesQjDksb4zTXRU0qxViQjF0+LocZlznkrsc6GExn6U/lR6kQK5BcYLnFpb1XSmtGCrgCIsLSxlgxYFrwIEopXeIqeIQFD3BRXSKcSzjen+b4QIbL17Zy27Xri8tlEgsiWqeGMmQ9RYGbL1/NfQeOMpzNF9sXvNafAqApEVnwKafGwqNc+vVYzLjMIc8e7WMwc+72qa8mhdVZBeKuQ97XYjwpMDhBfOnVnhT1UZf/9Nb1PN7RUzYFtCHmFj0VV2AgnWcgnad3OMvKljqaEkFmWX8qVyxya39sJJ20uS5a9EramhI83tFD1BUSEZe8rziOgB8YoIgrCz7l1Fh4lEu/HoulIs8hQ2ZYzkCAnOeHBYlBgSIEfWhcKVTg+9yz+1WuWb+4bAqoiBB1g6SHY/0Zwuxksr7yWl+awXTujDqEidJJC+KOy5riqAYtkhElk/ct5dRYkJT+exkPMy5zxK6D3bWVgz5HEQnSq5VgaczzRyStPQ1rVkQYSOX4xs9eJpnN81pfiheOD9LVm6Ih5tI9mKYu6nJyMDAsUSf4M9eSOMpYo1CqIdafytHWlCgG6ttb6zk1lOHkYCYoYPV8snmdUjdLw5hPlP57QZyy7osti80iv7fzAA88fRxvAaYPV8pkv5qcrzgoPuCrR2t9lL5QnXhFc5ys5zOU8cKYSaDD5ogQ0UCuJecpilIfPfO5arx00mvWL2bPKz04AhEHfA30yirtZmkY85HCvxe57ZVnyn1vxmWW+L2dB/jek8cm39CYlEKqg+crnb0pHMB1HU4NZVm/rJHFDVF6hgPNMd/XYtGpAK4rxFyHnK987oHnuCP8bqLalcc7emhrijGQypP1fGKuQ3NdhMc7evjk7F22YZxTmHGZJb7/lBmWahNxJJDDAcT3yWpgRpY0xMl5yrLGOC92DxF1wfEUDTVhljbGi2nLdz74PMmcX0wHL8i93AFFA9PZm2RJQ5yljYni3KpqWWKGMQEWc5kFdh3sxmpVq0vEESKuU9QQyyvEwuh/Kuexoa2JBz99LX/zoU1c1t6Kh+AQtAd+rT9Fx8khBlJZXuge4rW+FMf70wym88WizEL1P1hDLcOYDua51JDSqnBjZggjgf22xhh9qTy+BhI4WU9RZVQNzDXrF3PDXz1WlIBxUPJ+YJRcETJ5n+FsYDBclwlrV0ql9ksr8y1LzDDGx4xLDdh1sJsvPnQwXJIRcvmFVRBZKQWDUWo4xuKEGxY8v/pYILVfH49wcjAodIy5gRfjK7Q1BdX0/2f3q/Qmc0XPplCTKqFB8sIDBt/LhLUrJkxoGFPHjEuVKUq2D6RxBTRMpXUYCUQbAQKsaa3jWH+aSNgxMu/7ozLGoq4TpicHjbj+07Xrue/AUaKusG5pQ1l9r23bdzOYzhczxQDwAi/FV8XzAYWoAz6BhphPoXZFy3olC1GY0DBmghmXKnPXYx3kPI9M3g+fygMJEQu5nInrCu+/Yg1f3fUSOc8vn4YsYQFlSWvnN61pOcOLgMCodPYmOTmYIe/5RCMjIUUnlJRxHWHjimY6Tg6R9XyiofEp/Peqj1rtimFUAzMuVeZQ9yD9yVxxrWchGpVC6u94S14CxCOBxMvjHT18fMv5fOmRQ8XtI+FaloR7Xnne4lHLUGO9iLECoKeGMkGfG09R9dGS/w6uCKpKc12EEwNBUWTEDSr6fYX6+GitMsMwpkfNssVE5Jsi0i0iz5aMLRaRh0XkUPjaGo6LiHxFRA6LyNMicnnJPreE2x8SkVtKxq8QkWfCfb4iEjyCjjfHbJHN+1BSFb4QKXhqEYei9EopThhDiUccunqTfPLtF7K4IUpDzCXiCvGIw+qWOl6/opm2pjg7br16Qk+iVKFVJFA2dsLWxv4YA7+sKU5/Ksd5SxpZ1RwnFnVQguW31S11LKqLjsoUMwxjetTyDvgt4PoxY7cDP1LVDcCPws8ANwAbwp9bga9BYCiAzwNXAZuBz5cYi68BHyvZ7/pJ5pgVom7hqXth4EhgJMYiBNItvp75u/DDGEdTIlIMnF+4vJkVixJsXNHM+mWNNNdFK073LWh/FWiuixKRkfMQIB7K7a9preenf3hdIFjpOFywrHHUnKWNwQzDmD41My6q+hjQM2b4JuDu8P3dwPtKxu/RgN1Ai4isBN4FPKyqParaCzwMXB9+16yquzXodnbPmGOVm6Om7DrYzbbtuxnKeMFSkI6E710gHpF5a3AuXN7E6xbXj+pb7zoyymNwx1z8koYYsYhbjJfMpA95uToUD2iIuVyyehGXrF7EhSuaWdoYH2U4rH7FMGrHbK/dLFfVQqn6cWB5+H410FmyXVc4NtF4V5nxieY4AxG5VUT2ici+kydPTuNyAnYd7OYP7nuKJ470ksv7ZL2gpqI+jGh5QCav52T8JSIQdUZbhoI34EjYeyabpykRwREh6giJiIMrMsrYRFyHRCT4iUUc1i1tHBU4n0g4cjLKGaaI49CUGB1SLDUcuw520zuc4ZXTwxw6MchAKjslg2YYxsTMWUBfVVVEanq/nWwOVd0ObIegzfF05/ns95/h5FB21JgPJOdBE0nHcXAdIeePPOEXflGOQEPUpa0pQVdvkvqYS0Pc5dRQFleCYkW/KE0QpPhGXIfGcYLm0033LVeHctOlq7jvwNGyhY+lCQBrWuo4MZihqy/NhW2N/Ml7NlqmmGFUgdk2LidEZKWqHguXtrrD8aNAe8l2a8Kxo8CWMeO7wvE1ZbafaI6qk8p6PPTsMbr60rWaYs7JeT6u456R9SVhDc+a1vpiw63CTdt1AsFIxxFcVaKuE6QaA631UZY2xstqeM2EcoapXMrylo1tbNu+e1SL1ua6oLK/pT5mhsUwqsRsG5cHgFuAO8PX+0vGf1dEdhIE7/tD4/BD4C9KgvjvBD6jqj0iMiAiVwM/Bz4E/PUkc1QFz1cG0zkG03lyns/d//fVah7+rEOBdM4rGpZCGrFqUNioJaJpBQ/izgef59DJIaIirGmtI+I6dPWmaK2PsqwpEH8cr999NRnPEyrXotUC+YZRXWpmXERkB4HXsVREugiyvu4E7hWRjwKvAh8IN/8B8G7gMJAEPgwQGpEvAHvD7e5Q1UKSwO8QZKTVAQ+GP0wwx4xIZT0G0zmGs17xhtqXzHKoe7Aah58V3LARV4FExCGb9ydVDig1LDE3kE6JuQ4rGuNFfa4ChRt6QVetqzdJW1OCvmSWpY3xUdvO1Q29XItWC+QbRnWpmXFR1W3jfPW2Mtsq8PFxjvNN4JtlxvcBl5QZP11ujukw1kspcKw/xb37unjo2eNkziHdMC+UPMn5Yf2JQDTi4PnKokSE02HTrVIKy2GOBMbpwhXNxe8KPebLMdZr2LZ991lzQzchSsOoPVahX4ZyXgrASyeH+Oqjh3myq784FnMdst65Y2B8gr70EQl6zMdchxXN8UD0sTnBy6eGSef9sOuiE1TMu8LSxhhdfelp35DPphu6CVEaRu0x41JCXzJ7hpeiqjzd1c+OvZ3seXmkbCfiCI3xCPGoQ/dAZtbTjMcucVWCAA2xCPUxtxj7gBEPpGAAugfS+Bp0cPRRljYmiLjChW2NtNTHpnVDPttu6CZEaRi1xYxLSCbv0zM8kk68+6XTbP9pB0f7UuRK7uIRR1jSEKM5EUiNZPIe8YiQ80akRoo1IAp1seApfbLe8JUiwKK6CKmcD54/qYEpnAsi3PimFRw8PsSL3UP0pXIsb4oTcZ2iB1EwACPtAmBVU2BYcp7OOE3XbuiGsXAw4zKGnOez/ScdfP+p1/BKLELUFWKusHJRAkecIM3WEWIRh+FMHkQYyuTxQgXfxniEZY1xsp5PfSxCZ0+SvtSZMY1KiLrC4voYQ5k86bxH1lM83x/XW4q5Dp4fGB7HERpiLm/buIz9R/rD2o4EJwYydPWl2LCskT95z0VniEKODcjbspFhGFPBjEuIr8q9+zq5b38Xp0oKIhtiLq31MUSgZzhLzlMaE06xT0gym2fD8mZuu3b9qCWfa9Yv5sFnj/PK6WQgfaI65aWsqBtIxkRdh7bmBG3AQCpLV1+aqOOMWr4bi+MIDVGXp//7uwDGre1obYiXNRrmZRiGMRPMuIR0nBzm6z8ZUcNtirssbogRj4xUkyeiDoqQznlnBKVLb8alFeBL6qN0h8Yq7gpeaF1iblD5PlbbqpRYWHxYmjBwYiBD1BVWNCd49XSyrKR91vNxHeG337KuOGa1HYZhzCYLVxd+DL4q8YjDr122mjesaKKlPjAsIkELXU+VC5c3V6R/VSoBP5z1iEcc4q5DxHWK4pV5PzAY44lZxiMOSxuDLK5CD5JkNk/O91neFKcpEcV1ZJSkfUHLSwQ+dd0FfPLtFxa/M5FGwzBmE/NcQpY0xNjxsatoqY+xp6OHrzx6iKzn0xBzSefLeyjjUeolFLyIwvt4xCGd98Pgv+I6ELaACXqQaJAJtqI5TsQVWuqjLGsMepCsaa0n6gi5MBZUaLiFBunC65c1FjO/Sg0LnF2pwIZhzH/McwnpT+W445+eZ9/LPdzwxpX8+fsuYeWiOgbS+Skp9MJoLyHmBlIpqsH75c0JXAmMSd7ziTgOzXGXjSuaWNVSxxtWNLGhrTGoO2lK8KGrX0dLfay49PXuN64sKgAvbYzh+YqnytLG2ISqvmNVh2OuQ33U4bP3P8u27bvZdbBmEmyGYSxApLRIcCHT3P56vfyTX8fzlS/cdMkZbXTveqyDzt4k7RXUZ5TGXAZTuWLMJebAkqY4OU9Z1hhnKJOfsN6j9Dil3sbNl6/m8Y4eunqTNMYjqCrDWa/i2pHxjmu94w3DmCoisl9VN40dt2WxEEeEpkT0DDHFsf3ZK1HzLa0XOZ3MEQuzvjyFnuEcH99y/hnLVuUojd3AiNjj4x09RSXi6TDecWspImkYxsLClsXGMDaDamx/9vpYhKgrk/ZZ37KxjZb6GOctqef1K5q5cEUzb1jZzJrWOh7vGNugszxj2/eWO7/pUKvjGoZhFDDjMoaxGVQzuRHP9CZeqwwvyxwzDKPWmHEpoVxAfCY34pnexGfSV34ujmsYhlHAjEuI52vZrLCZ3IhnehOfSV/5uTiuYRhGAcsWC9m0aZPu27ev7HelOltTVfOdyb6GYRhnO+Nli5lxCZnIuBiGYRjlGc+42LKYYRiGUXXmrXERketF5AUROSwit8/1+RiGYSwk5qVxEREX+CpwA3ARsE1ELprbszIMw1g4zEvjAmwGDqtqh6pmgZ3ATXN8ToZhGAuG+WpcVgOdJZ+7wrFRiMitIrJPRPadPHly1k7OMAxjvrOgtcVUdTuwHUBETorIq3N8SpWwFDg11ycxR9i1L0zs2s9uXlducL4al6NAe8nnNeHYuKjqspqeUZUQkX3l0v4WAnbtdu0LjXP52ufrstheYIOIrBORGLAVeGCOz8kwDGPBMC89F1XNi8jvAj8EXOCbqvrcHJ+WYRjGgmFeGhcAVf0B8IO5Po8asH2uT2AOsWtfmNi1n4OY/IthGIZRdeZrzMUwDMOYQ8y4GIZhGFXHjMscIyLfFJFuEXm2ZGyxiDwsIofC19ZwXETkK6Fe2tMicnnJPreE2x8SkVvm4lqmioi0i8iPReQXIvKciHwqHJ/31y8iCRHZIyJPhdf+p+H4OhH5eXiN3w6zHRGRePj5cPj9eSXH+kw4/oKIvGuOLmnKiIgrIk+IyD+HnxfEtYvIKyLyjIg8KSL7wrH59zevqvYzhz/AtcDlwLMlY/8DuD18fzvwxfD9u4EHAQGuBn4eji8GOsLX1vB961xfWwXXvhK4PHzfBLxIoAU3768/vIbG8H0U+Hl4TfcCW8PxrwP/OXz/O8DXw/dbgW+H7y8CngLiwDrgJcCd6+ur8Hfw+8A/AP8cfl4Q1w68AiwdMzbv/ubNc5ljVPUxoGfM8E3A3eH7u4H3lYzfowG7gRYRWQm8C3hYVXtUtRd4GLi+5ic/Q1T1mKoeCN8PAs8TyPTM++sPr2Eo/BgNfxS4DrgvHB977YXfyX3A20REwvGdqppR1ZeBwwTaemc1IrIGeA/wjfCzsECufRzm3d+8GZezk+Wqeix8fxxYHr4fTzOtIi21s5lwqeMygif4BXH94bLQk0A3wc3hJaBPVfPhJqXXUbzG8Pt+YAnn6LUDfwX8N8APPy9h4Vy7Av8qIvtF5NZwbN79zc/bOpf5gqqqiMzrfHERaQS+C3xaVQeCh9KA+Xz9quoBbxaRFuB7wMa5PaPZQUTeC3Sr6n4R2TLHpzMXvEVVj4pIG/CwiBws/XK+/M2b53J2ciJ0fQlfu8Px8TTTpqyldrYgIlECw/L3qvqP4fCCuX4AVe0DfgxcQ7DsUXjoK72O4jWG3y8CTnNuXvsvAzeKyCsE7TCuA77Mwrh2VPVo+NpN8FCxmXn4N2/G5ezkAaCQ/XELcH/J+IfCDJKrgf7Qlf4h8E4RaQ2zTN4Zjp3VhOvmfwM8r6p/WfLVvL9+EVkWeiyISB3wDoKY04+Bm8PNxl574XdyM/CoBpHdB4CtYUbVOmADsGdWLmKaqOpnVHWNqp5HEKB/VFX/PQvg2kWkQUSaCu8J/lafZT7+zc91RsFC/wF2AMeAHMG66UcJ1pN/BBwCHgEWh9sKQYfNl4BngE0lx/kIQUDzMPDhub6uCq/9LQTrz08DT4Y/714I1w+8CXgivPZngc+F4+sJbpCHge8A8XA8EX4+HH6/vuRYfxz+Tl4Abpjra5vi72ELI9li8/7aw2t8Kvx5DvjjcHze/c2b/IthGIZRdWxZzDAMw6g6ZlwMwzCMqmPGxTAMw6g6ZlwMwzCMqmPGxTAMw6g6ZlwMowqIyHIR+QcR6QhlPR4XkV8rs915UqKAXTJ+h4i8vYJ53iwiKiJnlY6UYYzFjIthzJCwGPT7wGOqul5VryAoDlwzZrtx5ZZU9XOq+kgF020Dfha+lj0XEbF/18acY3+EhjFzrgOyqvr1woCqvqqqfy0ivyUiD4jIowRFcmURkW+JyM0icr2IfKdkfIuM9DsR4P3AbwHvEJFEOH6eBP1M7iEoyGwXkT8Qkb1hD5A/LTne90PP6rkS0UTDqDpmXAxj5lwMHJjg+8uBm1X1rRUc6xHgqlAaBOCDBPpbAL8EvKyqLwG7CCTrC2wA/reqXgy8Pvy8GXgzcIWIXBtu95HQs9oEfFJEllRwToYxZcy4GEaVEZGvStBhcm849LCqju3ZUxYNJOUfAn41XEZ7DyM6U9sYMTQ7Gb009qoG/T4g0Jl6J4G8zAECteUN4XefFJGngN0EwocbMIwaYJL7hjFzngN+o/BBVT8uIkuBfeHQ8BSPtxP4XYImcvtUdVBE3HCOm0Tkjwk0p5YURBDHzCHA/6Oqd5UeNJS3fztwjaomRWQXgW6XYVQd81wMY+Y8CiRE5D+XjNXP4Hg/IVhK+xgjnsrbgKdVtV1Vz1PV1xG0KjgjI41AHfcjYZ8cRGR12DtkEdAbGpaNBG1zDaMmmHExjBmigfrr+4C3isjLIrKHoFXtH46zy+tFpKvk5/1jjucB/wzcEL5CsAT2vTHH+S5lssZU9V8JetM/LiLPELQGbiJYbouIyPPAnQRLY4ZRE0wV2TAMw6g65rkYhmEYVceMi2EYhlF1zLgYhmEYVceMi2EYhlF1zLgYhmEYVceMi2EYhlF1zLgYhmEYVef/Bx+zQGw+ItUtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#scatter plot grlivarea/saleprice\n",
    "var = 'GrLivArea'\n",
    "#scatter plot\n",
    "sns.regplot(data = train, x= var, y=\"SalePrice\")\n",
    "#plt.scatter(train[var], train['SalePrice']);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train.drop(train[(train['GrLivArea']>4000) & (train['SalePrice']<200000)].index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data size is : (2917, 79)\n"
     ]
    }
   ],
   "source": [
    "ntrain = train.shape[0]\n",
    "ntest = test.shape[0]\n",
    "target = train.SalePrice.values\n",
    "data = pd.concat((train, test)).reset_index(drop=True)\n",
    "data.drop(['SalePrice', 'Id'], axis=1, inplace=True)\n",
    "print(\"data size is : {}\".format(data.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='1.2*'>1.2*. Xử lý data thiếu </a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def missing_values_table(df):\n",
    "        # Total missing values\n",
    "    mis_val = df.isnull().sum()        \n",
    "        # Percentage of missing values\n",
    "    mis_val_percent = 100 * df.isnull().sum() / len(df)     \n",
    "        # Make a table with the results\n",
    "    mis_val_table = pd.concat([mis_val, mis_val_percent], axis=1)     \n",
    "        # Rename the columns\n",
    "    mis_val_table_ren_columns = mis_val_table.rename(\n",
    "    columns = {0 : 'Missing Values', 1 : '% of Total Values'})      \n",
    "        # Sort the table by percentage of missing descending\n",
    "    mis_val_table_ren_columns = mis_val_table_ren_columns[\n",
    "            mis_val_table_ren_columns.iloc[:,1] != 0].sort_values(\n",
    "        '% of Total Values', ascending=False).round(1)      \n",
    "    return mis_val_table_ren_columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Missing Values</th>\n",
       "      <th>% of Total Values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PoolQC</th>\n",
       "      <td>2908</td>\n",
       "      <td>99.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MiscFeature</th>\n",
       "      <td>2812</td>\n",
       "      <td>96.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Alley</th>\n",
       "      <td>2719</td>\n",
       "      <td>93.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fence</th>\n",
       "      <td>2346</td>\n",
       "      <td>80.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FireplaceQu</th>\n",
       "      <td>1420</td>\n",
       "      <td>48.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LotFrontage</th>\n",
       "      <td>486</td>\n",
       "      <td>16.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageFinish</th>\n",
       "      <td>159</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageQual</th>\n",
       "      <td>159</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageCond</th>\n",
       "      <td>159</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageYrBlt</th>\n",
       "      <td>159</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Missing Values  % of Total Values\n",
       "PoolQC                  2908               99.7\n",
       "MiscFeature             2812               96.4\n",
       "Alley                   2719               93.2\n",
       "Fence                   2346               80.4\n",
       "FireplaceQu             1420               48.7\n",
       "LotFrontage              486               16.7\n",
       "GarageFinish             159                5.5\n",
       "GarageQual               159                5.5\n",
       "GarageCond               159                5.5\n",
       "GarageYrBlt              159                5.5"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing_values_table(data).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Phân tích kĩ data và ý nghĩa các trường dữ liệu có thể nhận thấy:\n",
    "\n",
    " + Một số giá trị NaN được cố ý để trống vì không có thông tin thích hợp, ví dụ ở trường \"Alley\" (mô tả loại hẻm vào nhà), giá trị NaN nghĩa là không có hẻm trong ngôi nhà cụ thể đó.\n",
    " + Một số giá trị NaN được để thiếu để ám chỉ giá trị 0 (thường các trường liên quan đến diện tích, diện tích gara, ...). \n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Chúng ta có 2 dạng giá trị NaN sau: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " + Giá trị có NaN được cố ý để trống vì không có giá trị phù hợp: với các trường này chúng ta thay NaN bằng một giá trị mới là \"None\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "NaN_None_columns = [\"Alley\", \"PoolQC\", \"MiscFeature\",\"Fence\",\n",
    "                   \"FireplaceQu\",\"GarageType\",\"GarageFinish\",\n",
    "                   \"GarageQual\",\"GarageCond\",'BsmtQual','BsmtCond',\n",
    "                    'BsmtExposure','BsmtFinType1','BsmtFinType2',\n",
    "                    'MasVnrType']\n",
    "## fill Nan the above columns by None\n",
    "for feature in NaN_None_columns:\n",
    "    data[feature]= data[feature].fillna(\"None\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " + Giá trị NaN xuất hiện ở trường diện tích vì nhà không phòng, gara , ...."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "NaN_zero_columns = ['BsmtFinSF1','BsmtFinSF2','BsmtUnfSF',\n",
    "                    'TotalBsmtSF','BsmtFullBath', 'BsmtHalfBath', \n",
    "                    'GarageYrBlt','GarageArea','GarageCars',\n",
    "                    'MasVnrArea']\n",
    "## fill Nan the above columns by 0\n",
    "for feature in NaN_zero_columns:\n",
    "    data[feature]= data[feature].fillna(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ngoài ra, dữ liệu thiếu của trường LotFrontage (khoảng cách từ nhà tới đường) được thay thế bằng khoảng cách trung bình của nhà xung quanh tới đường."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['LotFrontage'] = data.groupby('Neighborhood')['LotFrontage'].transform(lambda x: x.fillna(x.median()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Missing Values</th>\n",
       "      <th>% of Total Values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>MSZoning</th>\n",
       "      <td>4</td>\n",
       "      <td>0.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Utilities</th>\n",
       "      <td>2</td>\n",
       "      <td>0.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Functional</th>\n",
       "      <td>2</td>\n",
       "      <td>0.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Exterior1st</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Exterior2nd</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Electrical</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KitchenQual</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SaleType</th>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Missing Values  % of Total Values\n",
       "MSZoning                  4                0.1\n",
       "Utilities                 2                0.1\n",
       "Functional                2                0.1\n",
       "Exterior1st               1                0.0\n",
       "Exterior2nd               1                0.0\n",
       "Electrical                1                0.0\n",
       "KitchenQual               1                0.0\n",
       "SaleType                  1                0.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing_values_table(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Kiểm tra lại tập dữ liệu, sau ba bước điền NaN, còn 8 trường có tỷ lệ NaN thấp. Chúng ta sẽ thay giá trị Nan bởi giá trị có tần suất xuất hiện cao nhất. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "rate_nan_columns = ['MSZoning','Utilities','Functional',\n",
    "                    'Exterior1st','Exterior2nd','Electrical',\n",
    "                    'KitchenQual','SaleType']\n",
    "for feature in rate_nan_columns:\n",
    "    data[feature] = data[feature].fillna(data[feature].mode()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Kiểm tra tỉ lệ missing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Missing Values</th>\n",
       "      <th>% of Total Values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [Missing Values, % of Total Values]\n",
       "Index: []"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing_values_table(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Như vậy đã điền hết được các giá trị NaN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='1.1*'>1.1*. Xử lý trường dữ liệu số </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tiến hành : \n",
    "   + Chuẩn hóa các cột có độ chênh lệch (skewed) lớn: định nghĩa ở đây, các trường dữ liệu có $skew > 0.5$ \n",
    "   + Tạo những trường dữ liệu mới "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sử dụng [phép biến đổi](https://docs.scipy.org/doc/scipy/reference/stats.html#transformations) Box-Cox để chuẩn hóa các cột có dữ liệu chênh lệch\n",
    "Công thức của phép biến đổi Box-Cox ([Box-Cox Transformations](https://www.statisticshowto.com/box-cox-transformation/))\n",
    "$$y(\\lambda) =  \\left\\{\\begin{array}{cc}\\frac{y^{\\lambda}-1}{\\lambda}, \\quad if \\quad \\lambda \\neq 0\\\\\n",
    "\\log(y)\\quad if \\quad \\lambda = 0.\n",
    "\\end{array}\\right.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import skew  # for some statistics\n",
    "from scipy.special import boxcox1p\n",
    "from scipy.stats import boxcox_normmax"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Chúng ta sử dụng gói thư viện boxcox của scipy "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEGCAYAAABCa2PoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA//klEQVR4nO3deXzU1d3o8c93ZrLvJCFhCSQQdmQNi4ACUrdaxafVqmhrqy2PrW3vbZ+nfezTW2/b294uz33Z+rTaatVW2yoidaFURVRERXaEsCUQIAECWci+JzNz7h/zCyZhJgtkZpLJ9/16TTPz+53fyfdXQ7455/zOOWKMQSmllLpctmAHoJRSKjRoQlFKKdUvNKEopZTqF5pQlFJK9QtNKEoppfqFI9gBBFNKSorJzMwMdhhKKTWo7Nmz57wxJrXr8SGdUDIzM9m9e3eww1BKqUFFRIq8HdcuL6WUUv1CE4pSSql+oQlFKaVUv9CEopRSql9oQlFKKdUvNKEopZTqF5pQlFJK9QtNKEoppfqFJhSllFL9YkjPlFfB9/yOUxcdW7VgTBAiUUpdLk0oqt95SxKgiUKpUKddXkoppfqFJhSllFL9Qru8VEAZY3gvv5ztJytwugwutyF7eCw2kQtltMtMqcFJE4oKmJrGNr67bj9vHS4lzC7YRGhxuhmXGsMdORnERYYFO0Sl1GXQLi8VEK1ON3c/vZ1388r4XzdN4cCPrmf//76OW2eN4nRlI4+/d5yaprZgh6mUugyaUJTfGWN45eMzHDpbyxNfmMtXrhpHZJidyDA787OG8a9Xj6e5zcWfPzpJc5sr2OEqpS6Rdnmpi/T33JCPT1Wz/0wN/37dRFZMSbvo/MjEKO5eMJY/f3SS9fvP8vmcjEv+Xkqp4NEWivKrhhYnrx88x5hh0Xx9WbbPctnDY1k2aTj7TleTd642gBEqpfqLJhTlVxsPldDc5uLWWaOw2aTbsssmpZIWH8Gr+4ppc7kDFKFSqr9oQlF+c7a6iT1FVSwan0J6QmSP5R02G7fMHEVts5MdJysDEKFSqj/5dQxFRG4AHgXswFPGmF90OR8BPAfMBSqAO4wxhda57wP3Ay7gW8aYjd3VKSIrgP/CkyTrgS8ZYwr8eX/KN2MMbxw8R2SYneWThgO+55d0lJUSw/jUGLYcLWd+5jDCHfo3j1KDhd/+tYqIHXgMuBGYCtwlIlO7FLsfqDLGZAO/Bn5pXTsVuBOYBtwAPC4i9h7q/D1wtzFmFvA88L/8dW+qZ0dL6zle3sCKKcOJCrd7LdPS1ED+3o84svtDzhUewxgDwKempNHQ4mRnobZSlBpM/NlCmQ8UGGNOAIjIGmAlcLhDmZXAj6z364DfiYhYx9cYY1qAkyJSYNVHN3UaIN4qkwCc9dN9qR643J7WSXJMOPOzhl10vrW5iXfWPs3H77+Jy/nJ3JP0sdlce+dqMqfMZMywaHaerGTx+GREuh97UUoNDP5MKKOA0x0+nwEW+CpjjHGKSA2QbB3f3uXaUdZ7X3V+BXhdRJqAWmCht6BEZDWwGmDMGF3Kwx/2FlVRVtfCqvljcNg6N4Krykt4/v/9J5VlZ5mz9EamL1xORHQMxQVH2L7x7/z1Vw+x4vP3MW/qCv6+t5iiikYyU2KCdCdKqb4IpXko3wY+bYzZISLfBR7Bk2Q6McY8CTwJkJOTYwIbYuirbmzlrcMljB0WzbSR8Z3O1VVV8LdfPURTYx1f+N4vyZwy88K59DHjmX7lNfzj6Ud4+8Wn+NRdDiIcE9ldVKkJRalBwp8jnsVAxxlqo61jXsuIiANPV1VFN9d6PS4iqcBMY8wO6/iLwKL+uQ3VF//39SM0tbm4ZdbITl1VLmcbL/73j2ioq2bVv/2sUzJpFxEVzWe//n0mzr6St9c8QVZrIQeKa2hx6ux5pQYDfyaUXcAEEckSkXA8g+zru5RZD9xrvb8NeNd4RmbXA3eKSISIZAETgJ3d1FkFJIjIRKuua4Ejfrw35cX7R8tZu/sMS7JTGZEQ1encllf/wrmTR7nlK//OqHGTfdZhs9n57AMPkZYxjtLNz9LWWE9BWb2/Q1dK9QO/JRRjjBP4BrARzy/3tcaYQyLyExG5xSr2NJBsDbp/B3jIuvYQsBbPYPubwIPGGJevOq3jXwX+LiL7gS8A3/XXvamLFVU08M0XPmZSWhzXTB7e6Vzx8Ty2/nMts6++gSk5S3qsKywikpVf/S6tTQ1EHnyFvHN1/gpbKdWP/DqGYox5HXi9y7GHO7xvBm73ce3PgJ/1pk7r+CvAK5cZsroEJ8rruf/Z3YjAH7+Yw4cF5y+cM8aw8W+/JzYhiWvv+tde15mWkcXVK+/mvZef5UjuHtxzRvV8kVIqqHTWmLpkNU1tPPXBCVY+tpWapjaevjeHMcnRncoc3LaZ4hN5XHPbfURERfuoybsrb7iNqMRUXPte4/R57fZSaqALpae8lJ+53IZNh0vYWlDB3lNV5JXU4XIbslJiuG3uaPJL6skv+eQXv8vp5L2X/0z62GxmLFrR5+/nCA9n+ee+xOtP/xdb33sbPjOtP29HKdXPNKGoXtlTVMVDf8/lWFk9MeF2Zo1J5OvLxuM2MCoxyus1B7dvpvp8KXfc83XEdmmN4TmLr2Hjy89T+OGruN3fxHaJ9Sil/E8TiurRmapG/u/rR0iMDuOxVXO4YXo6dmvlYF/rc7ndLrZuWENaxjgmzOw6n7X3xGZjzJWf4eTrT7Dx7Xe48bprL7kupZR/6Z97qlv1LU7+/FEhidFhvPTAldw0Y8SFZNKd/L3bqCg5w5Kb77rspVNmL1qOOzqJx37/xIX1vpRSA48mFNWtt4+U0tzm4pkvzbtobkl3dr+znoTkNCbnLL7sGLKGx+Ecv5TjeYfIzc297PqUUv6hCUX5VFrbzK6TlczPSmZiWlyvrysvLqLwyH7mXnMTNpv3lYb7IjrcQfK0xdjCIlmzZs1l16eU8g9NKMqnd/PKiAiz8akuExV7svudf2B3hDH76hv6LZas9GTMmDm88cYbVFdX91u9Sqn+owlFedXY4uTwuVpmj0kiOqL3z260tTRzYNs7TJ1/NdFxCf0Wz9jkaJoyFtLS0sKrr77ab/UqpfqPJhTl1cenq3G5DTljk/p0Xd7ej2hpamTWVdf3azwjE6MwCSMZPX4yL7/8sg7OKzUAaUJRFzHGsLuoktFJUX0aiAfI/XATiSlpjJ10Rb/GlBIbQXS4nbQrruLYsWPk5eX1a/1KqcunCUVdpKS2mdLaFub2sXVSU1HGicMfM2PxtZc8kdEXmwhTR8TTkDadsLAwXnvttX6tXyl1+TShqIvkl3hW9506Ir6Hkp0d3L4ZjOGKxX1fZqU3po9KIL/SzdKly9iwYQNOp9Mv30cpdWk0oaiL5JXUMSoxirjIsD5dd3jn+4wcN4lhw0f6Ja5pI+NpbHUxf9m1VFRUsGPHjp4vUkoFjCYU1UlVQyunKxuZlN77eScAFSXFlBQVMG3BMv8EhqeFAhCVMZ3o6GjefPNNv30vpVTfaUJRnWw5Wo4BJvcxoRzeuQWAqfOu8kNUHtnDYwl32Mgvb2LFihVs2rSJtrY2v30/pVTfaEJRnbyXX0ZMhIORPlYQ9uXwzvfJmDiN+GGpfooMXtp9hpTYcN7LLydq3Fxqamq020upAUQTiupkV2EV41JisPVhQcfKsrOUnTnJlLk9b+97udLiIimtbWb8tLlEREXzxhtv+P17KqV6RxOKuqC4uoni6ibGJvdtZ8VjH28HYOLsK/0RVifD4yOpbXbShp3sGfPZsmULbrfb799XKdUzTSjqgt2FlQBkJsf06bqj+7aTOjqTpOEj/BFWJ2nxEQCU1TUzYdYCKioqdAVipQYITSjqgl2FlcRGOEhPiOz1NU0NdRTlH2DirIV+jOwTaXGe2EprW8ieMQ+73c57770XkO+tlOqeJhR1wa6TVcwZm9Sn8ZOC3F0Yt5tJAejuAkiIDiPcbqO0tpmomDjmzJnD5s2bA/K9lVLd04SiAKhpbCO/tI55fVxu5ejH24mJT2Jk1kQ/RdaZTYTh8RGU1jUDsHz5co4ePUpxcXFAvr9SyjfdU14BkFtcDcDsMUmcqmy86Ly3veNdzjaOH9jF1HlX9/vaXd1Ji4skv9SzPMyyZcv41a9+xXvvvcfdd98dsBiUUhfTFooC4GBxLQDTR/V+/a6i/AO0NDUG5OmujobHR1Df4qShxUlWVhaZmZna7aXUAKAJRQFw8GwNo5OiSIwO7/U1Rz/ejiM8gqyps/wXmBfDrYH58roWwNNK2blzJw0NDQGNQynVmSYUBcDhs7VMH9n7HRaNMRz9eDvjps0mLKL3T4X1h9Q4z6PD5fWehLJ8+XLa2trYunVrQONQSnWmCUVR19zGyfMNTBvZ++6uipIz1FSUkn3FPD9G5l1idBgOm1xoocyePZv4+Hjt9lIqyDShKA6fbR8/6X0L5cTBvQCMuyLHLzF1xyZCSmzEhYQSFhbG4sWL2bp1q24NrFQQaUJRHLISyrQ+DMifOLiHYWkjSUpN91dY3UqNi+C81eUFsGjRIsrLyzl27FhQ4lFKaUIZ8p7fcYp/7D9LXKSDtw+XeX08uCuXs43CvP2MmzY3ABF6lxIbQWVDKy1OF+BJKAAfffRR0GJSaqjThKIorW0mPb73A+tnCo7Q1tLMuOlz/BhV91LjIjBAUYVnzszIkSPJysrShKJUEGlCGeLcxlBe38Jw68mp3jh+cA9is5E5ZaYfI+te+5Nex8vqLxxbtGgRu3btorW1NVhhKTWkaUIZ4qob22hzGYb3oYVy4tBeRmdPISKqb6sS96eUWM98mePlnySUxYsX09zczN69e4MVllJDmiaUIa7MWhOrty2UxroazhUeC+r4CUCEw05CVBjHyz+ZzDh//nwcDod2eykVJJpQhriyWs+TUu2zz3ty8vDHYAzjpwc3oYCn2+tEhxZKTEwMM2fO1ISiVJBoQhniyupaiIt0EBVu71X5Ewf3Ehkdy4isCX6OrGcpsREcL2/oNPdk0aJFHD58mKqqqiBGptTQpAlliCura+51d5cxhhOH9pA1dTY2W+8SkD+lxnkWiSyr6zwfxRjDtm3bghiZUkOTJpQhzBhDWV1Lr7u7KkvPUlt5PuCLQfqSGnvxk17Tp08nLi5Ou72UCgK/7ociIjcAjwJ24CljzC+6nI8AngPmAhXAHcaYQuvc94H7ARfwLWPMxu7qFBEBfgrcbl3ze2PMf/vz/gYTbxMWqxtbaXW6GR7fuxbKqXzP3u1jJ8/o19gu1YVHh8vrWZSdAoDD4WDBggXaQlEqCPzWQhERO/AYcCMwFbhLRKZ2KXY/UGWMyQZ+DfzSunYqcCcwDbgBeFxE7D3U+SUgA5hsjJkCrPHXvYWK9tV62//S70lR3gFi4pNIHpHhz7B6LT7SQUy4vdOTXgALFizg7NmzuoujUgHmzy6v+UCBMeaEMaYVzy/4lV3KrASetd6vA1ZYLY2VwBpjTIsx5iRQYNXXXZ1fA35ijHEDGGPK/HhvIaGi3jMBMLkXCcUYQ1F+LmMmTUf6sOe8P4kI44fHdpqLAp7HhwF27doVjLCUGrL8mVBGAac7fD5jHfNaxhjjBGqA5G6u7a7O8cAdIrJbRN4QEa+PIYnIaqvM7vLy8ku6sVBR2dBKmF2Ii+y557P6fCm1leWMnTQwurvajUuJ4USXFkp2djaJiYns2LEjSFEpNTSF0qB8BNBsjMkB/gg8462QMeZJY0yOMSYnNTU1oAEONBX1LQyLCcfWixbHqbyBNX7SbnxqLMXVTTS2Oi8cs9lszJs3j507dwYxMqWGHn8mlGI8YxrtRlvHvJYREQeQgGdw3te13dV5BnjZev8KMLB+8w1AFQ2tDIvp3fhJYV4uUbHxpI4c4+eo+mb88FiAi1op8+fP13EUpQLMnwllFzBBRLJEJBzPIPv6LmXWA/da728D3jWeWWrrgTtFJEJEsoAJwM4e6nwVWG69Xwoc9c9thQa3MVQ2tJIc07s95E/l5zJm0hWIbWA1asenehKKr3EUbaUoFTh+++1gjYl8A9gIHAHWGmMOichPROQWq9jTQLKIFADfAR6yrj0ErAUOA28CDxpjXL7qtOr6BfA5ETkA/Bz4ir/uLRTUNTtxug3JsT0nlJqKMqrPlzJ20hUBiKxvxiZHY5OLWyjZ2dkkJSVpQlEqgPw6D8UY8zrwepdjD3d434xn3oi3a38G/Kw3dVrHq4GbLi/ioaPCemQ4uRddXkXt4ycDbEAeIDLMTsaw6ItaKDqOolTgDaz+CxUwFQ3WI8O96PIqyj9AZHQswzMy/RzVpRmXEnPRXBTQcRSlAk0TyhBVUd+KXYSE6LAey57KzyVj4vQBsX6XN+NTYzlRXo/bbTodnzdvHoA+PqxUgGhCGaIqG1pIignr8ZHhuqoKKkvPDsjxk3bjh8fS4nRTXN3U6biOoygVWJpQhqjKhlaG9aq7a2DOP+nI15NeNpuN+fPns3Pnzk5L3Cul/EMTyhBV1dhGYnQvEkpeLuGR0aSPGR+AqC7N+FTPVsRdn/QCT7fXuXPndBxFqQDQhDIEtbS5aGpzkdSbhJJ/gDETp2GzD8zxE4BhMeEkRodd1EIBnY+iVCBpQhmCqpraAEjsYUC+vqaKinOnGTOAx0/As0ik50mvixOKjqMoFTh+nYeiBqbqRs8jw0lR3SeUU/kHgIE5/6Rd+z4vgnCouPbC51ULPEvEiEincZSBslKyUqFIWyhDUFWj1ULpYVC+KD+XsIhIRmQGf//4nqTGRVDX4qS5zXXRufZxlDNnzgQhMqWGDk0oQ1B1Yyt2mxAb0X0DtSj/ABnZU7E7Bn5Dtn33xvIO+8u3W7BgAaDjKEr5myaUIai6sY3EqO7noDTW1VB+pnDAj5+0a991sn0Xyo7Gjx9PUlISu3fvDnRYSg0pmlCGoKrG1h6f8Dp19CAAmVNmBiKky5YUE45dxGsLRUTIycnRHRyV8jNNKENQdWNbj094FeXl4giPYGTWxABFdXnsNmFYTLjXhAKecZTi4mLOnj0b4MiUGjp6lVBE5GURuUlENAENcm0uN/Utzp4TSv4BRo+fgt3R81pfA0VqXITXLi+AnJwcAO32UsqPepsgHgdWAcdE5BciMsmPMSk/qrae8Oquy6upoY7S0ycYO3lwjJ+0S42LoLK+FZf74mVWJk6cSHx8vHZ7KeVHvUooxpi3jTF3A3OAQuBtEflIRL4sIoPnT1h1YQ5Kd8uunD56CIwZ0PNPvEmNjcBlDFXWPXZkt9uZM2eOtlCU8qNed2GJSDLwJTw7IX4MPIonwWzyS2TKL2qsWfIJ3UxqLMrPxe4IY9S4yYEKq1909+gweMZRCgsLKSsrC2RYSg0ZvR1DeQX4AIgGbjbG3GKMedEY800g1p8Bqv5V0+xJKPGRvueWFOUfYNT4yTjCe7ff/ECREttzQgHYs2dPwGJSaijpbQvlj8aYqcaYnxtjzgGISASAMSbHb9Gpflfb5CQm3I7D7v0/fUtTAyWFBQN6/xNfosLtxEY4fA7MT5kyhejoaB1HUcpPeptQfurl2Lb+DEQFRm1TW7fdXaePHcYY94De/6Q7qXERPlsoDoeDOXPmaEJRyk+6TSgiki4ic4EoEZktInOs1zI83V9qkKlpaiO+u/GTvFxsdgejx08JYFT9JzU2gvM+WijgeXy4oKCAqqqqAEal1NDQ0yJN1+MZiB8NPNLheB3wn36KSflRbXMbY5J9/y1QlH+AkVkTCYuIDGBU/Sc1LoLGVpfPHSnbx1F2797NtddeG+jwlApp3bZQjDHPGmOWA18yxizv8LrFGPNygGJU/aTN5aax1eWzy6u1pZlzhUcH3fyTjtqf9PK2NwrA9OnTiYiI0MeHlfKDblsoInKPMeavQKaIfKfreWPMI14uUwNUbVP7E17eE8qZY4dwu1yDbv5JR+1Peh0vq2de5rCLzoeHhzNr1iwdR1HKD3oalI+xvsYCcV5eahBpf2TYVwulKP8AYrMxOntqIMPqV4nRYThs4rOFAp5ur7y8POrq6gIYmVKhr9sWijHmCevrjwMTjvKnCy2UKO//2YvycxmROYGIqMH7vIVNhJTYCI6XN/gsk5OTgzGGPXv2sGzZssAFp1SI6+3Exl+JSLyIhInIOyJSLiL3+Ds41b9qm5wAJHjp8mprbeHsiaODurur3fD4CPJLfLc+Zs6cSVhYmHZ7KdXPejsP5TpjTC3wGTxreWUD3/VXUMo/apraiHDYiAizX3Su+PgRXM62QTmhsasRCVEUVzddWLesq8jISK644godmFeqn/U2obT3kdwEvGSMqfFTPMqPapt9z0EpyjuAiI2MidMDHFX/G5ngeeT58Nlan2XmzZvHoUOHaGjw3TWmlOqb3iaUDSKSB8wF3hGRVKDZf2Epf6jpZpZ8UX4uaWPGERkd4/X8YDIiMQqAQz0kFJfLxb59+wIUlVKhr7fL1z8ELAJyjDFtQAOw0p+Bqf5X29TmdfzE2dpK8fG8QbvcSlexEQ7S4yM5dNZ3Q3rWrFnY7XYdR1GqH/U0U76jyXjmo3S85rl+jkf5icttqGt2en3Cq/hkPs621pAYP2k3bWR8ty2UmJgYpk2bpuMoSvWj3j7l9Rfg/wFLgHnWS1cZHkTqW5wY8DqGUpSXCyKMCYHxk3bTRsZzvLyeplaXzzI5OTnk5ubS3Ky9t0r1h96OoeQAi40xXzfGfNN6fcufgan+VdvNxlqn8nNJG51FVGx8oMPym6kjE3AbyCvx3UrJycmhra2N3NzcAEamVOjqbUI5CKT7MxDlXzU+ll1xOds4XXAkZMZP2k0f5UmOB4p9j6PMnTsXEdFxFKX6SW/HUFKAwyKyE7iwNrgx5ha/RKX6na+tf8+ePIqztYUxITR+AjAqMYqU2HD2na7mi1d6LxMfH8/kyZN1HEWpftLbhPIjfwah/K+2uQ2HTYgO7zypsSj/AEBIDcgDiAizMhLZd7q623I5OTm89NJLtLa2Ej7ItjxWaqDp7WPDW/DMkA+z3u8C9voxLtXP2jfWEpFOx0/l5ZI6aizRcQlBisx/ZmUkcqK8gZrGNp9lcnJyaG5u5tChQwGMTKnQ1NunvL4KrAOesA6NAl71U0zKD2qb2i4aP3G7XJwuOBwS63d5MysjCYDc4mqfZTpuuKWUujy9HZR/EFgM1AIYY44Bw3u6SERuEJF8ESkQkYe8nI8QkRet8ztEJLPDue9bx/NF5Po+1PnfIuJ77fIhqtbLHJRzRcdobW5izCDeUKs7MzISEIF9p6p9lklKSiI7O1sH5pXqB71NKC3GmAsr7VmTG013F4iIHXgMuBGYCtwlIl032rgfqDLGZAO/Bn5pXTsVuBOYBtwAPC4i9p7qFJEcIKmX9zRkGGM8s+S7DMgXHtkPELItlPjIMManxvZqHGXPnj04nc7ABKZUiOptQtkiIv8JRInItcBLwD96uGY+UGCMOWElozVcvFzLSuBZ6/06YIV4OvlXAmuMMS3GmJNAgVWfzzqtZPNfwPd6eU9DRlVjG063uSihFB3ZT+qoscQmhG4Onp2RyN5TVRjj+++fefPm0djYSF5eXgAjUyr09DahPASUAweAfwVeB/5XD9eMAk53+HzGOua1jDHGCdQAyd1c212d3wDWG2POdReUiKwWkd0isru8vLyHWwgN52qagM5zUFzONk4dPUjm5JnBCisgcjKTqGps63HDLUC7vZS6TL19ysuNZxD+68aY24wxfzTd/ckXYCIyErgd+G1PZY0xTxpjcowxOampqf4PbgAoqfEsLdKxhXL25FHaWlvInDorSFEFRvu+8rsKK32WGT58OGPHjtWBeaUuU7cJRTx+JCLngXwg39qt8eFe1F0MZHT4PNo65rWMNS6TAFR0c62v47PxbPpVICKFQLSIFPQixiGhpNaTUDqu41V4ZD+IhNz8k66yUmJIiQ1n10nfCQU+GUdxu90Bikyp0NNTC+XbeJ7ummeMGWaMGQYsABaLyLd7uHYXMEFEskQkHM8g+/ouZdYD91rvbwPetVo+64E7rafAsoAJwE5fdRpj/mmMSTfGZBpjMoFGa6Bf4WmhCJ5l3dsVHtlHWsa4kFq/yxsRIWfsMHZ200IBzzhKTU0Nx44dC1BkSoWenhLKF4C7rIFxAIwxJ4B7gC92d6E1JvINYCNwBFhrjDkkIj8RkfYlW54Gkq3WxHfwjNVgjDkErAUOA28CDxpjXL7q7MsND0UlNc3ERTqw2zyTGp2trZw+dpisKaE9ftJuXtYwzlQ1XRhL8lrGmo+i4yhKXbqell4JM8ac73rQGFMuIt63/utc7nU8A/gdjz3c4X0znrEPb9f+DPhZb+r0Uia2p9iGkpLa5k7dXWeOH/bsHz9UEkqm5ym2nScrWTmr63MhHiNHjmTUqFHs2LGDe+65J5DhKRUyemqhtF7iOTWAlNQ0d3rCq/DIfkRsjJkY2uMn7aaOiCcu0sG24xXdlluwYAG7du3C5fK9h4pSyreeEspMEan18qoDhsZvoxDQtYVSmLefEVkTQmL/+N5w2G0sHJfM1uMXNbY7ufLKK6mpqdH5KEpdom4TijHGboyJ9/KKM8b02OWlgq+hxUlds/PCI8OtLc0UH88P+fknXS3JTuF0ZROnKhp9llmwYAEA27dvD1RYSoWU3k5sVIPUhUeGIz3DZaePHcLtcob8/JOuFmcnA3TbSklNTWX8+PGaUJS6RL3dD0UNUl0nNRYe3ofN7iBjwrRghuV3z+841emzMYbhcRFsLTjPXfPH+Lzuyiuv5O9//7vuj6LUJdAWSohrTyjtYyiFefsZNW4S4RGRwQwr4ESEJdkpbDtegdvte5GHBQsW0NTUpPvMK3UJNKGEuE+6vMJobmzg3MljZA6Rx4W7solQ0dDKr98+yvM7Tl3UigGYP38+NpuNHTt2BCFCpQY3TSghrqSmmYSoMMIdNk4dPYAx7iGbUMYP90xPOl7me7uc+Ph4pk6dyrZt2wIVllIhQxNKiCupbWZEgqd7q+jIfuyOMEaP77otzdCQEBVGSmxEtysPAyxcuJDc3FwaG30/EaaUupgmlBBXUtNMWrwnoZw8sp/R2VNwDOHB5vGpMZw834Czm0UgFy5cSFtbG3v37g1gZEoNfppQQlx7C6W+porSU8cZN21OsEMKquzhsbS63Jyp9L2u15w5cwgLC9NuL6X6SBNKCGtzuTlf30JafCQnD30MwPgrcoIcVXCNS4lFgIJy3+MoUVFRzJ49WxOKUn2kCSWEldW1YAykJ0Ry/OBuomLjSR8zPthhBVVUuJ2MYdEcLa3rttySJUs4cuQI5893v1yLUuoTmlBCWPsclLT4CE4c3Mu4aXMQm/4nn5gWR3FVE/UtTp9lFi9eDMDWrVsDFZZSg57+dglh7Qml5fwZGmqrGD99bpAjGhgmpcVhgGPdtFImT55McnKyJhSl+kATSghrn9R4/OAeALKmD+0B+XYjEiOJiXCQ301CsdlsLFq0iK1bt+q2wEr1kiaUEFZS00SEw8bendtJHTWW+KSUYIc0INhEmDg8lmOl9bi6WYZl8eLFVFZWcuTIkQBGp9TgpQklhJXUtpAeY2PPnj3a3dXFxPQ4mtpc7D9T7bOMjqMo1TeaUEJYSU0TMbVFtLa2Mk4TSicThnseH34vv9xnmZSUFKZMmcKHH34YuMCUGsQ0oYSwktpm3CV5hIeHM2aSbrDZUXS4g4xh0WzJL+u23OLFi/n4449paOh+uRallCaUkGWMobSmheqTB8nJySEsPCLYIQ04E9NiyS2uoaK+xWeZJUuW4HQ6dfVhpXpBE0qIqmxopa22nJrS01x11VXBDmdAmpgWhzGw5ajvbq/Zs2cTHR3NBx98EMDIlBqcNKGEqJLaZmylnqeTli1bFtxgBqiRiVGkxEbwTp7vbq/w8HAWLlzIli1bMMb3E2FKKU0oIaukphl76RFGjB5DZmZmsMMZkGwifGrKcLbkl9PidPkst3TpUs6dO8exY8cCGJ1Sg48mlBB1qqwK2/kClly9NNihDGjXTk2jvsXJjhOVPsu0t/A2b94coKiUGpw0oYSo3Tu2I24XN167ItihDGiLs1OICrOz6XCpzzLDhw9n+vTpmlCU6oEmlBCV//EOJDyKnLm63Ep3IsPsXD0xhbePlHY7RrJs2TJyc3OpqKgIYHRKDS6aUEKQ2+2m9OheEjKvICwsLNjhDHjXT0vnXE0z+05X+yyzfPlyjDFs2bIlcIEpNchoQglBhw4dwtlYy9gr5gc7lEFhxZQ0wu02/pl7zmeZKVOmkJ6ert1eSnVDE0oIeufdzRiEGTkLgh3KoJAQFcbVE1N4/cA53D4WixQRli5dykcffURra2uAI1RqcNCEEoLeefdd3MMyyR6VFuxQBo2bZozgbE0zH/fQ7dXY2MjOnTsDF5hSg4gmlBBTXFxMwdF83OlTGZUUFexwBo1PWd1e/9h/1meZBQsWEBkZqd1eSvmgCSXEbNq0CQDXyBmMStSE0pPnd5zi+R2n+Mf+c0xMi2Xt7tM8t63Qa9nIyEgWL17M22+/rZtuKeWFJpQQs3HjRlIyxmNikhmRGBnscAaVuWOTaGx1kXfO906O119/PWVlZezfvz+AkSk1OGhCCSElJSXs27ePlEnzGB4XQYTDHuyQBpXs4XHERTrYe6rKZ5nly5cTHh7Om2++GcDIlBocNKGEkPbuLsmYqeMnl8BuE2ZnJHG0tI6SmmavZWJjY1myZAlvvfWWdnsp1YUmlBCyceNGJk6cSIUk6PjJJZqfNQxj4Fkf4yjg6fYqKSnRbi+lutCEEiLKysrYu3cv1157HWermxidFB3skAalYTHhTB0Zz/M7TtHY6vRaZvny5YSFhbFx48YAR6fUwKYJJUS8/fbbGGOYdeVS2lyGzGRNKJdqSXYKNU1trNtzxuv5uLg47fZSygu/JhQRuUFE8kWkQEQe8nI+QkRetM7vEJHMDue+bx3PF5Hre6pTRP5mHT8oIs+IyJBaxOqtt95i/PjxEJ8OwNjkmCBHNHiNGRbNnDGJPL75OM1t3vdJue666zh37hwHDhwIcHRKDVx+SygiYgceA24EpgJ3icjULsXuB6qMMdnAr4FfWtdOBe4EpgE3AI+LiL2HOv8GTAauAKKAr/jr3gaasrIydu3axXXXXUdhRQMAmSnaQrlUIsL3bphMSW0zf/6o0GuZa665Rru9lOrCny2U+UCBMeaEMaYVWAOs7FJmJfCs9X4dsEJExDq+xhjTYow5CRRY9fms0xjzurEAO4HRfry3AeWf//wnbrebm2++maKKBiIcNtLidA7K5Vg4Lpllk1J5fHMBlQ0Xr90VHx/P4sWLeeONN3C5fO/2qNRQ4s+EMgo43eHzGeuY1zLGGCdQAyR3c22PdVpdXV8AvE4UEJHVIrJbRHaXl5f38ZYGptdee40ZM2aQlZVFYUUjY5Ojsdkk2GENet+/cQqNrS5+8o9DXs/fcsstlJSUsGPHjgBHptTAFIqD8o8D7xtjPvB20hjzpDEmxxiTk5qaGuDQ+l9eXh75+fmsXOlp/BVVNOj4ST+ZlB7Hg8uzeXXfWd72sqPjNddcQ3x8PK+++mrgg1NqAPJnQikGMjp8Hm0d81pGRBxAAlDRzbXd1iki/xtIBb7TL3cwCLz22muEhYVx44034nYbiioayUrRhNJfHlyezeT0OL67bj+nKxs7nYuIiODGG29k06ZN1NfXBylCpQYOfyaUXcAEEckSkXA8g+zru5RZD9xrvb8NeNcaA1kP3Gk9BZYFTMAzLuKzThH5CnA9cJcxZkg8y+l0OtmwYQNLly4lKSmJ0rpmWpxuxuojw/0m3GHjD/fMxeU2rP7LHupbOs9NufXWW2lubtbBeaXwY0KxxkS+AWwEjgBrjTGHROQnInKLVexpIFlECvC0Kh6yrj0ErAUO4xkLedAY4/JVp1XXH4A0YJuI7BORh/11bwPFRx99xPnz5y90dxWe9/wFfbys4cIquu0vdekyU2L477tmc7S0jvv+tIuGDkll5syZZGZmareXUoDDn5UbY14HXu9y7OEO75uB231c+zPgZ72p0zru13sZiF577TUSEhK4+uqrAc/4CUBybHgwwwoJ3pLwb+6Yxf9Y8zH3/XkXf/ryPKLDHYgIt956K7/5zW84ffo0GRkZXmpTamgIxUH5IaG2tpZ33nmHT3/604SHexLI8fJ6Ihw2EqKG1JzOgLl55kh+fccsdhVWcv+fd19YmuWWW25BRHjttdeCHKFSwaUJZZB65ZVXaGlp4bbbbrtwLL+0nuzhsdhEHxn2h+d3nKKhxcVtczPYfqKC6379Pk++f4IRI0awcOFCXnnlFZ2TooY0TSiDkNvt5oUXXmDWrFlMnfrJ4gPHSuuYmBYXxMiGhlkZidyzcCyltc38YctxCs83cMcdd3D27Fm2bNkS7PCUChpNKIPQtm3bKCoqYtWqVReO1TS1ca6mWRNKgEwZEc/9S8bR3Obis7//iJRJc0lPT+dvf/tbsENTKmiG3EB2KHjhhRcYNmwY119/Yc1MCso829ZOTIultLYlWKENKWOGRfPA0vGs23OGu5/exc3Lb2LDC09z4sQJtpdf/E9r1YIxQYhSqcDRFsogc/bsWTZv3sztt99+YTAeIL/EM7FOWyiBlRIbwd+/toiJaXGsqx6N3eHQVooasjShDDIvvvgiAJ///Oc7HT9aWkd0uF13agyC1LgI1qxeyKfnTqQlfSYvrnuZ+vq6YIelVMBpQhlEWltbWbduHcuXL2fkyJGdzh0trWPC8FhdFDJIosMd/G7VbO64axWu1mae/Mtaapvbgh2WUgGlYyiDyCuvvEJlZSX33HPPReeOltazbNLgX+xyMOo4CXLqtOm8OzqbykObeezdhXzhyizdjlkNGdpCGSTa2tr44x//yIwZM1iwYEGnc2W1zZyvb2Fyuo6fDATLb/k8Ul8Oxbk8+f4J9p+uDnZISgWEJpRB4p///CfFxcU88MADSJeJi/vP1ACe+REq+KbkLCF5RAaJhe8xKjGCF3ef5p0jpXjWPVUqdGlCGQRcLhdPPvkkkydPZtmyZRed33+6GrtNmDYyIfDBqYvYbHauunkV54sLuSq6lDljEnknr4z/+eI+n3vUKxUKNKEMAps2beLkyZOsXr36otYJwP4z1UxOjyMq3B6E6JQ30xYsZVjaSLZueIHPzh7FdVPTeG3fWe5+agcV9TpPSIUmTSgDnDGGJ554gqysLK677rqLzrvdhv2nq5mp3V0Dis1uZ8ln7qKkqIDjubtYNmk4j62aw8HiGm59fOuFiahKhRJNKAPcpk2byMvLY/Xq1djtF7dACisaqG12Mmt0YuCDU92afuU1JKak8f5rf8UYw00zRrBm9UKaWt187vfb2FVYGewQlepX+tjwANbW1sYjjzzC+PHj+cxnPuO1zP4z1QDaQhmA7A4HV91yN/945hEO7dgCC7/I7DFJvPL1Rdz7zE7ufmoHj94xi6rGi+er6DItajDSFsoAtm7dOoqKivi3f/s3HA7vuX9PURUx4Xayh8cGODrVGzOWfIr0sdm8s/YpmpqaAMgYFs26ry1i+sh4vv78Xj46fj7IUSrVPzShDFA1NTX89re/JScnx+uTXe0+PHaeheOSsesM+QHJZrNz/aoHqK0s55lnnrlwfFhMOM9/dSHXTkljQ+453jh4Drc+VqwGOU0oA9Tvfvc7ampq+MEPfuD1yS6A05WNFFY0smRCSoCjU30xZtIVTJl3FU899RTnzp27cDwyzM7v75nLwnHD+ODYedbuPo3T5Q5ipEpdHk0oA1BeXh4vvPACd9xxB5MnT/ZZbmuBp6tkSbYmlIHuU5//CsYYHnnkkU7H7Tbh5hkjuX5aOrlnanhuWxGtTk0qanDShDLAOJ1OfvjDH5KYmMi3vvWtbst+UHCetPgIHT8ZBBJT07nvvvvYsGEDW7du7XRORFg6MZXb5ozmeHk9f/6okPoWZ5AiVerSaUIZYP7yl79w8OBBfvCDH5CYmOiznNtt2JxXxsiEKF7YeZrnd5y68FID0+rVq8nKyuLhhx+moaHhovNzxibx+XkZnKps4ItP79DVitWgowllADl69Ci/+c1vWLFiBTfccEO3ZXcVVtLY6mKiLgg5aLy8v4ylq77J2XPnWP3vD3v9A2Dm6ETumj+GA8U1fPHpnTRoS0UNIppQBojW1la+973vERcXx49//GOfA/Ht/pF7ljC76ArDg0zGhGlcecNt7H3vdfL2fOS1zLSRCfxu1Rxyz1TzwF/30OLU9b/U4KAJZYD4+c9/Tn5+Pj/96U9JTk7utqzT5eaNAyVMTo8nwqHrdw02yz93L+ljs9nwzCNUny/1WqaivpV/mT2KD46d53O/38Zftxdpd6Ya8DShDADr169nzZo13Hfffd3OOWm37UQFFQ2tzBitqwsPRnZHGJ/92n/idrtY97v/Q1ur98Ui544dxo3T0zlYXMP6/Wd1+Xs14GlCCbJ9+/bxwx/+kJycHL797W/36pq/7zlDXISDiWna3TVYJaePYuXq73Gu8Bj//NOjPpPFVRNSuXpCKjtPVvLe0fIAR6lU32hCCaIzZ87wjW98g7S0NB599FGfy6t0VFLTzIbcc9yWM5owu/7nG8wmzb6SZZ+9lwPb3mHLK8/5LHf9tDRmZSSy6XApL+0+HcAIleob/Y0UJKWlpXz5y1/G6XTyhz/8gWHDhvXquue2FeI2hi8vyvJzhCoQltx8F7Ouup4P1j/PR6+/5LWMiPDZOaPITo3l+y8fYIu2VNQApQklCEpLS7nvvvuorKzkj3/8I+PGjevVdQ0tTv624xTXTU1nTHK0n6NUgSAifPrebzF1/lLeWfsUH25Y47Wcw2Zj1YIxTEiL42t/3cPB4poAR6pUzzShBNjp06f5whe+QElJCU888QRXXHFFr699/L0CapraeGDZeD9GqALN7nDwL//6H0xfuJzN6/7E+6/9zWu5yDA7f/7yPJKiw/nSn3ZxurIxwJEq1T1NKAG0bds2br/9dmpra/nTn/5ETk5Or689XdnIHz84ya2zRjJL9z4JOTa7nZWrv8sVi1aw5ZXn2PDMr3G2tl5ULi0+kmfvm0eby829z+yksuHiMkoFiyaUADDG8Oyzz/LVr36V1NRU1q5dy4wZM/p0/cOvHcQuwn/c6HuxSDW42Wx2bvnKv7Hk5rv4+P03efYX/05NRVmnMs/vOMXOk1XckZPBqcpGVv7uQ2q8bNClVDBoQvGzsrIyvvnNb/KLX/yC5cuXs2bNGsaM6dtufE99cJLN+eX8xw2TGJEQ5adI1UBgs9lZ/rkvcfs3H+b82dM89aNvcGTXBxc9VpyZEsOd88ZwtrqZO57cRlltc5AiVuoTmlD8xBjD2rVr+cxnPsOHH37Id7/7XR599FFiYmL6VM+Wo+X88s08rp+Wxr2LMv0TrBpwJs9dzP0PP0pcUgrrHvspLz76o4taK1NHxvPFRWM5VdnITb/9kB0nKoIUrVIeMpRn3+bk5Jjdu3f3a53GGLZs2cJjjz3GwYMHmT9/Pj/5yU8YO3Zsn+vaWnCe+/68i/GpsbyweiEJUWGdzutSHKHP7XKx461XPPNURJh/7a0svOFzRMfGXygzZ2wiX/vrXgorGlg1fwzfuXYiybERQYxahToR2WOMuWgQWBNKPyWU1tZWNm/ezNNPP82BAwcYPXo0Dz74ICtXruxxoceujDE8t62I/7PhMONSY3jhqwu9/oLQhDJ0VJeX8Pbapziy+0PCwiPIWXEz81bcQkLycFYtGENdcxuPbDrKsx8VEma3ceusUXx6xgjmZSYRHd7zhFml+kITiheXm1CMMeTl5bFhwwZeffVVKisrGTVqFA888AArV64kLCys50q6KCir41//sofj5Q1MTo/j9rkZRIXrApDKo6y4kA//8QKHdmwBYNzU2Xz9y6tYtmwZMTExFJTV8dQHJ3l5bzGtLjcCJMWEkx4fSXJsOCsmD2dkYtSFV1J0GC/s9D77ftWCvo31qaFDE4oXl5pQtm7dyqZNm9iyZQslJSU4HA6WL1/O5z73OZYsWYLd3rcE0OZy82HBeV7afZo3DpYQ4bBx7dR0FmQNw9bH1o0aGqrKS9j/4VvkfriJmooy7I4wxk66guwZ8xg7eSZJI8ZQWNlEcXUTJbUtlNY0U9XYitPd+d97ZJiN2IgwEqPCSIi2vlrvV80fQ1pCJLHhDmw2/TlUnwhKQhGRG4BHATvwlDHmF13ORwDPAXOBCuAOY0yhde77wP2AC/iWMWZjd3WKSBawBkgG9gBfMMZ0+5D+pSaUBx98kO3bt7No0SKWL1/O0qVLe1xyviNjDGdrmtlTVMVHBed581AJ1Y1txEc6+MKVY0mICic2QrspVM+M201R/gGO7d9BQe4uzp/1dIOGhUcwInMCwzOySE7PIHnEaIaljeT2q2dQUtvC2eomzlY3c7a6ie0nKqhuaqOmqY26Zu8bekWF2YkOt+M2hjC7DYdNsNsEh93G6KQowu02wh02wuw2YiIcxEc5iI8MIz7SQXxUmOd9VBhJ0WEkRYcTHxWGXZPUoBXwhCIiduAocC1wBtgF3GWMOdyhzNeBGcaYB0TkTuBfjDF3iMhU4AVgPjASeBuYaF3mtU4RWQu8bIxZIyJ/APYbY37fXYyXmlDKyspITEwkPDwc8GzH63Qb3Mbz1elyU9fspLqxjarGVqqb2iiva6HwfAOFFQ0cLa2jtNazZHm4w8aU9DhmjE5kwvBYHLrgo7oM1edLOVNwhOLjRyg+nkf52VO0Nn8yo97uCCM2IYmYhCRi4pOITUgiKjaeiMgoIqKisYdH4rJH0CbhTBw1jJpmQ7Mb2txCs1s4fr4JJzbc2HBhx43gdBlcgNMNLreh1WlodrpwugEfLWwRSIjyJJfE6M5fk6LDSIwOJyEqjOhwO1HhdqLDHReSWqT11WEXbCIIeL6KZykbt9vQ6nJ7Xk43bdbXVqeblgsvV6fPrdYxp8tgE7DZBLt46rfZBLsNwuy2C4nzwlcriXY61iG5htnF6xiqMQZjwGU8vzeMAbcxuNwGp8vQ5nbjdHV+3+ZyX/j90uYyGD5J7mF2Gw6752uYzfPeYZcL7zv+EdDXMV3v//28JxR//hk8HygwxpywAlgDrAQOdyizEviR9X4d8Dvx3O1KYI0xpgU4KSIFVn14q1NEjgDXAKusMs9a9XabUC7VwxtP897RPZ7/+G7PD0NvxEU6yEqJYdH4FGZlJFJW10J6fKT+pab6TWJKGokpaUxfuAzw/OJqqKmiorSYinOnqSo7R31NFQ01ldRWlHH2RB5NDXW4XRfvCvnOZcQRZr3aidhAwPofEKEVKEUoBYwIGPD8Uxps3fCD79+vAH949m8snTu9X+v1Z0IZBXQc7TsDLPBVxhjjFJEaPF1Wo4DtXa4dZb33VmcyUG2McXop34mIrAZWWx/rRSS/D/cEkAKc7+M1Fxy81AsD67LucZAI9XsM9fsDvcfLsiyn9+sIeuF1HsSQ66g3xjwJPHmp14vIbm9NvVCi9zj4hfr9gd7jQOTPDvtiIKPD59HWMa9lRMQBJOAZnPd1ra/jFUCiVYev76WUUsqP/JlQdgETRCRLRMKBO4H1XcqsB+613t8GvGs8TwmsB+4UkQjr6a0JwE5fdVrXbLbqwKrzNT/em1JKqS781uVljYl8A9iI5xHfZ4wxh0TkJ8BuY8x64GngL9ageyWeBIFVbi2eAXwn8KAxxgXgrU7rW/4HsEZEfgp8bNXtD5fcXTaI6D0OfqF+f6D3OOAM6YmNSiml+o9OelBKKdUvNKEopZTqF5pQ+kBEbhCRfBEpEJGHgh1PX4jIMyJSJiIHOxwbJiKbROSY9TXJOi4i8t/WfeaKyJwO19xrlT8mIvd6+17BICIZIrJZRA6LyCER+R/W8VC6x0gR2Ski+617/LF1PEtEdlj38qL1wArWQy0vWsd3iEhmh7q+bx3PF5Hrg3RLXomIXUQ+FpEN1udQu79CETkgIvtEZLd1LDR+Tj1LAOirpxeehwCOA+OAcGA/MDXYcfUh/quBOcDBDsd+BTxkvX8I+KX1/tPAG3gm1C4EdljHhwEnrK9J1vukYN+bFdsIYI71Pg7PEj1TQ+weBYi13ocBO6zY1wJ3Wsf/AHzNev914A/W+zuBF633U62f3wggy/q5tgf7/jrc53eA54EN1udQu79CIKXLsZD4OdUWSu9dWErGeBadbF9KZlAwxryP50m6jlbiWaYG6+utHY4/Zzy245njMwK4HthkjKk0xlQBm4Ab/B58Lxhjzhlj9lrv64AjeFZLCKV7NMaYeutj++omBs+yQ+us413vsf3e1wErRDovbWSMOQl0XNooqERkNHAT8JT1WQih++tGSPycakLpPW9LyXhd3mUQSTPGnLPelwBp1ntf9zoo/j+wuj5m4/kLPqTu0eoO2geU4fklchzfyw51WtoI6Li00UC9x98A3wPc1ufullUajPcHnj8C3hKRPeJZCgpC5Od0yC29orwzxhgRGfTPkItILPB34H8aY2qlw8qqoXCPxjMfa5aIJAKvAJODG1H/EZHPAGXGmD0isizI4fjTEmNMsYgMBzaJSF7Hk4P551RbKL3Xm6VkBptSq/mM9bXMOt7XpW8GBBEJw5NM/maMedk6HFL32M4YU41ndYgr8b3sUF+XNgq2xcAtIlKIp0v5Gjx7H4XK/QFgjCm2vpbh+aNgPiHyc6oJpfd6s5TMYNNx6ZuOy9WsB75oPWGyEKixmuMbgetEJMl6CuU661jQWX3nTwNHjDGPdDgVSveYarVMEJEoPPsCHcH3skN9XdooqIwx3zfGjDbGZOL59/WuMeZuQuT+AEQkRkTi2t/j+fk6SKj8nAb7qYDB9MLzxMVRPP3WPwh2PH2M/QXgHNCGp7/1fjz9ze8Ax/BsYjbMKivAY9Z9HgByOtRzH55BzgLgy8G+rw5xLcHTN50L7LNenw6xe5yBZ1mhXDy/hB62jo/D8wuzAHgJiLCOR1qfC6zz4zrU9QPr3vOBG4N9b17udRmfPOUVMvdn3ct+63Wo/fdIqPyc6tIrSiml+oV2eSmllOoXmlCUUkr1C00oSiml+oUmFKWUUv1CE4pSSql+oQlFqUskImki8ryInLCW0dgmIv/ipVymdFjlucPxn4jIp3rxfWaJiBGRoK/VpFR3NKEodQmsiZSvAu8bY8YZY+bimYw3uks5n8sbGWMeNsa83YtvdxfwofXVaywiov+WVdDpD6FSl+YaoNUY84f2A8aYImPMb0XkSyKyXkTexTNZzSsR+bOI3CaefXZe6nB8mXyyF4gAtwNfAq4VkUjreKZ49vp4Ds8kxwwR+a6I7LL2zfhxh/petVpQhzosRqhUv9OEotSlmQbs7eb8HOA2Y8zSXtT1NrDAWooD4A48a1kBLAJOGmOOA+/hWdq93QTgcWPMNGCS9Xk+MAuYKyJXW+Xus1pQOcC3RCS5FzEp1WeaUJTqByLymHh2UtxlHdpkjOm6/4xXxrP0+pvAzVYX2U18spbTXXySXNbQuduryHj2yADPWk7X4VmaZS+eVYgnWOe+JSL7ge14FhScgFJ+oMvXK3VpDgGfa/9gjHlQRFKA3dahhj7Wtwb4Bp5N0HYbY+pExG59j5Ui8gM86zolty8u2OV7CPBzY8wTHSu1loH/FHClMaZRRN7DswaWUv1OWyhKXZp3gUgR+VqHY9GXUd8WPN1kX+WTFskKINcYk2GMyTTGjMWzPP9FT5LhWWn2Pms/GERklLXfRgJQZSWTyXi2kVXKLzShKHUJjGdV1VuBpSJyUkR24tm69T98XDJJRM50eN3epT4XsAG40foKnu6tV7rU83e8PO1ljHkLzz7s20TkAJ4tcePwdKU5ROQI8As83V5K+YWuNqyUUqpfaAtFKaVUv9CEopRSql9oQlFKKdUvNKEopZTqF5pQlFJK9QtNKEoppfqFJhSllFL94v8DiwsMydxSfQQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAva0lEQVR4nO3deZjd4/nH8fedyB5ksTR7UoKKKjr2VpE0RRHUliaEIJJR0pW2+quqaou2lspJMlkkZIrYY0saCaULkqCWoCJkR1ZEIpHM/fvjeUZOxsycM8vZZj6v65przvc53+WeEeeeZzd3R0REpDpNch2AiIjkPyULERFJSclCRERSUrIQEZGUlCxERCQlJQsREUlJyUIkiZm5me1Zy2vfNbN+Vbz3TTN7s7JzzeyXZja+dhHXKL6jzWxppp8jDZOShRS8+MG70czWm9n7ZjbJzNrmOq5k7v6Mu+9dxXu/d/cLAcysZ0xYO9TmOWZ2npltjb+Lj8zsJTM7sRb3mWRmv6tNDNIwKVlIQ3GSu7cFDgKKgF9VPKG2H8AF6D/xd9EOmABMNbP2uQ1JCp2ShTQo7r4MeBzYDz5vVrrEzN4C3oplF5nZAjNbY2bTzKxzhducYGYLzWyVmd1gZk3idXuY2WwzWx3fKzWzdhWuPdjM5pvZWjO7zcxaxmurbAIys9+Y2ZR4+HT8vi7WDr4V4/xq0vm7mdkGM9s1xe+iDJgItAL2qOS5XzGzp8xsnZm9ZmYnx/JhwCDg8hjDw9U9RxoHJQtpUMysG3AC8GJS8SnAocC+ZnYs8AfgTKATsAi4q8JtTiXUTg4CBgBDy28fr+0MfAXoBvymwrWDgO8QPpz3opIaTgpHxe/t3L2tu/8jxjc46ZyBwCx3X1ndjWJN6kJgPTFRJr3XDHgY+DuwG3ApUGpme7t7CVAKXB9jOKmGP4M0QEoW0lA8aGbrgH8C/wB+n/TeH9x9jbtvJHyYT3T3F9x9E/AL4HAz65l0/nXx/MXATYQPZ9x9gbvPdPdN8YP6L8C3KsRxq7svcfc1wLXl19bRZGCgmVk8Pge4o5rzD4u/i/fi80919w8rngO0Bf7o7pvdfTbwSD3FKw1QY2nDlYbvFHd/oor3liS97gy8UH7g7uvNbDXQBXi3kvMXxWsws92Bm4FvAjsS/thaW82zPr+2Ltz9OTPbABxtZiuAPYFp1VzyrLt/I8VtOwNLYlNVuUWE34PIF6hmIY1B8tLKy4Ee5Qdm1gboCCxLOqdb0uvu8RoItRUHvuruOxGahoztVXVtbWJNNjk+7xzgXnf/tIb3rWg50K28Pybqzrbfg5ajlu0oWUhjcydwvpkdYGYtCAngOXd/N+mcn5lZ+9j/MRK4O5bvSGj//9DMugA/q+T+l5hZVzPrAFyZdG26VgJlwJcrlE8h9KUMBm6v4T0r8xywgdCJ3czMjgZOYlv/zfuVxCCNmJKFNCqxqer/gPuAFYSO6LMrnPYQMA94CXiUMPwU4GpCp/eHsfz+Sh7xN0Kn8ULgbaBGcxXcfQOhr+NfcZTSYbF8CaH5zIFnanLPKp6zmZAcjgdWAQngXHd/I54ygTAgYJ2ZPVjX50nhM21+JFIYzGwisNzdazrCSqTO1MEtUgDiaK3TgANzHIo0UmqGEslzZnYN8Cpwg7u/k+t4pHFSM5SIiKSkmoWIiKTUIPssdtllF+/Zs2euwxARKSjz5s1b5e6VrjnWIJNFz549mTt3bq7DEBEpKGa2qKr31AwlIiIpKVmIiEhKShYiIpKSkoWIiKSkZCEiIikpWYiINAClpdCzJzRpEr6Xltbv/Rvk0FkRkcaktBSGDYMNG8LxokXhGGDQoPp5hmoWIiIF7sortyWKchs2hPL6omQhIlLgFi+uWXltZDRZmNm7ZvaKmb1kZnNjWQczm2lmb8Xv7WO5mdktZrbAzF42s4OS7jMknv+WmQ3JZMwiIoWme/ealddGNmoWx7j7Ae5eFI9/Dsxy997ArHgMYceu3vFrGDAaQnIBrgIOBQ4BripPMCIiAtdeC61bb1/WunUory+5aIYaQNh8nvj9lKTy2z14FmhnZp2A7wAz3X2Nu68FZgLHZTlmEZG8NWgQlJRAjx5gFr6XlNRf5zZkfjSUA383MwfGunsJsLu7r4jvvwfsHl93AZYkXbs0llVVvh0zG0aokdC9PuteIiIFYNCg+k0OFWU6WXzD3ZeZ2W7ATDN7I/lNd/eYSOosJqISgKKiIu3oJCJSjzLaDOXuy+L3D4AHCH0O78fmJeL3D+Lpy4BuSZd3jWVVlYuISJZkLFmYWRsz27H8NdCfsI/wNKB8RNMQ4KH4ehpwbhwVdRjwYWyumgH0N7P2sWO7fywTEZEsyWQz1O7AA2ZW/py/uft0M5sDTDWzC4BFwJnx/MeAE4AFwAbgfAB3XxM3rJ8Tz/utu6/JYNwiIlKBuTe85v2ioiLXTnkiIjVjZvOSpjlsRzO4RUQkJSULERFJSclCRERSUrIQEZGUlCxERCQlJQsREUlJyUJERFJSshARkZSULEREJCUlCxERSUnJQkQkT5WWQs+e0KRJ+F5amrtYMr2fhYiI1EJpKQwbBhs2hONFi8IxZHaTo6qoZiEikoeuvHJboii3YUMozwUlCxGRPLR4cc3KM03JQkQkD3XvXrPyTFOyEBHJQ9deC61bb1/WunUozwUlCxGRHKts1NOgQVBSAj16gFn4XlKSm85t0GgoEZGcSjXqKVfJoSLVLEREcijfRj1VRclCRCSH8m3UU1WULEREcijfRj1VRclCRCSH8m3UU1WULEREcijfRj1VRaOhRERyLJ9GPVVFNQsREUlJyUJERFJSshARkZSULEREJKWMJwsza2pmL5rZI/G4l5k9Z2YLzOxuM2sey1vE4wXx/Z5J9/hFLH/TzL6T6ZhFRGR72ahZjAReTzq+DrjR3fcE1gIXxPILgLWx/MZ4Hma2L3A20Ac4DkiYWdMsxC0iIlFGk4WZdQW+C4yPxwYcC9wbT5kMnBJfD4jHxPf7xvMHAHe5+yZ3fwdYABySybhFRGR7ma5Z3ARcDpTF447AOnffEo+XAl3i6y7AEoD4/ofx/M/LK7nmc2Y2zMzmmtnclStX1vOPISLSuGUsWZjZicAH7j4vU89I5u4l7l7k7kW77rprNh4pItJoZHIG95HAyWZ2AtAS2Am4GWhnZjvE2kNXYFk8fxnQDVhqZjsAOwOrk8rLJV8jIiJZkLGahbv/wt27untPQgf1bHcfBDwJnB5PGwI8FF9Pi8fE92e7u8fys+NoqV5Ab+D5TMUtIiJflIt5FlcAPzazBYQ+iQmxfALQMZb/GPg5gLu/BkwF5gPTgUvcfWvWoxYRqSeVbaOa7yz88d6wFBUV+dy5c3MdhojIF1TcRhXCkuT5sNKsmc1z96LK3tMMbhGRLCqUbVQrUrIQEcmiQtlGtSIlCxGRLCqUbVQrUrIQEcmC8k7tRYvCjnjJ8nEb1YqULEREMqy8U3vRonDsvi1h5Os2qhVpW1URkQwpLQ0d1+VJIpl7SBTvvpv1sGpFyUJEpJ6VlsLIkbB6dfXn5XundjIlCxGRelTZPIqq5HundjL1WYiI1KPK5lFUphA6tZMpWYiI1KN0mpYKpVM7mZKFiEg9KC2FXXYJHddVad0apkwJndqFlChAfRYiInVWWgrnnw+ffVb1OR07ws03F16SKKdkISJSB6WlMGQIbK1iLeymTWHy5MJNEuXUDCUiUkvlI5+qShQAZWWFnyhAyUJEpMbKl+4YPDj1yKdCGh5bnRo1Q5lZE6Ctu3+UoXhERPJaTeZRNG9eWMNjq5OyZmFmfzOzncysDfAqMN/Mfpb50ERE8k+68yg6doSJExtGExSk1wy1b6xJnAI8DvQCzslkUCIi+SqdeRRTpsCqVQ0nUUB6yaKZmTUjJItp7v4Z0PD2YhURSUOHDtW/37Fjw0oS5dJJFmOBd4E2wNNm1gNQn4WINDqlpfBRNZ9+rVuHuRQNUcpk4e63uHsXdz/Bg0XAMVmITUQkL5TPzh48uOqJd4W4hEdNpBwNZWa7A78HOrv78Wa2L3A4MCHTwYmI5FpxMYweXf05ZoWzL0VtpdMMNQmYAXSOx/8DfpiheERE8kZpKYwZk/q8hjKXojrpJItd3H0qUAbg7luAauYriog0DCNHVr8wYLm8mUuxaBG8/npGbp1OsvjEzDoSR0CZ2WHAhxmJRkQkTxQXp97pDvJg9FNZGcyYAQMGwJe/DJdfnpHHpDOD+8fANGAPM/sXsCtwekaiERHJA/36waxZqc9r3jyHo5/WrIFJk0KHyoIFsOuu8POfh+nlGZAyWbj7C2b2LWBvwIA341wLEZEGpbQUhg6FzZtTn5uzJcfnzYNRo+DOO+HTT+HII+G3v4XTToMWLTL22HRGQ51boeggM8Pdb89QTCIiWZfOnhQQksSqVdmJ6XMbN8LUqZBIwPPPQ5s2YV30ESPga1/LSgjpNEMdnPS6JdAXeAGoNlmYWUvgaaBFfM697n6VmfUC7gI6AvOAc9x9s5m1iPf8OrAaOMvd3433+gVwAaFj/TJ3n5H2TygikoaRI1MnCrMsNzstXBiGY02YEJqd9tkHbrkFzj0Xdt45i4Gk1wx1afKxmbUjfNinsgk41t3Xx+VC/mlmjxP6QG5097vMbAwhCYyO39e6+55mdjZwHXBWnNdxNtCHMHz3CTPby901IktE6kVpaXqd2cOHZ6HZaetWmD49NDVNnw5NmsApp4Qe92OOCRkrB2qzn8UnhMUEqxVne6+Ph83ilwPHAvfG8smENacABsRj4vt9zcxi+V3uvsnd3wEWAIfUIm4RkUoNH576nL59QytQxqxaBdddB3vuCSeeCC++CP/3f2E47L33wrHH5ixRQHp9Fg+zbeHAJsC+wNR0bm5mTQlNTXsCo4C3gXVxrgbAUqBLfN0FWAJhLoeZfUhoquoCPJt02+RrRETqpF8/WL+++nP69oUnnsjAw93huedCFpo6FTZtgqOPhuuvD7WJZs0y8NDaSafP4k9Jr7cAi9x9aTo3j01FB8SmqweAfWocYZrMbBgwDKB7Y5hOKSJ1ls4Q2SlTMtD0tGFDGM00alSoQey4I1x4Yeiw7tOnnh9WP9Lps/hHXR/i7uvM7EnCmlLtzGyHWLvoCiyLpy0DugFLzWwHYGdCR3d5ebnka5KfUQKUABQVFWkJdRGpVnFx6kRR7xPu/ve/MC9i0iRYtw722y8cDxoUEkYeq7LPwsw+NrOPKvn62MxSLlFuZrvGGgVm1gr4NvA68CTbJvUNAR6Kr6fFY+L7s93dY/nZZtYijqTqDTxf459URCRKZ3FAqKeRT1u2wIMPQv/+sPfecOutcNxx8PTT8PLLocMkzxMFVFOzcPe6Rt8JmBz7LZoAU939ETObD9xlZr8DXmTb6rUTgDvMbAGwhjACCnd/zcymAvMJzWCXaCSUiNRGTSbdjRhRx1rF++/D+PEwdiwsWQJdu8I114Tmpi99qQ43zg3zdFbJAsxsN8I8CwDcPY3NBXOjqKjI586dm+swRCQPpFuLSFbrDm13+Ne/Ql/EffeFiRv9+oUgTjoJdkinmzh3zGyeuxdV9l46o6FOBv5MmOPwAdCD0JyUn70wItLo1aQGUVHz5rVIFOvXh57wRAJeeSVMmLvkktDEtPfeNQ8iD6WT5q4BDgOecPcDzewYYHBmwxIRqZ3S0rCjXW1NnFiDk+fPD9WWyZPh44/hgANg3DgYODAsydGApJMsPnP31WbWxMyauPuTZnZTpgMTEamJutQmyqXVT/HZZ/DQQ6Gp6amnQlXkzDNDU9Nhh+V04lwmpZMs1plZW8I6T6Vm9gFhFreISM6lu5x4KiNGpJihvXx52GS7pARWrAibbv/hD3DBBWF58AauymRhZmcADxOW29gI/AgYRJj/8NusRCciUo327cN0hbpo2TIMWqq0RuEeag+JBDzwQFi36bjjQsI4/nho2rRuDy8g1dUsvk9YomMGcCcww90nV3O+iEjGdekS/sivq7Ztw4KulSaJDz+EO+4ISeL116FDB/jRj0KH9R571P3hBajKSXnufiphTacngEsJM6vHxI2QRESyqrg4dAfUJVH07RsqC+6hP/oLieKVV0JC6NIFLr00ZJRJk2DpUrjhhkabKCBFn4W7f0RYCXZy3If7dOAWM+vg7t2qu1ZEpL7UtTbRuTMs+8IiQdHmzWFORCIB//xnaJcaODB0Yhx8cBUXNT5pzRAxs/bAacBZQAe2LTEuIpJRzZun3pSoOlVOsFu8OPQ9jBsHH3wQag1/+hOcd15YFEq2U10Hd1vgVGAgcCBhjaZrgKc83WnfIiK11KdPmMZQW5UmibKyMHRq1Ch4+OHQHnXiiaGNq3//sNGQVKq6msW7wHQgQejcrkNuFxFJra4T6qCKIbBr14a+h9Gj4a23wlDXK66AYcOgZ8+6PbCRqC5ZdHP3jVmLREQatboOg91hh5APtuu0fuGFkDn+9jfYuBEOPxyuugpOPx1atKhjxI1LdavOKlGISMbVpbmpWbNKZmx/+incc09IEs8+C61bh+rKiBFw4IF1jrexyu8lEEWkQWvdOvzBXxtfGOH0zjth4sSECbB6Ney1F9x0EwwZAu3a1UO0jZuShYjkRF0Sxef9EmVlMH16OHjssdBBPWBA6LA+9tgGu05TLlQ3GuphoMpRT+5+ckYiEpEGrTb7S5Rr1SpsX83q1XDDxHCjd96B3XeHX/0qdFh37Vqv8UpQXc3iT/H7acCXgCnxeCDwfiaDEpGGqS5zJkYMdxLnz4HzEnDXXbBpExx1VFjM79RTw80lY6rr4P4HgJn9ucLOSQ+bmbahE5G01bY20awZbF63ISSHRALGzAtLcAwdGm663371H6xUKp0+izZm9mV3XwhgZr2AhrWrh4hkTG1rE78e+BZXdxoDXW8L8yT23TdMphs8GHbaqf4DlWqlkyx+BDxlZgsBI2yrenFGoxKRglebfSaasJUBTR/l/n4JuHNGmDxx2mmhFnHUUeqwzqGUycLdp5tZb2CfWPSGu2/KbFgiUsiaNg0DldK1Kx9wIeO5mLH02LoYXukMV18NF10EnTplLlBJW8pkYWatgR8DPdz9IjPrbWZ7u/sjmQ9PRApN+n/8O0fwb4pJcAb30JzPwoJOxTfCSSeFDgvJG+k0Q90GzAMOj8fLgHsAJQsR2U46iaIN6/k+f6OYBAfwXz5kJ5pfNiJMnthnn9Q3kJxIZ4nFPdz9euAzAHffQOi7EBEBwgKAqRLF3rzBTYxkGV0o4WIM58Z9xrLzx8vg5puVKPJcOjWLzWbWijhBz8z2ANRnISJA9TOxm7KFATxEMQn6MpvNNOMezmAUl/DvssP5mjqsC0Y6yeIqwlLl3cysFDgSOC+TQYlIYajqs/5LrOAixjGMErqyjEV05xf8nglcwP59d+PflW1GJHmt2mRhZk2A8l3yDiM0P41091VZiE1E8lTlw2Kdo3iaYhKcxv00YwvT+Q7FJHiU71JGU7RtWuFKtQd3mZld7u5TgUezFJOI5LGKtYkd+YjBTKGYBPvxGmtozy1cxhiGs4DeQIo9sKUgpNMM9YSZ/RS4G/ikvNDd12QsKhHJOxXnTvThVYpJcA53sCPrmcvXOZ+J3M1ZbKT15+dVuQe2FJR0RkOdBVwCPE0YQjsPSLk2lJl1M7MnzWy+mb1mZiNjeQczm2lmb8Xv7WO5mdktZrbAzF42s4OS7jUknv+WmQ2pzQ8qIrXTr1+oTZSVQTM2cyZ38xTf4lW+ylAmch/f4xCe42DmMInzt0sUI0YoUTQU6czg7lXLe28BfuLuL5jZjsA8M5tJ6Byf5e5/NLOfAz8HrgCOB3rHr0OB0cChZtaB0MleRBiRNc/Mprn72lrGJSJp6NIFli+Pr1nKMEq4iHF04j0W0oufcT0TGcoaOlZ6vfonGpaazODu7u7D4tIfKWdwu/sKYEV8/bGZvQ50AQYAR8fTJgNPEZLFAOB2d3fgWTNrZ2ad4rkzy5u9YsI5DrizZj+qiKRjW5Jw+jKLYhKczDSaUMZjnMAoLmEG38GraJho0gS2bs1qyJIF6TRD3QZsBo6Ix8uA39XkIWbWEzgQeA7YPSYSgPeA3ePrLsCSpMuWxrKqyis+Y5iZzTWzuStXrqxJeCISmcEny9dxGTfzOl/hCb7NUTzNn/gpe/A2J/EI0zm+ykTRt68SRUOVTgf3Hu5+lpkNhDCD2yz9mTRm1ha4D/ihu3+UfKm7u5nVS2XV3UuAEoCioiJVgEVqoE8faDb/JcaSYBCltGED/+EwzuF27uEMNtEy5T3U7NSwpVOzqPUMbjNrRkgUpe5+fyx+PzYvEb9/EMuXAd2SLu8ay6oqF5G62rSJQVbKuPlH8BIHMpgp3MlADmIeR/AfpnBOykQxYoQSRWOQTrKoOIN7FnB5qoti7WMC8Lq7/yXprWlA+YimIcBDSeXnxlFRhwEfxuaqGUB/M2sfR071j2UiUkvf7v0uf7BfsLJlV0oZzC6s4ofcSBeWcRHjeZGDUt6jXbuQJBKJzMcruZfOaKiZZvYCNZ/BfSRwDvCKmb0Uy34J/BGYamYXAIuAM+N7jwEnAAuADcD58flrzOwaYE4877ea4yFSC2VlXNzr73x3cYIZPIJjTONkEhQzi75V9kNURjWJxse8iv/qyfMcKuPuL2QkonpQVFTkc+dqm3ARANas4Scdb2MEo9mTt3mf3RjHRYzlYpZu18KbmpJEw2Zm89y9qLL3qqtZ/Dl+b0mY4/BfQs1if8KkvMOruE5E8sHcuZQeOYrTNt/Fn/mUZ/gGv+J33M9pfEbzGt1KSUKqTBbufgyAmd0PHOTur8Tj/YDfZCU6EamZjRv5zw/vpmlJgkOYwwDaMInzGM0IXmH/Gt9OSULKpTN0du/yRAHg7q+a2VcyGJOI1NTbb3PDnmMYykQOZw3z+Qo/4K/cwTl8xM61uqUShSRLp0frFTMbb2ZHx69xwMuZDkxEUti6lRPtER6342HPPfkRNzKbYzmaJ+nDa4ziB7VKFFOmKFHIF6VTszgPGAGMjMdPE9ZtEpEc2NVWcgETGM4YHmERy+nEVfyGcVzECjrX+r5KEFKdVJsfNQUej/0XN2YnJBGpqF9f55PZz1JMgqVMpQWbmc0x/JQ/8RAD2EKzWt9bSULSkWrzo61mVmZmO7v7h9kKSkSCNvYJA7mTGxjFgbzER+xICcMYzQheZ99a37dVK9iwoR4DlQYvnWao9YR+i5lsv/nRZRmLSqQR69MHtsx/kxGMZhmTaMeHvMxXuZgxlDKIT2hbp/urJiG1kU6yuD9+iUgG9T92C22efJibSPBtnmAzzbiX00lQzL84kjDNqXaUIKSu0kkWdwN7xtcL3P3TDMYj0uh8yd7jQsYzgbF0YymL6caV/I4JXMD7fKnW923XDtZqizCpJ1UmCzPbAfg9MJSwhpMRFhO8DbjS3T/LTogiDY+Z8w3+ySWMYgn30Ywt/J1v8wNu5VG+y9a0/o6rnGoRkgnV/Yu8AdgR6OXuHwOY2U7An+LXyGquFZEKzKAtHzOYKbxMgq/yKmtpx1+5lDEM5y32qtV9990XXnutnoMVqaC6ZHEisJcnrTQYNy8aAbyBkoVIWsxgX17jr4zmXG5nJz7mBQ7kAsZzJwPZSOta3Vc1CMmm6pKFeyVL0sbhtPpnKpJCM/uMU3iQ2SQ4hqfYRHPu5iwSFPMch1LbDmslCcmF6pLFfDM7191vTy40s8GEmoWIVNC+PbRet4xhlLCIcXRmBe/Qkyv4IxMZyip2rdV9lSAk16pLFpcA95vZUGBeLCsCWgGnZjowkULRujVs3Ogcw5OMI8EpPEgTynic47mIcUznOMpoWqt7K0lIvqhuifJlwKFmdizQJxY/5u6zshKZSJ4zg534kAu4nWISfIU3WE0H/sKPGcvFLGSPWt9bSULyTTrbqs4GZmchFpG81749rFsH+/NfxpBgMFNowwae4xDOZTL3cAaf0qrG99WIJsl3tR/MLdKIdOkCq5Zv4nvcRzEJvsG/2EhL/sb3Gc0I5lHpTpQpqQYhhSL9HdpFGqF+/aC7LaZ4+ZUspjt/YxC78z4/5s90YRkXMqHGiaJv35AklCikkKhmIVJB69bw6cYy+vEElzKKE3kEgEc4kQTFzOTbeC3+zurcGZYtq+9oRbJDyUIkMoN2rGU4tzGC0fRmAR+wK9dxBWO5mMX0qNV9VYOQhkDJQho9MziIeYwnwUDupDUb+SdHchVXcx/fYzMtanxPJQhpaJQspFEygxZ8yplM5VlGcSjP8wmtuYNzGM0I/ssBNb6nRjRJQ6ZkIY1Gly6wfDn0YiHXMYahTGQXVvMGe3MZNzOZIXzEzjW+r2oR0hgoWUiDZnH5pSZs5TimU0KC43mcMprwEAMYxSU8yTHUZp0mJQlpTJQspEGxCp/5HVnFUCYynDF8mXdYwZe4hv9jHBexjK61eoaShDRGShZS0Comh8A5hOe5hFGcyVRasomn+BZXcB0PcgpbaFbj5yhBSGOXsUl5ZjbRzD4ws1eTyjqY2Uwzeyt+bx/LzcxuMbMFZvaymR2UdM2QeP5bZjYkU/FK4ejTJySJiomiFRsYygTmUsRzHMapPMB4LqQPr3IMT3EvZ9QoUWjynMg2mZzBPQk4rkLZz4FZ7t4bmBWPAY4HesevYcBoCMkFuAo4FDgEuKo8wUjj0a/ftuRgBvPnb/9+b/7Hn/kxy+jCBC6kBZsYQYLOLOdSbmX+5+tgpqdVq5AgnniiHn8IkQKXsWYod3/azHpWKB4AHB1fTwaeAq6I5bfHzZaeNbN2ZtYpnjvT3dcAmNlMQgK6M1NxS36ovHlpm6Zs4bs8yiWMoj8z+YwduI/vkaCYZ/gmNe2wbtUKNmyofbwiDV2214ba3d1XxNfvAbvH112AJUnnLY1lVZV/gZkNM7O5ZjZ35cqV9Ru1ZFxx8fa1h6rsxvv8kmtZyJd5iFP4Cq/zK66hG0sYyF08w1HUJFGUNzMpUYhUL2cd3O7u9bk9q7uXACUARUVFamXOU8XFMHp0Ta9yjuRfFJPgdO6lOZ/xBH0Zyc08zElsrcE/Y/U/iNROtpPF+2bWyd1XxGamD2L5MqBb0nldY9kytjVblZc/lYU4pRZqlwiq1ob1DKKUYhJ8jZdZx84kKGYMw3mTfdK6R7t2sHZt/cUk0lhluxlqGlA+omkI8FBS+blxVNRhwIexuWoG0N/M2seO7f6xTHIseURS+Vd9JYp9eJ1buJTldGYsw3GMiyihC8v4ETdVmyiSRzC5K1GI1JeM1SzM7E5CrWAXM1tKGNX0R2CqmV0ALALOjKc/BpwALAA2AOcDuPsaM7sGmBPP+215Z7dkT/kyGZm0A58xgIcoJsGxPMkmmjOVM0lQzLMcRqp+CDUviWSWeQP8v6yoqMjnzp2b6zAKTmkpDB0Kmzdn75mdWM5FjGMYJXRhOe/SgzEMZyJDWcluVV7XAP/ZiuScmc1z90p389IM7kasvvsY0ud8i39QTIJTeYBmbOFxjmM4Y3iMEyij6Reu6NtX8x5EcknJopHJRe2h3I58xDncQTEJ+jCf1XTgJn7IWC7mbfb8wvmqPYjkDyWLBq60FC6+GD75JHcx7McrFJPgHO6gLZ8whyLO4zbu5iysVSvNcRApANkeDSVZUloKLVrA4MG5SRTN2Mzw9nfh3zyKV9ifES0n0fa8M+D55znY5zDJz2OjK1GIFAoliwagtBR22WX7YayDB2e3qWnEiDhcdfES/MpfsXm3boxeOxCWLYMbboClS+G22+Dgg7MXlIjUGzVDFYjSUhg5Elavzm0clW4dWlYGs2bBqQmYNi1kjRNPDD3o/ftDE/1NIlLolCzyUL4khpQjkNauhcmTw5Cq//0vVG8uvzx0kvTsma0wRSQLlCzySC6TRMuWMH48DBqUxskvvgiJRAh440Y4/HC44w44/fRwIxFpcJQs8kRpKQwblt3VT2uUID79FO65JySJZ58Na3oPGhSamg48MOOxikhuKVnkUGkpXHklLFqUnefVKDmUe+cdGDsWJkyAVatgr73gpptgyJCwSp+INApKFllSnhgWL4YOHcIf6tka0tq2LYwZU4MkUVYGM2aEWsSjj4bhVQMGhFpE376pdyYSkQZHySKDkmsOZttmJGeyT6JjR7j55hrWHsqtXh2Gt44eDQsXwu67hx9g2DDo1i319SLSYClZ1KOKtYePP94216E+lq6oUyKozpw5MGoU3HUXbNoE3/wm/P73cOqp0Lx5PT9MRAqRkkU9qdhBXdfaQ9OmYVRqvSeGchs3huSQSMDcuaGtaujQMLvuq1/N0ENFpFBptlQtlJaGaQRNmoTv5TWK+hrJ1Lp1BhPFggXw05+GTSqGDg1B33prmGmdSChRiEilVLOooYo1iEWL6mfIa5MmoV+5Rw+49tp6ThRbt4aO6kQidFzvsENoYrrkEjjqKHVYi0hKqlkkqazGUFFlNYgNG0KzUSrln8kdO4Yvs5AcpkwJn+fu8O679ZgoPvgA/vAH+PKXw2imV16Bq68OGW7qVPjWt5QoRCQtqllEVdUYYPsP78WLK79+69bQfJScSJo1g512gjVroHv3DNQYKuMO//lPqEXcc0/oYT/2WPjLX+Dkk0NQIiI1pJpFVFWN4corty/r3r3y63v0gJKS8L28xnDbbWEeW1lZPdcYKvPJJzBuHBx0EBx5JDz8cFijaf78sMjf976nRCEitaaaRVRVjaFi+bXXfrGPonXrbbWGjNccKnrjjTAvYtIk+Ogj2H//MOP6+98PI5xEROqBkkXUvXvly25UrEmUJ4Py+RRZa15KtmVLWAo8kQi1hmbN4IwzwgzrI45QP4SI1Dsli6i6GkNFOalBAKxYERZ3Gjs2DHXt1i0EeMEFYba1iEiGKFlEeVFjqIw7PPNMmGF9//2hVtG/f6hVnHBCGAYrIpJh+qRJkrMaQ2U+/jjsEZFIhK3p2rWDyy6D4cOhd+9cRycijYySRb559dXQYX377bB+fRjdNGECnH12aBcTEckBJYt8sHkzPPhgaGp6+mlo0QLOOivMsD74YHVYi0jOKVnk0tKlYXLGuHHw3nvQqxdcfz2cf37Yz1pEJE8oWWSbO8yeHfoiHnoozNg74YQw7PU730lv3RARkSxTssiWdetCP0QiAW++GRaH+slPQod1r165jk5EpFoFkyzM7DjgZqApMN7d/5jjkNLz0kshQZSWhkkchx4aksYZZ4RNsUVECkBBJAszawqMAr4NLAXmmNk0d5+f28iqsGkT3HtvSBL//je0ahWW3xgxAr7+9VxHJyJSYwWRLIBDgAXuvhDAzO4CBgD5lSwWLQqzq8ePh5UrYc89w2qv550H7dvnOjoRkVorlGTRBViSdLwUODT5BDMbBgwD6F7V0rCZUFYGM2eGYa+PPhrKTjopdFj36xc2xxARKXCFkixScvcSoASgqKjIM/7ANWvCGuSjR8Pbb8Nuu8EvfhEWmMpmshIRyYJCSRbLgG5Jx11jWfbNnRv6Iu68Ez79FL7xDbjmGjjttDCZTkSkASqUZDEH6G1mvQhJ4mzg+1l7+saNYRvSRAKefx7atIEhQ0KH9de+lrUwRERypSCShbtvMbMfADMIQ2cnuvtrGX/wwoWhmWnixNDstM8+cMstcO65sPPOGX+8iEi+KIhkAeDujwGPZfxBW7fC44+HWsT06aGD+tRTQ4f10UdrnSYRaZQKJllkxZw5cOaZYcPsTp3g17+Giy6CLl1yHZmISE4pWSTbYw/Ya6+wmN8pp4TtSkVERMliOx06wIwZuY5CRCTvaMaYiIikpGQhIiIpKVmIiEhKShYiIpKSkoWIiKSkZCEiIikpWYiISEpKFiIikpK5Z37rh2wzs5XAolzHUcEuwKpcB1EDhRRvIcUKhRVvIcUKhRVvPsbaw913reyNBpks8pGZzXX3olzHka5CireQYoXCireQYoXCireQYgU1Q4mISBqULEREJCUli+wpyXUANVRI8RZSrFBY8RZSrFBY8RZSrOqzEBGR1FSzEBGRlJQsREQkJSWLLDKza8zsZTN7ycz+bmadcx1TVczsBjN7I8b7gJm1y3VM1TGzM8zsNTMrM7O8HI5oZseZ2ZtmtsDMfp7reKpjZhPN7AMzezXXsaRiZt3M7Ekzmx//DYzMdUzVMbOWZva8mf03xnt1rmNKh/osssjMdnL3j+Lry4B93X14jsOqlJn1B2a7+xYzuw7A3a/IcVhVMrOvAGXAWOCn7j43xyFtx8yaAv8Dvg0sBeYAA919fk4Dq4KZHQWsB2539/1yHU91zKwT0MndXzCzHYF5wCl5/Ls1oI27rzezZsA/gZHu/myOQ6uWahZZVJ4oojZA3mZqd/+7u2+Jh88CXXMZTyru/rq7v5nrOKpxCLDA3Re6+2bgLmBAjmOqkrs/DazJdRzpcPcV7v5CfP0x8DrQJbdRVc2D9fGwWfzK28+CckoWWWZm15rZEmAQ8Otcx5OmocDjuQ6iwHUBliQdLyWPP9AKlZn1BA4EnstxKNUys6Zm9hLwATDT3fM6XlCyqHdm9oSZvVrJ1wAAd7/S3bsBpcAP8jnWeM6VwBZCvDmVTrzSeJlZW+A+4IcVavF5x923uvsBhBr7IWaW1019ADvkOoCGxt37pXlqKfAYcFUGw6lWqljN7DzgRKCv50HnVg1+t/loGdAt6bhrLJN6ENv+7wNK3f3+XMeTLndfZ2ZPAscBeT2YQDWLLDKz3kmHA4A3chVLKmZ2HHA5cLK7b8h1PA3AHKC3mfUys+bA2cC0HMfUIMQO4wnA6+7+l1zHk4qZ7Vo+utDMWhEGPeTtZ0E5jYbKIjO7D9ibMGpnETDc3fPyr0szWwC0AFbHomfzdeQWgJmdCvwV2BVYB7zk7t/JaVAVmNkJwE1AU2Ciu1+b24iqZmZ3AkcTltF+H7jK3SfkNKgqmNk3gGeAVwj/bwH80t0fy11UVTOz/YHJhH8HTYCp7v7b3EaVmpKFiIikpGYoERFJSclCRERSUrIQEZGUlCxERCQlJQsREUlJyULympl1jKv0vmRm75nZsvh6nZlldaE4MzvFzPZNOv6tmdV4oqCZ9czlaq5m9ssKx/+O33Mal+Q3JQvJa+6+2t0PiEsjjAFujK8PYNuY+npjZtWtanAK8HmycPdfu/sT9R1DFmyXLNz9iFwFIoVDyUIKWVMzGxf3BPh7nA2Lme1hZtPNbJ6ZPWNm+8TynmY2O+7RMcvMusfySWY2xsyeA66v7HozOwI4Gbgh1mz2iNedHu9xsJn9O+5R8LyZ7Rif94yZvRC/qv1QtuBWC3tePGFmjyXd/10z2yW+LjKzp+LrQ8zsP2b2Ynz+3rH8PDO7P/4cb5nZ9bH8j0Cr+DOUxrL1lcTS1MKeJnPi7+viWN7JzJ6O179qZt+s439DKRTuri99FcQX8BvCXhUAPQkLHB4Qj6cCg+PrWUDv+PpQwr4cAA8DQ+LrocCD8fUk4BGgaYrrJwGnJ8UzCTgdaA4sBA6O5TsR1l1rDbSMZb2BuUmxv1rJz3caMJMws7czYSb66fG9d4Fd4usi4KnkZ8XX/YD74uvzYkw7Ay0JKwZ0i++tr/Dc9RXjAoYBv4qvWwBzgV7AT4ArY3lTYMdc/7vQV3a+tJCgFLJ33P2l+Hoe0DOuPHoEcE9YMggIH3YAhxM+kAHuAK5Putc97r41xfVV2RtY4e5zYNu+JWbWBrjVzA4AtgJ7pbjPUcCd7r4VWG5ms1OcDyEZTI7rjjlhb4Rys9z9wxjLfKAH2y+TXp3+wP7lNZv4nN6ENa4mxoX7Hkz6/UsDp2QhhWxT0uutQCtC0+o6D/0aNfFJ/F7b6yvzI8K6Sl+L9/20DvfawrZm45ZJ5dcAT7r7qRb2cngq6b2Kv5+a/P9uwKXuPuMLb4Rd9L4LTDKzv7j77TW4rxQo9VlIgxL/qn/HzM6Az/sBvhbf/jdhtVcIm089U8PrPwZ2rOSxbwKdzOzgeM2OsaN8Z0KNoww4h9BsU52ngbNif0En4Jik994Fvh5ffy+pfGe2LXV+Xor7l/ss1gyqMwMYUX6eme1lZm3MrAfwvruPA8YDB6X5TClwShbSEA0CLjCz/wKvsW370kuB883sZcKH98gaXn8X8LPYmbxH+cketkk9C/hrvGYm4a//BDAklu3DttpLVR4A3gLmA7cD/0l672rgZjObS6gllLse+IOZvUj6NYcS4OXyDu4qjI9xvBCH046N9z8a+G983lnAzWk+UwqcVp0VyVNmNgl4xN3vzXUsIqpZiIhISqpZiIhISqpZiIhISkoWIiKSkpKFiIikpGQhIiIpKVmIiEhK/w+15xEugfGWDAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#histogram and normal probability plot\n",
    "sns.distplot(data['GrLivArea'], fit=norm);\n",
    "fig = plt.figure()\n",
    "res = stats.probplot(data['GrLivArea'], plot=plt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7i0lEQVR4nO3deXyT153v8c9PsuR9t8HGxthm38FsASdAIAuBDEmapm26pk2abmk7zUx7c2fa3k5nbjqTmTa306RtliZt0iF7k9CErECgbGYHA47BGPCKbbzi3ZbO/UOCGOMNsPzI6Pd+vfyy9TxH0hch+6fnnOc5R4wxKKWUClw2qwMopZSylhYCpZQKcFoIlFIqwGkhUEqpAKeFQCmlAlyQ1QEuVUJCgklPT7c6hlJKDSt79uw5Y4xJ7GnfsCsE6enp7N692+oYSik1rIjIqd72adeQUkoFOC0ESikV4LQQKKVUgNNCoJRSAU4LgVJKBTgtBEopFeC0ECilVIDTQqCUUgFOC4FSSgW4YXdlsVJXozU5RT1u//yCtCFOogKRFgKlhlBvf/CVspJ2DSmlVIDTQqCUUgFOC4FSSgU4LQRKKRXgdLBYKYu4jSG3pJ59xbU0tHQSGRLEdeMTGTciwupoKsBoIVDKAvUtHby8u5gTZ5qICXWQFB3C6fpWntl6guyx8ayaMcrqiCqA+KwQiMgzwK1ApTFmWg/7Bfg1sBJoBu4xxuz1VR6l/EVdcztPbC6kpd3FnVmpzE6LwSZCh8vN27nlbD1eTXJ0KFljYq2OqgKEL8cI/gis6GP/LcB479f9wO98mEUpv9DU1skzW0/Q1uni/sWZzBkTi00EAIfdxuqZo8hICOfNA6WcaWyzOK0KFD4rBMaYzUBNH01uA54zHjuAGBFJ9lUepaxmjOEv+0qpbe7gy9ekMyom9KI2NhE+O3c0AB/lVw11RBWgrBwjSAGKu9wu8W4r795QRO7Hc9RAWppecq+Gp5d2FZNX3sDKaUmkJ4T32i4q1MGcMXHsOlHD7z46TnSo46I2OvWEGkzDYrDYGPMk8CTA3LlzjcVxlOqT2+3mgw8+4LXXXuPgwYN0dHQQn5BISdh4xsy8nkXjEvp9jGvHJZBTWM3242dYMU0PlJVvWVkISoHRXW6nercpNWyVlpby4IMPcvDgQVJTU7nxxhuJiIjgna37cOdvoKZwC3ud9zFn2a2Id2ygJ3HhTqaOimLXyVpunJKE3dZ7W6WulJWFYC3wgIi8CCwA6o0xF3ULKTVcHDhwgG9/+9u0t7fz8MMPs3r1aux2OwWVjTxZv5nb7gghf90zvPP8Y1SVnWLFF76N2Hofpps5OoZDZQ2cONOk1xYon/Ll6aMvAEuBBBEpAf4P4AAwxvweWIfn1NECPKePftVXWZTytWPHjvGNb3yDqKgo/vznP5ORkXF+3283FhAcZOOf7srmvcljWf/yH9j+zisYt5tbvvzdXo8Mxo+IxGEXDpfVayFQPuWzQmCMubuf/Qb4jq+eX6mh8tQHB3n6Z9/BLXZu+96/sb3Szrk6UFrXwtoDZXx5YToJEcGICMs/cy8IbF/3CokpY5h3w209Pq4zyMaEkZEcKW/g72aOOn+aqVKDTecaUuoKGGP46x9+SVNDPZ/7wb8Sm5h0wf4//O0EAPdd98kRgoiw/NNfY/ysa3hvze8pKcjr9fGnjormbGsnxTXNvvkHKIUWAqWuyJo1azh2YCc3fPbrJI8Zd8G+xrZOXtpVxOqZoy66ZkBsNm6//0dExsSz9un/oqO954vHJo6MRICjFY2++icopYVAqctVWVnJo48+Sua0Ocy7YfVF+9fuL6Op3cWXFo7p8f4hYeHc+rUfUH26hE1vPN9jm1CnnVExoZw40zSo2ZXqSguBUpfpkUceoaOjg1u+9J0eB3xf2FnEpKRIZo2O6fUxxk6bw8zrbiLnvdepPt3z2dMZCeEU1zbT4XIPVnSlLqCFQKnLsG/fPt5++23uu+8+4kamXLQ/t6Se3NJ6Pr8grc/rBQCW3flVghwO1r/8dI/7MxPCcbmNjhMon9FCoNQlMsbw6KOPkpCQwL333ttjm1f3FBMcZOO2WRcXie4iYuLIXvVZ8vduoyg/96L9Y+LDEaBQu4eUj2ghUOoSbdu2jV27dvHNb36TsLCwi/a73Ia3DpZzw+SRPc4T1JMFN3+K8KhYNr3x54v2hTrtJMeE6DiB8hktBEpdAmMMjz32GMnJydx11109timobKS6qZ3bZg18cRmHM5hFK+/iZN5+io4eumh/Rnw4xTXNuNw61ZYafFoIlLoEe/bsYf/+/dx77704nc4e2xwoqSM61MHSiSMu6bHnXL+K8KgY/vbm/1y0LzU2jE63oaKh9bJyK9UXLQRKXYKnn36a2NhYPvWpT/W4v8Pl5khZAyunJ+EMurRfL0dwCPNvuoPCw3upKC68YF9KrOc6hNLalssLrlQfhsU01Er5g4KCAjZt2sR3v/tdQkMvXlQG4FhFI+0uNyEOO2tyii75OeYsXcmWtWvIef91Vt/7D+e3x4c7CXHYKKlrZh5xl/1vUKonekSg1ACtWbMGp9PJ3Xf3Po3WkfJ6Qhw2MhMub5K40IgoZl57E4e2b6Sx7pMF/kSE1JgwPSJQPqGFQKkBaGxs5M0332TlypXExva8qLzLbcgrP8vkpKgrWj9g/k234+rsYO+mdy7YnhIbyumGVr2wTA06LQRKDcCbb75Jc3MzX/jCF3ptc+JMEy0dLqaMirqi54pPSiVj6mz2bXoHt9t1fntKTChuA+X1OmCsBpcWAqX6YYzhxRdfZPr06UybNq3XdkfK63HYhfEjIq/4ObOWrqKhporjB3ef35bqHTAuqdUrjNXg0kKgVD9yc3MpKCjg05/+dK9t3MZwpKyB8SMiL/lsoZ5MnL2Q8KhY9ny07vy26FAHoQ47p/WIQA0yLQRK9eO1114jNDSUlStX9tqmtLaFhtZOpl5ht9A59qAgZl57IwUHd9LUUAd4BoyTokP0WgI16LQQKNWH5uZm3n77bW6++WYiIno/E+hwWQM2gUlJg1MIAKZnL8e43RzO2XR+W1JUCBUNbbj1CmM1iLQQKNWHDRs20NTUxB133NFrG2MMh8vqyUyMINRpH7TnHpGSzsi0seRuX39+W1JUCO0uNyV6GqkaRFoIlOrD22+/TVJSEnPnzu21TeXZNqqb2pmSPHhHA+dMX7iMssJ8qk+XADAyOgSAvNMNg/5cKnBpIVCqF3V1dWzdupVbbrkFm633X5XDZQ0IXPFpoz2Zds31iNjI3b4BgJFRwQDknz476M+lApcWAqV68f7779PR0cGqVav6bHekvJ7RcWFEhQxsyulLERkbT8aUWRzatgFjDMFBduLCnVoI1KDSQqBUL9atW8eYMWOYMmVKr22Ka5opq2v1SbfQOdMWLaO2qpzS43kAjIwK4WPtGlKDSAuBUj2orKxk586drFq1qs+lJt8/UgEwaKeN9mRSVjZBzmAObvMMGidFBXOyupnWDlc/91RqYLQQKNWDd999F2NMv91C7+SWkxQVQnxEsM+yBIeGMTFrIUd2bsbtcpEUHYrLbSiobPTZc6rAooVAqR68/fbbTJ48mczMzF7blNe3sPtULdNTo32eZ8q8xbQ0NnAqP/f8gPHHOk6gBokWAqW6KS4u5uDBg31eSQzwTu5pAKaP8n0hGDttDkHOYD7es4X48GCcQTbydZxADRItBEp188EHHwCwYsWKPtu9nVvO5OQoEiJ91y10jiM4hHHT55K/Zxs2DONHROgRgRo0WgiU6mbDhg1MnjyZ1NTUXtuU17ew51Qtq6YnDVmuSXOu5WxdNaWF+UxKitJTSNWg0UKgVBfV1dXs3buX5cuX99lunbdbaOX05KGIBcD4mfOx2YP4eM9WJiVFUnm2jZqm9iF7fnX18mkhEJEVIpIvIgUi8lAP+9NEZKOI7BORgyLSd6esUj62ceNGjDH9FoK3D5YxJTmKzMTLW5LycoSER5AxZRYf79nChJGe59XrCdRg8FkhEBE78DhwCzAFuFtEul+Z82PgZWPMbOBzwG99lUepgVi/fj0pKSlMnDix1zZldS3sLapj1YyhOxo4Z9KcbGory3E2eo5ItHtIDQZfHhHMBwqMMYXGmHbgReC2bm0McO5KnGigzId5lOpTU1MT27ZtY/ny5X1eRLYutxwY2m6hcyZmLQIRdm/bRGyYg6MVWgjUlfNlIUgBirvcLvFu6+pnwBdFpARYB3y3pwcSkftFZLeI7K6qqvJFVqXYunUr7e3t/XYLvb6vlGkpUWQkhA9Rsk+ER8WQNmEaH3zwARNGRuoRgRoUQRY//93AH40xvxSRhcDzIjLNGOPu2sgY8yTwJMDcuXN1RQ41KNbkFF1w+40X1xIaEUVWVlav9zlUWs/hsgZ+fttUX8fr1cSsRXzwwhOsdDbz3slOjDF9HsEo1R9fHhGUAqO73E71buvqXuBlAGPMdiAESPBhJqV65Ors5NiBHCbMWkBQUO+fj17aVYwzyMZtM7sf3A6d8TMXANBZeoTGtk7KdA1jdYV8WQh2AeNFJENEnHgGg9d2a1MELAcQkcl4CoH2/aghV3T0EK3NjZ4++F60drh4Y38pt0xLIjps8KecHqj4pBTS09M5nb8XgKPaPaSukM8KgTGmE3gAeA/Iw3N20GER+bmIrPY2+wfg6yJyAHgBuMcYo10/asgdO5CDPchBxtTeu4X+sreUs62d3D0/bQiT9WzJkiXkHdgDnW3k64CxukI+HSMwxqzDMwjcddtPu/x8BMj2ZQalBqLgwE7SJ8/EGRzS43632/CHLYVMS4liQUbcEKe72NKlS/nTn/5EfNMpjlb0PjGeUgOhVxargFdTWUb16RLGzZjfa5uPjlZyvKqJ+67N9IuB2aysLMLDwwmvyddTSNUV00KgAl7BgV0AjJs5r8f9xhh+u/E4ydEhllxE1hOn08m1115L48lcjp0+i8utParq8mkhUAGv4OBO4pNSiRsxqsf9H+ZVsvtULd9dNh6H3T9+ZdbkFBGcOo2Whhraq0t4fGPBRafDKjVQ/vGuVsoiHW2tnMw7wLiZPXcLudyGR979mMyEcD4zt/fZSK0wdsY8EMFecYSKBj2FVF0+LQQqoJ3MO4Crs4NxM3ruFnp26wmOVTbyw5snEuQnRwPnRETHkpw+AXtFnhYCdUX8652t1BA7dnAnjuAQ0iZMu2jf8apG/vO9fG6YPJIV04Zu3YFLMWHWAmy1xZRVnLE6ihrGtBCogGWMoeDALjKnZhHkcF6wr6Xdxfdf3EeIw87Dd0zzizOFejJu+lzAUH70gNVR1DCmhUAFrDNlRdRXV1zULeR2Gx58eT+Hyxr45V0zGRHV87UF/iApfRxBoRE0FR2i0+Xu/w5K9UALgQpYBQe9p412KQQut+FHrx3knUOn+eeVk7lhykir4g2IzWZn5LgZ2CqPUnW2zeo4apiyevZRpXyut9MqCw/tITFlDFFxiQC0d7p5eXcxR8ob+PsbxnPvtRlDGfOyjZs+l9LcbRw7mg83976gjlK90UKgAlJHexun8nOZu/zvAKhrbuf5Hac4Xd/KrTOSGREZwgs7iy+4z+cXWD/HUE+mz5nPpjWewgar+22vVHdaCFRAKsrPxdXZwdhpczhV3cSfc4rodLn58sJ0JiZFWh3vksTGJ2KLGUVVgQ4Yq8ujYwQqIBUe2oM9yEFHbAbPbD1BSJCNby0ZO+yKwDlR6dNoPV1Ac3Oz1VHUMKSFQAWk44f2MCJjMi/sLScu3Mk3loz167OD+pMyaRa4Xfxt2w6ro6hhSAuBCjgNtWeoKj1FRVg6USEO7r02k4jg4d1LOm7yDIzdwbvrP7I6ihqGtBCogHM8dw8AbfHj+fyCtGFfBABGxUfhjh/Lrh3brY6ihiEtBCrg7Nu9AxMcyYpFs0mODrU6zqCIC3fCyIlUny6htLT70uBK9W34fxRS6hK0d3RS+vEBnKlTuWZswiXd15+nebaJEJs5nYbcN9myZQuf/exnrY6khhE9IlAB5YNte6C9iXkLFmLz0/mDLldy6hgkPJYtW7ZYHUUNM1oIVMDodLnZvzsHgGsWLrI4zeBLig6lI2EC27fvoKOjw+o4ahjRQqACxoGSOjrK8ogdlUF4VIzVcQZdUnQIrhETaWpq5ODBg1bHUcOIFgIVEIwxbD5Sgr3mJJNn9bwIzXA3KiYUd+J4xGbT7iF1SbQQqIBwsrqZ6hNHwLgZO32O1XF8IiI4iKT4WKJTxrFt2zar46hhRAuBCgh7TtXgPHMUhzOY1HFTrI7jM9NSonAlTuDQoUPU1dVZHUcNE1oI1FWvtcNFbmk9wTUFjJk086LVyK4mU0dFUx2ejtvtJicnx+o4apjQQqCuerkl9XQ2nKG9roLMaVlWx/GpaSnRuGJGExoWztatW62Oo4aJARUCEfmLiKwSES0catjZX1JHdMMJADKnXZ3jA+dMHRUFNjujJ81k69atGGOsjqSGgYH+Yf8t8HngmIj8u4joMkhqWDhd38rJM01E1B8nMjaBhOTRVkfyqeToEOLCnYSkTKGsrIyTJ09aHUkNAwMqBMaYD40xXwCygJPAhyKyTUS+KiIOXwZU6kq8nVuOMW4ai/PInDobucquJu5ORJg6KorayHQAPXtIDciAu3pEJB64B7gP2Af8Gk9h+MAnyZQaBG8dLGNEZxVtzY1kTL26xwfOmToqmhOtoaSmjtZCoAZkoGMErwN/A8KAvzPGrDbGvGSM+S4Q0cf9VohIvogUiMhDvbT5jIgcEZHDIrLmcv4RSvWkuKaZfUV1xDedBCAzQArBtJQoOlyGybPmsmOHTjeh+jfQI4KnjDFTjDG/MMaUA4hIMIAxZm5PdxARO/A4cAswBbhbRKZ0azMe+N9AtjFmKvD3l/WvUKoHbx0sB6C9LI+RaWOvymklejJtVDQAcZnTaW5u5sABXctY9W2gheDfetjW3woY84ECY0yhMaYdeBG4rVubrwOPG2NqAYwxlQPMo1S//nqgjFlJIZwu/DhgjgYA0uLCiAgOoiUmE7vdrqeRqn71WQhEJElE5gChIjJbRLK8X0vxdBP1JQUo7nK7xLutqwnABBHZKiI7RGRFLznuF5HdIrK7qqqqn6dVCgoqGzlS3sBU5xncrs6r/vqBrmw2YcqoKI7VdjJ9+nQdJ1D96u+I4Gbgv4BU4FfAL71fDwL/NAjPHwSMB5YCdwNPiUhM90bGmCeNMXONMXMTExMH4WnV1e6tg2WIgFQeJcjhJG38NKsjDalpo6I5Ut7AwkWLdLoJ1a8+C4Ex5k/GmOuBe4wx13f5Wm2M+Us/j10KdD1pO9W7rasSYK0xpsMYcwI4iqcwKHXZjDH89UAZCzLi2L87h7QJ0whyXr3TSvRkWkoUrR1uxkzJ0ukmVL/66xr6ovfHdBF5sPtXP4+9CxgvIhki4gQ+B6zt1uYNPEcDiEgCnq6iwkv8Nyh1gbzysxyvamJJmpPjx48HzGmj4FlOc01OEcU1LQBsrAghODRMxwlUn/pbszjc+73XU0R7Y4zpFJEHgPcAO/CMMeawiPwc2G2MWevdd5OIHAFcwA+NMdWX+lxKdfXXg2XYbUJEneczRSCND5wTH+Ek1GGnpL6N9Mmzzk83cbVfUKcuT5+FwBjzhPf7v1zOgxtj1gHrum37aZefDZ7xhv6OLpQakHPdQteOS+Dgno3Ex8czMjXD6lhDzibC6LhQimubyZ6WxTt7t3Hq1CnS09Otjqb80EAvKHtERKJExCEi60Wkqku3kVJ+Y29RLSW1LayansT27dtZuHAhYgvMuRJHx4VR2dBGysTZANo9pHo10N+Qm4wxDcCteOYaGgf80FehlLpcr+4pIdRhZ3zwWaqrq8nOzrY6kmXSYsMwQJMjhtGjdboJ1buBFoJzXUirgFeMMfU+yqPUZWvtcPHWgXJumZbE3l07AFi4cKHFqayTGuu51Ke4tplFixaRk5Oj002oHg20ELwlIh8Dc4D1IpIItPoullKX7v0jFZxt6+TOOals27aNsWPHMnLkSKtjWSbUaScxMpiiak8haGpq4uDBg1bHUn5ooNNQPwQsAuYaYzqAJi6eLkIpS728q5hR0SHMHhXOnj17Arpb6Jy0uDCKa5tZsGABNptNxwlUj/o7fbSrSXiuJ+h6n+cGOY9S563JKepx++cXpF207XhVI1sKzvCPN01g3769tLW1sWjRIl9H9HtpsWHsOVVLbUcQM2bMYOvWrXzve9+zOpbyMwM9a+h5PFNNXAvM8371OOuoUlb4nx1FOOzCZ+Z5BkUdDgfz5s2zOpblRsd5xgn2FdeSnZ2t002oHg10jGAunqmiv22M+a73Sz9WKL/Q3N7JK3uKWTEtmRGRIWzfvp1Zs2YRFtbfvIhXvxFRwTiDbOw9VceiRYt0ugnVo4EWgkNAki+DKHW5XthZzNnWTu5ZNIbq6mry8vJ0fMDLJkJqbCj7imuZPn06ERERehqpushAxwgSgCMishNoO7fRGLPaJ6mUGqDWDhdPbj7ONZlxzBkTx1tvvQWg4wNdpMWF8bdjZ+g0NhYsWMCWLVt0ugl1gYEWgp/5MoRSl+vVPSVUNLTxy7tmAZ7F2qOjo5kyZUrfdwwgabFhuNyGAyV1ZGdns379ep1uQl1goKePbsJzRbHD+/MuYK8PcynVrw6Xm999dJxZo2PIHhePMYZt27axcOFC7Ha71fH8xpj4cEQgp7Dm/JGSnkaquhroWUNfB14FnvBuSsEzhbRSlnlzfxmldS18d9k4RITCwkIqKioC+mrinoQ67UxKiiLnRDVpaWmkpqbqOIG6wEAHi78DZAMNAMaYY8AIX4VSqj8ut+G3GwuYnBzFskmet+K5T7k6UHyxBRlx7C2qpcNlyM7O1ukm1AUGWgjavAvQA+C9qMz4JpJS/VuXW07hmabzRwPgKQRpaWmkpHRfGltdkxlPa4ebgyV1Ot2EushAC8EmEfknPIvY3wi8AvzVd7GU6p3bbXhsQwHjRkSwYqrnrOa2tjZ27tzJ4sWLLU7nn+ZnxAGQc6KGa665RqebUBcYaCF4CKgCcoFv4Fls5se+CqVUXz7MqyC/4izfuX4sNpvnaGDXrl20trZy3XXXWZzOP8WFO5k4MpIdhdVERUUxffp0LQTqvAGdPmqMcYvIG8Abxpgq30ZSqm//+tYR4sKdNLa6zs9H9N7LbxMcHMz8+fMtTue/FmTG8eqeEjpcbrKzs/n9739PfX090dHRVkdTFutv8XoRkZ+JyBkgH8j3rk72077up5SvFNU0U1zbQva4BOy2Ty6IKsjdzfz58wkJCbEwnX9bkBFPc7uLQ6X1ZGdn63QT6rz+jgh+gOdsoXnGmBMAIpIJ/E5EfmCMedTXAZXqamvBGUIcNrLSYs5vq60sp+Z0CaHLbu11xlJ14TjBvYs8001s3bqVm266yeJkymr9jRF8Cbj7XBEAMMYUAl8EvuzLYEp1V9fczuGyeuaNiSM46JMLxgoO7gJg3AydbbQviZHBjE0MJ6ewGofDccF0Eyqw9VcIHMaYM903escJHL6JpFTP9hbV4jaeUyG7Op67i9jEZOJG6mmj/VmQGc+uk7V0utwsWrSIsrIyTp06ZXUsZbH+CkH7Ze5TalAZY9hXVEdGQjix4c7z2zvb2zmZd4CxM+bqJGoDsCAjjsa2To6UN5y/8G7Lli0Wp1JW668QzBSRhh6+zgLThyKgUuAZJK5uaicrLfbC7Udz6WhvY9x07RYaiIVjPUdTWwrOMGbMGMaMGcPmzZstTqWs1udgsTFGZ+5SfmFfUR0OuzBtVNQF2wtyd2MPcpA+eaZFyYaHroPoSVEhvLq7hJhQJ0uWLOHFF1+kpaWF0NBQCxMqKw30gjKlLONyGw6V1TM5OYpgx4WfTQoO7mLMpBk4gvW00YEaPyKCUzXNtHe6Wbx4Me3t7ezYscPqWMpCWgiU3yuqaaa53cXUURde+FRXdZrq8mLGTtflsy/FuJERuNyGE2eamDdvHmFhYdo9FOC0ECi/l1fegF2E8SMiLthekOs9bVQLwSVJjw8nyCYUVJ7F6XSycOFCNm3apKeRBjAtBMqvGWPIK28gMzGckIu6hXYTkzCS+OTRFqUbnhx2GxkJ4eRXNAKwePFiysvLKSgosDiZsooWAuXXKs+2Ud3UzuTkCweJOzvaOZm3n7Ez5ulpo5dhYlIkZxrbOHmm6fyMrdo9FLh8WghEZIWI5ItIgYg81Ee7O0XEiIge46sLHK04C8CkpMgLthfl59LR1qqnjV6mSUmewrr+40qSkpKYOHEimzZtsjiVsorPCoGI2IHHgVuAKcDdInLRiuIiEgl8H9DZr9RFjlc1khDhJCbMecH2o/tzCHI4yZgyy5pgw1xcuJMRkcGsz6sAYMmSJezdu5ezZ89anExZwZdHBPOBAmNMoXd1sxeB23po96/AfwCtPsyihqFOt5uTZ5oZm3jhILExhqP7dpAxNUtPG70Ck5Ki2HmihobWDhYvXozL5dI1CgKULwtBClDc5XaJd9t5IpIFjDbGvN3XA4nI/SKyW0R2V1XpcgiBoqSmhXaX+6JCUFlygvrqCibOvsaiZFeHycmRdLoNGz+uZObMmcTExLBhwwarYykLWDZYLCI24FfAP/TX1hjzpDFmrjFmbmJiou/DKb9wvKoRATITwy/YfnSf5+Kn8TMXWJDq6jE6LowRkcG8k3uaoKAglixZwubNm3VR+wDky0JQCnQ9ry/Vu+2cSGAa8JGInASuAdbqgLE653hVI6NiQglzXjgTytH9OxiVOZGImDiLkl0dbCKsnJ7MxvxKmto6WbZsGfX19ezdu9fqaGqI+bIQ7ALGi0iGiDiBzwFrz+00xtQbYxKMMenGmHRgB7DaGLPbh5nUMNHa4aK4toWMhAuPBs7WVVNWmM+EWdotNBhWTk+mrdPN+o8ryc7Oxul0sn79eqtjqSHms0JgjOkEHgDeA/KAl40xh0Xk5yKy2lfPq64Oh0rrcbkN6fFhF2wvOLATgAmzF1oR66ozd0wsIyKDeetAGeHh4SxcuJANGzboVcYBxqdjBMaYdcaYCcaYscaY/+vd9lNjzNoe2i7VowF1zu5TtQCkxV94RJC/bwcxCSMZkZpuQaqrj80m3DpjFB/lV1Hb1M6yZcsoLS3l6NGjVkdTQ0ivLFZ+affJWuLDnUQEfzI+0NHWyonDe5kwe6FeTTyIPj0nlXaXmzf3l3L99dcjIto9FGC0ECi/Y4xhz6ka0rsdDRQe3ktnRzvjZ+nZQoNpyqgopqdE8/LuEhITE5kxYwYbN260OpYaQloIlN85XtVEbXMHY7qND+Tt3kJIeARjJs6wKNnV6zNzUzlS3sCh0nqWLVvGoUOHqKiosDqWGiJaCJTf2Xt+fOCTQuDq7ODovh1MnL0Qe1CfC+upy7B6ZgohDhvPbT/JsmXLALR7KIBoIVB+Z19xLVEhQSREBJ/fduLIftpampg091oLk129osMc3JmVyhv7y4gakUpGRgbvv/++1bHUENFCoPzO/uJ6Zo6OwdZlQDhv9xacIWFkTs2yMNnV7WvXZtDe6eZ/coq4+eab2bVrF9XV1VbHUkNAC4HyK83tneSfbmDW6Jjz29wuF/l7tzFh1gKCHM7e76yuyNjECJZNGsHzO06xZNkNuN1uPvzwQ6tjqSGghUD5lUOlDbgNFxSCU/m5tDQ2aLfQEPjO9eOoaWpn6xkn6enpvPvuu1ZHUkNAC4HyK/uLPQPFM7sUgrzdf8PhDNa1iYfAnDGxXD8xkSc3n2Dp8hvYuXMnNTU1VsdSPqanXyi/cqC4ntTY0PMDxcbtJn/PNsbOmKdrD/jAmpyii7ZNGRXNxvwqalOnnO8e+sxnPmNBOjVU9IhA+ZX9xXUXHA0UFxyhsb6GydotNGRSYkK5fdYo/lJoGJU6mvfee8/qSMrHtBAov1F5tpXSuhZmpcac33Z4x0cEOZy69sAQe+iWyQTZbdhGzyInJ4fa2lqrIykf0kKg/MaB4noAZqXFAODq7OTIrs1MmH0NwaFhfdxTDbYNH1eyeHwiBc5xuFwuHn7q5R67kdTVQQuB8hsHiuuw24Rpo6IBz9xCzWfrmbZwmcXJAlP2uARGpY+FiET2b9MlLK9mWgiU39hfXMfEkZGEOu0AHNqxkZDwCD1byCJ2m/DpOaNxjc6iJD+XujOVVkdSPqKFQPkFt9twoOSTgeLm5mby925jyrzF2IMc1oYLYCOjQli49CbA8OH771gdR/mIFgLlFwrPNHG2tZPZ3kKwceNGOtpamXbN9dYGU9w4fypBiRnk5WzkdH2r1XGUD2ghUH7hQHEd8MmFZG+99RZRcQmkTZhmXSgFeBa5X3j9zVBfznd++1fcbl3G8mqjhUD5hX3FtUQEBzFuRAS1tbVs2bKFqQuuR2z6FvUH8xcvR2x2Dmxbzx+3nbQ6jhpk+lum/MK+ojpmjo7GbhPeffddOjs7tVvIj4RFRDF+xjzCTx/k3985wtGKs1ZHUoNIC4GyXHN7Jx+fPktWWiwAr732GpMmTWJkWqbFyVRX0xctp72xlrC6E/z9i/tp73RbHUkNEi0EynIHS+pxuQ2z02LIy8vj8OHD3HnnnbpAvZ8ZP2sBkZGRTO88ypHyBn71wVGrI6lBopPOKcs9u+UEAIWVTfzx5T9iD3LQljyLUItzqQs5nMFMnLeUfZvfZdbXbuWJTcdxuQ0ZCeF8fkGa1fHUFdAjAmW5oppm4sOdOMRF7vYNTJqbTWhElNWxVA9mL70FV2cHqQ1HiA138pe9JXS4tItouNNCoCxljKGotoW0uDA+3rOV1uZGZi9eYXUs1YuktLGMypjAwb+9y+qZyVQ3tbPpaJXVsdQV0kKgLFVc00JTWyej48LYv/ldYhKTSJ800+pYqg9ZS1dSVXqK0IZiZqRGs+loFcerGq2Opa6AFgJlqX3eFcli3PWczDvArOtu1msH/NzUBUtxhoSyb9M7rJqejMMu/OSNQxijF5oNV/obpyy191QtTruNkj0bEbEx89obrY6k+uEMCWXaNddzeOdmHK42bp6axLbj1by+r9TqaOoyaSFQltpXXMeoSDv7Nq9jYtZCouISrY6kBiBr6Uo629vI3b6eeelxzBwdw3+8+zEt7S6ro6nLoIVAWaa1w8WRsgZCy/fR2tTI/JvusDqSGqDk9PEkZ0xg1/q/IsbwzysnU9HQxrPbTlgdTV0GnxYCEVkhIvkiUiAiD/Ww/0EROSIiB0VkvYiM8WUe5V9yS+vpdLmpObCekWljdYK5YWbBjXdQXV5MQe5uCiobmTgykv9ef4yn/1aoq5kNMz4rBCJiBx4HbgGmAHeLyJRuzfYBc40xM4BXgUd8lUf5n72narGdKaChopgFN96uVxIPM1PmX0dkbAI5770GwM1Tk2jrcOvppMOQL48I5gMFxphCY0w78CJwW9cGxpiNxphm780dQKoP8yg/k3OihqiSbYRFRjN1wVKr46hLZA9yMO+G1Zw4sp+K4kKSokOYNTqG7cerqWtutzqeugS+LAQpQHGX2yXebb25F9AlkAJEp8tNTu5R2ooPkbV0FUFOp9WR1GXIWroShzOYnPf+AsANU0ZigA0f67KWw4lfDBaLyBeBucB/9rL/fhHZLSK7q6r0sPNqcLisgfa8jdhsNuYuu9XqOOoyhYZHMvO6mzi04yMa62qIDXMyPz2OvUW1lNa1WB1PDZAvC0EpMLrL7VTvtguIyA3APwOrjTFtPT2QMeZJY8xcY8zcxEQ9vfBq8OG+Y9hP5XDLqr8jMjbe6jjqCsy/8Q5crk52rf8rANeNTwDgqc2FVsZSl8CXhWAXMF5EMkTECXwOWNu1gYjMBp7AUwT0WDKAvPXKGsS4+N53vmV1FHWF4pNSmJi1iF0fvklrUyMxYU5mjY7lxV1FVDf2+NlO+RmfFQJjTCfwAPAekAe8bIw5LCI/F5HV3mb/CUQAr4jIfhFZ28vDqatIRdUZTu/bQNrMa0lL0+mLrwaLV3+BtpYmct5/3XN7QgJtnW6e3XrS2mBqQHy6HoExZh2wrtu2n3b5+QZfPr/yT//52JMYVwefv+deq6OoQZI0ZiwT52ST8/5fmH/T7YyIjOTmKUn8aftJvrEkk8gQh9URVR/8YrBYBY66ujref/NVTMoMPrUky+o4ahAtuf2LtLU0nz+DKDMxnLOtnfzwlYOsySk6/6X8jxYCNaSee+45OtpamLzsTv2UeJUZOTqTSXOvZecHb9DS2EBqbBjjEiPYWnCGTl28xq9pIVBDprKykmeffZbOUTNZea0eDVyNFt/mOSrY4T0quG58AmfbOjlQUm9xMtUXLQRqyPz617+mo9NF55SVXD9xhNVxlA+MHJ3BlPlLyHnvLzTUVDFuRARJUSFsLTij6xX4MS0Eakjk5eXx+uuvkzh7OSmpqYwbEWF1JOUjy+/6Gm63m/WvPIOIkD0ugdMNrRRU6ipm/koLgfI5YwyPPPIIwWERnEpcRGZCOC/sLNbBw6tUTGISC2/5NIe2b6Ck4AgzU6OJDA5iS8EZq6OpXmghUD63efNmduzYQfp1n8LtCGN6aozVkZSPZa/6LBExcbz3P7/DLrBwbDzHKhs5Xd9qdTTVAy0EyqdaWlp4+OGHSU9Ppz55LvHhTkZFh1gdS/mYMySU5XfdS9mJoxzcvp756XE47KJHBX5KC4Hyqccee4yioiL+/kf/zImaNqanRuu6AwFi+sJlpGROYv1LT0NHM1lpsRworqOyQY8K/I0WAuUzhw8f5o9//CN33XUXR92e6Ymz0mKtjqWGiNhsrPrq92lpOsu7f36ca8cl4DaGP20/aXU01Y0WAuUTHR0d/PjHPyY+Pp4fPPgPvLCziHGJESREBFsdTQ2hkaMzuW715zm84yOq8nczOTmKP+8oorm90+poqgstBMonnnnmGT7++GN++tOfsruslbL6VuZnxFkdS1kge9XnGJk2lnV/+m/mJjupb+ng5V3F/d9RDRktBGrQ7du3j9/85jesWLGCG264gaf/VsjIqGAmJ0dZHU1ZwB4UxOr7/oGWprMcfusZ5qTF8OTmQto7ddoJf6GFQA2q2tpaHnzwQVJSUvj5z3/OtuNnyDlRwzcWj8Vu00HiQJWUNpYlt3+JI7s2M6PtEGX1rbyx76J1qpRFtBCoQeN2u3nooYeorq7mV7/6FREREfy/D44xIjKYzy/QdQcCXfaqzzJuxnxefeYxxtnP8PhHBXToZHR+QQuBGjRPP/00mzdv5qGHHmLq1Km8f6SCnSdr+M714whx2K2OpywmNhu3f+NHJCcn0/TR05wqq+CV3SVWx1JoIVCDZN26dTz66KOsXLmSu+++m7OtHfyfNw8zKSlSjwbUeaHhkfz617+mpeks8bkv8v/eP0xrh8vqWAFPC4G6Yjt37uShhx5izpw5PPzww4gID6/Lo+JsK/9+5wwcdn2bqU9MnjyZf/u3f6O5NJ+6jc/y1OYCqyMFPP0NVVfk6NGjPPDAA6SlpfH4448THBzMy7uKeWFnMfcvzmTW6BirIyo/dOutt/LQQw9hL8/l8f/6BcU1TVZHCmhaCNRlO3r0KPfeey+hoaE89dRTREdHs+XYGX78xiGuG5/AD2+aaHVE5ce+8pWv8MWvfh05mcNXH/wpbrcOHFtFC4G6LPv37+fLX/4ydrudZ599luTkZDbmV3Lvn3aRmRjOb+6eTZB2Cal+/NMPf8DMpbdSmrOOe777I1wuHS+wQpDVAdTws337dh544AESEhJ45plnSE4exeMbC/jl+/lMTo7ijlkprMs9bXVMNQyICH9+7D9Ydo9h14a3uf9bTTz+348SEqIz1A4l/cimBswYw3PPPcf9999PSkoKzz//PE1BUXzmie3853v5rJyezMvfWEhYsH6+UAMXZLfxwm/+FdvsT7Htb5v4yle/RmVlpdWxAooWAjUgTU1N/OM//iO/+MUvWLJkCY889hS/3Hyamx/dzOGyBu7MSmFhZjxv7i+zOqoahlJiQnnqX3+Aa/6XyD10mNtvv4OtW7daHStgyHBbUHru3Llm9+7dVscIKHv37uUnP/kJJ0+e5Av3fovK5EX8Nfc0dhHmpceydOIIwvUoQF2hzy9I4839pTz41LtE7F9DR2059913Hw888ABOp9PqeMOeiOwxxsztcZ8WAtWbs2fP8stf/pKXXnqJiNhEIq79Eidso3DabczPiCN7XALRoQ6rY6qrxLkLD989VM73nt9JWN5a2o5tY8yYMfzkJz8hOzvb4oTDW1+FQD/GqYu0t7fz8ssv88QTT1BdU8OoebdwfMS1nA0O5frMBLLHxus4gPKZFdOSefmBJXz/xShKEqZSlf9X7rvvPm6++Wa+//3vk5GRYXXEq44eEajzWlpaWLt2LU888QTl5eVEjZ5IZcYKIpMzmJcex6KxCYQ6dc4g5RvdpyJpbOvkN+uP8ezmY5ijG3Ee24BxdXDTzSv49re+yYQJEyxKOjxp15Dq04kTJ3jppZd4/fXXaWhoIDx5LDXpy4lIm8J912VyT3Y6bx0otzqmClBLJibyx60neGlLHs2H1uM4uQ0628iYPJPP3HUXn7tjlZ5uOgBaCNQFjDEUFhbywQcf8P7775OXl4fdHkTE2CwqE+cQkzaJe6/L5MsLxxAZ4hkDWJNTZHFqFeg63W4Kq5rYX1DK0e3v4Srcga25BhyhRGXOImPmAmbPW8gPV07HpmtfXEQLQYDr6OigsLCQQ4cOsXPnTnJycqioqAAgdvQE6uMm05Q0k1EjR5A1Jpa5Y+JwBumZxcp/GWOoqG9hz+5dHNu5gbMnD0J7M8YWhC0hneSxU8nKmsMNi+aQPTWNCB3Tsq4QiMgK4NeAHXjaGPPv3fYHA88Bc4Bq4LPGmJN9PWagFwJjDM3tLhpaOzjb2klDi+d7fUs7FWdqqCwvp6bqNDVV5dSUF1NdcoL6imLcLs9i4fbQSEgcR0tMBq6kqYRFx7FiahJ3ZKWwaGwCL+lasmoY6uzo4Ejufg7u3s7pgkO0VBWB92+bCYnCEZdK5rhxTJ04gcnjxpA1MZPxGakBdVqqJYVAROzAUeBGoATYBdxtjDnSpc23gRnGmG+KyOeAO4wxn+3rcQerEBhjcBtwuQ1uY3C5DS5jcLs9P3e6DR0uNx0uQ6fLTbvLTUenmw6Xm/ZONx2dLjqNd5IsAxiDwfNaiucmwrnX1niezw0tHS5a2ztpbuuksbWNltY2GlvbaW5to6W1nebWdlrb2mht66ClrY3W1lZaW5ppa22ho62F9tYWTGcb0t6MtDUi7U3Q1oS0nUVc7Rf+G4MjICYFiUnBHptCUPxoYkamkhARzLLJI5g1OoastNgLFo3RLiB1NWhraaIw/wjHjuZTeqqQ+tNFdNSWgfvCuYxsoVE4wqIIDo/EERqBIzQcZ1gEztAIGjrt2IOc2BzB2IMc2B3B2BxOxiXHEBwcgtMRRFBQEMGOIBxBdpyOIJxBdoIdDhyOIJwOO8GOIJxBQTgdQTiCbATZbDjsQpDdRpBNCLLLBdscNu8+u+Cw2bz7BZEr7+qyqhAsBH5mjLnZe/t/AxhjftGlzXveNttFJAg4DSSaPkJdbiH49s8eZcOrf/TeMuc/LXxy2/v9/KZPfhb8q/vMFuTEERpBSEQUIRFRhEfGEB4VTWxiEnEjkohLTCIucSSh4ZFWR1XKb3R0dHCquITSsjIqTp+mrrqSlrozSHsTbc1ncbU24Wptwt3WdFHBGAyGbn/ML/jjLhd8o2vb8+2E1V/+No/86BuX9fxWFYJPAyuMMfd5b38JWGCMeaBLm0PeNiXe28e9bc50e6z7gfu9NycC+T08ZQJwpoft/sKf82m2y+PP2cC/82m2y3Ml2cYYYxJ72jEsRlCMMU8CT/bVRkR291bt/IE/59Nsl8efs4F/59Nsl8dX2Xx5akgpMLrL7VTvth7beLuGovEMGiullBoiviwEu4DxIpIhIk7gc8Dabm3WAl/x/vxpYENf4wNKKaUGn8+6howxnSLyAPAentNHnzHGHBaRnwO7jTFrgT8Az4tIAVCDp1hcrj67jvyAP+fTbJfHn7OBf+fTbJfHJ9mG3QVlSimlBpdePqqUUgFOC4FSSgW4YVcIRCRGRF4VkY9FJM974VrX/SIi/y0iBSJyUESy/CjbUhGpF5H93q+fDmG2iV2ed7+INIjI33drY8lrN8BsVr52PxCRwyJySEReEJGQbvuDReQl7+uWIyLpfpTtHhGp6vK63TdU2bzP/31vtsPd/0+9+638fe0v25C+50TkGRGp9F5fdW5bnIh8ICLHvN9je7nvV7xtjonIV3pq0ydjzLD6Av4E3Of92QnEdNu/EngHz6V51wA5fpRtKfCWH7yGdjxXcY/xl9duANksee2AFOAEEOq9/TJwT7c23wZ+7/35c8BLfpTtHuAxi95n04BDQBieE1M+BMb5w3tugNmG9D0HLAaygENdtj0CPOT9+SHgP3q4XxxQ6P0e6/059lKee1gdEYhINJ4X6w8Axph2Y0xdt2a3Ac8Zjx1AjIgk+0k2f7EcOG6MOdVtuyWv3QCzWSkICPVe6xIGlHXbfxueDwEArwLLRQZhcpjByWalyXj+sDcbYzqBTcCnurWx6j03kGxDyhizGc/Zk111fW/9Cbi9h7veDHxgjKkxxtQCHwArLuW5h1UhADKAKuBZEdknIk+LSHi3NilA1yk0S7zb/CEbwEIROSAi74jI1CHI1ZPPAS/0sN2q166r3rKBBa+dMaYU+C+gCCgH6o0x73drdv518/5RqQfi/SQbwJ3ebpdXRWR0D/t95RBwnYjEi0gYnk//3Z/fqvfcQLKB9b+vI40x51aFOg2M7KHNFb+Gw60QBOE5dPqdMWY20ITncMkfDCTbXjxdHjOB3wBvDGlCQDwX960GXhnq5+5PP9ksee28fbK34Sn0o4BwEfniUDx3fwaY7a9AujFmBp5Pin9iiBhj8oD/AN4H3gX2A4M/m9tlGGA2y39fuzKefiCfnO8/3ApBCVBijMnx3n4Vzx/frgYytYUl2YwxDcaYRu/P6wCHiCQMQbaubgH2GmMqethn1Wt3Tq/ZLHztbgBOGGOqjDEdwF+ARd3aWDVVSr/ZjDHVxpg2782n8az9MWSMMX8wxswxxiwGavFMTd+VZe+5/rL5ye9rxbmuMu/3yh7aXPFrOKwKgTHmNFAsIhO9m5YDR7o1Wwt82Xs2wjV4Dpd9vuDuQLKJSNK5vmMRmY/n9R/quZXupveuF0teuy56zWbha1cEXCMiYd7nXw7kdWtj1VQp/Wbr1t++uvt+XxOREd7vaXj64Nd0a2LZe66/bH7y+9r1vfUV4M0e2rwH3CQisd6jxJu82wZuqEbEB+sLmAXsBg7iOVSLBb4JfNO7X4DHgeNALjDXj7I9ABwGDgA7gEVD/NqF43kjR3fZ5i+vXX/ZLHvtgH8BPsbTr/w8EAz8HFjt3R+CpzurANgJZPpRtl90ed02ApOG+D33NzwfiA4Ay/3sPddftiF9z+H5EFQOdODpYbgXz1jTeuAYnjOb4rxt5+JZ9fHcfb/mff8VAF+91OfWKSaUUirADauuIaWUUoNPC4FSSgU4LQRKKRXgtBAopVSA00KglFIBTguBCigiMlJE1ohIoYjsEZHtInJHD+3Su84C2WX7z0XkhgE8zywRMSJySXO+KGUFLQQqYHgvDnoD2GyMyTTGzMEzt1Fqt3a9LuFqjPmpMebDATzd3cAW7/ces4iI/v4pv6BvRBVIlgHtxpjfn9tgjDlljPmNeObtXysiG/BcwNMjEfmjiHxaRFaIyCtdti8Vkbe8PwtwF54poG8U7xoB3qOMfBF5Ds8FYKNF5Icisss7Kdy/dHm8N7xHLIdF5P7BfRmUupAWAhVIpuKZSKw3WcCnjTFLBvBYHwILusww+1ngRe/Pi/DMAXQc+AhY1eV+44HfGmOmAhO9t+fjuSp9jogs9rb7mveIZS7wPRHx+WymKnBpIVABS0Qe904xvMu76QNjTPf54HtkPNNNvwv8nbcraRWfzANzN58UhRe5sHvolPHMuw+eOWFuAvbhKVCT8BQG8PzxPze1wegu25UadL32hSp1FToM3HnuhjHmO97ZJHd7NzVd4uO9iGc+mhpgtzHmrIjYvc9xm4j8M565dOJFJLKH5xDgF8aYJ7o+qIgsxTOz6EJjTLOIfIRnPiOlfEKPCFQg2QCEiMi3umwLu4LH24SnO+nrfHIEsBw4aIwZbYxJN8aMAV4DLjozCc8MkV8TkQgAEUnxzogZDdR6i8AkPEs4KuUzWghUwDCeGRZvB5aIyAkR2YlnoZb/1ctdJopISZevu7o9ngt4C886Cm95N98NvN7tcV6jh7OHjGc1sTXAdhHJxbOGRSSeLqcgEckD/h1P95BSPqOzjyqlVIDTIwKllApwWgiUUirAaSFQSqkAp4VAKaUCnBYCpZQKcFoIlFIqwGkhUEqpAPf/AaTgbXZgqLdAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAx3UlEQVR4nO3dedyU8/7H8denlDZEZSkqbkoLJSF+OCjHcmw59qhsUQ5ZQ2mlyHqsJXQq3Ry7g1M4snNEpVJREqFwkvZFy/35/XFdU3PfzXbnnntm7nk/H4953DPXXNd3Pt3V9Znvbu6OiIjkr0qZDkBERDJLiUBEJM8pEYiI5DklAhGRPKdEICKS55QIRETynBKB5A0zczPbeyuv/c7MOsR57wgzmx3rXDPrbWaPb13EpYrvKDP7Md2fIxWTEoFktfCmusbMVprZL2Y2ysxqZTquaO7+gbs3jfPeEHe/BMDMGofJaJut+Rwz62pmG8PfxXIzm2pmJ21FOaPM7LatiUEqJiUCyQUnu3stoA3QFril5Albe3PNQf8Nfxe1gSeAZ81sx8yGJLlOiUByhrsvAMYDLWFTU88VZvY18HV47FIzm2tmv5nZK2ZWv0QxJ5rZPDP71czuMrNK4XUFZva2mS0O3ys0s9olrj3IzGaZ2RIz+4eZVQuvjdssY2YDzGxs+PL98OfS8Fv9n8I494s6f2czW21m9ZL8LoqAkUB1oCDG5zYzs3fNbKmZzTSzU8Lj3YBOQK8whlcTfY7kByUCyRlmtgdwIvB51OHTgEOA5mZ2DHA7cBawGzAf+GeJYjoS1CraAKcCF0WKD6+tDzQD9gAGlLi2E3AcwY23CTFqJkkcGf6s7e613P29ML7zo845F5jg7osSFRTWgC4BVhImwaj3qgCvAm8COwNXAoVm1tTdRwCFwJ1hDCeX8s8gFZASgeSCl81sKfAh8B4wJOq92939N3dfQ3CjHunuU9z9d+Bm4FAzaxx1/tDw/O+BvxPceHH3ue7+H3f/PbwJ3wv8qUQcD7n7D+7+GzA4cu0fNBo418wsfH0B8GSC89uFv4ufw8/v6O7LSp4D1ALucPd17v428FoZxSsVUL60q0puO83d34rz3g9Rz+sDUyIv3H2lmS0GGgDfxTh/fngNZrYLcD9wBLAdwZekJQk+a9O1f4S7TzSz1cBRZvYTsDfwSoJLPnH3w5MUWx/4IWw+iphP8HsQ2YJqBJLropfPXQg0irwws5pAHWBB1Dl7RD1vGF4DQS3Dgf3cfXuC5hqjuHjXbk2s0UaHn3cB8Ly7ry1luSUtBPaI9H+EGrL596Alh6UYJQKpSJ4GLjSz1ma2LcHNfaK7fxd1zg1mtmPY39ATeCY8vh1Be/syM2sA3BCj/CvMbHcz2wnoE3VtqhYBRcBeJY6PJei7OB8YU8oyY5kIrCboEK5iZkcBJ7O5v+SXGDFIHlMikAojbD7qC7wA/ETQqXtOidP+BUwGpgL/JhiCCTCQoAN5WXj8xRgf8RRBB+w84BugVGPx3X01Qd/CR+Fonnbh8R8ImrQc+KA0Zcb5nHUEN/4TgF+BR4DO7v5VeMoTBJ3rS83s5T/6eZL7TBvTiGSemY0EFrp7aUciifxh6iwWybBwVNPpwAEZDkXylJqGRDLIzG4FZgB3ufu3mY5H8pOahkRE8pxqBCIieS7n+gjq1q3rjRs3znQYIiI5ZfLkyb+6e8w1rHIuETRu3JhJkyZlOgwRkZxiZvPjvaemIRGRPKdEICKS55QIRETynBKBiEieUyIQEclzSgQiIlmusBAaN4ZKlYKfhYVlW37ODR8VEcknhYXQrRusXh28nj8/eA3QqVPZfIZqBCIiWaxPn81JIGL16uB4WVEiEBHJYt9/X7rjW0OJQEQkizVsWLrjW0OJQEQkiw0eDDVqFD9Wo0ZwvKwoEYiIZLFOnWDECGjUCMyCnyNGlF1HMWjUkIhI1uvUqWxv/CWpRiAikueUCERE8pwSgYhIBqR7tnBpqI9ARKSclcds4dJQjUBEpJyVx2zh0lAiEBFJo1hNQOUxW7g00poIzKynmc0ws5lmdnWM948ys2VmNjV89EtnPCIi5SnSBDR/PrhvbgLaaafY5yecLfzLL7BkSVriTFsiMLOWwKXAwUAr4CQz2zvGqR+4e+vwMShd8YiIlLd4TUBQitnCK1ZA//5QUFC204mjpLNG0AyY6O6r3X0D8B5weho/T0Qkq8Rr6vnttxRmC69bBw89FCSAQYPgL3+Byy5LS5zpTAQzgCPMrI6Z1QBOBPaIcd6hZjbNzMabWYs0xiMiUq4SLRjXqRN89x0UFQU/NyWBoiL45z+hWTO48kpo2RI+/RSeeQb22SctcaYtEbj7l8BQ4E3gdWAqsLHEaVOARu7eCngQeDlWWWbWzcwmmdmkRYsWpStkEZEyVeoF4956Cw46CM49F2rVgvHjYcKE4FgapbWz2N2fcPcD3f1IYAkwp8T7y919Zfh8HFDFzOrGKGeEu7d197b16tVLZ8giImUm5QXjPv8cjjsOjj0Wfv0VxowJjh1/fHBhmqV1QpmZ7ezu/zOzhgT9A+1KvL8r8Iu7u5kdTJCYFqczJhGR8pRwwbh586BvX3jqqWAo0b33QvfuUK1aucaY7pnFL5hZHWA9cIW7LzWzywHcfThwBtDdzDYAa4Bz3N3THJOISGYtWgS33QbDhsE220Dv3tCrF+ywQ0bCSXfT0BHu3tzdW7n7hPDY8DAJ4O4PuXuL8P127v5xOuMREUm3hGsIrVoVJICCgmBEUNeuMHdu0GmQoSQAWmtIRKTMxFtDyDas57w1T8CAAcHEsI4dYcgQ2HffjMYboUQgIlJGtpxA5pyw+gXaXdIbNnwNhx8OL70Ehx6aqRBj0lpDIiJ/UKQ5aP78zcf+xLt8Qjue50xWb6gKr74K77+fdUkAlAhEREolctM3C/p5zeCCCzYngf2Yzr85kXc5mvos5EJGckrDaXDSSeUyFHRrqGlIRCRFJfsANoZTZN2hIfMZRD8u4EmWsQM3cCcP8Tcq1ajOiCGZizkVSgQiIikoLIQuXTbf/CN2YjG9GcLfeAjHuIsbuIObWMqONGoUDAjKxGYzpaFEICKSQGEh9OwJi0tMda3OanpyPzdxB7VYySi6MoAB/BguqdaoUbCGUC5QIhARiaNHDxg+PGj6iajMBroyioH0pwELeYWT6c0QZtJy0zkJ1xPKQuosFhGJUlgIdesG/brDhkUnAedUXuYL9uNxLmU+jTiC9zmVV5hJy039wHHXE8piSgQiIqHCQrjwwi2bgf6PD/mQw3mZjgCcxkv8Hx/xIUcAwc3/ySeDpFFsSekcoUQgIhLq2RPWr9/8uhmzeJlT+ZAj2JNvuZQR7McX/IvTqFHDGDs2d2/+0ZQIREQI+gMiNYEG/MjjXMwX7MdRvMvNDGFv5vI4l7KRbahTJ/eafxJRZ7GI5K2SI4Jqs4SbuIOreIBKFPF3rmYIvfmNOkDQb3D55fDIIxkMOg2UCEQkL0X6A9avh21Zy994iN4MoTZLGcv59OVWvqfRpvPr1IH77684tYBoSgQikncKC6FzZ6BoI114kkH0oyE/MI4TuJnbmU6rTedWqhRsGFYRE0CE+ghEJC8UFgbbAJvB+ec7JxS9xjRaMYoL+ZldOZq3+QvjiiUBs4qfBECJQETyQKQGsGoVHMInvMtRvMbJVGUdZ/AchzCRdzl6i+suv7ziJwFQIhCRPNCzJ+xdNJvn+SufcChNmU13HqEFM3mBM4AtVwXt3r3idQrHo0QgIhVWjx5Q3xYyePFlzKQFf+ZN+jKIvZnLcLqzgSpbXFOnDowdmz9JANRZLCIV1LUXL6P+yLuYy71swwYe5goG04dF7Bzz/KpVYeTI/GgKKkmJQEQqlt9/57ljhtH749uoy2Ke4lz6civzKIh7Sa1aweJy+ZgEIM1NQ2bW08xmmNlMM7s6xvtmZg+Y2Vwzm25mbdIZj4hUYEVFjOowlu+qNeXMj69hCm1ow2Q68VTcJFCrVtAMtGJF/iYBSGMiMLOWwKXAwUAr4CQz27vEaScA+4SPbsCwdMUjIhVHYSFsu20wvNPMOc7eYGrlNnSdcAG/sRPH8ibH8SafE/+7ZZ06SgAR6awRNAMmuvtqd98AvAecXuKcU4ExHvgEqG1mu6UxJhHJcR06wPnnw7p10JbPeIsOvMHxbM9yzuUp2jKJtzg2YRmVKwezhCWQzkQwAzjCzOqYWQ3gRAi37tmsAfBD1Osfw2PFmFk3M5tkZpMWLVqUtoBFJDtF1wAmTIAC5vJPzuYzDmZ/pnMlD7AvX/FPzsWT3Nbq1IHRo1UTiJa2zmJ3/9LMhgJvAquAqcDGhBfFL2sEMAKgbdu2nuR0EakgOnQIbvwRO/ML/RhEN0bwO9sykH7cw3WsYPuUysunuQGlkdbOYnd/wt0PdPcjgSXAnBKnLKB4LWH38JiI5KEePSLt/pu//QPUYgX9GcA3FHAZj/IYl7I3cxnAQCWBMpDuUUM7hz8bEvQPPFXilFeAzuHooXbAMnf/KZ0xiUj2iSSAYSWGi1RhHVfwEN9QwAAGMo4Tac4sruARfmHXlMru3j3YPEZJIL50zyN4wczqAOuBK9x9qZldDuDuw4FxBH0Hc4HVwIVpjkdEskyNGrBmTfFjRhFn8SyD6UMB83iHoziJoXzGwSmVWVH3DUiXtCYCdz8ixrHhUc8duCKdMYhIdmrRAmbN2vL4MUxgKDfSlslMZz9OYByvczyx1gOKaN8e3norfbFWdFprSETKTXQfQMkk0JrPeZ3jmEAH6rGICxjDAXzO65xAvCRQrVowIUxJ4I/REhMiknY77ghLl8Z+rzHfchu30ImnWMxOXMO9DKM7v1Mtbnnq+C1bSgQikjaJEkBdFtGHwfTgETawDUO4maHcyHJ2iHn+NtvAqFEa/58OSgQikhYWp0m/Bqu4hvvoxZ3UZBVPcDED6c/CLeeS6pt/OVEiEJEyFa8WsA3ruZgn6M9AduNnXqQjvRnCbPbd4lx1/pYvJQIRKRMNGsDChbHecf7KCwymD02Zwwcczum8yCccusWZ1avD6tVpD1VK0KghEflDWrQImoFiJYEjeY9PaMfznMkGtuFkXuFI3o+ZBLp3VxLIFNUIRGSrxesH2I/p3M7N/IVx/MDuXMhIxtCZIioXO69+fVigRWUyTjUCESm1HXeMnQQaMp9RdGEqrTmMj7mBO2nCHEZxYbEkMHZssOyDkkB2UCIQkZRFJoSV7AzeicXczXXMoQln8wx3cQN7MY+7uYG1VN90XmTdHw0BzS5qGhKRlFSuDEVFxY9VZzVX8QA3cQfbsYLRdKE/A/lxi61HglqAEkB2Uo1ARJIyK54EKrOBi3mcr9mHO7iZ9zmSVkzjYkZukQRUC8h+pUoEZlbJzFJb/FtEcl5kRNBmzin8i+nsz+NcynwacQTvcyqvMJOWxa5t317LP+eKpInAzJ4ys+3NrCbB9pOzzOyG9IcmIplUcmG4w/iIDzmcf3EalSiiIy/yf3zEhxRfZLh27SABaEJY7kilRtDc3ZcDpwHjgT2BC9IZlIhkTocOxWsBzZjFy5zKRxzOnnzLpYygJTN4mY6UXBXUHZYsKd945Y9LpbO4iplVIUgED7n7ejPTvsEiFVB0AmjAjwykP10ZxUpqcTNDuJ+erKHGFtc1bw4zZ5ZjoFKmUkkEjwLfAdOA982sEbA8nUGJSPmKTgC1WcKNDKUn91OJIu6nJ4Ppw2/UiXmt62thzkvaNOTuD7h7A3c/0QPzgaPLITYRSbMGDTYngW1Zy3XczTcU0Is7eY4zacpsruPemEkg0hcguS+VzuJdzOwJMxsfvm4OdEl7ZCKSNpFdwhYuhEpspAujmEMT7uYGPqEdB/A5XRjDfBrHvL57d/UFVCSpdBaPAt4A6oev5wBXpykeEUmjGjWim4GcE/k3U2nNKC7kZ3blaN7mL4xjOq3ilqEhoRVPKomgrrs/CxQBuPsGYGMqhZvZNWY208xmmNnTZlatxPtdzWyRmU0NH5eU+k8gIkkVFgYJYM2a4PUhfMK7HMW/OYlqrOVMnuUQJvJuglbfyLwAqXhS6SxeZWZ1AAcws3bAsmQXmVkD4CqC4adrzOxZ4ByCGka0Z9z9b6WKWkRSFr1RTBNmM4Te/JUX+Zld6M4jPM4lbKBKwjKUACq2VBLBtcArQIGZfQTUA84oRfnVzWw9UAOIuW2FiKRHpBloV36iPwO5hMdZQ3X6Moj7uIZV1Ep4vRJAfkiaCNx9ipn9CWhKMHtktruvT+G6BWZ2N/A9sAZ4093fjHHqX83sSIK+h2vc/YeSJ5hZN6AbQMOGDZN9tEjeiySA7VnGDdzFNdxHFdbzCD24jVtYxM4Jr9c+AfnFPEnKN7POsY67+5gk1+0IvACcDSwFngOed/exUefUAVa6++9mdhlwtrsfk6jctm3b+qRJkxLGLJKvIgmgKr/TnWHcwm3UZTFPcS59uZV5FCQtQ7WAisnMJrt721jvpdJZfFDU4whgAHBKCtd1AL5190VhDeJF4LDoE9x9sbv/Hr58HDgwhXJFpITIcFCjiPMo5Cv25e9cw+ccQBsm04mnEiaByJwAJYH8lErT0JXRr82sNvDPFMr+HmhnZjUImobaA8W+ypvZbu7+U/jyFODLFMoVkVD0UNA/8yZDuZHWTGMKB9CNEbzFsUnL0M1ftmY/glUEC88l5O4TgeeBKcAX4WeNMLNBZhapUVwVDi+dRjDCqOtWxCOSdyI1AIADmcRbdOANjmd7lnMuT9GWSUmTgGoAEpFKH8GrhENHCW7mzYFn3f2mNMcWk/oIJF9VrQrro4ZpFDCXwfThbJ5lEXUZRD8e5TLWUzVpWUoA+SdRH0Eqw0fvjnq+AZjv7j+WSWQiklTJTeJ35hf6ciuX8SjrqMog+nI317OC5HtGabtIiSWVPoL3yiMQESmuZAKoxQqu4x6u526qsZYRdGMQ/fiFXZOW1b69NoqR+OImAjNbweYmoWJvAe7u2rJSpIyVvPkDVGEd3RhBPwaxM4t4ljO5hdv4miZJy6tSBdatS0OgUqHE7Sx29+3cffsYj+2UBETKVnTn76ZjFHEWzzCL5jzElcyiOQczkbN5NmkSGDs26AdQEpBUpNJHAICZ7QxsWjTO3b9PS0QieSRWDQDgGCYwlBtpy2Smsx8nMI7XOZ6SW0OWpCYg2RpJE0E41PMegmWo/wc0Ihjv3yK9oYlUXPESQGs+5w5u4jjeZD4NuYAxPMV5FFE5YXmVKsHGlNYEFtlSKvMIbgXaAXPcfU+CiWGfpDUqkQoqVhMQQGO+5UnO53Pa0JZJXMs9NGU2Y7kgYRKIzAVQEpA/IpVEsN7dFwOVzKySu78DxByLKiKxFd8QZrO6LOI+rmY2TTmdFxnCzRTwDfdxLb9TbcsLCL79azKYlKVU+giWmlkt4H2g0Mz+RzC7WERSECsB1GAV13AfvbiTmqziCS5mIP1ZSIO45aj9X9Il0fDRM4FXgVMJ1gq6BugE7AAMKpfoRHJU5cpQVLTl8W1Yz0WMZAAD2I2feYnT6M0QvqJZzHKqV4fVq9McrOS9RDWC84CHCfYrfhp4w91Hl0tUIjkqXicweNj005umzOFD/o+/8gL/Lb4g7ya1a2tzeCk/ieYRdAT2Bt4CrgR+NLPh4SY1IhIlXicwwJG8xye04wXOYAPbcDKvcAQfxEwCVaoEbf9KAlKeEnYWu/tydx/t7icALYHPgQfMbItdxETyUdWq8RNAS77gVU7iPY6iPgu5kJHsz3Re42RizQfQBDDJlJSWoQ53GzudYLexnQiWlxbJW5FRQOtjbNq6B9/zD7oyjVb8Hx/Ri6E0YQ6juHCLoaDaEEayQaLO4lpAR+Bc4ACCDexvBd71ZGtXi1Rg8WoAO7GYm7mdv/EQAHdzPXdwE0vYKeb5+l8k2SJRjeA74DjgEaChu1/m7u8oCUi+ijcXoBpr6MVQvqGAa7mXpziPffiaG7kzZhKIrAMkki0SjRraw93XlFskIlmqRg1YE+N/QmU20IXRDKQ/u7OAVzmJm7mdmbSMWY5u/pKtEo0aUhKQvBYZCbRlEnBO4V9MZ3+e4BJ+YA+O5D1O4dWYSUB9AJLttmbPYpEKL14/wGF8xAccwb84jUoU0ZEXOYyP+YAjtzhXCUByhRKBSJR48wGaMYuXOI2POJy9mEc3HqUlM3iZjpQcCqoEILkm0aih6E3rt+DupyQr3MyuAS4Jy/kCuNDd10a9vy0wBjgQWAyc7e7fpRq8SFkpLITzz9/yeAN+ZAADuJB/sJJa9GYw99OT1dTc4lzd/CVXJeosjmxafzqwKzA2fH0u8Euygs2sAXAV0Nzd15jZs8A5wKio0y4Glrj73mZ2DjCUYK6CSLmJtS7QDizlJu6gJ/dTiSLupydD6M1i6sYsQ0lAclncRBDZtN7M7nH36GWnXzWzSaUov7qZrQdqAAtLvH8qMCB8/jzwkJmZhqhKeSnZDLQta7mCh+nDYGqzlEI60ZdbmU/jmNfrX6pUBKn0EdQ0s70iL8xsT4hRLy7B3RcQ1Cq+B34Clrn7myVOawD8EJ6/AVgG1ClZlpl1M7NJZjZp0aJFKYQsklx0EqjERjozmjk04R6uZyKH0IYpdObJmElA/QBSkaSSCK4B3jWzd83sPeAd4OpkF4XLUpwK7EmwzWVNM4vRCpucu49w97bu3rZevXpbU4QIAC1alOwQdk7k30ylNaPpyi/swjFM4ETGM43Wxa7VhjBSUSXdmMbdXzezfYB9w0NfufvvKZTdAfjW3RcBmNmLwGFs7msAWADsQbCy6TYEex0sLkX8Iilp0QJmzSp+7GAmMpQbOYr3mEsBZ/EMz3Em8RaEE6moktYIzKwGcAPwN3efBjQ0s5NSKPt7oJ2Z1TAzI9jr+MsS57wCdAmfnwG8rf4BKWtmxZNAE2bzHGcwkXY040t68DDN+JLnOIuSSSCyLLRIRZZK09A/gHXAoeHrBcBtyS5y94kEHcBTCIaOVgJGmNkgM4sMPX0CqGNmc4FrgZtKF75IYtH9ALvyE8O4nJm04DjeoB8DKeAbhtGDDVTZ4lotCy35IpU9iwvc/WwzOxfA3VeH3/CTcvf+QP8Sh/tFvb8WODPVYEVKI/KvdDuWcwN3cS33UpV1DKM7t9KXRewc91rVAiSfpJII1plZdcLJZWZWAKTSRyCSEZEEUJXfuZzh9OVW6rKYpzmHW7iNeRQkvF5JQPJNKk1D/YHXgT3MrBCYAPRKa1QiWyGyTLRRxHkU8hX7cj9XM5XWHMgkzuPphEmgfXslAclPCWsEZlYJiOxO1o6gJ62nu/9aDrGJpCyoBTh/5k3u4CYOYCpTOIBujOAtjk14rW7+ku+S7VlcBPRy98Xu/m93f01JQLJFgwab5wQcyCTeogNvcDw7sIzzKKQtkxImAc0JEAmk0jT0lpldb2Z7mNlOkUfaIxOJo0OH4Oa/cCEUMJenOYdJHMT+TOcq7qcZX/I05+EJ/nkrAYhslkpncWQRuCuijjmwV4xzRdIq0hG8M7/Ql1u5jEdZR1UG0Ze7uZ4VbJ+0DCUBkeJSmVm8Z3kEIpJIJAHUYgXXcQ/XczfVWMsIunErffmZ3ZKWUbs2LFmS3jhFclHSRBDOLL6WYAP7buFyE03d/bW0Ryd5rWpVWL8+eF6FdVzKY/RjELvwP57jDPowmK9pkrScKlU0MUwkkdLMLD4sfJ3SzGKRP8IsSAJGEWfxDLNozsP8jS9pxiF8wlk8l1IS0OxgkeRSSQQF7n4nsB6CmcXEWpVLpAxERgIBHMMEPuVgnuEcVlODExjH0bzDpxyStJzISqEiklwqiUAziyWtCgs3DwNduBBaMZXxHM8EOlCPRXRmNAfwOa9zAsm+g0SGhG7cWD6xi1QEmlksGdWjx+a9ghvzLU9yPlM5gIP4jGu5h6bM5kk6U0TlhOVoToDI1ktl1NB/zGwKmlksZahHDxg2LHheh1+5hdvowSNsYBuGcDN30otl1E5aTvXqsHp1emMVqejiJgIza1Pi0E/hz4Zm1tDdp6QvLKnIIn0ANVjF1fydGxlKTVYxkosYwAAW0iBpGfXrw4IFaQ5UJE8kqhHcE/6sBrQFphHUCPYHJrF5fwKRlDRoEPQBbMN6LmIkAxjAbvzMS5xGb4bwFc2SlqHmH5GyF7ePwN2PdvejCWoCbcI9gw8EDiAYQiqSkkhn8MKFzum8wAxa8iiX8w0FHMZHnM5LSZNA/fpKAiLpksoSE03d/YvIC3efYWbJv7pJ3oueEHYE73MnvWjHRGbSnJN5hdc4iVRGIisBiKRXKqOGvjCzx83sqPDxGDA93YFJbtq0J0A4IawlX/AqJ/E+f2J3fuQinqAV03iNk0mUBCKjgJQERNIvlRpBV6A70DN8/T4wLF0BSW6qXBmKija/3oPvGUQ/OjOGZexAL4byIFeylupJy9LNX6R8JduYpjIwPuwruK98QpJcsuOOsHRp1Gt+42Zu50oeBOBurucObmIJyVcuVwIQyYxkG9NsBIrMbIfSFmxmTc1satRjuZldXeKco8xsWdQ5/eIUJxnUo8fm5p6Sj0gSqMYaejGUeezFddzD05xLE+ZwI3cmTQLqCBbJrFSahlYS9BP8B1gVOejuVyW6yN1nA61hU81iAfBSjFM/cPeTUg1YykeHDjBhQvLzKrOBLoxmIP3ZnQW8ykn0Zggz2C/ptZoMJpIdUkkEL4aPP6I98I27z/+D5UiapZoAwDmZV7mdm2nBLP5LO87jKT7gyJQ+RzUAkeyRSiJ4Btg7fD7X3dduxeecAzwd571DzWwasBC43t1nljzBzLoB3QAaNmy4FR8vqYhM+ErmMD5iKDdyOB8xmyaczgu8REeSDQVt3x7eeqtsYhWRshO3j8DMtjGzO4EfgdHAGOAHM7vTzKqk+gFmVhU4BXguxttTgEbu3gp4EHg5VhnuPiKc0Na2Xr16qX60pCB6A/hkSaAZs3iJ0/iIw9mLeXTjUVowk5c4nXhJoH37zcNAlQREslOizuK7gJ2APd39QHdvAxQAtYG7S/EZJwBT3P2Xkm+4+3J3Xxk+HwdUMbO6pShbttLm2b7Jz63PAh7jEr5gP47mHXozmH34msfoxsY4lUrd/EVyR6JEcBJwqbuviBxw9+UEcwpOLMVnnEucZiEz29UsWILMzA4O41lcirKlFKJH/0SWfk5kB5YyhJuZy950ZgwPcBUFfMPt9GY1NWNeE6kBiEjuSNRH4O5b/pd2941mltJ/dTOrCRwLXBZ17PKwnOHAGUB3M9sArAHOifWZ8sel2v4PsC1ruYKH6cNgarOUQjrRj0F8x54ANG8OM7foyRGRXJWoRjDLzDqXPGhm5wNfpVK4u69y9zruvizq2PAwCeDuD7l7C3dv5e7t3P3j0v4BJL7CQth229SbgCqxkc6MZg5NuIfrmcghtGEKnXmSgvZ7bmruURIQqVgS1QiuAF40s4uAyeGxtkB1oGO6A5M/JvVhoADOCYznDm5if75gEgfSlVG8wzHUrg2+JI2BikjGxU0E7r4AOMTMjgFahIfHuXvKtxfJjNI0Ax3MRIZyI0fxHnMp4Cye4XnOwKmk4Z4ieSKVrSrfBt4uh1ikDHTokFoSaMJsBtOHM3iBX9iZHjzM2GqXsnxNyiODRaSCSGUZaskRqTQH7cpPPEJ3ZtKC43iD/gyggG+Y076HkoBInkplZrFkuVQSwHYs5wbu4lrupSrrGEZ3fuxyC0NH7cLA8glTRLKUEkEOKyyECy5IPG6/Kr9zOcO5hduox688zTmM3vs2Xv+6oPwCFZGspqahHJUsCRhFnEchX9KM+7maabSi+0GTONefVhIQkWKUCHJQYWEwMzh2EnCO5U0mcyCFnM8yduDPvMEvT/6HYZ8eWN6hikgOUCLIMYWF0HmLaX6BNkzmPxzLmxxHbZZyHoUcyGT27v5nOp2ffJN4EclPSgQ5oLAQatXavEZQ9N7AAAXM5WnOYTJtacU0ruJ+9uUrnuY8Lu9eiUceyUzcIpIb1Fmc5Xr0gGHDYr9Xj//Rj0FcxqOsoyqD6MvdXM8Ktqd7d5QARCQlSgRZrLAQhg/f8ngtVnAt93I9d1OdNTzGpQyiHz+zm2YDi0ipKRFksZ49i3cIb8N6LuUx+jOQXfgfz3EGfRjM1zQBtAOYiGwd9RFkWGEhNG68eZ+A6MficGcGo4gzeZZZNOcRruBLmnEIn3AWz21KAjVrKgmIyNZRjSCDCguhWzdYvTr+OUfzNkO5kYOYxHT240T+zXhOIHpryKpV4dFH0x+viFRMqhFkSGQYaLwksD/TGM/xvE17duZ/dGY0B/A54zmR6CRQpw6MHAmdOpVP3CJS8ahGkAGJRgI14jtupS+dKGQJO3It9/AIPfidapvPaQTffVc+sYpIxadEUI4KC4MO4MUxdmWuw6/0YTA9eIQiKjGUGxnKjSyjdrHzqlaFwYPLJ14RyQ9KBOUkXn9ADVZxNX+nF3dSi5WM5CIGMICFNNiijFq1guGkagYSkbKkPoJy0qdP8SRQmQ1cygi+Zh8GcwtvcwwtmUE3HtsiCdSpA2PHwooVSgIiUvbSlgjMrKmZTY16LDezq0ucY2b2gJnNNbPpZtYmXfFk2vz5kWdOR15kJi0YwWXMYy8O4yNO5yXmVW3G2LFs2iQ+8vj1VyUAEUmftCUCd5/t7q3dvTVwILAaeKnEaScA+4SPbkCcLtTcVlgYzAs4gvf5L4fyIn9lI5U5hX9xBB/wXw6jVi2N/hGRzCivpqH2wDfuPr/E8VOBMR74BKhtZruVU0zlZvQNM/iXn8z7/Ind+ZGLeIL9mc6rnEKdOqZmHxHJqPLqLD4HeDrG8QbAD1GvfwyP/RR9kpl1I6gx0LBhwzSFmAbffw/9+/P6T6NZzvb0YigPciVrqb7plF9/zWB8IiKUQ43AzKoCpwDPbW0Z7j7C3du6e9t69eqVXXDp8ttvcMMN0KQJGwuf5j67jr2Yx130KpYEGjXKYIwiIqHyqBGcAExx919ivLcA2CPq9e7hsdy0Zg088ADccQcsW8Y3h3fhxM8GMmf9lrWYGjU0H0BEskN59BGcS+xmIYBXgM7h6KF2wDJ3/ynOudlrw4agp3effeCmm+Dww2HaNNp//w/mrN0yCVSuDCNGqE9ARLJDWhOBmdUEjgVejDp2uZldHr4cB8wD5gKPAT3SGU+Zc4dXXoFWreDii2H33eHdd+HVVymcvl/UkNHiioqUBEQke6S1acjdVwF1ShwbHvXcgSvSGUPafPQR3Hhj8LNJE3jhBejYEcw2zSKOJ5f6u0Wk4tPM4tL68ks47bSg+eebb4L1n2fMgNNPDyYLsOUs4mjqGxCRbKNEkKoFC+DSS6FlS3j7bbjtNpg7N/jqX6XKptMKC4nbJATqGxCR7KNF55JZuhSGDoW//x02boSrrgq+8tetu+mUwsLg0Pz5myoFMTVqpCQgItlHiSCetWvh4YeDdpylS4M7+KBBsOeexU4ruapo9B7D0dQkJCLZSk1DJW3cCGPGQNOmcP31cPDBMGUKPPlksSQQ2Wv4/PMTbzUZoSYhEclWSgQR7jBuHBxwAHTpAvXqBbvBv/46tG5d7NRILSBRX0A0NQmJSDZTIgCYOBGOPhr+8pfg6/0zz8Cnn0L79jFPTzQqqCQ1CYlItsvvRDBnDpx5JrRrFwwLfeghmDULzjoLKsX/1Xz/feJiIx3GjRqpSUhEsl9+JoKffoLu3aF5cxg/HgYMCIaCXnFFsClwEokmhDVqFHQnuAcbzCsJiEi2y69EsHw59O0Le+8Njz8eJINvvoH+/WG77VIuZvDgoMknWo0awXaSuvmLSK7Jn0QwfjwUFAQTwU4+OWgKevBB2GWXlC6PjBKqVCnoI+jSJfj2b6YmIBHJbfkzj6CgANq0Cb7Ot21bqktLzhWYPx9Gj9bNX0QqBvN4M6CyVNu2bX3SpEnl+pmNG8ceKtqoUdAUJCKS7cxssrvH/BacP01Df0C8UULJRg+JiOQCJYIUxBslpOWkRaQiUCIoIbpTuHHj4HW8UUKaKCYiFYESQZTopSPcg5+RDWZGjNAoIRGpmNRZHEWdwiJSUamzOEXqFBaRfKREEEWdwiKSj5QIoqhTWETyUVoTgZnVNrPnzewrM/vSzA4t8f5RZrbMzKaGj37pjCeZTp3UKSwi+SfdS0zcD7zu7meYWVWgRoxzPnD3k9IcR8o6ddKNX0TyS9oSgZntABwJdAVw93XAunR9noiIbJ10Ng3tCSwC/mFmn5vZ42ZWM8Z5h5rZNDMbb2YtYhVkZt3MbJKZTVq0aFEaQxYRyT/pTATbAG2AYe5+ALAKuKnEOVOARu7eCngQeDlWQe4+wt3bunvbevXqpTFkEZH8k85E8CPwo7tPDF8/T5AYNnH35e6+Mnw+DqhiZnXTGJOIiJSQtkTg7j8DP5hZ0/BQe2BW9DlmtqtZsMOvmR0cxrM4XTGJiMiW0j1q6EqgMBwxNA+40MwuB3D34cAZQHcz2wCsAc7xXFvzQkQkx6V1HoG7Tw3b9vd399PcfYm7Dw+TAO7+kLu3cPdW7t7O3T9ORxyxVhQVEZFAhd+qMtY2k5EVRTVfQEQkD5aY6NNncxKIWL06OC4iInmQCLSiqIhIYhU+EWhFURGRxCp8ItCKoiIiiVX4RKAVRUVEEqvwo4ZAK4qKiCRS4WsEIiKSmBKBiEieUyIQEclzSgQiInlOiUBEJM9Zri32aWaLgPmZjqOEusCvmQ4iRbkUKyjedMqlWCG34s3GWBu5e8ydvXIuEWQjM5vk7m0zHUcqcilWULzplEuxQm7Fm0uxgpqGRETynhKBiEieUyIoGyMyHUAp5FKsoHjTKZdihdyKN5diVR+BiEi+U41ARCTPKRGIiOQ5JYIyYma3mtl0M5tqZm+aWf1MxxSPmd1lZl+F8b5kZrUzHVMiZnammc00syIzy8oheWZ2vJnNNrO5ZnZTpuNJxMxGmtn/zGxGpmNJxsz2MLN3zGxW+G+gZ6ZjSsTMqpnZp2Y2LYx3YKZjSoX6CMqImW3v7svD51cBzd398gyHFZOZ/Rl42903mNlQAHe/McNhxWVmzYAi4FHgeneflOGQijGzysAc4FjgR+Az4Fx3n5XRwOIwsyOBlcAYd2+Z6XgSMbPdgN3cfYqZbQdMBk7L4t+tATXdfaWZVQE+BHq6+ycZDi0h1QjKSCQJhGoCWZth3f1Nd98QvvwE2D2T8STj7l+6++xMx5HAwcBcd5/n7uuAfwKnZjimuNz9feC3TMeRCnf/yd2nhM9XAF8CDTIbVXweWBm+rBI+svZeEKFEUIbMbLCZ/QB0AvplOp4UXQSMz3QQOa4B8EPU6x/J4ptVrjKzxsABwMQMh5KQmVU2s6nA/4D/uHtWxwtKBKViZm+Z2YwYj1MB3L2Pu+8BFAJ/y+ZYw3P6ABsI4s2oVOKV/GVmtYAXgKtL1L6zjrtvdPfWBDXtg80sq5vfIE+2qiwr7t4hxVMLgXFA/zSGk1CyWM2sK3AS0N6zoKOoFL/bbLQA2CPq9e7hMSkDYVv7C0Chu7+Y6XhS5e5Lzewd4HggqzvmVSMoI2a2T9TLU4GvMhVLMmZ2PNALOMXdV2c6ngrgM2AfM9vTzKoC5wCvZDimCiHsfH0C+NLd7810PMmYWb3IKDwzq04wgCBr7wURGjVURszsBaApweiW+cDl7p6V3wrNbC6wLbA4PPRJto5wAjCzjsCDQD1gKTDV3Y/LaFAlmNmJwN+BysBIdx+c2YjiM7OngaMIlkr+Bejv7k9kNKg4zOxw4APgC4L/WwC93X1c5qKKz8z2B0YT/DuoBDzr7oMyG1VySgQiInlOTUMiInlOiUBEJM8pEYiI5DklAhGRPKdEICKS55QIJGPMrE64WutUM/vZzBaEz5eaWbkuKmZmp5lZ86jXg8ys1JPczKxxJlf1NLPeJV5/HP7MaFyS3ZQIJGPcfbG7tw6n4w8H7guft2bzmPEyY2aJZtKfBmxKBO7ez93fKusYykGxRODuh2UqEMkdSgSSrSqb2WPhmu5vhrM0MbMCM3vdzCab2Qdmtm94vLGZvR3usTDBzBqGx0eZ2XAzmwjcGet6MzsMOAW4K6yRFITXnRGWcZCZfRyuMf+pmW0Xft4HZjYlfCS84VrgIQv2LHjLzMZFlf+dmdUNn7c1s3fD5web2X/N7PPw85uGx7ua2Yvhn+NrM7szPH4HUD38MxSGx1bGiKWyBXtSfBb+vi4Lj+9mZu+H188wsyP+4N+h5Ap310OPjD+AAQR7DQA0JlgMr3X4+lng/PD5BGCf8PkhBPsqALwKdAmfXwS8HD4fBbwGVE5y/SjgjKh4RgFnAFWBecBB4fHtCdboqgFUC4/tA0yKin1GjD/f6cB/CGac1ieYIX1G+N53QN3weVvg3ejPCp93AF4In3cNY9oBqEYwk32P8L2VJT53Zcm4gG7ALeHzbYFJwJ7AdUCf8HhlYLtM/7vQo3weWnROstW37j41fD4ZaByuQHkY8FywBA0Q3MgADiW42QI8CdwZVdZz7r4xyfXxNAV+cvfPYPO+E2ZWE3jIzFoDG4EmSco5Enja3TcCC83s7STnQ3CjHx2uY+UEa9tHTHD3ZWEss4BGFF8KO5E/A/tHaiTh5+xDsGbSyHCRt5ejfv9SwSkRSLb6Per5RqA6QVPmUg/6EUpjVfhza6+P5RqCdXpaheWu/QNlbWBzM221qOO3Au+4e0cL1uJ/N+q9kr+f0vxfNuBKd39jizeC3cv+Aowys3vdfUwpypUcpT4CyRnht/FvzexM2NTu3ip8+2OCVT8h2Bjog1JevwLYLsbHzgZ2M7ODwmu2CzuddyCoKRQBFxA0pSTyPnB22D6/G3B01HvfAQeGz/8adXwHNi9n3TVJ+RHrw2/0ibwBdI+cZ2ZNzKymmTUCfnH3x4DHgTYpfqbkOCUCyTWdgIvNbBowk81bQl4JXGhm0wluzPE2OY93/T+BG8KO2YLIyR5sPXk28GB4zX8IvrU/AnQJj+3L5lpHPC8BXwOzgDHAf6PeGwjcb2aTCL7dR9wJ3G5mn5P6N/4RwPRIZ3Ecj4dxTAmHlD4aln8UMC38vLOB+1P8TMlxWn1UJAPMbBTwmrs/n+lYRFQjEBHJc6oRiIjkOdUIRETynBKBiEieUyIQEclzSgQiInlOiUBEJM/9PzhvT20zovP3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#histogram and normal probability plot\n",
    "sns.distplot(boxcox1p(data['GrLivArea'],boxcox_normmax(data['GrLivArea']+ 1)), fit=norm);\n",
    "fig = plt.figure()\n",
    "res = stats.probplot(boxcox1p(data['GrLivArea'],boxcox_normmax(data['GrLivArea']+ 1)), plot=plt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Áp dụng biến đổi Box-Cox cho tất cả trường dữ liệu có chỉ số skew lớn hơn 0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Áp dụng  Box-Cox transform thay vì log1p transform\n",
    "transform = boxcox1p\n",
    "numeric_dtypes = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']\n",
    "numerics = []\n",
    "for i in data.columns:\n",
    "    if data[i].dtype in numeric_dtypes:\n",
    "        numerics.append(i)\n",
    "skew_features = data[numerics].apply(lambda x: skew(x)).sort_values(ascending=False)\n",
    "\n",
    "high_skew = skew_features[skew_features > 0.5]\n",
    "skew_index = high_skew.index\n",
    "\n",
    "for i in skew_index:\n",
    "    data[i] = transform(data[i], boxcox_normmax(data[i] + 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tạo những cột feature mới "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " #### Từ thông tin nếu 'YearRemodAdd' (Năm tu sửa) == 'YearBuilt' (Năm xây dựng) ==> Nhà không tu sửa. Tạo cột dữ liệu nhà có tu sửa hay không ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    " data['YrBltdiffRemod'] = (data['YearBuilt']==data['YearRemodAdd'])*1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tạo cột dữ liệu tổng diện tích nhà ở bằng tổng diện tích tầng hầm, tầng 1 và tầng 2. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['TotalSF'] = data['TotalBsmtSF'] + data['1stFlrSF'] + data['2ndFlrSF']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "## total footage\n",
    "data['Total_sqr_footage'] = (data['BsmtFinSF1'] + data['BsmtFinSF2'] +\n",
    "                                 data['1stFlrSF'] + data['2ndFlrSF'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#total bathroom \n",
    "data['Total_Bathrooms'] = (data['FullBath'] + (0.5 * data['HalfBath']) +\n",
    "                               data['BsmtFullBath'] + (0.5 * data['BsmtHalfBath']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "## total porch\n",
    "data['Total_porch_sf'] = (data['OpenPorchSF'] + data['3SsnPorch'] +\n",
    "                              data['EnclosedPorch'] + data['ScreenPorch'] +\n",
    "                              data['WoodDeckSF'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Từ dữ liệu diện tích các hạng mục nhà, tạo những trường dữ liệu mới. Ví dụ: có bể bơi hay không?, có tầng 2 hay không , etc, ...  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['haspool'] = data['PoolArea'].apply(lambda x: 1 if x > 0 else 0)\n",
    "data['has2ndfloor'] = data['2ndFlrSF'].apply(lambda x: 1 if x > 0 else 0)\n",
    "data['hasgarage'] = data['GarageArea'].apply(lambda x: 1 if x > 0 else 0)\n",
    "data['hasbsmt'] = data['TotalBsmtSF'].apply(lambda x: 1 if x > 0 else 0)\n",
    "data['hasfireplace'] = data['Fireplaces'].apply(lambda x: 1 if x > 0 else 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tạo trường dữ liệu năm sử dụng = năm bán - năm xây dựng"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['YearBuiltDur'] = data['YrSold'] - data['YearBuilt']\n",
    "data['YearRemodAddDur'] = data['YrSold'] - data['YearRemodAdd']\n",
    "data['GarageYrBltDur'] = data['YrSold'] - data['GarageYrBlt']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# <a id='1.1*'>1.1*. Xử lý trường dữ liệu phân loại </a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Các bước xử lý : \n",
    "   + Xóa các trường dữ liệu có chênh lệch data lớn dễ gây overfitting dữ liệu.\n",
    "   + Những trường phân loại mang tính đánh giá chất lượng, sử dụng label encoding dựa trên chất lượng.\n",
    "   + Ở trường phân loại có những dữ liệu có tần suất xuất hiện thấp, gộp chung các phân loại này thành một. \n",
    "   + One hot encoding các trường dữ liệu còn lại.\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quan sát các trường dữ liệu phân loại  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>LotConfig</th>\n",
       "      <th>LandSlope</th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Condition1</th>\n",
       "      <th>...</th>\n",
       "      <th>GarageType</th>\n",
       "      <th>GarageFinish</th>\n",
       "      <th>GarageQual</th>\n",
       "      <th>GarageCond</th>\n",
       "      <th>PavedDrive</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>CollgCr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Veenker</td>\n",
       "      <td>Feedr</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>CollgCr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Crawfor</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Detchd</td>\n",
       "      <td>Unf</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NoRidge</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 43 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  MSZoning Street Alley LotShape LandContour Utilities LotConfig LandSlope  \\\n",
       "0       RL   Pave  None      Reg         Lvl    AllPub    Inside       Gtl   \n",
       "1       RL   Pave  None      Reg         Lvl    AllPub       FR2       Gtl   \n",
       "2       RL   Pave  None      IR1         Lvl    AllPub    Inside       Gtl   \n",
       "3       RL   Pave  None      IR1         Lvl    AllPub    Corner       Gtl   \n",
       "4       RL   Pave  None      IR1         Lvl    AllPub       FR2       Gtl   \n",
       "\n",
       "  Neighborhood Condition1  ... GarageType GarageFinish GarageQual GarageCond  \\\n",
       "0      CollgCr       Norm  ...     Attchd          RFn         TA         TA   \n",
       "1      Veenker      Feedr  ...     Attchd          RFn         TA         TA   \n",
       "2      CollgCr       Norm  ...     Attchd          RFn         TA         TA   \n",
       "3      Crawfor       Norm  ...     Detchd          Unf         TA         TA   \n",
       "4      NoRidge       Norm  ...     Attchd          RFn         TA         TA   \n",
       "\n",
       "  PavedDrive PoolQC Fence MiscFeature SaleType SaleCondition  \n",
       "0          Y   None  None        None       WD        Normal  \n",
       "1          Y   None  None        None       WD        Normal  \n",
       "2          Y   None  None        None       WD        Normal  \n",
       "3          Y   None  None        None       WD       Abnorml  \n",
       "4          Y   None  None        None       WD        Normal  \n",
       "\n",
       "[5 rows x 43 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical_features = data.select_dtypes(exclude=[np.number])\n",
    "categorical_columns = categorical_features.columns\n",
    "data[categorical_columns].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Xóa những cột dữ liệu phân loại có sự chênh lệch quá cao về quy mô giữa các phân lớp dữ liệu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Street', 'Utilities', 'Condition2', 'PoolArea', 'PoolQC', 'haspool']\n"
     ]
    }
   ],
   "source": [
    "overfit = []\n",
    "for i in data.columns:\n",
    "    counts = data[i].value_counts()\n",
    "    zeros = counts.iloc[0]\n",
    "    if zeros / len(data) * 100 > 99:\n",
    "        overfit.append(i)\n",
    "overfit = list(overfit)\n",
    "print(overfit)\n",
    "data = data.drop(overfit, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Label encoding các trường dữ liệu có tính chất thứ tự "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_LotShape(x):\n",
    "    if x == 'Reg':\n",
    "        return 4\n",
    "    elif x == 'IR1' :\n",
    "        return 3\n",
    "    elif x == 'IR2' :\n",
    "        return 2\n",
    "    else :\n",
    "        return 1\n",
    "data['LotShape'] = data['LotShape'].apply(clean_LotShape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_ExterCond(x):\n",
    "    if x == 'Ex':\n",
    "        return 5\n",
    "    elif x == 'Gd' :\n",
    "        return 4\n",
    "    elif x == 'TA' :\n",
    "        return 3\n",
    "    elif x == 'Fa' :\n",
    "        return 2\n",
    "    elif x == 'Po' :\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "data['ExterCond'] = data['ExterCond'].apply(clean_ExterCond)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_ExterQual(x):\n",
    "    if x == 'Ex':\n",
    "        return 5\n",
    "    elif x == 'Gd' :\n",
    "        return 4\n",
    "    elif x == 'TA' :\n",
    "        return 3\n",
    "    elif x == 'Fa' :\n",
    "        return 2\n",
    "    elif x == 'Po' :\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "\n",
    "data['ExterQual'] = data['ExterQual'].apply(clean_ExterQual)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_BsmtQual(x):\n",
    "    if x == 'Ex':\n",
    "        return 5\n",
    "    elif x == 'Gd' :\n",
    "        return 4\n",
    "    elif x == 'TA' :\n",
    "        return 3\n",
    "    elif x == 'Fa' :\n",
    "        return 2\n",
    "    elif x == 'Po' :\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "data['BsmtQual'] = data['BsmtQual'].apply(clean_BsmtQual)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_BsmtCond(x):\n",
    "    if x == 'Ex':\n",
    "        return 5\n",
    "    elif x == 'Gd' :\n",
    "        return 4\n",
    "    elif x == 'TA' :\n",
    "        return 3\n",
    "    elif x == 'Fa' :\n",
    "        return 2\n",
    "    elif x == 'Po' :\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "data['BsmtCond'] = data['BsmtCond'].apply(clean_BsmtCond)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_BsmtFinType1(x):\n",
    "    if x == 'GLQ':\n",
    "        return 6\n",
    "    elif x == 'ALQ' :\n",
    "        return 5\n",
    "    elif x == 'BLQ' :\n",
    "        return 4\n",
    "    elif x == 'Rec':\n",
    "        return 3\n",
    "    elif x == 'LwQ':\n",
    "        return 2\n",
    "    elif x == 'Unf':\n",
    "        return 1\n",
    "    else:\n",
    "        return 0   \n",
    "data['BsmtFinType1'] = data['BsmtFinType1'].apply(clean_BsmtFinType1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_BsmtFinType2(x):\n",
    "    if x == 'GLQ':\n",
    "        return 6\n",
    "    elif x == 'ALQ' :\n",
    "        return 5\n",
    "    elif x == 'BLQ' :\n",
    "        return 4\n",
    "    elif x == 'Rec':\n",
    "        return 3\n",
    "    elif x == 'LwQ':\n",
    "        return 2\n",
    "    elif x == 'Unf':\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "    \n",
    "data['BsmtFinType2'] = data['BsmtFinType2'].apply(clean_BsmtFinType2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_HeatingQC(x):\n",
    "    if x == 'Ex':\n",
    "        return 5\n",
    "    elif x == 'Gd' :\n",
    "        return 4\n",
    "    elif x == 'TA' :\n",
    "        return 3\n",
    "    elif x == 'Fa' :\n",
    "        return 2\n",
    "    elif x == 'Po' :\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "    \n",
    "data['HeatingQC'] = data['HeatingQC'].apply(clean_HeatingQC)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Nhóm các dữ liệu có tần suất xuất hiện thấp thành một phân lớp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "def group_rare_value(df, frequency_threshold=15):\n",
    "    '''\n",
    "    Replace value of categorical columns in df by 'rare_value\n",
    "    if the frequency of the value in df_ref is smaller than frequency_threshold\n",
    "    '''\n",
    "    col_list = list(df.dtypes[df.dtypes == 'object'].index)\n",
    "    for col in col_list:\n",
    "        value_counts = df[col].value_counts()\n",
    "        if(len(value_counts[value_counts < frequency_threshold]) > 0):\n",
    "            rare_values = value_counts[value_counts < frequency_threshold].index\n",
    "            for rare_value in rare_values:\n",
    "                df.loc[df[col] == rare_value, col] = 'rare_value'\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = group_rare_value(data, frequency_threshold = 15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "86"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### One-hot-encoding các trường dữ liệu còn lại\n",
    "\n",
    "![onehot](https://i.imgur.com/TW5m0aJ.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2917, 235)\n"
     ]
    }
   ],
   "source": [
    "data = pd.get_dummies(data)\n",
    "print(data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sử dụng các thuật toán lightgbm cùng các tham số như tuần 1. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### log1p giá trị trường target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "target = np.log1p(train['SalePrice'].values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Chia lại tập train và test sau khi xử lý toàn bộ dữ liệu "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set = data[:ntrain]\n",
    "test_set = data[ntrain:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tham số và hàm số huấn luyện của thuật toán lightgbm trên một fold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "## parameters of lightgbm \n",
    "params = {\n",
    "        \"objective\" : \"regression\",\n",
    "        \"metric\" : \"rmse\",\n",
    "        \"max_bin\" : 200,\n",
    "        \"num_leaves\" : 8,\n",
    "        \"learning_rate\" : 0.01,\n",
    "        \"bagging_fraction\" : 0.75,\n",
    "        \"bagging_freq\" : 5,\n",
    "        \"bagging_seed\" : 7,\n",
    "        \"feature_fraction\" :0.5,\n",
    "        \"feature_fraction_seed\": 7,       \n",
    "        \"verbosity\" : -1 } \n",
    "def single_fold_lgb(train_X, train_y, valid_X, valid_y, params):    \n",
    "    lgtrain = lgb.Dataset(train_X, label=train_y)\n",
    "    lgval = lgb.Dataset(valid_X, label=valid_y)\n",
    "    evals_result = {}\n",
    "    model = lgb.train(params, lgtrain, 2000, valid_sets=[lgval], early_stopping_rounds=100, verbose_eval=200, evals_result=evals_result)\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.155381\n",
      "[400]\tvalid_0's rmse: 0.124352\n",
      "[600]\tvalid_0's rmse: 0.119188\n",
      "[800]\tvalid_0's rmse: 0.117491\n",
      "[1000]\tvalid_0's rmse: 0.116552\n",
      "[1200]\tvalid_0's rmse: 0.116087\n",
      "[1400]\tvalid_0's rmse: 0.115781\n",
      "Early stopping, best iteration is:\n",
      "[1337]\tvalid_0's rmse: 0.115651\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.170159\n",
      "[400]\tvalid_0's rmse: 0.134041\n",
      "[600]\tvalid_0's rmse: 0.124464\n",
      "[800]\tvalid_0's rmse: 0.121412\n",
      "[1000]\tvalid_0's rmse: 0.120399\n",
      "[1200]\tvalid_0's rmse: 0.119827\n",
      "Early stopping, best iteration is:\n",
      "[1172]\tvalid_0's rmse: 0.119753\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.162125\n",
      "[400]\tvalid_0's rmse: 0.128809\n",
      "[600]\tvalid_0's rmse: 0.121635\n",
      "[800]\tvalid_0's rmse: 0.119814\n",
      "[1000]\tvalid_0's rmse: 0.118638\n",
      "[1200]\tvalid_0's rmse: 0.118273\n",
      "Early stopping, best iteration is:\n",
      "[1140]\tvalid_0's rmse: 0.118193\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.160851\n",
      "[400]\tvalid_0's rmse: 0.128823\n",
      "[600]\tvalid_0's rmse: 0.122036\n",
      "[800]\tvalid_0's rmse: 0.120513\n",
      "[1000]\tvalid_0's rmse: 0.120122\n",
      "Early stopping, best iteration is:\n",
      "[1007]\tvalid_0's rmse: 0.120061\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.175689\n",
      "[400]\tvalid_0's rmse: 0.13665\n",
      "[600]\tvalid_0's rmse: 0.125362\n",
      "[800]\tvalid_0's rmse: 0.122519\n",
      "[1000]\tvalid_0's rmse: 0.120649\n",
      "[1200]\tvalid_0's rmse: 0.119658\n",
      "[1400]\tvalid_0's rmse: 0.119087\n",
      "Early stopping, best iteration is:\n",
      "[1444]\tvalid_0's rmse: 0.118862\n",
      "Full Out-Of-Fold score :  0.118513\n"
     ]
    }
   ],
   "source": [
    "kf = KFold(n_splits=5, shuffle = True, random_state = 2020)\n",
    "def kfold_model_lgb(params):\n",
    "    oof_preds = np.zeros(train_set.shape[0]) ## out of fold prediction \n",
    "    for train_index, valid_index in kf.split(train_set):\n",
    "        X_train , X_valid  = train_set.loc[train_index,:], train_set.loc[valid_index,:]\n",
    "        y_train, y_valid = target[train_index], target[valid_index]\n",
    "        model = single_fold_lgb(X_train, y_train, X_valid, y_valid, params) ## get testing prediction and model  \n",
    "        oof_preds[valid_index] = model.predict(train_set.loc[valid_index,:],num_iteration=model.best_iteration)  # prediction of out of fold\n",
    "    print('Full Out-Of-Fold score : %9.6f'% (mean_squared_error(target, oof_preds) ** .5))\n",
    "    return oof_preds\n",
    "\n",
    "oof_preds = kfold_model_lgb(params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### So sánh với kết quả tuần trước, kết quả đạt tốt hơn với những feature mới, đạt 0.1185"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Các thuật toán thường được dùng khi xây dựng mô hình regression \n",
    "+ [Linear regression](https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/)\n",
    "+ [Lasso Regression](https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/)\n",
    "+ [Ridge Regression](https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/)\n",
    "+ [Elastic Net Regression](https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/)\n",
    "+ [Gradient Boosting regression](https://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_regression.html)\n",
    "+ [CatBoostRegressor](https://catboost.ai/docs/concepts/python-reference_catboostregressor.html)\n",
    "+ [Xgboost](https://xgboost.readthedocs.io/en/latest/parameter.html)\n",
    "+ [Lightgbm](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='2*'>2*. Lightgbm và Bayesian Optimization  </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id='2.1*'>2.1*. Giới thiệu tổng quan về Lightgbm  </a>\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bagging and Boosting "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/bagging-and-boosting.png\" alt=\"drawing\" width=\"500\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ **Bagging** và **Boosting** đều là thuật toán học giám sát dựa vào nguyên lý tập hợp (ensembling): Xây dựng thuật toán cuối cùng ( gọi là strong learner) dựa vào tổng hợp các thuạt toán nhỏ hơn ( gọi là weak learners)\n",
    "\n",
    "---\n",
    "\n",
    "+ **Bagging**: các thuật toán con được xây dưng một cách song song không phụ thuộc vào nhau. Dữ liệu huấn luyện của các mô hình con là dữ liệu bất kì trong dữ liệu huấn luyện gốc (lấy một cách bất kì theo cách lấy vật trong túi - **bagging**) \n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "+ **Boosting**: Xây dựng thuật toán mạnh dựa vào chuỗi (sequence) các thuật toán yếu và có cập nhật trọng số (update weight) vào dữ liệu. Mỗi thuật toán con có liên kết với thuật toán trước và sau đó, thúc đẩy độ chính xác hơn (**boosting**)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/boosting.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n",
    "**Hình mô tả nguyên lý thuật toán Bagging** - các tập dữ liệu được lấy một cách độc lập và mô hình con được huấn luyện độc lập với nhau"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/bagging.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n",
    "**Hình mô tả nguyên lý thuật toán Boosting** -Các mô hình con có mối quan hệ sequence với nhau, dữ liệu được update trọng số sau mỗi lần huấn luyện tập con "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/adaboost.png\" alt=\"drawing\" width=\"500\"/>\n",
    "\n",
    "**Ví dụ điển hình nguyên lý bagging: Random forest** là thuật toán dùng nguyên lý  bagging và sử dụng decision tree ở mỗi thuật toán con (weak learners)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/random_foret.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n",
    "**Ví dụ điển hình nguyên lý boosting : Adaboost** (Adaptive + Boosting) - là thuật toán dùng nguyên lý boosting và sử dụng weak learner là decision tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Tổng quan Adaboost and Gradient Boosting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Giống nhau: Đều dựa vào nguyên lý Boosting \n",
    "+ Khác nhau : Cách tạo ra thuật toán con (weak learners) trong các bước lặp "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adaboost = Adaptive + Boosting                        \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/adaboost_final.png\" alt=\"drawing\" width=\"600\"/>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Adaboost: Mẫu dữ liệu huấn luyện được thay đổi qua từng bước lặp , tăng trọng số ở mẫu đoán sai, giảm trọng số ở mẫu đoán đúng giúp mô hình con tiếp theo học tốt hơn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Adaboost: Mô hình huấn luyện cuối cùng được kết hợp từ những weak learners"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Boosting = Gradient Descent + Boosting "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hình mô tả nguyên lý hoạt động của Gradient Boosting \n",
    "\n",
    "<img src=\"./imgs/gradientboosting.png\" alt=\"drawing\" width=\"600\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Gradient Boosting : Không thay đổi mẫu huấn luyện (không có quá trình update trọng số  của mẫu dữ liệu huấn luyện). \n",
    "#### Định nghĩa hàm loss (để tính toán sai lệch), tại mỗi bước lặp sử dụng gradient descent để update mô hình weak learner. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./imgs/gradient boosting.png\" alt=\"drawing\" width=\"400\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lightgbm - Xgboost "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Đều là các thuật toán xây dựng trên nền tảng của Gradient Boosting.\n",
    "\n",
    "---------\n",
    "\n",
    "+ Khác biệt lớn nhất với thuật toán Gradient Boosting thông thường: Ở mỗi bước lặp, khi tối ưu hàm thất thoát, sử dụng thuật toán tối ưu bậc hai, e.g.: Newton, quasi-Newton (các thuật toán khác thường bậc 1) dẫn tới kết quả tối ưu tốt hơn. \n",
    "\n",
    "---------\n",
    "+ Điểm khác nhau của  Lightgbm - Xgboost  ở cách chia lá ở thuật toán weak learner: Thuật toán cây của Lightgbm phát triển cây theo chiều dọc trong khi thuật toán Xgboost phát triển cây theo chiều ngang\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "![](./imgs/xgboost-lightgbm.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Các tham số quan trọng của Lightgbm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "+ num_leaves: số lượng lá cần sử dụng trong thuật toán. num_leaves lớn sẽ cải thiện độ chính xác, nhưng num_leaves lớn sẽ gây nên hiện tượng overfitting\n",
    "+ min_child_samples: Số lượng mẫu (dữ liệu) tối thiểu để nhóm thành một lá. Tham số có thể hỗ trợ rất nhiều việc chống overfitting: min_child_samples lớn trên mỗi lá sẽ làm giảm hiện tượng overfitting (nhưng đồng thời có thể dẫn đến underfitting nếu số min_child_samples quá lớn).\n",
    "+ max_depth: Tham số này nhằm kiểm soát độ sâu của lá. Tham số max_depth thấp tương ứng với cây không bị chia quá phức tạp, giúp cho mô hình chống overfitting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tham số thay đổi để tăng độ chính xác (theo documentation của [lightgbm](https://lightgbm.readthedocs.io/en/latest/Parameters-Tuning.html) ) : \n",
    "+ max_bin: Sử dụng  max_bin lớn có thể tăng độ chính xác (nhưng tốc độ huấn luyện có thể chậm hơn)\n",
    "+ learning_rate: Sử dụng learning_rate nhỏ cùng số bước lặp (num_iterations) lớn có thể tăng độ chính xác\n",
    "+ Sử dụng num_leaves lớn có thể tăng độ chính xác (nhưng cũng có thể gây nên overfitting)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Một số phương án hiệu chỉnh tham số để  tránh hiện tượng overfitting \n",
    "+ Sử dụng max_bin nhỏ \n",
    "+ Sử dụng num_leaves nhỏ \n",
    "+ bagging_fraction: cho phép kiểm soát việc lấy mẫu con dữ liệu khi huấn luyện mô hình. Ví dụ bagging_fraction = 0.5 nghĩa là mỗi bước lặp sẽ lấy 50% dữ liệu để tiến hành chia lá. bagging_fraction nhỏ có thể giúp tránh được overfitting.\n",
    "+ feature_fraction: kiểm soát việc lấy mẫu con ở các trường dữ liệu khi tiến hành huấn luyện. feature_fraction nhỏ hơn có thể giúp mô hình giảm overfitting.\n",
    "+ Sử dụng tham số lambda_l1, lambda_l2 và min_gain_to_split để chính quy hoá (regularization).\n",
    "+ Sử dụng max_depth để kiểm soát độ sâu của nhánh khi chia lá."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "## the fucntion for hidden print\n",
    "class HiddenPrints:\n",
    "    '''To hide all the printouts'''\n",
    "\n",
    "    def __enter__(self):\n",
    "        self._original_stdout = sys.stdout\n",
    "        sys.stdout = open(os.devnull, 'w')\n",
    "\n",
    "    def __exit__(self, exc_type, exc_val, exc_tb):\n",
    "        sys.stdout.close()\n",
    "        sys.stdout = self._original_stdout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id='2.2*'>2.2*. Giới thiệu tổng quan về tối ưu Bayes ([Bayesian Optimization](https://thetalog.com/thetaflow/bayesian-optimization-visualizer/?fbclid=IwAR1soc93sJ-b5nAC5dxxJ95Im3YrA3K3SXTNIoDPzZ3SBag0-jyj4zSHl88))  </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tối ưu hóa Bayes là một chiến lược thiết kế tuần tự để tối ưu hóa toàn cục của các hàm số mà chúng ta không hiểu rõ tính chất (black-box-function)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sử dụng gói thư viện BayesianOptimization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bayes_opt import BayesianOptimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ví dụ cơ bản gói thư viện BayesianOptimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Giả sử chúng ta có black box function như ở dưới. \n",
    "\n",
    "$$ f(x,y) = -x^2 -(y-1)^2 +1 $$\n",
    "\n",
    "Ở đây chúng ta hiểu rõ tính chất hàm \\(f\\). Nhưng ở ví dụ này chúng ta giả sử không biết rõ về hàm \\(f\\) và chúng ta muốn tìm \n",
    "\n",
    "$$Max_{x,y} f(x,y)$$\n",
    "\n",
    " với $x\\in (2,4)$, và $y \\in (-3,3)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Chúng ta định nghĩa black box function và grid_search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def black_box_function(x, y):\n",
    "    \"\"\"Function with unknown internals we wish to maximize.\n",
    "\n",
    "    This is just serving as an example, for all intents and\n",
    "    purposes think of the internals of this function, i.e.: the process\n",
    "    which generates its output values, as unknown.\n",
    "    \"\"\"\n",
    "    return -x ** 2 - (y - 1) ** 2 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "grid_search = {'x': (2, 4), 'y': (-3, 3)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sử dụng tối ưu Bayes nhờ thư viện BayesianOptimization. Hàm BayesianOptimization gồm 2 tham số quan trọng: \n",
    "   +  f: Hàm black-box\n",
    "   +  pbounds : Không gian cần tìm kiếm "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = BayesianOptimization(\n",
    "    f=black_box_function,\n",
    "    pbounds=grid_search,\n",
    "    random_state=1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Để tìm giá trị max của hàm black-box trong khảng giá trị $(2,4)\\times (-3,3)$ chúng ta  sử dụng lệnh "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|   iter    |  target   |     x     |     y     |\n",
      "-------------------------------------------------\n",
      "| \u001b[0m 1       \u001b[0m | \u001b[0m-7.135   \u001b[0m | \u001b[0m 2.834   \u001b[0m | \u001b[0m 1.322   \u001b[0m |\n",
      "| \u001b[0m 2       \u001b[0m | \u001b[0m-7.78    \u001b[0m | \u001b[0m 2.0     \u001b[0m | \u001b[0m-1.186   \u001b[0m |\n",
      "| \u001b[0m 3       \u001b[0m | \u001b[0m-16.13   \u001b[0m | \u001b[0m 2.294   \u001b[0m | \u001b[0m-2.446   \u001b[0m |\n",
      "| \u001b[0m 4       \u001b[0m | \u001b[0m-8.341   \u001b[0m | \u001b[0m 2.373   \u001b[0m | \u001b[0m-0.9266  \u001b[0m |\n",
      "| \u001b[0m 5       \u001b[0m | \u001b[0m-7.392   \u001b[0m | \u001b[0m 2.794   \u001b[0m | \u001b[0m 0.2329  \u001b[0m |\n",
      "| \u001b[0m 6       \u001b[0m | \u001b[0m-15.02   \u001b[0m | \u001b[0m 4.0     \u001b[0m | \u001b[0m 0.8624  \u001b[0m |\n",
      "| \u001b[95m 7       \u001b[0m | \u001b[95m-3.799   \u001b[0m | \u001b[95m 2.184   \u001b[0m | \u001b[95m 0.83    \u001b[0m |\n",
      "| \u001b[0m 8       \u001b[0m | \u001b[0m-5.343   \u001b[0m | \u001b[0m 2.0     \u001b[0m | \u001b[0m 2.531   \u001b[0m |\n",
      "| \u001b[0m 9       \u001b[0m | \u001b[0m-14.29   \u001b[0m | \u001b[0m 3.36    \u001b[0m | \u001b[0m 3.0     \u001b[0m |\n",
      "| \u001b[95m 10      \u001b[0m | \u001b[95m-3.363   \u001b[0m | \u001b[95m 2.0     \u001b[0m | \u001b[95m 1.603   \u001b[0m |\n",
      "=================================================\n"
     ]
    }
   ],
   "source": [
    "optimizer.maximize(init_points=5,n_iter=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Giá trị tốt nhất sử dụng tối ưu bayes đạt: $-3.363$ khi $x=2$, và $y = 1.6$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id='2.2*'>2.2*. Áp dụng tối ưu bayes để tìm kiếm tham số tối ưu cho thuật toán lightgbm  </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ở đây chúng ta sẽ áp dụng \"giá trị tốt nhất trên cross-validation của thuật toán lightgbm\" như hàm  black-box \n",
    "#### Grid_seach là  không gian các tham số muốn tối ưu cho thuật toán lightgbm\n",
    "\n",
    "Cụ thể: \n",
    "+ hàm black-box sẽ là hàm số \"kfold_model_gbm\" ở mục trước \n",
    "+ grid_seach sẽ bao gồm \n",
    "    + 'feature_fraction': (0.1, 0.9),\n",
    "    + 'max_depth': (4, 8),\n",
    "    + 'num_leaves' : (4,16),\n",
    "    + 'max_bin' : (50,200),\n",
    "    + 'learning_rate' : (0.01, 0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bayes_parameter_opt_lgb(train_set, target, init_round=15, opt_round=25 ,output_process=False):\n",
    "    # prepare data\n",
    "    kf = KFold(n_splits=5, shuffle = True, random_state = 2020)\n",
    "    def single_fold_lgb(train_X, train_y, valid_X, valid_y, params): \n",
    "        \"\"\"\n",
    "        args: \n",
    "        + train   : training dataset \n",
    "        + train_y : target of training dataset\n",
    "        + valid   : validation dataset\n",
    "        + valid_y : target of validation dataset\n",
    "        +params   : important parameters of lgb model \n",
    "        output:\n",
    "        model after training with training dataset \n",
    "        \"\"\"\n",
    "        lgtrain = lgb.Dataset(train_X, label=train_y)\n",
    "        lgval = lgb.Dataset(valid_X, label=valid_y)\n",
    "        evals_result = {}\n",
    "        model = lgb.train(params, lgtrain, 2000, valid_sets=[lgval], early_stopping_rounds=100, verbose_eval=200, evals_result=evals_result)\n",
    "        return model\n",
    "    \n",
    "    grid_search =  {'feature_fraction': (0.1, 0.9),\n",
    "                    'max_depth': (4, 8),\n",
    "                    'num_leaves' : (4,16),\n",
    "                    'max_bin' : (50,200),\n",
    "                    'learning_rate' : (0.01, 0.1)}\n",
    "    \n",
    "    def kfold_model(feature_fraction, max_depth, num_leaves, max_bin,learning_rate):\n",
    "        \"\"\"\n",
    "    \n",
    "        args: \n",
    "        + all parameters we want to search for the lightgbm model. Here, we want to tune : \n",
    "        feature_fraction, max_depth, num_leaves, max_bin, learning_rate.\n",
    "        ouput:\n",
    "        The value we want to get maximum when we apply baysian optimization. \n",
    "        Here we use the MNLSE metric, then the output should be -MNLSE of the validation dataset. \n",
    "    \n",
    "        \"\"\"\n",
    "        oof_preds = np.zeros(train_set.shape[0]) ## out of fold prediction \n",
    "        ## define parameters\n",
    "        params = {\n",
    "        \"objective\" : \"regression\",\n",
    "        \"metric\" : \"rmse\",\n",
    "        \"max_bin\" : 100,\n",
    "        \"num_leaves\" : 8,\n",
    "        \"learning_rate\" : 0.01,\n",
    "        \"bagging_fraction\" : 0.75,\n",
    "        \"bagging_freq\" : 5,\n",
    "        \"bagging_seed\" : 7,\n",
    "        \"feature_fraction\" :0.2,\n",
    "        \"feature_fraction_seed\": 7,       \n",
    "        \"verbosity\" : -1 }\n",
    "        # update parameter search \n",
    "        \n",
    "        params['feature_fraction'] = max(min(feature_fraction, 1), 0)\n",
    "        params['max_depth'] = int(round(max_depth))\n",
    "        params['num_leaves'] = int(round(num_leaves))\n",
    "        params['max_bin'] = int(round(max_bin))\n",
    "        params['learning_rate'] = learning_rate\n",
    "        \n",
    "        for train_index, valid_index in kf.split(train_set):\n",
    "            X_train , X_valid  = train_set.loc[train_index,:], train_set.loc[valid_index,:]\n",
    "            y_train, y_valid = target[train_index], target[valid_index]\n",
    "            with HiddenPrints():\n",
    "                model = single_fold_lgb(X_train, y_train, X_valid, y_valid, params) ## get testing prediction and model  \n",
    "            oof_preds[valid_index] = model.predict(train_set.loc[valid_index,:],num_iteration=model.best_iteration)  # prediction of out of fold\n",
    "        score = -(mean_squared_error(target, oof_preds) ** .5)\n",
    "        return score\n",
    "    \n",
    "    optim = BayesianOptimization(kfold_model, grid_search , random_state=0)\n",
    "    # optimize\n",
    "    optim.maximize(init_points=init_round, n_iter=opt_round,acq='ei')\n",
    "    # output optimization process\n",
    "    if output_process == True: optim.points_to_csv(\"./data/bayes_opt_result.csv\")\n",
    "\n",
    "    return optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|   iter    |  target   | featur... | learni... |  max_bin  | max_depth | num_le... |\n",
      "-------------------------------------------------------------------------------------\n",
      "| \u001b[0m 1       \u001b[0m | \u001b[0m-0.1203  \u001b[0m | \u001b[0m 0.5391  \u001b[0m | \u001b[0m 0.07437 \u001b[0m | \u001b[0m 140.4   \u001b[0m | \u001b[0m 6.18    \u001b[0m | \u001b[0m 9.084   \u001b[0m |\n",
      "| \u001b[0m 2       \u001b[0m | \u001b[0m-0.1205  \u001b[0m | \u001b[0m 0.6167  \u001b[0m | \u001b[0m 0.04938 \u001b[0m | \u001b[0m 183.8   \u001b[0m | \u001b[0m 7.855   \u001b[0m | \u001b[0m 8.601   \u001b[0m |\n",
      "| \u001b[95m 3       \u001b[0m | \u001b[95m-0.1193  \u001b[0m | \u001b[95m 0.7334  \u001b[0m | \u001b[95m 0.0576  \u001b[0m | \u001b[95m 135.2   \u001b[0m | \u001b[95m 7.702   \u001b[0m | \u001b[95m 4.852   \u001b[0m |\n",
      "| \u001b[95m 4       \u001b[0m | \u001b[95m-0.1183  \u001b[0m | \u001b[95m 0.1697  \u001b[0m | \u001b[95m 0.01182 \u001b[0m | \u001b[95m 174.9   \u001b[0m | \u001b[95m 7.113   \u001b[0m | \u001b[95m 14.44   \u001b[0m |\n",
      "| \u001b[0m 5       \u001b[0m | \u001b[0m-0.1205  \u001b[0m | \u001b[0m 0.8829  \u001b[0m | \u001b[0m 0.08192 \u001b[0m | \u001b[0m 119.2   \u001b[0m | \u001b[0m 7.122   \u001b[0m | \u001b[0m 5.419   \u001b[0m |\n",
      "| \u001b[0m 6       \u001b[0m | \u001b[0m-0.1186  \u001b[0m | \u001b[0m 0.6119  \u001b[0m | \u001b[0m 0.0229  \u001b[0m | \u001b[0m 191.7   \u001b[0m | \u001b[0m 6.087   \u001b[0m | \u001b[0m 8.976   \u001b[0m |\n",
      "| \u001b[0m 7       \u001b[0m | \u001b[0m-0.1193  \u001b[0m | \u001b[0m 0.3116  \u001b[0m | \u001b[0m 0.07968 \u001b[0m | \u001b[0m 118.4   \u001b[0m | \u001b[0m 6.274   \u001b[0m | \u001b[0m 4.225   \u001b[0m |\n",
      "| \u001b[0m 8       \u001b[0m | \u001b[0m-0.1197  \u001b[0m | \u001b[0m 0.5941  \u001b[0m | \u001b[0m 0.06509 \u001b[0m | \u001b[0m 142.5   \u001b[0m | \u001b[0m 7.775   \u001b[0m | \u001b[0m 12.18   \u001b[0m |\n",
      "| \u001b[95m 9       \u001b[0m | \u001b[95m-0.1182  \u001b[0m | \u001b[95m 0.3876  \u001b[0m | \u001b[95m 0.04933 \u001b[0m | \u001b[95m 154.6   \u001b[0m | \u001b[95m 4.241   \u001b[0m | \u001b[95m 12.0    \u001b[0m |\n",
      "| \u001b[0m 10      \u001b[0m | \u001b[0m-0.1197  \u001b[0m | \u001b[0m 0.6365  \u001b[0m | \u001b[0m 0.02893 \u001b[0m | \u001b[0m 69.34   \u001b[0m | \u001b[0m 5.262   \u001b[0m | \u001b[0m 8.365   \u001b[0m |\n",
      "| \u001b[0m 11      \u001b[0m | \u001b[0m-0.119   \u001b[0m | \u001b[0m 0.5562  \u001b[0m | \u001b[0m 0.04947 \u001b[0m | \u001b[0m 198.3   \u001b[0m | \u001b[0m 4.408   \u001b[0m | \u001b[0m 6.507   \u001b[0m |\n",
      "| \u001b[0m 12      \u001b[0m | \u001b[0m-0.1189  \u001b[0m | \u001b[0m 0.229   \u001b[0m | \u001b[0m 0.06878 \u001b[0m | \u001b[0m 87.99   \u001b[0m | \u001b[0m 5.865   \u001b[0m | \u001b[0m 6.933   \u001b[0m |\n",
      "| \u001b[95m 13      \u001b[0m | \u001b[95m-0.1172  \u001b[0m | \u001b[95m 0.2272  \u001b[0m | \u001b[95m 0.01993 \u001b[0m | \u001b[95m 148.4   \u001b[0m | \u001b[95m 4.553   \u001b[0m | \u001b[95m 6.359   \u001b[0m |\n",
      "| \u001b[0m 14      \u001b[0m | \u001b[0m-0.1205  \u001b[0m | \u001b[0m 0.395   \u001b[0m | \u001b[0m 0.08389 \u001b[0m | \u001b[0m 64.57   \u001b[0m | \u001b[0m 7.352   \u001b[0m | \u001b[0m 5.153   \u001b[0m |\n",
      "| \u001b[0m 15      \u001b[0m | \u001b[0m-0.1226  \u001b[0m | \u001b[0m 0.8812  \u001b[0m | \u001b[0m 0.05218 \u001b[0m | \u001b[0m 196.5   \u001b[0m | \u001b[0m 6.419   \u001b[0m | \u001b[0m 12.87   \u001b[0m |\n",
      "| \u001b[0m 16      \u001b[0m | \u001b[0m-0.1195  \u001b[0m | \u001b[0m 0.4463  \u001b[0m | \u001b[0m 0.05003 \u001b[0m | \u001b[0m 51.39   \u001b[0m | \u001b[0m 7.063   \u001b[0m | \u001b[0m 13.16   \u001b[0m |\n",
      "| \u001b[0m 17      \u001b[0m | \u001b[0m-0.1197  \u001b[0m | \u001b[0m 0.6402  \u001b[0m | \u001b[0m 0.0301  \u001b[0m | \u001b[0m 161.7   \u001b[0m | \u001b[0m 5.956   \u001b[0m | \u001b[0m 8.657   \u001b[0m |\n",
      "| \u001b[0m 18      \u001b[0m | \u001b[0m-0.1188  \u001b[0m | \u001b[0m 0.1334  \u001b[0m | \u001b[0m 0.01163 \u001b[0m | \u001b[0m 65.88   \u001b[0m | \u001b[0m 4.089   \u001b[0m | \u001b[0m 4.93    \u001b[0m |\n",
      "| \u001b[0m 19      \u001b[0m | \u001b[0m-0.1197  \u001b[0m | \u001b[0m 0.6918  \u001b[0m | \u001b[0m 0.02413 \u001b[0m | \u001b[0m 186.8   \u001b[0m | \u001b[0m 7.079   \u001b[0m | \u001b[0m 5.013   \u001b[0m |\n",
      "| \u001b[0m 20      \u001b[0m | \u001b[0m-0.12    \u001b[0m | \u001b[0m 0.6055  \u001b[0m | \u001b[0m 0.04228 \u001b[0m | \u001b[0m 95.49   \u001b[0m | \u001b[0m 7.117   \u001b[0m | \u001b[0m 10.9    \u001b[0m |\n",
      "| \u001b[0m 21      \u001b[0m | \u001b[0m-0.1206  \u001b[0m | \u001b[0m 0.7991  \u001b[0m | \u001b[0m 0.04251 \u001b[0m | \u001b[0m 98.71   \u001b[0m | \u001b[0m 6.459   \u001b[0m | \u001b[0m 9.037   \u001b[0m |\n",
      "| \u001b[0m 22      \u001b[0m | \u001b[0m-0.1205  \u001b[0m | \u001b[0m 0.6413  \u001b[0m | \u001b[0m 0.09038 \u001b[0m | \u001b[0m 199.6   \u001b[0m | \u001b[0m 6.36    \u001b[0m | \u001b[0m 6.498   \u001b[0m |\n",
      "| \u001b[0m 23      \u001b[0m | \u001b[0m-0.1196  \u001b[0m | \u001b[0m 0.4364  \u001b[0m | \u001b[0m 0.08441 \u001b[0m | \u001b[0m 174.6   \u001b[0m | \u001b[0m 7.764   \u001b[0m | \u001b[0m 4.539   \u001b[0m |\n",
      "| \u001b[0m 24      \u001b[0m | \u001b[0m-0.122   \u001b[0m | \u001b[0m 0.1045  \u001b[0m | \u001b[0m 0.07444 \u001b[0m | \u001b[0m 121.3   \u001b[0m | \u001b[0m 6.295   \u001b[0m | \u001b[0m 11.56   \u001b[0m |\n",
      "| \u001b[0m 25      \u001b[0m | \u001b[0m-0.1226  \u001b[0m | \u001b[0m 0.5847  \u001b[0m | \u001b[0m 0.09103 \u001b[0m | \u001b[0m 79.44   \u001b[0m | \u001b[0m 6.035   \u001b[0m | \u001b[0m 14.92   \u001b[0m |\n",
      "| \u001b[0m 26      \u001b[0m | \u001b[0m-0.1213  \u001b[0m | \u001b[0m 0.4263  \u001b[0m | \u001b[0m 0.0661  \u001b[0m | \u001b[0m 163.6   \u001b[0m | \u001b[0m 5.326   \u001b[0m | \u001b[0m 15.08   \u001b[0m |\n",
      "| \u001b[0m 27      \u001b[0m | \u001b[0m-0.1194  \u001b[0m | \u001b[0m 0.6634  \u001b[0m | \u001b[0m 0.09611 \u001b[0m | \u001b[0m 135.3   \u001b[0m | \u001b[0m 7.707   \u001b[0m | \u001b[0m 4.668   \u001b[0m |\n",
      "| \u001b[0m 28      \u001b[0m | \u001b[0m-0.1202  \u001b[0m | \u001b[0m 0.3235  \u001b[0m | \u001b[0m 0.09152 \u001b[0m | \u001b[0m 148.7   \u001b[0m | \u001b[0m 4.065   \u001b[0m | \u001b[0m 6.887   \u001b[0m |\n",
      "| \u001b[0m 29      \u001b[0m | \u001b[0m-0.1186  \u001b[0m | \u001b[0m 0.2866  \u001b[0m | \u001b[0m 0.08075 \u001b[0m | \u001b[0m 148.4   \u001b[0m | \u001b[0m 4.725   \u001b[0m | \u001b[0m 6.974   \u001b[0m |\n",
      "| \u001b[0m 30      \u001b[0m | \u001b[0m-0.1198  \u001b[0m | \u001b[0m 0.8102  \u001b[0m | \u001b[0m 0.04195 \u001b[0m | \u001b[0m 148.3   \u001b[0m | \u001b[0m 4.993   \u001b[0m | \u001b[0m 5.95    \u001b[0m |\n",
      "| \u001b[0m 31      \u001b[0m | \u001b[0m-0.119   \u001b[0m | \u001b[0m 0.2948  \u001b[0m | \u001b[0m 0.08162 \u001b[0m | \u001b[0m 172.7   \u001b[0m | \u001b[0m 4.72    \u001b[0m | \u001b[0m 6.222   \u001b[0m |\n",
      "| \u001b[0m 32      \u001b[0m | \u001b[0m-0.1203  \u001b[0m | \u001b[0m 0.4466  \u001b[0m | \u001b[0m 0.07168 \u001b[0m | \u001b[0m 147.9   \u001b[0m | \u001b[0m 4.17    \u001b[0m | \u001b[0m 6.531   \u001b[0m |\n",
      "| \u001b[0m 33      \u001b[0m | \u001b[0m-0.1231  \u001b[0m | \u001b[0m 0.4475  \u001b[0m | \u001b[0m 0.0892  \u001b[0m | \u001b[0m 174.9   \u001b[0m | \u001b[0m 7.496   \u001b[0m | \u001b[0m 14.82   \u001b[0m |\n",
      "| \u001b[0m 34      \u001b[0m | \u001b[0m-0.1213  \u001b[0m | \u001b[0m 0.1563  \u001b[0m | \u001b[0m 0.08802 \u001b[0m | \u001b[0m 148.2   \u001b[0m | \u001b[0m 4.781   \u001b[0m | \u001b[0m 6.91    \u001b[0m |\n",
      "| \u001b[0m 35      \u001b[0m | \u001b[0m-0.1186  \u001b[0m | \u001b[0m 0.2417  \u001b[0m | \u001b[0m 0.05013 \u001b[0m | \u001b[0m 91.39   \u001b[0m | \u001b[0m 5.228   \u001b[0m | \u001b[0m 5.256   \u001b[0m |\n",
      "| \u001b[0m 36      \u001b[0m | \u001b[0m-0.1185  \u001b[0m | \u001b[0m 0.2747  \u001b[0m | \u001b[0m 0.08604 \u001b[0m | \u001b[0m 166.8   \u001b[0m | \u001b[0m 5.66    \u001b[0m | \u001b[0m 4.26    \u001b[0m |\n",
      "| \u001b[0m 37      \u001b[0m | \u001b[0m-0.1201  \u001b[0m | \u001b[0m 0.2698  \u001b[0m | \u001b[0m 0.07564 \u001b[0m | \u001b[0m 185.3   \u001b[0m | \u001b[0m 6.159   \u001b[0m | \u001b[0m 11.15   \u001b[0m |\n",
      "| \u001b[0m 38      \u001b[0m | \u001b[0m-0.1182  \u001b[0m | \u001b[0m 0.3895  \u001b[0m | \u001b[0m 0.02202 \u001b[0m | \u001b[0m 167.0   \u001b[0m | \u001b[0m 5.546   \u001b[0m | \u001b[0m 4.154   \u001b[0m |\n",
      "| \u001b[0m 39      \u001b[0m | \u001b[0m-0.1182  \u001b[0m | \u001b[0m 0.5758  \u001b[0m | \u001b[0m 0.01678 \u001b[0m | \u001b[0m 92.55   \u001b[0m | \u001b[0m 4.357   \u001b[0m | \u001b[0m 6.424   \u001b[0m |\n",
      "| \u001b[0m 40      \u001b[0m | \u001b[0m-0.1194  \u001b[0m | \u001b[0m 0.2549  \u001b[0m | \u001b[0m 0.0813  \u001b[0m | \u001b[0m 167.1   \u001b[0m | \u001b[0m 5.379   \u001b[0m | \u001b[0m 4.011   \u001b[0m |\n",
      "=====================================================================================\n"
     ]
    }
   ],
   "source": [
    "optim = bayes_parameter_opt_lgb(train_set, target, init_round=15, opt_round=25 ,output_process=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### In ra các các tham số  tối ưu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'target': -0.11719952351472296, 'params': {'feature_fraction': 0.22717566691641577, 'learning_rate': 0.019933762704787465, 'max_bin': 148.44943841979102, 'max_depth': 4.552731805394455, 'num_leaves': 6.358988340160642}}\n"
     ]
    }
   ],
   "source": [
    "print(optim.max)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kiểm tra lại kết quả\n",
    "\n",
    "#### Sử dụng lại Kfold và tham số của mô hình tuần 1. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "kf = KFold(n_splits=5, shuffle = True, random_state = 2020)\n",
    "params = {\n",
    "        \"objective\" : \"regression\",\n",
    "        \"metric\" : \"rmse\",\n",
    "        \"max_bin\" : 148,\n",
    "        \"num_leaves\" : 8,\n",
    "        \"learning_rate\" : 0.02,\n",
    "        \"bagging_fraction\" : 0.75,\n",
    "        \"bagging_freq\" : 5,\n",
    "        \"bagging_seed\" : 7,\n",
    "        \"feature_fraction\" :0.5,\n",
    "        \"feature_fraction_seed\": 7,       \n",
    "        \"verbosity\" : -1 }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Update tham số cần tối ưu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "params['feature_fraction'] = max(min(optim.max[\"params\"][\"feature_fraction\"], 1), 0)\n",
    "params['max_depth'] = int(round(optim.max[\"params\"][\"max_depth\"]))\n",
    "params['num_leaves'] = int(round(optim.max[\"params\"][\"num_leaves\"]))\n",
    "params['max_bin'] = int(round(optim.max[\"params\"][\"max_bin\"]))\n",
    "params['learning_rate'] = optim.max[\"params\"][\"learning_rate\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Chạy lại mô hình dựa trên Kfold và tham số ở trên "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.12988\n",
      "[400]\tvalid_0's rmse: 0.11911\n",
      "[600]\tvalid_0's rmse: 0.117124\n",
      "[800]\tvalid_0's rmse: 0.116261\n",
      "[1000]\tvalid_0's rmse: 0.115754\n",
      "Early stopping, best iteration is:\n",
      "[952]\tvalid_0's rmse: 0.115673\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.140574\n",
      "[400]\tvalid_0's rmse: 0.122653\n",
      "[600]\tvalid_0's rmse: 0.119384\n",
      "[800]\tvalid_0's rmse: 0.117754\n",
      "[1000]\tvalid_0's rmse: 0.117179\n",
      "[1200]\tvalid_0's rmse: 0.116663\n",
      "[1400]\tvalid_0's rmse: 0.116203\n",
      "Early stopping, best iteration is:\n",
      "[1489]\tvalid_0's rmse: 0.116041\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.134852\n",
      "[400]\tvalid_0's rmse: 0.120704\n",
      "[600]\tvalid_0's rmse: 0.117466\n",
      "[800]\tvalid_0's rmse: 0.115958\n",
      "[1000]\tvalid_0's rmse: 0.115579\n",
      "Early stopping, best iteration is:\n",
      "[1045]\tvalid_0's rmse: 0.115488\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.135146\n",
      "[400]\tvalid_0's rmse: 0.121824\n",
      "[600]\tvalid_0's rmse: 0.120142\n",
      "[800]\tvalid_0's rmse: 0.120019\n",
      "Early stopping, best iteration is:\n",
      "[715]\tvalid_0's rmse: 0.119922\n",
      "Training until validation scores don't improve for 100 rounds\n",
      "[200]\tvalid_0's rmse: 0.144218\n",
      "[400]\tvalid_0's rmse: 0.125357\n",
      "[600]\tvalid_0's rmse: 0.121578\n",
      "[800]\tvalid_0's rmse: 0.120446\n",
      "[1000]\tvalid_0's rmse: 0.119598\n",
      "[1200]\tvalid_0's rmse: 0.119024\n",
      "Early stopping, best iteration is:\n",
      "[1110]\tvalid_0's rmse: 0.118819\n",
      "Full Out-Of-Fold score :  0.117200\n"
     ]
    }
   ],
   "source": [
    "oof_preds = kfold_model_lgb(params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sau khi finetune parameter, kết quả trên tập validation đã có những cải thiện đáng kể"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# QA"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}