08_AutoSklearn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下ごしらえ"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits\n",
    "digits=load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data_train,data_test,target_train,target_test=train_test_split(\n",
    "    digits.data,digits.target,train_size=0.95)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 今回のお題"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from autosklearn.classification import AutoSklearnClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearnのAPIは基本的には共通。モデル間の比較もしやすいはず。  \n",
    "当然、全部まとめてやりたいって考えが出てくる。  \n",
    "説明は[PyLadysTokyoのスライド](https://t.co/jiFNTz60cy)が詳しい。  \n",
    "中身は私もよくわかっていない。。。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "clf = AutoSklearnClassifier(time_left_for_this_task=360, per_run_time_limit=36)\n",
    "# 時間は短くしてあります。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "いつもどおりfit。  \n",
    "outputがいっぱい出る。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:53:09,780:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Starting to create dummy predictions.\n",
      "[INFO] [2016-05-21 11:53:09,975:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Finished creating dummy prediction 1/2.\n",
      "[INFO] [2016-05-21 11:53:10,171:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Finished creating dummy prediction 2/2.\n",
      "[INFO] [2016-05-21 11:53:14,572:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Start Ensemble with 355.19sec time left\n",
      "[INFO] [2016-05-21 11:53:14,591:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Start SMAC with 355.17sec time left\n",
      "[ERROR] [2016-05-21 11:53:14,773:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:53:14,785:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Training default configurations on a subset of 563/1707 data points.\n",
      "[INFO] [2016-05-21 11:53:14,969:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.003355\n",
      "\tMembers: [0]\n",
      "\tWeights: [ 1.]\n",
      "\tIdentifiers: (1, 2)[INFO] [2016-05-21 11:53:14,973:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 3 on SUBSET with size 563 and time limit 18s.\n",
      "\n",
      "[INFO] [2016-05-21 11:53:14,974:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: random_forest\n",
      "  classifier:random_forest:bootstrap, Value: True\n",
      "  classifier:random_forest:criterion, Value: gini\n",
      "  classifier:random_forest:max_depth, Constant: None\n",
      "  classifier:random_forest:max_features, Value: 1.0\n",
      "  classifier:random_forest:max_leaf_nodes, Constant: None\n",
      "  classifier:random_forest:min_samples_leaf, Value: 1\n",
      "  classifier:random_forest:min_samples_split, Value: 2\n",
      "  classifier:random_forest:min_weight_fraction_leaf, Constant: 0.0\n",
      "  classifier:random_forest:n_estimators, Constant: 100\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.01\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: no_preprocessing\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n",
      "[INFO] [2016-05-21 11:53:14,976:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.003355\n",
      "[INFO] [2016-05-21 11:53:14,978:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 0.207753 seconds\n",
      "[INFO] [2016-05-21 11:53:14,981:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (2)!.\n",
      "[INFO] [2016-05-21 11:53:14,988:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (2)!\n",
      "[INFO] [2016-05-21 11:53:16,042:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 3. configuration on SUBSET. Duration 1.017266; loss 0.047365; status 1; additional run info: ;duration: 1.0172662734985352;num_run:00003 \n",
      "[INFO] [2016-05-21 11:53:16,164:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000\n",
      "[INFO] [2016-05-21 11:53:16,168:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 4 on SUBSET with size 563 and time limit 18s.\n",
      "[INFO] [2016-05-21 11:53:16,169:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: sgd\n",
      "  classifier:sgd:alpha, Value: 0.0001\n",
      "  classifier:sgd:average, Value: True\n",
      "  classifier:sgd:eta0, Value: 0.01\n",
      "  classifier:sgd:fit_intercept, Constant: True\n",
      "  classifier:sgd:learning_rate, Value: optimal\n",
      "  classifier:sgd:loss, Value: hinge\n",
      "  classifier:sgd:n_iter, Value: 5\n",
      "  classifier:sgd:penalty, Value: l2\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.1\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: no_preprocessing\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n",
      "[INFO] [2016-05-21 11:53:16,580:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 4. configuration on SUBSET. Duration 0.218824; loss 0.063154; status 1; additional run info: ;duration: 0.2188243865966797;num_run:00004 \n",
      "[INFO] [2016-05-21 11:53:16,582:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000\n",
      "[INFO] [2016-05-21 11:53:16,586:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 5 on SUBSET with size 563 and time limit 18s.\n",
      "[INFO] [2016-05-21 11:53:16,588:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: extra_trees\n",
      "  classifier:extra_trees:bootstrap, Value: False\n",
      "  classifier:extra_trees:criterion, Value: gini\n",
      "  classifier:extra_trees:max_depth, Constant: None\n",
      "  classifier:extra_trees:max_features, Value: 1.0\n",
      "  classifier:extra_trees:min_samples_leaf, Value: 5\n",
      "  classifier:extra_trees:min_samples_split, Value: 5\n",
      "  classifier:extra_trees:min_weight_fraction_leaf, Constant: 0.0\n",
      "  classifier:extra_trees:n_estimators, Constant: 100\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.1\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: select_percentile_classification\n",
      "  preprocessor:select_percentile_classification:percentile, Value: 2.0\n",
      "  preprocessor:select_percentile_classification:score_func, Value: chi2\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n",
      "[ERROR] [2016-05-21 11:53:17,167:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:53:18,769:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 5. configuration on SUBSET. Duration 1.807020; loss 0.708506; status 1; additional run info: ;duration: 1.8070201873779297;num_run:00005 \n",
      "[INFO] [2016-05-21 11:53:18,771:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000\n",
      "[INFO] [2016-05-21 11:53:18,774:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 6 on SUBSET with size 563 and time limit 18s.\n",
      "[INFO] [2016-05-21 11:53:18,776:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: gaussian_nb\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.1\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: no_preprocessing\n",
      "  rescaling:__choice__, Value: standardize\n",
      "\n",
      "[INFO] [2016-05-21 11:53:18,967:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.952635 1: 0.952635 2: 0.952635 3: 0.952635 4: 0.952635 5: 0.952635 6: 0.952635 7: 0.956582 8: 0.956582 9: 0.956582 10: 0.956582 11: 0.958555 12: 0.962502 13: 0.962502 14: 0.962502 15: 0.958555 16: 0.958555 17: 0.958555 18: 0.958555 19: 0.960529 20: 0.960529 21: 0.960529 22: 0.960529 23: 0.962502 24: 0.962502 25: 0.962502 26: 0.962502 27: 0.962502 28: 0.962502 29: 0.962502 30: 0.962502 31: 0.960529 32: 0.962502 33: 0.962502 34: 0.962502 35: 0.962502 36: 0.962502 37: 0.962502 38: 0.962502 39: 0.962502 40: 0.962502 41: 0.962502 42: 0.962502 43: 0.962502 44: 0.962502 45: 0.962502 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502\n",
      "\tMembers: [1, 1, 1, 1, 1, 1, 0, 2, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 2, 1, 0, 1, 1, 1, 1, 1, 0, 2, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 2, 1, 1]\n",
      "\tWeights: [ 0.14  0.78  0.08]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 4)\n",
      "[INFO] [2016-05-21 11:53:18,972:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502\n",
      "[INFO] [2016-05-21 11:53:18,973:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 1.808918 seconds\n",
      "[INFO] [2016-05-21 11:53:18,976:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (4)!.\n",
      "[INFO] [2016-05-21 11:53:18,977:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (4)!\n",
      "[ERROR] [2016-05-21 11:53:18,981:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:53:19,166:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 6. configuration on SUBSET. Duration 0.192416; loss 0.211170; status 1; additional run info: ;duration: 0.19241595268249512;num_run:00006 \n",
      "[INFO] [2016-05-21 11:53:19,169:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000\n",
      "[INFO] [2016-05-21 11:53:19,170:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Metadata directory: /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/autosklearn/metalearning/files/acc_metric_multiclass.classification_dense\n",
      "[INFO] [2016-05-21 11:53:20,989:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.952635 1: 0.952635 2: 0.952635 3: 0.952635 4: 0.952635 5: 0.954608 6: 0.954608 7: 0.954608 8: 0.954608 9: 0.956582 10: 0.956582 11: 0.958555 12: 0.960529 13: 0.960529 14: 0.960529 15: 0.960529 16: 0.960529 17: 0.960529 18: 0.960529 19: 0.960529 20: 0.960529 21: 0.960529 22: 0.960529 23: 0.960529 24: 0.960529 25: 0.962502 26: 0.962502 27: 0.962502 28: 0.960529 29: 0.962502 30: 0.962502 31: 0.962502 32: 0.962502 33: 0.962502 34: 0.962502 35: 0.962502 36: 0.962502 37: 0.962502 38: 0.962502 39: 0.962502 40: 0.962502 41: 0.962502 42: 0.962502 43: 0.962502 44: 0.962502 45: 0.962502 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502\n",
      "\tMembers: [1, 1, 1, 1, 1, 3, 1, 1, 0, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 0, 2, 3, 3, 0, 1, 1, 1, 1, 1, 1, 2, 0, 1, 1, 0, 1, 1, 1, 1, 2, 0, 1, 1, 1, 1]\n",
      "\tWeights: [ 0.12  0.72  0.06  0.1 ]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 4) (1, 5)\n",
      "[INFO] [2016-05-21 11:53:20,993:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502\n",
      "[INFO] [2016-05-21 11:53:20,994:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 2.014419 seconds\n",
      "[INFO] [2016-05-21 11:53:21,165:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (5)!.\n",
      "[INFO] [2016-05-21 11:53:21,166:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (5)!\n",
      "[ERROR] [2016-05-21 11:53:21,171:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:53:22,796:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Start calculating metafeatures for 6be0c6791d6c4877e53f8a237238200e\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/numpy/lib/nanfunctions.py:1147: RuntimeWarning: Degrees of freedom <= 0 for slice.\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice.\", RuntimeWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/numpy/lib/nanfunctions.py:675: RuntimeWarning: Mean of empty slice\n",
      "  warnings.warn(\"Mean of empty slice\", RuntimeWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:53:22,979:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Calculating Metafeatures (categorical attributes) took  0.18\n",
      "[INFO] [2016-05-21 11:53:23,588:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.952635 1: 0.952635 2: 0.952635 3: 0.952635 4: 0.952635 5: 0.954608 6: 0.954608 7: 0.954608 8: 0.954608 9: 0.956582 10: 0.956582 11: 0.958555 12: 0.960529 13: 0.960529 14: 0.960529 15: 0.960529 16: 0.960529 17: 0.960529 18: 0.960529 19: 0.960529 20: 0.960529 21: 0.960529 22: 0.960529 23: 0.960529 24: 0.960529 25: 0.962502 26: 0.962502 27: 0.962502 28: 0.962502 29: 0.962502 30: 0.962502 31: 0.962502 32: 0.962502 33: 0.962502 34: 0.964476 35: 0.962502 36: 0.962502 37: 0.960529 38: 0.960529 39: 0.962502 40: 0.962502 41: 0.962502 42: 0.962502 43: 0.962502 44: 0.962502 45: 0.962502 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502\n",
      "\tMembers: [1, 1, 1, 1, 1, 3, 1, 1, 0, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 0, 2, 3, 3, 4, 1, 1, 1, 1, 1, 2, 0, 0, 1, 1, 1, 1, 3, 1, 1, 1, 3, 1, 1, 1, 3]\n",
      "\tWeights: [ 0.08  0.7   0.04  0.16  0.02]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)\n",
      "[INFO] [2016-05-21 11:53:23,593:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502\n",
      "[INFO] [2016-05-21 11:53:23,594:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 2.425635 seconds\n",
      "[INFO] [2016-05-21 11:53:23,597:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (6)!.\n",
      "[INFO] [2016-05-21 11:53:23,598:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (6)!\n",
      "[INFO] [2016-05-21 11:53:23,771:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Calculating Metafeatures (encoded attributes) took  0.77sec\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:53:24,211:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Time left for 6be0c6791d6c4877e53f8a237238200e after finding initial configurations: 340.55sec\n",
      "[INFO] [2016-05-21 11:53:24,215:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Metafeatures for dataset 6be0c6791d6c4877e53f8a237238200euiaeo\n",
      "  KurtosisMean: 79.9312077309\n",
      "  ClassProbabilityMin: 0.09431751611013474\n",
      "  SkewnessMean: 3.70700734283\n",
      "  ClassEntropy: 3.32137035936\n",
      "  NumberOfCategoricalFeatures: 0\n",
      "  KurtosisSTD: 249.225704121\n",
      "  SymbolsMean: 0\n",
      "  SymbolsSTD: 0\n",
      "  PercentageOfFeaturesWithMissingValues: 0.0\n",
      "  NumberOfInstancesWithMissingValues: 0.0\n",
      "  InverseDatasetRatio: 26.671875\n",
      "  NumberOfFeatures: 64.0\n",
      "  RatioNumericalToNominal: 0.0\n",
      "  LogInverseDatasetRatio: 3.28360963943\n",
      "  NumberOfFeaturesWithMissingValues: 0.0\n",
      "  KurtosisMin: -3.0\n",
      "  NumberOfNumericFeatures: 64\n",
      "  SymbolsSum: 0.0\n",
      "  SkewnessSTD: 7.95909987575\n",
      "  LogNumberOfInstances: 7.44249272279\n",
      "  NumberOfClasses: 10.0\n",
      "  SkewnessMax: 39.5696193954\n",
      "  SymbolsMin: 0\n",
      "  ClassProbabilitySTD: 0.00277323969034\n",
      "  ClassProbabilityMax: 0.10369068541300527\n",
      "  PercentageOfInstancesWithMissingValues: 0.0\n",
      "  LandmarkRandomNodeLearner: 0.102491005855\n",
      "  ClassProbabilityMean: 0.1\n",
      "  PercentageOfMissingValues: 0.0\n",
      "  LogDatasetRatio: -3.28360963943\n",
      "  NumberOfMissingValues: 0.0\n",
      "  SymbolsMax: 0\n",
      "  NumberOfInstances: 1707.0\n",
      "  RatioNominalToNumerical: 0.0\n",
      "  DatasetRatio: 0.03749267721148213\n",
      "  SkewnessMin: -1.33853375774\n",
      "  KurtosisMax: 1601.02833276\n",
      "  LogNumberOfFeatures: 4.15888308336\n",
      "\n",
      "[INFO] [2016-05-21 11:53:24,224:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] ['3_acc', '971_acc', '718_acc', '180_acc', '1018_acc', '797_acc', '401_acc', '1036_acc', '184_acc', '300_acc', '897_acc', '46_acc', '722_acc', '30_acc', '934_acc', '723_acc', '351_acc', '903_acc', '912_acc', '1112_acc', '354_acc', '913_acc', '21_acc', '1139_acc', '1021_acc', '36_acc', '803_acc', '837_acc', '847_acc', '396_acc', '12_acc', '843_acc', '392_acc', '833_acc', '1120_acc', '737_acc', '16_acc', '31_acc', '1000_acc', '1049_acc', '813_acc', '904_acc', '735_acc', '44_acc', '1040_acc', '1020_acc', '846_acc', '38_acc', '741_acc', '182_acc', '293_acc', '1116_acc', '966_acc', '60_acc', '980_acc', '772_acc', '727_acc', '1041_acc', '761_acc', '554_acc', '57_acc', '185_acc', '1053_acc', '715_acc', '32_acc', '1119_acc', '1069_acc', '395_acc', '1128_acc', '979_acc', '181_acc', '849_acc', '910_acc', '23_acc', '6_acc', '1114_acc', '1142_acc', '819_acc', '1002_acc', '978_acc', '1056_acc', '393_acc', '799_acc', '871_acc', '914_acc', '740_acc', '977_acc', '976_acc', '959_acc', '752_acc', '390_acc', '398_acc', '881_acc', '1138_acc', '1166_acc', '917_acc', '823_acc', '273_acc', '866_acc', '399_acc', '28_acc', '22_acc', '1130_acc', '389_acc', '6be0c6791d6c4877e53f8a237238200euiaeo', '1134_acc', '14_acc', '958_acc', '1146_acc', '821_acc', '1111_acc', '822_acc', '930_acc', '179_acc', '26_acc', '901_acc', '993_acc', '845_acc', '807_acc', '995_acc', '1019_acc', '751_acc', '1161_acc', '962_acc', '816_acc', '743_acc', '679_acc', '734_acc', '953_acc', '24_acc', '720_acc', '806_acc', '991_acc', '391_acc', '923_acc', '728_acc', '18_acc', '1067_acc', '1050_acc', '1068_acc', '357_acc']\n",
      "[INFO] [2016-05-21 11:55:50,225:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 7. configuration (default configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:55:50,228:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: random_forest\n",
      "  classifier:random_forest:bootstrap, Value: True\n",
      "  classifier:random_forest:criterion, Value: gini\n",
      "  classifier:random_forest:max_depth, Constant: None\n",
      "  classifier:random_forest:max_features, Value: 1.0\n",
      "  classifier:random_forest:max_leaf_nodes, Constant: None\n",
      "  classifier:random_forest:min_samples_leaf, Value: 1\n",
      "  classifier:random_forest:min_samples_split, Value: 2\n",
      "  classifier:random_forest:min_weight_fraction_leaf, Constant: 0.0\n",
      "  classifier:random_forest:n_estimators, Constant: 100\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.01\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: no_preprocessing\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n",
      "[INFO] [2016-05-21 11:55:52,170:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 7. configuration. Duration 1.624692; loss 0.035524; status 1; additional run info: ;duration: 1.6246922016143799;num_run:00007 \n",
      "[INFO] [2016-05-21 11:55:52,172:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 8. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:55:52,176:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: libsvm_svc\n",
      "  classifier:libsvm_svc:C, Value: 2.820950683561126\n",
      "  classifier:libsvm_svc:coef0, Value: 0.7525658406326312\n",
      "  classifier:libsvm_svc:degree, Value: 5\n",
      "  classifier:libsvm_svc:gamma, Value: 1.924396413436426\n",
      "  classifier:libsvm_svc:kernel, Value: poly\n",
      "  classifier:libsvm_svc:max_iter, Constant: -1\n",
      "  classifier:libsvm_svc:shrinking, Value: True\n",
      "  classifier:libsvm_svc:tol, Value: 0.00037682073690554086\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:use_minimum_fraction, Value: False\n",
      "  preprocessor:__choice__, Value: select_percentile_classification\n",
      "  preprocessor:select_percentile_classification:percentile, Value: 83.60106850579331\n",
      "  preprocessor:select_percentile_classification:score_func, Value: f_classif\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/feature_selection/univariate_selection.py:113: UserWarning: Features [ 0 32 39 56] are constant.\n",
      "  UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:55:52,388:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:55:53,001:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 8. configuration. Duration 0.778786; loss 0.011841; status 1; additional run info: ;duration: 0.7787859439849854;num_run:00008 \n",
      "[INFO] [2016-05-21 11:55:53,165:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 9. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:55:53,167:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: liblinear_svc\n",
      "  classifier:liblinear_svc:C, Value: 0.5815868776595028\n",
      "  classifier:liblinear_svc:dual, Constant: False\n",
      "  classifier:liblinear_svc:fit_intercept, Constant: True\n",
      "  classifier:liblinear_svc:intercept_scaling, Constant: 1\n",
      "  classifier:liblinear_svc:loss, Value: squared_hinge\n",
      "  classifier:liblinear_svc:multi_class, Constant: ovr\n",
      "  classifier:liblinear_svc:penalty, Value: l2\n",
      "  classifier:liblinear_svc:tol, Value: 0.05261321190581748\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:use_minimum_fraction, Value: False\n",
      "  preprocessor:__choice__, Value: polynomial\n",
      "  preprocessor:polynomial:degree, Value: 2\n",
      "  preprocessor:polynomial:include_bias, Value: True\n",
      "  preprocessor:polynomial:interaction_only, Value: False\n",
      "  rescaling:__choice__, Value: none\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/class_weight.py:62: DeprecationWarning: The class_weight='auto' heuristic is deprecated in 0.17 in favor of a new heuristic class_weight='balanced'. 'auto' will be removed in 0.19\n",
      "  \" 0.19\", DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:55:56,196:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.964476 1: 0.964476 2: 0.964476 3: 0.966450 4: 0.966450 5: 0.970397 6: 0.970397 7: 0.968423 8: 0.970397 9: 0.972370 10: 0.972370 11: 0.972370 12: 0.970397 13: 0.970397 14: 0.970397 15: 0.970397 16: 0.970397 17: 0.970397 18: 0.970397 19: 0.970397 20: 0.968423 21: 0.966450 22: 0.964476 23: 0.966450 24: 0.966450 25: 0.966450 26: 0.966450 27: 0.966450 28: 0.966450 29: 0.966450 30: 0.966450 31: 0.966450 32: 0.966450 33: 0.966450 34: 0.966450 35: 0.966450 36: 0.966450 37: 0.966450 38: 0.966450 39: 0.966450 40: 0.966450 41: 0.964476 42: 0.962502 43: 0.960529 44: 0.960529 45: 0.960529 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502\n",
      "\tMembers: [5, 5, 5, 1, 5, 5, 5, 5, 4, 1, 1, 5, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 5, 5, 1, 5, 3, 4, 2, 1, 5, 1, 5, 5, 0, 5, 5, 1, 4, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1]\n",
      "\tWeights: [ 0.1   0.48  0.02  0.02  0.06  0.32]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 4) (1, 5) (1, 6) (1, 7)\n",
      "[INFO] [2016-05-21 11:55:56,365:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502\n",
      "[INFO] [2016-05-21 11:55:56,366:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 3.981098 seconds\n",
      "[INFO] [2016-05-21 11:55:56,370:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (7)!.\n",
      "[INFO] [2016-05-21 11:55:56,371:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (7)!\n",
      "[ERROR] [2016-05-21 11:55:56,377:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:55:58,583:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 9. configuration. Duration 5.013263; loss 0.011841; status 1; additional run info: ;duration: 5.013262987136841;num_run:00009 \n",
      "[INFO] [2016-05-21 11:55:58,586:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 10. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:55:58,588:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: adaboost\n",
      "  classifier:adaboost:algorithm, Value: SAMME\n",
      "  classifier:adaboost:learning_rate, Value: 1.2306208006800998\n",
      "  classifier:adaboost:max_depth, Value: 6\n",
      "  classifier:adaboost:n_estimators, Value: 499\n",
      "  imputation:strategy, Value: median\n",
      "  one_hot_encoding:use_minimum_fraction, Value: False\n",
      "  preprocessor:__choice__, Value: extra_trees_preproc_for_classification\n",
      "  preprocessor:extra_trees_preproc_for_classification:bootstrap, Value: True\n",
      "  preprocessor:extra_trees_preproc_for_classification:criterion, Value: entropy\n",
      "  preprocessor:extra_trees_preproc_for_classification:max_depth, Constant: None\n",
      "  preprocessor:extra_trees_preproc_for_classification:max_features, Value: 3.5347851525007146\n",
      "  preprocessor:extra_trees_preproc_for_classification:min_samples_leaf, Value: 6\n",
      "  preprocessor:extra_trees_preproc_for_classification:min_samples_split, Value: 8\n",
      "  preprocessor:extra_trees_preproc_for_classification:min_weight_fraction_leaf, Constant: 0.0\n",
      "  preprocessor:extra_trees_preproc_for_classification:n_estimators, Constant: 100\n",
      "  rescaling:__choice__, Value: standardize\n",
      "\n",
      "[INFO] [2016-05-21 11:56:00,765:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.988159 1: 0.988159 2: 0.988159 3: 0.988159 4: 0.988159 5: 0.988159 6: 0.988159 7: 0.988159 8: 0.988159 9: 0.988159 10: 0.988159 11: 0.988159 12: 0.988159 13: 0.988159 14: 0.988159 15: 0.988159 16: 0.988159 17: 0.988159 18: 0.988159 19: 0.988159 20: 0.988159 21: 0.988159 22: 0.988159 23: 0.988159 24: 0.988159 25: 0.988159 26: 0.988159 27: 0.988159 28: 0.988159 29: 0.988159 30: 0.988159 31: 0.988159 32: 0.988159 33: 0.988159 34: 0.988159 35: 0.988159 36: 0.988159 37: 0.988159 38: 0.988159 39: 0.988159 40: 0.988159 41: 0.988159 42: 0.988159 43: 0.988159 44: 0.988159 45: 0.988159 46: 0.988159 47: 0.988159 48: 0.988159 49: 0.988159\n",
      "\tMembers: [6, 6, 6, 6, 0, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 0, 6, 6, 5, 6, 0, 6, 6, 6, 0, 5, 6, 6, 6, 0, 5, 6, 6]\n",
      "\tWeights: [ 0.2   0.    0.    0.    0.    0.18  0.62]\n",
      "\tIdentifiers: (1, 2) (1, 7) (1, 8)\n",
      "[INFO] [2016-05-21 11:56:00,770:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.988159\n",
      "[INFO] [2016-05-21 11:56:00,771:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 4.396333 seconds\n",
      "[INFO] [2016-05-21 11:56:00,774:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (8)!.\n",
      "[INFO] [2016-05-21 11:56:00,775:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (8)!\n",
      "[ERROR] [2016-05-21 11:56:00,781:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:56:04,798:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.988159 1: 0.988159 2: 0.988159 3: 0.988159 4: 0.988159 5: 0.988159 6: 0.988159 7: 0.988159 8: 0.988159 9: 0.988159 10: 0.988159 11: 0.988159 12: 0.988159 13: 0.988159 14: 0.988159 15: 0.988159 16: 0.988159 17: 0.988159 18: 0.988159 19: 0.988159 20: 0.988159 21: 0.988159 22: 0.988159 23: 0.988159 24: 0.988159 25: 0.988159 26: 0.988159 27: 0.988159 28: 0.988159 29: 0.988159 30: 0.988159 31: 0.988159 32: 0.988159 33: 0.988159 34: 0.988159 35: 0.988159 36: 0.988159 37: 0.988159 38: 0.988159 39: 0.988159 40: 0.988159 41: 0.988159 42: 0.988159 43: 0.988159 44: 0.988159 45: 0.988159 46: 0.988159 47: 0.988159 48: 0.988159 49: 0.988159\n",
      "\tMembers: [6, 6, 6, 6, 0, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 0, 6, 6, 5, 6, 0, 6, 6, 6, 0, 5, 6, 6, 6, 0, 5, 6, 6]\n",
      "\tWeights: [ 0.2   0.    0.    0.    0.    0.18  0.62  0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 7) (1, 8)\n",
      "[INFO] [2016-05-21 11:56:04,965:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.988159\n",
      "[INFO] [2016-05-21 11:56:04,967:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 4.187559 seconds\n",
      "[INFO] [2016-05-21 11:56:04,968:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:56:12,565:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 10. configuration. Duration 13.453381; loss 0.029603; status 1; additional run info: ;duration: 13.453380584716797;num_run:00010 \n",
      "[INFO] [2016-05-21 11:56:12,568:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 11. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:56:12,569:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: sgd\n",
      "  classifier:sgd:alpha, Value: 0.0002011208878338422\n",
      "  classifier:sgd:average, Value: False\n",
      "  classifier:sgd:eta0, Value: 0.0812309539678548\n",
      "  classifier:sgd:fit_intercept, Constant: True\n",
      "  classifier:sgd:l1_ratio, Value: 0.49756924007684844\n",
      "  classifier:sgd:learning_rate, Value: invscaling\n",
      "  classifier:sgd:loss, Value: hinge\n",
      "  classifier:sgd:n_iter, Value: 554\n",
      "  classifier:sgd:penalty, Value: elasticnet\n",
      "  classifier:sgd:power_t, Value: 0.2567923580475304\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.0007915092939712602\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: nystroem_sampler\n",
      "  preprocessor:nystroem_sampler:coef0, Value: -0.6384046151685252\n",
      "  preprocessor:nystroem_sampler:degree, Value: 3\n",
      "  preprocessor:nystroem_sampler:gamma, Value: 0.22795987796995726\n",
      "  preprocessor:nystroem_sampler:kernel, Value: poly\n",
      "  preprocessor:nystroem_sampler:n_components, Value: 3207\n",
      "  rescaling:__choice__, Value: none\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/kernel_approximation.py:463: UserWarning: n_components > n_samples. This is not possible.\n",
      "n_components was set to n_samples, which results in inefficient evaluation of the full kernel.\n",
      "  warnings.warn(\"n_components > n_samples. This is not possible.\\n\"\n",
      "You are already timing task: index_run6\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:56:13,168:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:56:21,963:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.988159 1: 0.988159 2: 0.988159 3: 0.988159 4: 0.988159 5: 0.988159 6: 0.988159 7: 0.988159 8: 0.988159 9: 0.988159 10: 0.988159 11: 0.988159 12: 0.988159 13: 0.988159 14: 0.988159 15: 0.988159 16: 0.988159 17: 0.988159 18: 0.988159 19: 0.988159 20: 0.988159 21: 0.988159 22: 0.988159 23: 0.988159 24: 0.988159 25: 0.988159 26: 0.988159 27: 0.988159 28: 0.988159 29: 0.988159 30: 0.988159 31: 0.988159 32: 0.988159 33: 0.988159 34: 0.988159 35: 0.988159 36: 0.988159 37: 0.988159 38: 0.988159 39: 0.988159 40: 0.988159 41: 0.988159 42: 0.988159 43: 0.988159 44: 0.988159 45: 0.988159 46: 0.988159 47: 0.988159 48: 0.988159 49: 0.988159\n",
      "\tMembers: [6, 6, 6, 6, 0, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 0, 6, 6, 5, 6, 0, 6, 6, 6, 0, 5, 6, 6, 6, 0, 5, 6, 6]\n",
      "\tWeights: [ 0.2   0.    0.    0.    0.    0.18  0.62  0.    0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 7) (1, 8)\n",
      "[INFO] [2016-05-21 11:56:22,166:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.988159\n",
      "[INFO] [2016-05-21 11:56:22,168:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 9.002641 seconds\n",
      "[INFO] [2016-05-21 11:56:22,170:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.\n",
      "[INFO] [2016-05-21 11:56:49,615:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 11. configuration. Duration 37.038066; loss 2.000000; status 2; additional run info:  \n",
      "[INFO] [2016-05-21 11:56:49,618:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 12. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:56:49,763:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: proj_logit\n",
      "  classifier:proj_logit:max_epochs, Value: 2\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.0011870322904213703\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: no_preprocessing\n",
      "  rescaling:__choice__, Value: none\n",
      "\n",
      "[INFO] [2016-05-21 11:56:50,385:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 12. configuration. Duration 0.387105; loss 0.055260; status 1; additional run info: ;duration: 0.38710546493530273;num_run:00012 \n",
      "[INFO] [2016-05-21 11:56:50,388:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 13. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:56:50,389:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: adaboost\n",
      "  classifier:adaboost:algorithm, Value: SAMME\n",
      "  classifier:adaboost:learning_rate, Value: 1.0081104516473922\n",
      "  classifier:adaboost:max_depth, Value: 6\n",
      "  classifier:adaboost:n_estimators, Value: 468\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:use_minimum_fraction, Value: False\n",
      "  preprocessor:__choice__, Value: liblinear_svc_preprocessor\n",
      "  preprocessor:liblinear_svc_preprocessor:C, Value: 1.1828431725901418\n",
      "  preprocessor:liblinear_svc_preprocessor:dual, Constant: False\n",
      "  preprocessor:liblinear_svc_preprocessor:fit_intercept, Constant: True\n",
      "  preprocessor:liblinear_svc_preprocessor:intercept_scaling, Constant: 1\n",
      "  preprocessor:liblinear_svc_preprocessor:loss, Value: squared_hinge\n",
      "  preprocessor:liblinear_svc_preprocessor:multi_class, Constant: ovr\n",
      "  preprocessor:liblinear_svc_preprocessor:penalty, Constant: l1\n",
      "  preprocessor:liblinear_svc_preprocessor:tol, Value: 0.0022792606924326923\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "You are already timing task: index_run6\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:56:52,240:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:56:57,164:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.988159 1: 0.990132 2: 0.990132 3: 0.992106 4: 0.992106 5: 0.992106 6: 0.992106 7: 0.992106 8: 0.992106 9: 0.992106 10: 0.992106 11: 0.992106 12: 0.992106 13: 0.992106 14: 0.992106 15: 0.992106 16: 0.992106 17: 0.992106 18: 0.992106 19: 0.992106 20: 0.992106 21: 0.992106 22: 0.992106 23: 0.992106 24: 0.992106 25: 0.992106 26: 0.992106 27: 0.992106 28: 0.992106 29: 0.992106 30: 0.992106 31: 0.992106 32: 0.992106 33: 0.992106 34: 0.992106 35: 0.992106 36: 0.992106 37: 0.992106 38: 0.992106 39: 0.992106 40: 0.992106 41: 0.992106 42: 0.992106 43: 0.992106 44: 0.992106 45: 0.992106 46: 0.992106 47: 0.992106 48: 0.992106 49: 0.992106\n",
      "\tMembers: [6, 9, 1, 7, 8, 8, 7, 1, 7, 0, 1, 6, 7, 1, 1, 1, 7, 0, 1, 7, 1, 7, 1, 6, 7, 0, 1, 7, 1, 1, 7, 1, 5, 7, 1, 5, 7, 5, 7, 1, 7, 1, 6, 0, 1, 7, 1, 5, 7, 1]\n",
      "\tWeights: [ 0.08  0.38  0.    0.    0.    0.08  0.08  0.32  0.04  0.02]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 7) (1, 8) (1, 9) (1, 10) (1, 12)\n",
      "[INFO] [2016-05-21 11:56:57,169:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.992106\n",
      "[INFO] [2016-05-21 11:56:57,170:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 4.932312 seconds\n",
      "[INFO] [2016-05-21 11:56:57,173:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (11)!.\n",
      "[INFO] [2016-05-21 11:56:57,175:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (11)!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:57:08,021:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 13. configuration. Duration 17.245130; loss 0.017762; status 1; additional run info: ;duration: 17.24513030052185;num_run:00013 \n",
      "[INFO] [2016-05-21 11:57:08,024:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 14. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:57:08,025:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: lda\n",
      "  classifier:lda:n_components, Value: 242\n",
      "  classifier:lda:shrinkage, Value: None\n",
      "  classifier:lda:tol, Value: 0.0011339574479631279\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.004889144803037605\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: gem\n",
      "  preprocessor:gem:N, Value: 8\n",
      "  preprocessor:gem:precond, Value: 0.39344596751061517\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n",
      "[ERROR] [2016-05-21 11:57:09,368:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:57:20,786:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 14. configuration. Duration 12.396692; loss 0.009868; status 1; additional run info: ;duration: 12.396692276000977;num_run:00014 \n",
      "[INFO] [2016-05-21 11:57:20,788:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 15. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:57:20,790:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: libsvm_svc\n",
      "  classifier:libsvm_svc:C, Value: 2099.097215335698\n",
      "  classifier:libsvm_svc:coef0, Value: 0.32850098053260424\n",
      "  classifier:libsvm_svc:degree, Value: 3\n",
      "  classifier:libsvm_svc:gamma, Value: 2.809557625370355\n",
      "  classifier:libsvm_svc:kernel, Value: poly\n",
      "  classifier:libsvm_svc:max_iter, Constant: -1\n",
      "  classifier:libsvm_svc:shrinking, Value: False\n",
      "  classifier:libsvm_svc:tol, Value: 0.003213729117706896\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.0033839910940432396\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: select_rates\n",
      "  preprocessor:select_rates:alpha, Value: 0.08594367101091721\n",
      "  preprocessor:select_rates:mode, Value: fpr\n",
      "  preprocessor:select_rates:score_func, Value: chi2\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n",
      "[INFO] [2016-05-21 11:57:20,992:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.988159 1: 0.990132 2: 0.990132 3: 0.992106 4: 0.992106 5: 0.992106 6: 0.992106 7: 0.992106 8: 0.992106 9: 0.992106 10: 0.992106 11: 0.992106 12: 0.992106 13: 0.992106 14: 0.992106 15: 0.992106 16: 0.992106 17: 0.992106 18: 0.992106 19: 0.992106 20: 0.992106 21: 0.992106 22: 0.992106 23: 0.992106 24: 0.992106 25: 0.992106 26: 0.992106 27: 0.992106 28: 0.992106 29: 0.992106 30: 0.992106 31: 0.992106 32: 0.992106 33: 0.992106 34: 0.992106 35: 0.992106 36: 0.992106 37: 0.992106 38: 0.992106 39: 0.992106 40: 0.992106 41: 0.992106 42: 0.992106 43: 0.992106 44: 0.992106 45: 0.992106 46: 0.992106 47: 0.992106 48: 0.992106 49: 0.992106\n",
      "\tMembers: [6, 9, 1, 7, 8, 8, 7, 1, 7, 0, 1, 6, 7, 1, 1, 1, 7, 0, 1, 7, 1, 7, 1, 6, 7, 0, 1, 7, 1, 1, 7, 1, 5, 7, 1, 5, 7, 5, 7, 1, 7, 1, 6, 0, 1, 7, 1, 5, 7, 1]\n",
      "\tWeights: [ 0.08  0.38  0.    0.    0.    0.08  0.08  0.32  0.04  0.02  0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 7) (1, 8) (1, 9) (1, 10) (1, 12)\n",
      "[INFO] [2016-05-21 11:57:20,998:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.992106\n",
      "[INFO] [2016-05-21 11:57:21,164:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 11.798627 seconds\n",
      "[INFO] [2016-05-21 11:57:21,165:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/class_weight.py:62: DeprecationWarning: The class_weight='auto' heuristic is deprecated in 0.17 in favor of a new heuristic class_weight='balanced'. 'auto' will be removed in 0.19\n",
      "  \" 0.19\", DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:57:22,002:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 15. configuration. Duration 0.992510; loss 0.013815; status 1; additional run info: ;duration: 0.9925096035003662;num_run:00015 \n",
      "[INFO] [2016-05-21 11:57:22,004:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 16. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:57:22,006:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: liblinear_svc\n",
      "  classifier:liblinear_svc:C, Value: 62.21750028494345\n",
      "  classifier:liblinear_svc:dual, Constant: False\n",
      "  classifier:liblinear_svc:fit_intercept, Constant: True\n",
      "  classifier:liblinear_svc:intercept_scaling, Constant: 1\n",
      "  classifier:liblinear_svc:loss, Value: squared_hinge\n",
      "  classifier:liblinear_svc:multi_class, Constant: ovr\n",
      "  classifier:liblinear_svc:penalty, Value: l2\n",
      "  classifier:liblinear_svc:tol, Value: 0.0002178180133457927\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.028777961041830738\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: polynomial\n",
      "  preprocessor:polynomial:degree, Value: 3\n",
      "  preprocessor:polynomial:include_bias, Value: False\n",
      "  preprocessor:polynomial:interaction_only, Value: False\n",
      "  rescaling:__choice__, Value: none\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are already timing task: index_run7\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:57:23,175:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:57:29,391:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079\n",
      "\tMembers: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7]\n",
      "\tWeights: [ 0.02  0.32  0.    0.02  0.    0.08  0.04  0.34  0.16  0.    0.    0.02\n",
      "  0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14)\n",
      "[INFO] [2016-05-21 11:57:29,397:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079\n",
      "[INFO] [2016-05-21 11:57:29,566:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 6.393615 seconds\n",
      "[INFO] [2016-05-21 11:57:29,569:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (14)!.\n",
      "[INFO] [2016-05-21 11:57:29,571:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (14)!\n",
      "[INFO] [2016-05-21 11:57:59,866:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 16. configuration. Duration 37.818645; loss 2.000000; status 2; additional run info:  \n",
      "\n",
      "[INFO] [2016-05-21 11:57:59,953:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 17. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:57:59,954:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: extra_trees\n",
      "  classifier:extra_trees:bootstrap, Value: True\n",
      "  classifier:extra_trees:criterion, Value: gini\n",
      "  classifier:extra_trees:max_depth, Constant: None\n",
      "  classifier:extra_trees:max_features, Value: 2.3779010566365115\n",
      "  classifier:extra_trees:min_samples_leaf, Value: 2\n",
      "  classifier:extra_trees:min_samples_split, Value: 4\n",
      "  classifier:extra_trees:min_weight_fraction_leaf, Constant: 0.0\n",
      "  classifier:extra_trees:n_estimators, Constant: 100\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.004305770290473698\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: select_rates\n",
      "  preprocessor:select_rates:alpha, Value: 0.30934089491790895\n",
      "  preprocessor:select_rates:mode, Value: fdr\n",
      "  preprocessor:select_rates:score_func, Value: f_classif\n",
      "  rescaling:__choice__, Value: none\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/feature_selection/univariate_selection.py:113: UserWarning: Features [ 0 32 39 56] are constant.\n",
      "  UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:58:06,028:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 17. configuration. Duration 5.939049; loss 0.027630; status 1; additional run info: ;duration: 5.939048528671265;num_run:00017 \n",
      "[INFO] [2016-05-21 11:58:06,030:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 18. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:58:06,031:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: proj_logit\n",
      "  classifier:proj_logit:max_epochs, Value: 11\n",
      "  imputation:strategy, Value: median\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.002883367159521145\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: gem\n",
      "  preprocessor:gem:N, Value: 16\n",
      "  preprocessor:gem:precond, Value: 0.30628439346357783\n",
      "  rescaling:__choice__, Value: standardize\n",
      "\n",
      "[ERROR] [2016-05-21 11:58:06,964:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:58:22,168:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079\n",
      "\tMembers: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7]\n",
      "\tWeights: [ 0.02  0.32  0.    0.02  0.    0.08  0.04  0.34  0.16  0.    0.    0.02\n",
      "  0.    0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14)\n",
      "[INFO] [2016-05-21 11:58:22,367:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079\n",
      "[INFO] [2016-05-21 11:58:22,368:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 15.600882 seconds\n",
      "[INFO] [2016-05-21 11:58:22,370:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.\n",
      "[INFO] [2016-05-21 11:58:36,171:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 18. configuration. Duration 29.602208; loss 0.015788; status 1; additional run info: ;duration: 29.602208137512207;num_run:00018 \n",
      "[INFO] [2016-05-21 11:58:36,174:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 19. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:58:36,175:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: libsvm_svc\n",
      "  classifier:libsvm_svc:C, Value: 295.4185183083245\n",
      "  classifier:libsvm_svc:gamma, Value: 0.05098015552849704\n",
      "  classifier:libsvm_svc:kernel, Value: rbf\n",
      "  classifier:libsvm_svc:max_iter, Constant: -1\n",
      "  classifier:libsvm_svc:shrinking, Value: False\n",
      "  classifier:libsvm_svc:tol, Value: 0.01616175671031427\n",
      "  imputation:strategy, Value: median\n",
      "  one_hot_encoding:use_minimum_fraction, Value: False\n",
      "  preprocessor:__choice__, Value: feature_agglomeration\n",
      "  preprocessor:feature_agglomeration:affinity, Value: cosine\n",
      "  preprocessor:feature_agglomeration:linkage, Value: complete\n",
      "  preprocessor:feature_agglomeration:n_clusters, Value: 216\n",
      "  preprocessor:feature_agglomeration:pooling_func, Value: mean\n",
      "  rescaling:__choice__, Value: none\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are already timing task: index_run8\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:58:36,583:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/class_weight.py:62: DeprecationWarning: The class_weight='auto' heuristic is deprecated in 0.17 in favor of a new heuristic class_weight='balanced'. 'auto' will be removed in 0.19\n",
      "  \" 0.19\", DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] [2016-05-21 11:58:39,183:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 19. configuration. Duration 2.800076; loss 0.988751; status 1; additional run info: ;duration: 2.8000760078430176;num_run:00019 \n",
      "[INFO] [2016-05-21 11:58:39,185:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 20. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:58:39,186:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: none\n",
      "  classifier:__choice__, Value: sgd\n",
      "  classifier:sgd:alpha, Value: 0.00039961166337423236\n",
      "  classifier:sgd:average, Value: False\n",
      "  classifier:sgd:epsilon, Value: 0.0017527970920483906\n",
      "  classifier:sgd:eta0, Value: 0.060041614581907\n",
      "  classifier:sgd:fit_intercept, Constant: True\n",
      "  classifier:sgd:l1_ratio, Value: 0.36339240847373755\n",
      "  classifier:sgd:learning_rate, Value: optimal\n",
      "  classifier:sgd:loss, Value: modified_huber\n",
      "  classifier:sgd:n_iter, Value: 939\n",
      "  classifier:sgd:penalty, Value: elasticnet\n",
      "  imputation:strategy, Value: most_frequent\n",
      "  one_hot_encoding:use_minimum_fraction, Value: False\n",
      "  preprocessor:__choice__, Value: select_percentile_classification\n",
      "  preprocessor:select_percentile_classification:percentile, Value: 76.84823099284878\n",
      "  preprocessor:select_percentile_classification:score_func, Value: chi2\n",
      "  rescaling:__choice__, Value: normalize\n",
      "\n",
      "[INFO] [2016-05-21 11:58:43,769:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079\n",
      "\tMembers: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7]\n",
      "\tWeights: [ 0.02  0.32  0.    0.02  0.    0.08  0.04  0.34  0.16  0.    0.    0.02\n",
      "  0.    0.    0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14)\n",
      "[INFO] [2016-05-21 11:58:43,775:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079\n",
      "[INFO] [2016-05-21 11:58:43,776:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 7.196891 seconds\n",
      "[INFO] [2016-05-21 11:58:43,778:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are already timing task: index_run8\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:58:45,790:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[INFO] [2016-05-21 11:58:53,191:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection:\n",
      "\tTrajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079\n",
      "\tMembers: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7]\n",
      "\tWeights: [ 0.02  0.32  0.    0.02  0.    0.08  0.04  0.34  0.16  0.    0.    0.02\n",
      "  0.    0.    0.    0.  ]\n",
      "\tIdentifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14)\n",
      "[INFO] [2016-05-21 11:58:53,196:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079\n",
      "[INFO] [2016-05-21 11:58:53,197:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 7.408631 seconds\n",
      "[INFO] [2016-05-21 11:58:53,198:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.\n",
      "[INFO] [2016-05-21 11:59:02,578:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 20. configuration. Duration 23.025394; loss 0.041445; status 1; additional run info: ;duration: 23.02539372444153;num_run:00020 \n",
      "[INFO] [2016-05-21 11:59:02,581:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 21. configuration (meta-learning configuration) with time limit 36s.\n",
      "[INFO] [2016-05-21 11:59:02,583:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration:\n",
      "  balancing:strategy, Value: weighting\n",
      "  classifier:__choice__, Value: sgd\n",
      "  classifier:sgd:alpha, Value: 0.0005401543374603144\n",
      "  classifier:sgd:average, Value: False\n",
      "  classifier:sgd:eta0, Value: 0.04377140004056303\n",
      "  classifier:sgd:fit_intercept, Constant: True\n",
      "  classifier:sgd:learning_rate, Value: optimal\n",
      "  classifier:sgd:loss, Value: log\n",
      "  classifier:sgd:n_iter, Value: 943\n",
      "  classifier:sgd:penalty, Value: l2\n",
      "  imputation:strategy, Value: mean\n",
      "  one_hot_encoding:minimum_fraction, Value: 0.0007887810786977907\n",
      "  one_hot_encoding:use_minimum_fraction, Value: True\n",
      "  preprocessor:__choice__, Value: select_rates\n",
      "  preprocessor:select_rates:alpha, Value: 0.17377268248209546\n",
      "  preprocessor:select_rates:mode, Value: fwe\n",
      "  preprocessor:select_rates:score_func, Value: chi2\n",
      "  rescaling:__choice__, Value: min/max\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are already timing task: index_run8\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ERROR] [2016-05-21 11:59:03,389:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285\n",
      "[ERROR] [2016-05-21 11:59:04,772:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Caught error! \n"
     ]
    }
   ],
   "source": [
    "clf.fit(data_train,target_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "いっぱいモデルが出てる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MyDummyClassifier(configuration=2, random_states=None)\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'none',\n",
      "  'classifier:__choice__': 'random_forest',\n",
      "  'classifier:random_forest:bootstrap': 'True',\n",
      "  'classifier:random_forest:criterion': 'gini',\n",
      "  'classifier:random_forest:max_depth': 'None',\n",
      "  'classifier:random_forest:max_features': 1.0,\n",
      "  'classifier:random_forest:max_leaf_nodes': 'None',\n",
      "  'classifier:random_forest:min_samples_leaf': 1,\n",
      "  'classifier:random_forest:min_samples_split': 2,\n",
      "  'classifier:random_forest:min_weight_fraction_leaf': 0.0,\n",
      "  'classifier:random_forest:n_estimators': 100,\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:minimum_fraction': 0.01,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'no_preprocessing',\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'weighting',\n",
      "  'classifier:__choice__': 'proj_logit',\n",
      "  'classifier:proj_logit:max_epochs': 2,\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:minimum_fraction': 0.0011870322904213703,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'no_preprocessing',\n",
      "  'rescaling:__choice__': 'none'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'none',\n",
      "  'classifier:__choice__': 'adaboost',\n",
      "  'classifier:adaboost:algorithm': 'SAMME',\n",
      "  'classifier:adaboost:learning_rate': 1.0081104516473922,\n",
      "  'classifier:adaboost:max_depth': 6,\n",
      "  'classifier:adaboost:n_estimators': 468,\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:use_minimum_fraction': 'False',\n",
      "  'preprocessor:__choice__': 'liblinear_svc_preprocessor',\n",
      "  'preprocessor:liblinear_svc_preprocessor:C': 1.1828431725901418,\n",
      "  'preprocessor:liblinear_svc_preprocessor:dual': 'False',\n",
      "  'preprocessor:liblinear_svc_preprocessor:fit_intercept': 'True',\n",
      "  'preprocessor:liblinear_svc_preprocessor:intercept_scaling': 1,\n",
      "  'preprocessor:liblinear_svc_preprocessor:loss': 'squared_hinge',\n",
      "  'preprocessor:liblinear_svc_preprocessor:multi_class': 'ovr',\n",
      "  'preprocessor:liblinear_svc_preprocessor:penalty': 'l1',\n",
      "  'preprocessor:liblinear_svc_preprocessor:tol': 0.0022792606924326923,\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'none',\n",
      "  'classifier:__choice__': 'lda',\n",
      "  'classifier:lda:n_components': 242,\n",
      "  'classifier:lda:shrinkage': 'None',\n",
      "  'classifier:lda:tol': 0.0011339574479631279,\n",
      "  'imputation:strategy': 'most_frequent',\n",
      "  'one_hot_encoding:minimum_fraction': 0.004889144803037605,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'gem',\n",
      "  'preprocessor:gem:N': 8,\n",
      "  'preprocessor:gem:precond': 0.39344596751061517,\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'weighting',\n",
      "  'classifier:__choice__': 'sgd',\n",
      "  'classifier:sgd:alpha': 0.0001,\n",
      "  'classifier:sgd:average': 'True',\n",
      "  'classifier:sgd:eta0': 0.01,\n",
      "  'classifier:sgd:fit_intercept': 'True',\n",
      "  'classifier:sgd:learning_rate': 'optimal',\n",
      "  'classifier:sgd:loss': 'hinge',\n",
      "  'classifier:sgd:n_iter': 5,\n",
      "  'classifier:sgd:penalty': 'l2',\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:minimum_fraction': 0.1,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'no_preprocessing',\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'weighting',\n",
      "  'classifier:__choice__': 'libsvm_svc',\n",
      "  'classifier:libsvm_svc:C': 2099.097215335698,\n",
      "  'classifier:libsvm_svc:coef0': 0.32850098053260424,\n",
      "  'classifier:libsvm_svc:degree': 3,\n",
      "  'classifier:libsvm_svc:gamma': 2.809557625370355,\n",
      "  'classifier:libsvm_svc:kernel': 'poly',\n",
      "  'classifier:libsvm_svc:max_iter': -1,\n",
      "  'classifier:libsvm_svc:shrinking': 'False',\n",
      "  'classifier:libsvm_svc:tol': 0.003213729117706896,\n",
      "  'imputation:strategy': 'most_frequent',\n",
      "  'one_hot_encoding:minimum_fraction': 0.0033839910940432396,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'select_rates',\n",
      "  'preprocessor:select_rates:alpha': 0.08594367101091721,\n",
      "  'preprocessor:select_rates:mode': 'fpr',\n",
      "  'preprocessor:select_rates:score_func': 'chi2',\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'weighting',\n",
      "  'classifier:__choice__': 'extra_trees',\n",
      "  'classifier:extra_trees:bootstrap': 'False',\n",
      "  'classifier:extra_trees:criterion': 'gini',\n",
      "  'classifier:extra_trees:max_depth': 'None',\n",
      "  'classifier:extra_trees:max_features': 1.0,\n",
      "  'classifier:extra_trees:min_samples_leaf': 5,\n",
      "  'classifier:extra_trees:min_samples_split': 5,\n",
      "  'classifier:extra_trees:min_weight_fraction_leaf': 0.0,\n",
      "  'classifier:extra_trees:n_estimators': 100,\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:minimum_fraction': 0.1,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'select_percentile_classification',\n",
      "  'preprocessor:select_percentile_classification:percentile': 2.0,\n",
      "  'preprocessor:select_percentile_classification:score_func': 'chi2',\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'none',\n",
      "  'classifier:__choice__': 'libsvm_svc',\n",
      "  'classifier:libsvm_svc:C': 2.820950683561126,\n",
      "  'classifier:libsvm_svc:coef0': 0.7525658406326312,\n",
      "  'classifier:libsvm_svc:degree': 5,\n",
      "  'classifier:libsvm_svc:gamma': 1.924396413436426,\n",
      "  'classifier:libsvm_svc:kernel': 'poly',\n",
      "  'classifier:libsvm_svc:max_iter': -1,\n",
      "  'classifier:libsvm_svc:shrinking': 'True',\n",
      "  'classifier:libsvm_svc:tol': 0.00037682073690554086,\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:use_minimum_fraction': 'False',\n",
      "  'preprocessor:__choice__': 'select_percentile_classification',\n",
      "  'preprocessor:select_percentile_classification:percentile': 83.60106850579331,\n",
      "  'preprocessor:select_percentile_classification:score_func': 'f_classif',\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'weighting',\n",
      "  'classifier:__choice__': 'gaussian_nb',\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:minimum_fraction': 0.1,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'no_preprocessing',\n",
      "  'rescaling:__choice__': 'standardize'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'weighting',\n",
      "  'classifier:__choice__': 'liblinear_svc',\n",
      "  'classifier:liblinear_svc:C': 0.5815868776595028,\n",
      "  'classifier:liblinear_svc:dual': 'False',\n",
      "  'classifier:liblinear_svc:fit_intercept': 'True',\n",
      "  'classifier:liblinear_svc:intercept_scaling': 1,\n",
      "  'classifier:liblinear_svc:loss': 'squared_hinge',\n",
      "  'classifier:liblinear_svc:multi_class': 'ovr',\n",
      "  'classifier:liblinear_svc:penalty': 'l2',\n",
      "  'classifier:liblinear_svc:tol': 0.05261321190581748,\n",
      "  'imputation:strategy': 'most_frequent',\n",
      "  'one_hot_encoding:use_minimum_fraction': 'False',\n",
      "  'preprocessor:__choice__': 'polynomial',\n",
      "  'preprocessor:polynomial:degree': 2,\n",
      "  'preprocessor:polynomial:include_bias': 'True',\n",
      "  'preprocessor:polynomial:interaction_only': 'False',\n",
      "  'rescaling:__choice__': 'none'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'none',\n",
      "  'classifier:__choice__': 'random_forest',\n",
      "  'classifier:random_forest:bootstrap': 'True',\n",
      "  'classifier:random_forest:criterion': 'gini',\n",
      "  'classifier:random_forest:max_depth': 'None',\n",
      "  'classifier:random_forest:max_features': 1.0,\n",
      "  'classifier:random_forest:max_leaf_nodes': 'None',\n",
      "  'classifier:random_forest:min_samples_leaf': 1,\n",
      "  'classifier:random_forest:min_samples_split': 2,\n",
      "  'classifier:random_forest:min_weight_fraction_leaf': 0.0,\n",
      "  'classifier:random_forest:n_estimators': 100,\n",
      "  'imputation:strategy': 'mean',\n",
      "  'one_hot_encoding:minimum_fraction': 0.01,\n",
      "  'one_hot_encoding:use_minimum_fraction': 'True',\n",
      "  'preprocessor:__choice__': 'no_preprocessing',\n",
      "  'rescaling:__choice__': 'min/max'})\n",
      "SimpleClassificationPipeline(configuration={\n",
      "  'balancing:strategy': 'none',\n",
      "  'classifier:__choice__': 'adaboost',\n",
      "  'classifier:adaboost:algorithm': 'SAMME',\n",
      "  'classifier:adaboost:learning_rate': 1.2306208006800998,\n",
      "  'classifier:adaboost:max_depth': 6,\n",
      "  'classifier:adaboost:n_estimators': 499,\n",
      "  'imputation:strategy': 'median',\n",
      "  'one_hot_encoding:use_minimum_fraction': 'False',\n",
      "  'preprocessor:__choice__': 'extra_trees_preproc_for_classification',\n",
      "  'preprocessor:extra_trees_preproc_for_classification:bootstrap': 'True',\n",
      "  'preprocessor:extra_trees_preproc_for_classification:criterion': 'entropy',\n",
      "  'preprocessor:extra_trees_preproc_for_classification:max_depth': 'None',\n",
      "  'preprocessor:extra_trees_preproc_for_classification:max_features': 3.5347851525007146,\n",
      "  'preprocessor:extra_trees_preproc_for_classification:min_samples_leaf': 6,\n",
      "  'preprocessor:extra_trees_preproc_for_classification:min_samples_split': 8,\n",
      "  'preprocessor:extra_trees_preproc_for_classification:min_weight_fraction_leaf': 0.0,\n",
      "  'preprocessor:extra_trees_preproc_for_classification:n_estimators': 100,\n",
      "  'rescaling:__choice__': 'standardize'})\n"
     ]
    }
   ],
   "source": [
    "for key in clf.models_.keys():print(clf.models_[key])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "多分モデルのアンサンブルをとっている（はず）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n"
     ]
    }
   ],
   "source": [
    "pred=clf.predict(data_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "いつもは97.5％くらいだけど今回はたまたまよい?  \n",
    "正答率もまあまあ。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC.\n",
      "  \"SVC.\", ChangedBehaviorWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.score(data_test,target_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Environment (automl)",
   "language": "",
   "name": "automl"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}