2-Keras-XOR.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras XOR Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've seen how to construct a simple network using raw Theano (although for people who are used to writing code in C++, maybe raw seems like a relative term), let's see how we can quickly implement high-level ideas using the [Keras](https://github.com/fchollet/keras) **deep learning** framework. We'll build the exact same network, but this time using Keras, and see how it compares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using Theano backend.\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "\n",
    "import numpy as np\n",
    "from keras.engine import Input, Model\n",
    "from keras.layers import Dense"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our training data as Numpy arrays, unlike in the Theano example, where we initialized them as shared variables. Keras has a very well-organized system for building networks, which is awesome."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = np.asarray([[0, 1], [1, 0], [0, 0], [1, 1]])\n",
    "y = np.asarray([[0], [0], [1], [1]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's define our model, the same way we defined it in the Theano example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "input_layer = Input(shape=(2,))\n",
    "hidden_layer = Dense(5, activation='relu')(input_layer)\n",
    "output_layer = Dense(1, activation='sigmoid')(hidden_layer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can easily choose which optimizer and loss function we want to use, instead of having to implement them ourselves. For the sake of consistency, let's use the same *subgradient descent* optimizer and *mean squared error* loss function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = Model(input=input_layer, output=output_layer)\n",
    "model.compile(optimizer='sgd', loss='mse')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, let's train the network. One good thing about Keras is that, if we haven't seen an error up until now and we haven't strayed too far outside the boundaries of what's supported, our network should work without any debugging (that is, all the input and output dimensions will be figured out for us). The `compile` step does this. The functions needed for training and testing the network are built only when they are needed, because they are the computationally heavy part of this process; hopefully by the time we're building them, our network is already flawless."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error before: 0.280486553907\n",
      "Error after: 0.0372566692531\n"
     ]
    }
   ],
   "source": [
    "print('Error before: {}'.format(model.evaluate([X], [y], verbose=0)))\n",
    "model.fit([X], [y], nb_epoch=10000, verbose=0)\n",
    "print('Error after: {}'.format(model.evaluate([X], [y], verbose=0)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}