sagemaker-python-sdk/tensorflow_iris_dnn_classifier_using_estimators/tensorflow_iris_dnn_classifier_using_estimators_elastic_inference_local.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Amazon Elastic Inference with TensorFlow on an Amazon SageMaker Notebook Instance\n",
    "\n",
    "This notebook demonstrates how to enable and utilize Amazon Elastic Inference with our predefined SageMaker TensorFlow containers on your SageMaker notebook instance. This notebook will train locally on your notebook instance and make inferences to the EI accelerator attached to your notebook instance.\n",
    "\n",
    "Amazon Elastic Inference (EI) is a resource you can attach to your Amazon EC2 instances to accelerate your deep learning (DL) inference workloads. EI allows you to add inference acceleration to an Amazon SageMaker hosted endpoint or Jupyter notebook for a fraction of the cost of using a full GPU instance. Since EI is only meant for inferences, no training logic changes are needed. For more information please visit: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html\n",
    "\n",
    "This notebook is an adaption of the [SageMaker TensorFlow Iris DNN classifier](https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/tensorflow_iris_dnn_classifier_using_estimators/tensorflow_iris_dnn_classifier_using_estimators.ipynb), with changes showing the easy changes needed to enable and use EI with TensorFlow on SageMaker.\n",
    "\n",
    "1. [The Iris dataset](#The-Iris-dataset)\n",
    "1. [Setup](#Setup)\n",
    "1. [tf.estimator](#tf.estimator)\n",
    "1. [Construct a deep neural network classifier](#Construct-a-deep-neural-network-classifier)\n",
    "    1. [Complete neural network source code](#Complete-neural-network-source-code)\n",
    "    1. [Using a tf.estimator in SageMaker](#Using-a-tf.estimator-in-SageMaker)\n",
    "    1. [Describe the training input pipeline](#Describe-the-training-input-pipeline)\n",
    "    1. [Describe the serving input pipeline](#Describe-the-serving-input-pipeline)\n",
    "1. [Train a Model on Amazon SageMaker using TensorFlow custom code](#Train-a-Model-on-Amazon-SageMaker-using-TensorFlow-custom-code)\n",
    "1. [Deploy the trained Model to an Endpoint with an attached EI accelerator](#Deploy-the-trained-Model-to-an-Endpoint-with-an-attached-EI-accelerator)\n",
    "    1. [Using EI with a SageMaker notebook instance](#Using-EI-with-a-SageMaker-notebook-instance)\n",
    "    1. [Invoke the Endpoint to get inferences locally](#Invoke-the-Endpoint-to-get-inferences-locally)\n",
    "    1. [Delete the Endpoint](#Delete-the-Endpoint)\n",
    "\n",
    "If you are familiar with SageMaker and already have a trained model, skip ahead to the [Creating-an-inference-endpoint section](#Creating-an-inference-endpoint-with-EI)\n",
    "\n",
    "For this example, we will be utilizing the SageMaker Python SDK, which helps deploy your models for training and hosting in SageMaker.\n",
    "\n",
    "In this tutorial, you'll use tf.estimator to construct a\n",
    "[neural network](https://en.wikipedia.org/wiki/Artificial_neural_network)\n",
    "classifier and train it on the\n",
    "[Iris data set](https://en.wikipedia.org/wiki/Iris_flower_data_set) to\n",
    "predict flower species based on sepal/petal geometry. You'll write code to\n",
    "perform the following five steps:\n",
    "\n",
    "1.  Deploy a TensorFlow container in SageMaker\n",
    "2.  Load CSVs containing Iris training/test data from a S3 bucket into a TensorFlow `Dataset`\n",
    "3.  Construct a `tf.estimator.DNNClassifier` neural network classifier\n",
    "4.  Train the model using the training data\n",
    "5.  Host the model in an endpoint with EI\n",
    "6.  Classify new samples invoking the endpoint\n",
    "\n",
    "This tutorial is a simplified version of TensorFlow's [get_started/estimator](https://www.tensorflow.org/get_started/estimator#fit_the_dnnclassifier_to_the_iris_training_data) tutorial but using SageMaker and the SageMaker Python SDK to simplify training and hosting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Iris dataset\n",
    "\n",
    "The [Iris data set](https://en.wikipedia.org/wiki/Iris_flower_data_set) contains\n",
    "150 rows of data, comprising 50 samples from each of three related Iris species:\n",
    "*Iris setosa*, *Iris virginica*, and *Iris versicolor*.\n",
    "\n",
    "![Petal geometry compared for three iris species: Iris setosa, Iris virginica, and Iris versicolor](https://www.tensorflow.org/images/iris_three_species.jpg) **From left to right,\n",
    "[*Iris setosa*](https://commons.wikimedia.org/w/index.php?curid=170298) (by\n",
    "[Radomil](https://commons.wikimedia.org/wiki/User:Radomil), CC BY-SA 3.0),\n",
    "[*Iris versicolor*](https://commons.wikimedia.org/w/index.php?curid=248095) (by\n",
    "[Dlanglois](https://commons.wikimedia.org/wiki/User:Dlanglois), CC BY-SA 3.0),\n",
    "and [*Iris virginica*](https://www.flickr.com/photos/33397993@N05/3352169862)\n",
    "(by [Frank Mayfield](https://www.flickr.com/photos/33397993@N05), CC BY-SA\n",
    "2.0).**\n",
    "\n",
    "Each row contains the following data for each flower sample:\n",
    "[sepal](https://en.wikipedia.org/wiki/Sepal) length, sepal width,\n",
    "[petal](https://en.wikipedia.org/wiki/Petal) length, petal width, and flower\n",
    "species. Flower species are represented as integers, with 0 denoting *Iris\n",
    "setosa*, 1 denoting *Iris versicolor*, and 2 denoting *Iris virginica*.\n",
    "\n",
    "Sepal Length | Sepal Width | Petal Length | Petal Width | Species\n",
    ":----------- | :---------- | :----------- | :---------- | :-------\n",
    "5.1          | 3.5         | 1.4          | 0.2         | 0\n",
    "4.9          | 3.0         | 1.4          | 0.2         | 0\n",
    "4.7          | 3.2         | 1.3          | 0.2         | 0\n",
    "&hellip;     | &hellip;    | &hellip;     | &hellip;    | &hellip;\n",
    "7.0          | 3.2         | 4.7          | 1.4         | 1\n",
    "6.4          | 3.2         | 4.5          | 1.5         | 1\n",
    "6.9          | 3.1         | 4.9          | 1.5         | 1\n",
    "&hellip;     | &hellip;    | &hellip;     | &hellip;    | &hellip;\n",
    "6.5          | 3.0         | 5.2          | 2.0         | 2\n",
    "6.2          | 3.4         | 5.4          | 2.3         | 2\n",
    "5.9          | 3.0         | 5.1          | 1.8         | 2\n",
    "\n",
    "For this tutorial, the Iris data has been randomized and split into two separate\n",
    "CSVs:\n",
    "\n",
    "*   A training set of 120 samples\n",
    "    iris_training.csv\n",
    "*   A test set of 30 samples\n",
    "    iris_test.csv\n",
    "\n",
    "These files are provided in the SageMaker sample data bucket:\n",
    "**s3://sagemaker-sample-data-{region}/tensorflow/iris**. Copies of the bucket exist in each SageMaker region. When we access the data, we'll replace {region} with the AWS region the notebook is running in."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Let's start by creating a SageMaker session and specifying the IAM role arn used to give training and hosting access to your data. See the [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) for how to create these. Note, if more than one role is required for notebook instances, training, and/or hosting, please replace the `sagemaker.get_execution_role()` with a the appropriate full IAM role arn string(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "isConfigCell": true
   },
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "\n",
    "sagemaker_session = sagemaker.Session()\n",
    "\n",
    "bucket = sagemaker_session.default_bucket()\n",
    "prefix = 'sagemaker/DEMO-tensorflow-iris'\n",
    "\n",
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook shows how to use the SageMaker Python SDK to run your code in a local container before deploying to SageMaker's managed training or hosting environments. Just change your estimator's train_instance_type to local or local_gpu. For more information, see [local mode](https://github.com/aws/sagemaker-python-sdk#local-mode).\n",
    "\n",
    "To use Amazon Elastic Inference locally change your `accelerator_type` to `local_sagemaker_notebook` when calling `deploy()`.\n",
    "\n",
    "***`local_sagemaker_notebook` will only work if you created your notebook instance with an EI accelerator attached to it.***\n",
    "\n",
    "In order to use this feature you'll need to install docker-compose (and nvidia-docker if training with a GPU). Running following script will install docker-compose or nvidia-docker-compose and configure the notebook environment for you.\n",
    "\n",
    "Note, you can only run a single local notebook at a time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!/bin/bash ./setup.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# tf.estimator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tf.estimator framework makes it easy to construct and train machine learning models via its high-level Estimator API. Estimator offers classes you can instantiate to quickly configure common model types such as regressors and classifiers:\n",
    "\n",
    "\n",
    "*   **```tf.estimator.LinearClassifier```**:\n",
    "    Constructs a linear classification model.\n",
    "*   **```tf.estimator.LinearRegressor```**:\n",
    "    Constructs a linear regression model.\n",
    "*   **```tf.estimator.DNNClassifier```**:\n",
    "    Construct a neural network classification model.\n",
    "*   **```tf.estimator.DNNRegressor```**:\n",
    "    Construct a neural network regression model.\n",
    "*   **```tf.estimator.DNNLinearCombinedClassifier```**:\n",
    "    Construct a neural network and linear combined classification model.\n",
    "*   **```tf.estimator.DNNRegressor```**:\n",
    "    Construct a neural network and linear combined regression model.\n",
    "    \n",
    "More information about estimators can be found [here](https://www.tensorflow.org/extend/estimators)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Construct a deep neural network classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete neural network source code \n",
    "\n",
    "Here is the full code for the neural network classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat \"iris_dnn_classifier.py\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With few lines of code, using SageMaker and TensorFlow, you can create a deep neural network model, ready for training and hosting. Let's give a deeper look at the code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using a tf.estimator in SageMaker\n",
    "Using a TensorFlow estimator in SageMaker is very easy, you can create one with few lines of code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def estimator(model_path, hyperparameters):\n",
    "    feature_columns = [tf.feature_column.numeric_column(INPUT_TENSOR_NAME, shape=[4])]\n",
    "    return tf.estimator.DNNClassifier(feature_columns=feature_columns,\n",
    "                                      hidden_units=[10, 20, 10],\n",
    "                                      n_classes=3,\n",
    "                                      model_dir=model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code above first defines the model's feature columns, which specify the data\n",
    "type for the features in the data set. All the feature data is continuous, so\n",
    "`tf.feature_column.numeric_column` is the appropriate function to use to\n",
    "construct the feature columns. There are four features in the data set (sepal\n",
    "width, sepal height, petal width, and petal height), so accordingly `shape`\n",
    "must be set to `[4]` to hold all the data.\n",
    "\n",
    "Then, the code creates a `DNNClassifier` model using the following arguments:\n",
    "\n",
    "*   `feature_columns=feature_columns`. The set of feature columns defined above.\n",
    "*   `hidden_units=[10, 20, 10]`. Three\n",
    "    [hidden layers](http://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw),\n",
    "    containing 10, 20, and 10 neurons, respectively.\n",
    "*   `n_classes=3`. Three target classes, representing the three Iris species.\n",
    "*   `model_dir=model_path`. The directory in which TensorFlow will save\n",
    "    checkpoint data during model training. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Describe the training input pipeline\n",
    "\n",
    "The `tf.estimator` API uses input functions, which create the TensorFlow\n",
    "operations that generate data for the model.\n",
    "We can use `tf.estimator.inputs.numpy_input_fn` to produce the input pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_input_fn(training_dir, hyperparameters):\n",
    "    training_set = tf.contrib.learn.datasets.base.load_csv_with_header(\n",
    "        filename=os.path.join(training_dir, 'iris_training.csv'),\n",
    "        target_dtype=np.int,\n",
    "        features_dtype=np.float32)\n",
    "\n",
    "    return tf.estimator.inputs.numpy_input_fn(\n",
    "        x={INPUT_TENSOR_NAME: np.array(training_set.data)},\n",
    "        y=np.array(training_set.target),\n",
    "        num_epochs=None,\n",
    "        shuffle=True)()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Describe the serving input pipeline\n",
    "\n",
    "After traininng your model, SageMaker will host it in a TensorFlow serving. You need to describe a serving input function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def serving_input_fn(hyperparameters):\n",
    "    feature_spec = {INPUT_TENSOR_NAME: tf.FixedLenFeature(dtype=tf.float32, shape=[4])}\n",
    "    return tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to submit the script for training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train a Model on Amazon SageMaker using TensorFlow custom code\n",
    "\n",
    "We can use the SDK to run our training script locally.\n",
    "\n",
    "1. Pass the path to the `iris_dnn_classifier.py` file, which contains the functions for defining your estimator, to the sagemaker.TensorFlow init method.\n",
    "2. Pass the S3 location that we uploaded our data to previously to the `fit()` method.\n",
    "3. Pass `local` into the `train_instance_type`. By passing local, training will be done inside of a Docker container on this notebook instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tensorflow import TensorFlow\n",
    "\n",
    "iris_estimator = TensorFlow(entry_point='iris_dnn_classifier.py',\n",
    "                            role=role,\n",
    "                            framework_version='1.12',\n",
    "                            train_instance_count=1,\n",
    "                            train_instance_type='local',\n",
    "                            training_steps=1000,\n",
    "                            evaluation_steps=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "import boto3\n",
    "\n",
    "# use the region-specific sample data bucket\n",
    "region = boto3.Session().region_name\n",
    "train_data_location = 's3://sagemaker-sample-data-{}/tensorflow/iris'.format(region)\n",
    "\n",
    "iris_estimator.fit(train_data_location)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy the trained Model to an Endpoint with an attached EI accelerator\n",
    "\n",
    "The `deploy()` method creates an endpoint which serves prediction requests in real-time.\n",
    "\n",
    "The only change required for utilizing EI with our SageMaker TensorFlow containers only requires providing an `accelerator_type` parameter, which determines which type of EI accelerator to attach to your endpoint. The supported types of accelerators can be found here: https://aws.amazon.com/sagemaker/pricing/instance-types/\n",
    "\n",
    "No code changes are necessary for your model, as our predefined TensorFlow containers utilizes TensorFlow serving, which has been modified to utilize EI for inference, as long as an EI accelerator is attached to the endpoint.\n",
    "\n",
    "## Using EI with a SageMaker notebook instance\n",
    "\n",
    "Here we're going to utilize the EI accelerator attached to our local SageMaker notebook instance. This can be done by using `local_sagemaker_notebook` as the value for `accelerator_type`. This will make an inference request against the TensorFlow Serving endpoint running on this Notebook Instance with an attached EI.\n",
    "\n",
    "An EI accelerator must be attached in order to make inferences using EI.\n",
    "\n",
    "As of now, an EI accelerator attached to a notebook will initialize for the first deep learning framework used to inference against EI. If you wish to use EI with another deep learning framework, please either restart or create a new notebook instance with the new EI.\n",
    "\n",
    "***`local_sagemaker_notebook` will only work if you created your notebook instance with an EI accelerator attached to it.*** \n",
    "\n",
    "***Please restart or create a new notebook instance if you wish to use EI with a different framework than the first framework used on this notebook instance as specified when calling `deploy()` with `local_sagemaker_notebook`for `accelerator_type`.***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "iris_predictor = iris_estimator.deploy(initial_instance_count=1,\n",
    "                                       instance_type='local',\n",
    "                                       accelerator_type='local_sagemaker_notebook')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Invoke the Endpoint to get inferences locally\n",
    "\n",
    "Invoking prediction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "iris_predictor.predict([6.4, 3.2, 4.5, 1.5]) #expected label to be 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Delete the Endpoint\n",
    "\n",
    "After you have finished with this example, remember to delete the prediction endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(iris_predictor.endpoint)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "\n",
    "iris_predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_tensorflow_p27",
   "language": "python",
   "name": "conda_tensorflow_p27"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  },
  "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 2
}