01a - Image Analysis with Computer Vision.ipynb

{"cells":[{"cell_type":"markdown","source":["# Image Analysis with the Computer Vision Service\n","\n","![A robot holding a picture](./images/computer_vision.jpg)\n","\n","*Computer Vision* is a branch of artificial intelligence (AI) that explores the development of AI systems that can \"see\" the world, either in real-time through a camera or by analyzing images and video. This is made possible by the fact that digital images are essentially just arrays of numeric pixel values, and we can use those pixel values as *features* to train machine learning models that can classify images, detect discrete objects in an image, and even generate text-based summaries of photographs.\n","\n","## Use the Computer Vision Cognitive Service\n","\n","Microsoft Azure includes a number of *cognitive services* that encapsulate common AI functions, including some that can help you build computer vision solutions.\n","\n","The *Computer Vision* cognitive service provides an obvious starting point for our exploration of computer vision in Azure. It uses pre-trained machine learning models to analyze images and extract information about them.\n","\n","For example, suppose Northwind Traders has decided to implement a \"smart store\", in which AI services monitor the store to identify customers requiring assistance, and direct employees to help them. By using the Computer Vision service, images taken by cameras throughout the store can be analyzed to provide meaningful descriptions of what they depict.\n","\n","### Create a Cognitive Services Resource\n","\n","Let's start by creating a **Cognitive Services** resource in your Azure subscription:\n","\n","1. In another browser tab, open the Azure portal at https://portal.azure.com, signing in with your Microsoft account.\n","2. Click the **&#65291;Create a resource** button, search for *Cognitive Services*, and create a **Cognitive Services** resource with the following settings:\n","    - **Name**: *Enter a unique name*.\n","    - **Subscription**: *Your Azure subscription*.\n","    - **Location**: *Choose any available region*:\n","    - **Pricing tier**: S0\n","    - **Resource group**: *Create a resource group with a unique name*.\n","3. Wait for deployment to complete. Then go to your cognitive services resource, and on the **Overview** page, click the link to manage the keys for the service. You will need the endpoint and keys to connect to your cognitive services resource from client applications.\n","\n","### Get the Key and Endpoint for your Cognitive Services resource\n","\n","To use your cognitive services resource, client applications need its endpoint and authentication key:\n","\n","1. In the Azure portal, on the **Keys and Endpoint** page for your cognitive service resource, copy the **Key1** for your resource and paste it in the code below, replacing **YOUR_COG_KEY**.\n","2. Copy the **endpoint** for your resource and and paste it in the code below, replacing **YOUR_COG_ENDPOINT**.\n","3. Run the code below by selecting the cell and then clicking the **Run cell** (&#9655) button to the left of the cell."],"metadata":{}},{"cell_type":"code","source":["cog_key = 'YOUR_COG_KEY'\n","cog_endpoint = 'YOUR_COG_ENDPOINT'\n","\n","print('Ready to use cognitive services at {} using key {}'.format(cog_endpoint, cog_key))"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599691487445}}},{"cell_type":"markdown","source":["Now that you've set up the key and endpoint, you can use the computer vision service to analyze an image.\n","\n","To do this from Python, you'll need to run the following cell to install the Azure Cognitive Services Computer Vision package. "],"metadata":{}},{"cell_type":"code","source":["! pip install azure-cognitiveservices-vision-computervision "],"outputs":[],"execution_count":null,"metadata":{"tags":[]}},{"cell_type":"markdown","source":["Now you're ready to go!\r\n","\r\n","Run the following cell to get a description for an image in the */data/vision/store_cam1.jpg* file."],"metadata":{}},{"cell_type":"code","source":["from azure.cognitiveservices.vision.computervision import ComputerVisionClient\n","from msrest.authentication import CognitiveServicesCredentials\n","from python_code import vision\n","import os\n","%matplotlib inline\n","\n","# Get the path to an image file\n","image_path = os.path.join('data', 'vision', 'store_cam1.jpg')\n","\n","# Get a client for the computer vision service\n","computervision_client = ComputerVisionClient(cog_endpoint, CognitiveServicesCredentials(cog_key))\n","\n","# Get a description from the computer vision service\n","image_stream = open(image_path, \"rb\")\n","description = computervision_client.describe_image_in_stream(image_stream)\n","\n","# Display image and caption (code in helper_scripts/vision.py)\n","vision.show_image_caption(image_path, description)\n"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599691518279}}},{"cell_type":"markdown","source":["That seems reasonably accurate.\n","\n","Let's try another image."],"metadata":{}},{"cell_type":"code","source":["# Get the path to an image file\n","image_path = os.path.join('data', 'vision', 'store_cam2.jpg')\n","\n","# Get a description from the computer vision service\n","image_stream = open(image_path, \"rb\")\n","description = computervision_client.describe_image_in_stream(image_stream)\n","\n","# Display image and caption (code in helper_scripts/vision.py)\n","vision.show_image_caption(image_path, description)"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599691524330}}},{"cell_type":"markdown","source":["Again, the suggested caption seems to be pretty accurate.\n","\n","## Analyze image features\n","\n","So far, you've used the Computer Vision service to generate a descriptive caption for a couple of images; but there's much more you can do. The Computer Vision service provides analysis capabilities that can extract detailed information like:\n","\n","- The locations of common types of object detected in the image.\n","- Location and approximate age of human faces in the image.\n","- Whether the image contains any 'adult', 'racy', or 'gory' content.\n","- Relevant tags that could be associated with the image in a database to make it easy to find.\n","\n","Run the following code to analyze an image of a shopper."],"metadata":{}},{"cell_type":"code","source":["# Get the path to an image file\n","image_path = os.path.join('data', 'vision', 'store_cam1.jpg')\n","\n","# Specify the features we want to analyze\n","features = ['Description', 'Tags', 'Adult', 'Objects', 'Faces']\n","\n","# Get an analysis from the computer vision service\n","image_stream = open(image_path, \"rb\")\n","analysis = computervision_client.analyze_image_in_stream(image_stream, visual_features=features)\n","\n","# Show the results of analysis (code in helper_scripts/vision.py)\n","vision.show_image_analysis(image_path, analysis)"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599691530747}}},{"cell_type":"markdown","source":["## Learn More\n","\n","In addition to the capabilities you've explored in this notebook, the Computer Vision cognitive service includes the ability to:\n","\n","- Identify celebrities in images.\n","- Detect brand logos in an image.\n","- Perform optical character recognition (OCR) to read text in an image.\n","\n","To learn more about the Computer Vision cognitive service, see the [Computer Vision documentation](https://docs.microsoft.com/azure/cognitive-services/computer-vision/)\n"],"metadata":{}}],"metadata":{"language_info":{"name":"python","version":"3.6.9","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kernelspec":{"name":"python3-azureml","language":"python","display_name":"Python 3.6 - AzureML"},"kernel_info":{"name":"python3-azureml"},"nteract":{"version":"nteract-front-end@1.0.0"}},"nbformat":4,"nbformat_minor":2}