Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding script to build Docker images with ETA installed #440

Merged
merged 14 commits into from
Mar 3, 2020
Merged
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.git/modules
91 changes: 91 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Dockerfile for building a base image with an ETA environment atop a
# Debian, Ubuntu, or related Linux distribution.
#
# ARGs:
# BASE_IMAGE
# TENSORFLOW_VERSION
#
# TensorFlow version notes:
# - For CPU-only images, use tensorflow==1.12.0
# - For GPU-enabled images, use the TensorFlow version compatible with the
# CUDA version you are using:
# - CUDA 8: tensorflow-gpu==1.4.0
# - CUDA 9: tensorflow-gpu==1.12.0
# - CUDA 10: tensorflow-gpu==1.14.0
#
# Copyright 2017-2020, Voxel51, Inc.
# voxel51.com
#
# Brian Moore, [email protected]
#

#
# The base image to build from, which must be a Debian, Ubuntu, or related
# Linux distribution
#

ARG BASE_IMAGE
FROM $BASE_IMAGE

#
# Install ETA
#
# Notes:
# ETA supports Python 2.7.X or Python 3.6.X
#
# `ppa:deadsnakes/ppa` is used in order to install Python 3.6 on Ubuntu 16.04
# https://askubuntu.com/questions/865554/how-do-i-install-python-3-6-using-apt-get
#
# `https://bootstrap.pypa.io/get-pip.py` is used to install pip on Python 3.6
# https://askubuntu.com/questions/889535/how-to-install-pip-for-python-3-6-on-ubuntu-16-10
#
# numpy==1.16.0 is enforced as a last step because tensorflow requires this
# version to function properly, and some commands here seem to mess with the
# numpy version installed via the `requirements.txt` file
#

RUN apt-get update \
&& apt-get -y --no-install-recommends install software-properties-common \
&& add-apt-repository -y ppa:deadsnakes/ppa \
&& apt-get update \
&& apt-get -y --no-install-recommends install \
sudo \
build-essential \
pkg-config \
ca-certificates \
cmake \
cmake-data \
unzip \
pciutils \
git \
curl \
wget \
python3.6 \
python3.6-dev \
libcupti-dev \
ffmpeg \
imagemagick \
&& ln -s /usr/bin/python3.6 /usr/local/bin/python \
&& curl https://bootstrap.pypa.io/get-pip.py | python

WORKDIR /usr/src/app
COPY . eta/

ARG TENSORFLOW_VERSION
RUN pip --no-cache-dir install --upgrade pip setuptools \
&& pip --no-cache-dir install -r eta/requirements.txt \
&& pip --no-cache-dir install --upgrade setuptools \
&& pip --no-cache-dir install -e eta/. \
&& pip --no-cache-dir install -I $TENSORFLOW_VERSION \
&& pip --no-cache-dir install --upgrade numpy==1.16.0 \
&& curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.6.1/protoc-3.6.1-linux-x86_64.zip \
&& unzip protoc-3.6.1-linux-x86_64.zip -d protoc3 \
&& rm -rf protoc-3.6.1-linux-x86_64.zip \
&& mv protoc3/bin/* /usr/local/bin \
&& mv protoc3/include/* /usr/local/include \
&& rm -rf protoc3 \
&& cd eta/tensorflow/models \
&& protoc research/object_detection/protos/*.proto \
--proto_path=research \
--python_out=research \
&& rm -rf /var/lib/apt
21 changes: 15 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,28 @@ ETA is very portable:
- Supports CUDA 8, 9 and 10 (for GPU installations)


## Installation
## Docker Installation

0. _(Optional but highly recommended)_ Activate a
[virtual environment](docs/virtualenv_guide.md)
If you prefer to operate via Docker, see the
[Docker Build Guide](docs/docker_build_guide.md) for simple instructions for
building a Docker image with an ETA environment installed.

1. Clone the repository:

## Local Installation

The following instructions describe how to install ETA locally on your machine.

0. _(Optional but highly recommended)_
[Activate a virtual environment](docs/virtualenv_guide.md)

1. Clone the repository

```shell
git clone https://github.com/voxel51/eta
cd eta
```

2. Run the install script:
2. Run the install script

```shell
bash install.bash
Expand Down Expand Up @@ -98,7 +107,7 @@ file to configure various package-level constants. Many advanced ETA features
such as pipeline building, model management, etc. require a properly configured
environment to function.

To setup your environment, create a copy the example configuration file
To setup your environment, create a copy the example configuration file:

```shell
cp config-example.json config.json
Expand Down
93 changes: 93 additions & 0 deletions docs/docker_build_guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Docker Build Guide

This document guides you through building a Docker image with a pre-installed
ETA environment.


## Dependencies

- [Docker Community Edition](https://hub.docker.com/search/?type=edition&offering=community)
- To build/run GPU-enabled images, you must install `nvidia-docker` by
following [these instructions](https://github.com/NVIDIA/nvidia-docker)


## Setup

In order to build an ETA Docker image, you must have ETA cloned on your machine
with a valid ETA `config.json` file installed:

```shell
# Clone repository
git clone https://github.com/voxel51/eta
cd eta

# Clone submodules
git submodule init
git submodule update

# Install your ETA config (customize as necessary)
cp config-example.json config.json
```


## Building an image

Follow the instructions below to build an ETA Docker image for your desired
environment.

These commands should be run from the root directory of your local ETA clone.


#### CPU example

The following snippet builds a CPU-enabled image on Ubuntu 16.04:

```shell
BASE_IMAGE=ubuntu:16.04
TENSORFLOW_VERSION=tensorflow==1.12.0
TAG=voxel51/eta:cpu-ubuntu-16.04

docker build \
--build-arg BASE_IMAGE="${BASE_IMAGE}" \
--build-arg TENSORFLOW_VERSION="${TENSORFLOW_VERSION}" \
--tag "${TAG}" \
.
```

#### GPU example

The following snippet builds a GPU-enabled image on Ubuntu 16.04 with CUDA 9.0:

```shell
BASE_IMAGE=nvidia/cuda:9.0-cudnn7-runtime-ubuntu16.04
TENSORFLOW_VERSION=tensorflow-gpu==1.12.0
TAG=voxel51/eta:gpu-ubuntu-16.04-cuda-9.0

docker build \
--build-arg BASE_IMAGE="${BASE_IMAGE}" \
--build-arg TENSORFLOW_VERSION="${TENSORFLOW_VERSION}" \
--tag "${TAG}" \
.
```


## Using an image

After you have built an ETA image, you run the image with an interactive
shell as follows:

```shell
# For CPU images
docker run -it $TAG

# For GPU images
docker run -it --runtime=nvidia $TAG
```


## Copyright

Copyright 2017-2020, Voxel51, Inc.<br>
voxel51.com

Brian Moore, [email protected]