GCP-jupyter-notebook

This ansible script provisions a machine on the Google Cloud Platform (GCP) with:

• Oracle Java 8

• spark_version: "2.1.0-bin-hadoop2.6"

• Anaconda3-4.3.1-Linux-x86_64.sh (python3)

• Jupyter Scala binaries for Scala 2.11 when using python3

• The Anacoda R Essentials bundle

License

The MIT License (MIT). See LICENSE.adoc

How to run

To get this project working, perform the following steps:

# Step 1 (in your local machine)
# create the jupyter machine
$ gcloud compute instances create jupyter --scopes compute-rw --image-family ubuntu-1604-lts --image-project ubuntu-os-cloud --zone us-central1-f --scopes https://www.googleapis.com/auth/devstorage.read_write --machine-type n1-standard-1

# Step 2 (in your local machine)
# SSH into the jupyter machine
$ gcloud compute ssh jupyter --zone us-central1-f

# Step 3 (in the jupyter VM)
# clone repo
$ git clone https://github.com/yencarnacion/GCP-jupyter-notebook

# Step 4 (in the jupyter VM)
# provision the machine
$ cd GCP-jupyter-notebook
$ sudo bash go.sh

# Step 5 (in the jupyter VM)
# get the jupyter notebook login token
$ sudo su ubuntu
$ /usr/lib/anaconda/bin/jupyter notebook list

# Step 6 (in your local machine)
# See Securely Connecting to VM Instances
# Forward ports from VM to local machine
# https://cloud.google.com/solutions/connecting-securely
$ gcloud compute ssh jupyter \
    --zone us-central1-f \
    --ssh-flag="-L" \
    --ssh-flag="8888:localhost:8888" \
    --ssh-flag="-N"

To Use:

• open http://localhost:8888?token=<your token> in your local machine browser

• change the directory to /home/ubuntu in the Jupiter Notebook

Notes about Anaconda R

# The R installed by Anaconda can be found at:
/usr/lib/anaconda/lib/R/bin/R
# it is different from the system installed R

# If you want to install R packages, you may want to open an R notebook and execute:
system("mkdir -p ~/anaconda3/lib/R/library", intern=TRUE)
system("echo R_LIBS=~/anaconda3/lib/R/library > ~/.Renviron", intern=TRUE)
# Then do a Kernel>Restart and in the first cell of your notebook put:
.libPaths("~/anaconda3/lib/R/library")
# You can then install R packages as follows:
install.packages("png","~/anaconda3/lib/R/library")
# or
/usr/lib/anaconda/lib/R/bin/R CMD INSTALL --library=~/anaconda3/lib/R/library brocolors_0.1.tar.gz

Other Notes:

• Last run successfully on March 26, 2017

• You need the GCP IAM 'Compute Instance Admin' and the 'Storage Object Admin' permissions

• remember to do gcloud compute instances delete jupyter --zone us-central1-f when you are done