Skip to content

Building a scalable backend using Python, BigQuery and Task queues to store high volume data for analytics.

Notifications You must be signed in to change notification settings

rd4704/big-data-collection

Repository files navigation

Big Data Collection on GCP

A Google Cloud Platform App

Works with Google App Engine Standard platform

Get Started

Create an App Engine application on GCP

Enable billing

Prepare service account

Get private key (p12 format)

Convert P12 to PEM key

cat privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > pk.pem

Place privatekey.pem file in the root directory

Prepare app libraries and dependencies by running the command

pip install -r requirements.txt -t lib

Download and install GoogleAppEngineLauncher from here

Add application in GoogleAppEngineLauncher, configure ports

Browse

You can test the application by using the test shell script file go-crazy.sh which makes post request to your GAE application as many times as specified in the argument

About

Building a scalable backend using Python, BigQuery and Task queues to store high volume data for analytics.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published