Data Compass

Thanks for your interest in using the Data Compass software, which is part of IREX's Data Compass tool. The Data Compass tool helps teams and organizations improve their strategic use of data. As part of that tool, this software helps them survey and analyze how data flows between actors in a system. This software source code is made available under an Affero General Public License, and you're invited to modify and use it for your needs.

If you would like to use the software without deploying it yourself, please contact IREX for a license to use their readily available software. IREX is a global development and education organization. Please contact [email protected] with inquiries.

Technology

This project is based on Python with Django framework, Postgresql database and Webpack for frontend assets management.

Development Installation

Database Setup

PostgreSQL is used as a primary database engine.

On ubuntu or Debian based system to install and start Postgresql you can run something like

sudo apt update
sudo apt install postgresql postgresql-contrib libpq-dev
sudo service postgresql start

After installing Postgresql, you will need to initialize the database.

Login as Postgresql admin user (postgres)

sudo su -l postgres

While logged in as postgres create the project database

createdb datacompass

Connect to the database shell

psql datacompass

While you are in the database shell create the database user and grant appropriate privileges to the user.

CREATE USER datacompass WITH PASSWORD 'datacompass';
GRANT ALL PRIVILEGES ON DATABASE datacompass TO datacompass;
exit;

When running tests locally against postgresql database, you should allow the database user to have the CREATEDB role.

CREATE USER datacompass WITH PASSWORD 'datacompass';
GRANT ALL PRIVILEGES ON DATABASE datacompass TO datacompass;
ALTER USER datacompass CREATEDB;
exit;

Logout as postgres user

exit

You may use any database name, user name or password, just Make sure you keep the credentials because you will need them later in your project configuration.

Install system wide Python dependencies

Install Python development header files (python-dev) and Python package Installer pip

sudo apt install python3-dev python3-pip libz-dev libjpeg-dev libfreetype6-dev

Setup a Python virtual environment

It is recommended to isolate project dependencies in order to avoid potential dependency conflicts. One of the simplest ways to achieve that is by using Python virtual environments.

For development installation you may optionally use Virtualenvwrapper for convenience.

You can create a virtual environment for the project using any of your favorite tools.

Project setup

Download the source code

git clone https://github.com/tehamalab/datacompass.git

Go to project root

cd data-compass

make sure your python virtual environment is active then use pip to install project requirements.

pip install -r requirements/development.txt

Change your project settings according to your requirements.

Example; to enable debug mode

# .env file

DJANGO_DEBUG=True

Project setting which can modified using

• using system environment variables
• using environment variables written in .env file at the project root

To check if things are OK run

./manage.py check

Create database tables

./manage.py migrate

Create a superuser for admin access

./manage.py createsuperuser

NOTE: When you are executing manage.py ... commands make sure the vertualenv is active.

Starting the development server

Django comes with an inbuilt server which can be used during development. You shouldn't be using this server on production sites.

To start the development server go to your project root directory run

./manage.py runserver

Working with frontend assets

The most frontend Javascript, CSS (SaSS) and static images for UI files are managed using Webpack.

If you want to modify frontend assets; Install relevant dependencies using

npm install

To build static bundles which could be served in production run

npm run build

To build for development with live updates preview run

npm run watch

Running tests

To run unit tests make sure you database user has permission to create a database. On your database shell, You can give your user permission to create database executing something like:

ALTER USER datacompass CREATEDB;

To run all tests against multiple versions of Django and Python, use tox

tox

To run basic unit tests

./manage.py test

To check Python coding style, use flake8

flake8

To automatically sort imports, use isort

isort -rc .

Building Documentation

The project uses Sphinx for managing and compiling documentation.

To build the HTML documentation, Install documentation dependencies:

pip install -r requirements/docs.txt

Build the documentation:

make docs

The HTML docs will be created in docs/_build/html/ folder

Deployment

Data Compas can be deployed using any standard Django deployment. For more information on Django deployment please look for the available resources on the Internet including https://docs.djangoproject.com/en/3.0/howto/deployment/