diff --git a/INSTALL.md b/INSTALL.md index ff8ce88446..b8c18ebbfd 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -5,41 +5,39 @@ Use the provided bash script: ```bash -curl -fsSL https://get.fmtm.dev | bash +# Download the script +curl -L https://get.fmtm.dev -o install.sh +# OR (alternative URL) +curl -L https://fmtm.hotosm.org/install.sh -# OR - -curl -fsSL https://fmtm.hotosm.org/install.sh | bash +# Run the script +bash install.sh # Then follow the prompts ``` +> Note: it is best to run this script as a user other than root. +> However, if you run as root, a user svcfmtm will be created for you. + ## Manual Way -> NOTE: This is an installation guide to quickly get the fmtm app up and running. -> For a detailed guide on how to install the fmtm app using different -> methods and contributing, checkout the -> [dev docs](https://hotosm.github.io/fmtm/dev/Setup/) +If more details are required, check out the +[dev docs](https://hotosm.github.io/fmtm/dev/Setup/) -# Table of Contents +### Table of Contents 1. [Software Requirements](#software-requirements) -2. [Setting up the Backend](#setting-up-the-backend) +2. [Setting up FMTM](#setting-up-fmtm) - [Fork and Clone the FMTM repository](#fork-and-clone-the-fmtm-repository) - - [Development: Setup Your Local Environment](#development-setup-your-local-environment) + - [Development: Setup Your Local Environment](#setup-your-local-environment) - [Start the API with Docker](#start-the-api-with-docker) - - [Setup ODK Central User](#setup-odk-central-user) - - [Import Test Data](#import-test-data) - - [Check Authentication](#check-authentication) - -3. [Setting up the Frontend](#setting-up-the-frontend) + - [Setup ODK Central User (Optional)](#setup-odk-central-user-optional) + - [Import Test Data (Optional)](#import-test-data-optional) + - [Check Authentication (Optional)](#check-authentication-optional) - - [Fork and Clone the FMTM repository](#fork-and-clone-the-fmtm-repository-1) - - [Start the Frontends with Docker](#start-the-frontends-with-docker) - -# 1. Software Requirements +## Software Requirements Before you can install and use this application, you will need to have the following software @@ -48,181 +46,192 @@ installed and configured on your system: - [Git](https://git-scm.com/) - [Docker](https://docs.docker.com/) -To install Git, please follow the instructions on the official Git website: - - https://git-scm.com/downloads - -To install Docker, please follow the instructions on the official Docker website: +To install Git, please follow the instructions on the +[official Git website](https://git-scm.com/downloads) - https://docs.docker.com/engine/install/ +To install Docker, please follow the instructions on the +[official Docker website](https://docs.docker.com/engine/install/) -# 2. Setting up the Backend +## Setting up FMTM -## Fork and Clone the FMTM repository +### Fork and Clone the FMTM repository -### 1. Fork the repository +#### 1. Fork the repository Forking creates a copy of the repository in your own GitHub account. -Go to the [Field Mapping Tasking Manager repository](https://github.com/hotosm/fmtm) and click the "Fork" button in the top right corner of the page. +Go to the [Field Mapping Tasking Manager repository](https://github.com/hotosm/fmtm) +and click the "Fork" button in the top right corner of the page. -### 2. Clone the forked repository +#### 2. Clone the forked repository Clone the forked repository to your local machine using the following command: -`git clone https://github.com//fmtm.git` +```bash +git clone https://github.com//fmtm.git + +# If you wish to deploy for production, change to the main branch +git checkout main +``` Make sure to replace `` with your GitHub username. -## Development: Setup Your Local Environment +### Setup Your Local Environment These steps are essential to run and test your code! -### 1. Setup OSM OAUTH 2.0 +#### 1. Setup OSM OAUTH 2.0 -The FMTM uses OAUTH2 with OSM to authenticate users. To properly configure your FMTM project, you will need to create keys for OSM. +The FMTM uses OAUTH2 with OSM to authenticate users. -1. [Login to OSM](https://www.openstreetmap.org/login) (_If you do not have an account yet, click the signup button at the top navigation bar to create one_). Click the drop down arrow on the extreme right of the navigation bar and select My Settings. +To properly configure your FMTM project, you will need to create keys for OSM. -2. Register your FMTM instance to OAuth 2 applications. Put your login redirect url as `http://127.0.0.1:7051/osmauth/`, For Production replace the URL as production API Url +1. [Login to OSM](https://www.openstreetmap.org/login) + (_If you do not have an account yet, click the signup + button at the top navigation bar to create one_). + Click the drop down arrow on the top right of the navigation bar + and select My Settings. -> Note: `127.0.0.1` is required instead of `localhost` due to OSM restrictions. +2. Register your FMTM instance to OAuth 2 applications. + Put your login redirect url as `http://127.0.0.1:7051/osmauth/` if running locally, + or for production replace with https://{YOUR_DOMAIN}/osmauth/ -image + > Note: `127.0.0.1` is required for debugging instead of `localhost` + > due to OSM restrictions. -3. Right now _read user preferences permission_ is enough later on fmtm may need permission to modify the map option which should be updated on OSM_SCOPE variable on .env , Keep read_prefs for now. + ![image](https://user-images.githubusercontent.com/36752999/216319298-1444a62f-ba6b-4439-bb4f-2075fdf03291.png) -4. Now Copy your Client ID and Client Secret. Put them in the `OSM_CLIENT_ID` and `OSM_CLIENT_SECRET` of your `.env` file +3. Only the _read user preferences permission_ is required as of now. -### 2. Create an `.env` File +4. Now save your Client ID and Client Secret for the next step. -Environmental variables are used throughout this project. To get started, create `.env` file in the top level dir, a sample is located at `.env.example` +#### 2. Create an `.env` File - cp .env.example .env +Environmental variables are used throughout this project. +To get started, create `.env` file in the top level dir, +a sample is located at `.env.example`. -Your env should look like this: +This can be created interactively by running: -```dotenv -### ODK Central ### -ODK_CENTRAL_URL=https://proxy -ODK_CENTRAL_USER=`` -ODK_CENTRAL_PASSWD=`` +```bash +bash scripts/gen-env.sh +``` -### FMTM ### -# DEBUG=True -# LOG_LEVEL=DEBUG -FMTM_DOMAIN=fmtm.localhost +### Start the API with Docker -### OSM ### -OSM_CLIENT_ID=`` -OSM_CLIENT_SECRET=`` -OSM_URL=https://www.openstreetmap.org -OSM_SCOPE=read_prefs -OSM_LOGIN_REDIRECT_URI=http://127.0.0.1:7051/osmauth/ -OSM_SECRET_KEY= +This is the easiest way to get started with FMTM. -### S3 File Storage ### -S3_ENDPOINT="http://s3:9000" -S3_ACCESS_KEY=`` -S3_SECRET_KEY=`` +Docker runs each service inside **containers**, fully isolated from your +host operating system. -### Database (optional) ### -CENTRAL_DB_HOST=central-db -CENTRAL_DB_USER=odk -CENTRAL_DB_PASSWORD=odk -CENTRAL_DB_NAME=odk +##### Prerequisite -FMTM_DB_HOST=fmtm-db -FMTM_DB_USER=fmtm -FMTM_DB_PASSWORD=fmtm -FMTM_DB_NAME=fmtm -``` +You will need to [Install Docker](https://docs.docker.com/engine/install/) +and ensure that it is running on your local machine. -## Start the API with Docker +Then from the command line, navigate to the top level directory of the FMTM project. -The easiest way to get up and running is by using the FMTM Docker deployment. Docker creates a virtual environment, isolated from your computer's environment, installs all necessary dependencies, and creates a container for the database, the api, and the frontend. These containers talk to each other via the URLs defined in the docker-compose file and your env file. +##### Select the install type -### Starting the Containers +Determine the what type of FMTM install you would like: -1. You will need to [Install Docker](https://docs.docker.com/engine/install/) and ensure that it is running on your local machine. -2. From the command line, navigate to the top level directory of the FMTM project. -3. From the command line run: `docker compose pull`. - This will pull the latest container builds from **main** branch. -4. Once everything is pulled, from the command line run: `docker compose up -d api` -5. If everything goes well you should now be able to **navigate to the project in your browser:** `http://api.fmtm.localhost:7050/docs` +```text +main - the latest production +staging - the latest staging +development - the latest development (warning: may be unstable) +local test - used during development, or to start a test version +``` -> Note: If those link doesn't work, check the logs with `docker logs fmtm_api`. +The corresponding docker-compose files are: -## Setup ODK Central User +```text +main - docker-compose.main.yml +staging - docker-compose.staging.yml +development - docker-compose.development.yml +local test - docker-compose.yml +``` -The FMTM uses ODK Central to store ODK data. +Set your selection to a terminal variable to make the next step easier: -- By default, the docker setup includes a Central server. -- The credentials should have been provided in your `.env` file to automatically create a user. -- To create a user manually: +```bash +export GIT_BRANCH={your_selection} + +# E.g. +export GIT_BRANCH=development +``` + +##### Pull the Images ```bash -docker compose exec central odk-cmd --email YOUREMAIL@ADDRESSHERE.com user-create -docker-compose exec central odk-cmd --email YOUREMAIL@ADDRESSHERE.com user-promote +docker compose -f "docker-compose.${GIT_BRANCH}.yml" pull ``` -> Note: Alternatively, you may use an external Central server and user. +> This will pull the latest containers for the branch you selected. -## Import Test Data +##### Build the Frontend -Some test data is available to get started quickly. +Before we can run, you need to build your version of the frontend. -- Navigate to the `import-test-data` endpoint in the API docs page: - -- Click `Try it out`, then `execute`. +This is because the frontend contains variable specific to your deployment. -## Check Authentication +```bash +docker compose -f "docker-compose.${GIT_BRANCH}.yml" build ui +``` -Once you have deployed, you will need to check that you can properly authenticate. +##### Start the Containers -1. Navigate to `http://api.fmtm.localhost/docs` +```bash +docker compose -f "docker-compose.${GIT_BRANCH}.yml" up -d +``` - Three endpoints are responsible for oauth - image +You should see the containers start up in order. -2. Hit `/auth/osm_login/` : This will give you the Login URL where you can supply your osm username/password +Once complete, you should now be able to **navigate to the project in your browser:** - Response should be like this : +```text +https://{YOUR_DOMAIN} - {"login_url": "https://www.openstreetmap.org/oauth2/authorize/?response_type=code&client_id=xxxx"} +# For the local test setup, this will be +http://fmtm.localhost:7050 +``` - Now Copy your login_url and hit it in new tab, and you will be redirected to OSM for your LOGIN. Give FMTM the necessary permission +> Note: If those link doesn't work, check the logs with `docker logs fmtm-api`. +> Note: Use `docker ps` to view all container names. - After successful login, you will get your `access_token` for FMTM Copy it and now you can use it for rest of the endpoints that need authorizations +### Setup ODK Central User (Optional) -3. Check your access token: Hit `/auth/me/` and pass your `access_token` You should get your osm id, username and profile picture id +The FMTM uses ODK Central to store ODK data. -That's it, you have successfully set up the backend!! +- By default, the docker setup includes a Central server. +- The credentials should have been provided in your `.env` + file to automatically create a user. +- To create a user manually: -# 3. Setting up the Frontend +```bash +docker compose exec central odk-cmd --email YOUREMAIL@ADDRESSHERE.com user-create +docker-compose exec central odk-cmd --email YOUREMAIL@ADDRESSHERE.com user-promote +``` -## Fork and Clone the FMTM repository +> Note: Alternatively, you may use an external Central server and user. -### 1. Fork the repository +### Import Test Data (Optional) -Forking creates a copy of the repository in your own GitHub account. -Go to the [Field Mapping Tasking Manager repository](https://github.com/hotosm/fmtm) and click the "Fork" button in the top right corner of the page. +If running a local test version, test data is available to get started quickly. -### 2. Clone the forked repository +- Navigate to the `import-test-data` endpoint in the API docs page: + +- Click `Try it out`, then `execute`. -Clone the forked repository to your local machine using the following command: +### Check Authentication (Optional) -`git clone https://github.com//fmtm.git` +Once you have deployed, you will need to check that you can properly authenticate. -Make sure to replace `` with your GitHub username. +1. Navigate to your frontend (e.g. `http://fmtm.localhost:7050`) -## Start the Frontend with Docker +2. Click the 'Sign In' button and follow the popup prompts. -1. You will need to [Install Docker](https://docs.docker.com/engine/install/) and ensure that it is running on your local machine. -2. From the command line: navigate to the top level directory of the FMTM project. -3. From the command line run: `docker compose build ui` - This is essential, as the development container for the frontend is different to production. -4. Once everything is built, from the command line run: `docker compose up -d ui` +3. If successful, you should see your username in the header. -5. If everything goes well you should now be able to **navigate to the project in your browser:** +4. If you see an error instead, double check your credentials and + redirect URL in the openstreetmap.org settings. -That's it, you have successfully set up the frontend!! +That's it, you have successfully set up FMTM!! diff --git a/docs/dev/Backend.md b/docs/dev/Backend.md index 1a3f18fff9..48aee8573d 100644 --- a/docs/dev/Backend.md +++ b/docs/dev/Backend.md @@ -20,7 +20,7 @@ The easiest way to get up and running is by using the FMTM Docker deployment. Do 5. Once everything is pulled, from the command line run: `docker compose up -d api` 6. If everything goes well you should now be able to **navigate to the project in your browser:** `http://api.fmtm.localhost:7050/docs` -> Note: If that link doesn't work, check the logs with `docker log fmtm_api`. +> Note: If that link doesn't work, check the logs with `docker log fmtm-api`. > Note: the database host `fmtm-db` is automatically resolved by docker compose to the database container IP. diff --git a/docs/dev/Production.md b/docs/dev/Production.md index 040ebe721b..67cd54c4a7 100644 --- a/docs/dev/Production.md +++ b/docs/dev/Production.md @@ -142,7 +142,19 @@ runs, and you can access the working website from the domain name! - To access the database via GUI tool such as PGAdmin, it is possible using port tunneling. ```bash -ssh username@server.domain -N -f -L 5430:localhost:5432 +ssh username@server.domain -N -f -L {local_port}:localhost:{remote_port} + +# Example +ssh root@fmtm.hotosm.org -N -f -L 5430:localhost:5433 ``` This will map port 5432 on the remote machine to port 5430 on your local machine. + +## Manual Database Backups + +```bash +backup_filename="fmtm-db-backup-$(date +'%Y-%m-%d').sql.gz" +echo $backup_filename + +docker exec -i -e PGPASSWORD=PASSWORD_HERE fmtm_db pg_dump --verbose --format c -U fmtm fmtm | gzip -9 > "$backup_filename" +```