Sarthi

Vercel for Backend! Easily setup preview environments with just Docker 🐳

Self-host Ephemeral (Preview) Environments with ease and forget about server management. Sarthi uses other open-source projects to export logs, enable monitoring, manage secrets and create preview environments. Let devs focus on building stuff in isolated environments rather than bashing heads together to manage conflicts.

It is meant to be used along with sarthi-deploy GitHub Action for setting up preview environments in your project. Every time there is a new branch or a PR created, Sarthi GHA will create a preview environment for that. It also takes care of deleting preview environments when respective branches or PRs are merged.

PS: Service Developers can directly jump to the Developer Guide

Pre-requisites 🛠️

1. Dockerized projects with a docker-compose.
   • It is MANDATORY to have a docker-compose file at the root of the project's folder.
2. A public Linux machine (preferred Ubuntu 20+ LTS versions) and user with root access.
3. A wildcard subdomain pointing to the above machine (\*.sarthi.your-domain.io)

General Flow

1. Create a public machine (preferred 4GB RAM, Ubuntu 20+ LTS versions) and map a wildcard domain to it.
2. Set up the project using the setup-sarthi.sh script present in the root folder.
3. Get the generated SECRET_TEXT and the deployed SERVER_URL after running the installation script.
4. Set up the Sarthi GitHub Action in your projects (No external bots - all your data within your servers).
5. Environment secrets for preview deployments are fetched from the Vault and added in a .env file and placed along with your docker-compose file.
   • Sarthi searches for .env.sample or sample.env and adds your secrets to project_name_branchname_hash path in Vault.
6. On any new events (PR open, close) Sarthi by default creates a unique URL for every service that is exposed in the docker-compose of the project.
7. You are ready to roll 🚀. GitHub Actions will comment on the deployment status according to different events!

Sarthi-Deploy GHA will not clutter your PR with comments - it will keep updating its earlier comment

Setup Instructions ⚙️

1. SSH into your server and clone the project.

git clone https://github.com/tushar5526/sarthi.git

2. Run the setup script.

chmod +x setup-sarthi.sh
chmod +x setup-vault.sh
sudo ./setup-sarthi.sh

3. Follow the prompts and specify the values, you will be requested to specify the wild card domain name created earlier. (using localhost is possible, but that would require setting up dnsmaq)

Services Installed 🤖

The following services are exposed:

1. Grafana + Loki to export service logs from the deployed environments. http://grafana.sarthi.your_domain.io
   • A dashboard named Service Logs is pre-seeded in Grafana. You can use this to filter service logs based on deployments, containers etc.

   

2. Portainer for admin access to manage deployments if needed. http://portainer.sarthi.your_domain.io

3. Hashicorp Vault to specify environment secrets. http://hashicorp.sarthi.your_domain.io

   • For each deployed branch/PR a path will be created by default in the vault where developers can specify branch-specific secrets.
   • 👉 PS: Hashicorp vault gets sealed on restarts. Unseal keys are generated by the setup script and stored in a keys.txt on the server. There is no RBAC yet and the root token is used to modify the env vars for different deployments. Root tokens can be found in keys.txt

   

4. Sarthi Backend for GHA. http://api.sarthi.your_domain.io

Developer Guide

Exposing services

1. Every service in docker-compose of which ports are exposed, is exposed to developers via a unique URL by Sarthi.
2. Sarthi currently only support fetching secrets from the vault and storing them in .env before deploying, so it's recommended to avoid env_file command or use it with .env files.

Secrets Discovery and namespacing

1. For each PR, Sarthi creates a preview environment using the docker-compose specified.
2. Before Submitting a Pull Request, install pre-commit using pip3 install pre-commit and install the pre-commit git hooks: pre-commit install.
3. Sarthi finds the secret for the service as follows.
   • Check the vault under the project/feature-branch namespace and find secrets there.
   • There is a default namespace reserved for developers to specify default secrets for all the PS. Secrets defined under project/default-dev-secrets are used if project/feature-branch secret path is empty.
   • If the default namespace is not configured as well, Sarthi automatically tries to find sample.env, env.sample, .env.sample and similar sample env files in the root directory and loads those sample environment variables to both default-dev-secrets and project/feature-branch

Tips 💡

1. Use docker-compose's service discovery to connect within the same services in your projects.

Contributor's Guide

A Makefile is provided at the project's root that can be used to set up the local environment for Sarthi easily. It needs to have docker installed on your system. Supported dev environments are either Mac or Linux, I have not tested it on Windows. Read more about Makefile.

High-Level Architecture

License 📄

This action is licensed under some specific terms. Check here for more information.