Skip to content

This project was created for final paper at the Federal University of Itajubá (UNIFEI). REST API data analysis system

License

Notifications You must be signed in to change notification settings

carloshssouza/data-controller-system

Repository files navigation

Logo

RESTGuardian

Summary


Description

RESTGuardian is a personal data and sensitive data controller system in REST API responses. It sits between your frontend and backend, analyzing the responses returned from the APIs and generating errors if any personal or sensitive data is leaked in the JSON of the response. This system is composed of three parts: Backend, Frontend and Proxy Server. This project is open source and free to use. This repository contains all the necessary files and documentation related to the project.

Technologies used

  • Typescript
  • NodeJS
    • Express
    • Socket.io
    • http-proxy
  • MongoDB
  • React (Vite)
  • gRPC (Protobuf)
  • Docker
  • Docker Compose

Objective

This system aims to not allow excessive exposure of data generated by REST APIs. The system can be used in the development environment, preventing insecure APIs from going to production. Not used in production, the system can be used to analyze the APIs of your backend and identify which APIs are returning personal and sensitive data, so you can fix them before going to production. This system have the objective to help you to be compliant with the Brazilian General Data Protection Law (LGPD) and try to prevent the OWASP Top 10 - 2019 - A3:2019-Excessive Data Exposure vulnerability. See more about this vulnerability in the link: https://github.com/OWASP/API-Security/blob/master/2019/en/src/0xa3-excessive-data-exposure.md


Installation

Now we will see how to install the project to analyze the responses of your APIs.

Requirements

You need to have installed the following tools:

  • Docker and Docker Compose
  • MongoDB(can be installed with Docker, local or cloud service)

How to install

  1. Create a docker-compose.yml file in the root of your project and add the following code:
version: "3.9"

services:
 backend-web-service:
   container_name: restguardian-backend-web-service
   image: carlos94souza/restguardian-backend-server:v1.0
   command: yarn start:prod
   environment:
     - EXPRESS_PORT=8000 //8000 is a example. Pass the port to run the express server (You can keep this for default)
     - PREFIX=/api/v1 // Pass the prefix to the express server(You can keep this for default)
     - GRPC_PORT=8080 // Pass the port to run the grpc server (You can keep this for default)
     - SECRET_KEY=secretkey // Pass the secret key to jwt
     - MONGO_URL=mongodb://mongo:27017/restguardian // Pass the mongo url(Can be a local mongo, docker mongo or a mongo atlas cloud)
   ports:
     - 8000:8000 // Expose the port to the express
     - 8080:8080 // Expose the port to the grpc
   networks:
     - my-network
   volumes:
     - backend-web-service-data:/usr/backend-web-service
 
 frontend-web-service:
   container_name: restguardian-frontend-web-service
   image: carlos94souza/restguardian-frontend-app:v1.0
   command: yarn start:prod
   environment:
     - VITE_BASE_URL=http://localhost:8000/api/v1 // Pass the base url to the vite(This need to be the same as the host express, so you can keep this for default)
     - VITE_BASE_URL_SOCKET=http://localhost:8000 // Pass the base url to the vite(without prefix)
     - VITE_DEV_SERVER_PORT=3001 // Pass the port to run the vite
   ports: 
     - 3001:3001 // Expose the port to the vite
   volumes: 
     - frontend-web-service-data:/usr/frontend-web-service
   networks:
     - my-network
   depends_on:
     - backend-web-service

 proxy-server:
   container_name: restguardian-proxy-server
   image: carlos94souza/restguardian-proxy-server:v1.0
   command: yarn start:prod
   environment:
     - TARGET=http://172.17.0.1:3333 // Pass the target to the proxy server (This is the host from your backend to be analyzed)
     - PROXY_PORT=8888 // Pass the port to the proxy server (Your frontend will call this port instead your backend port)
     - GRPC_HOST=restguardian-backend-web-service:8080 // Pass the grpc host
   ports:
     - 8888:8888 // Expose the port to the proxy server
   volumes:
     - proxy-server-data:/usr/proxy-server
   networks:
     - my-network
   depends_on:
     - backend-web-service
   restart: always

networks:
 my-network:
   driver: bridge

volumes:
 backend-web-service-data:
 frontend-web-service-data:
 proxy-server-data:
  1. Run the command docker-compose up -d in the same directory of the docker-compose.yml file.

  2. Now you can access the frontend and backend in the ports you provided in the docker compose file


How to configure

  1. Access the frontend in the port you passed in the docker compose file, so you will se the login page. Enter with the default credentials: admin and admin.(This user is created in the first run of the backend)

  2. Now you will see the dashboard page. Click in the APIs button in the navbar.

  3. According to your backend APIs, you need to register all the routes and indicate whether or not it allows the return of personal and sensitive data. Follow the gif below to register the routes.

  • Route name: Give a name to the route.
  • Endpoint path: The path of the route. You don't need pass the query string. For params is accepted : and {} Example:
    • /users CORRECT
    • /users/:id CORRECT
    • /users/:id/posts CORRECT
    • /users/:id/posts?limit=10&offset=0 WRONG -/user/{id}/posts CORRECT
  • Request Type: The type of the request. Example:
    • GET
    • POST
    • PUT
    • DELETE
    • PATCH
  • Allow Personal Data: If the route allows the return of personal data, check this option. This will control if the route can return personal data in the response, if can not return will, generate a error.
  1. Click in the button Add to register the route. You can register how many routes you want and see all in the table below. Follow the gif:
Logo
  1. For default, the backend generate a restrict data list for personal and sensitive data for default. You can see this list in the Configuration button in the navbar. This application follow the Brazilian General Data Protection Law (LGPD) rules. But, you can add, edit or delete as per your need. Can you check the site https://www.lgpdbrasil.com.br/ for more information about the LGPD and https://www.gov.br/cidadania/pt-br/acesso-a-informacao/lgpd/classificacao-dos-dados for more information about the data classification. Follow the gif below to add more data to the list.
Logo

6. Now, on the dashboard page you have the APIs registered and you can now make a request for the routes(Make sure your backend is running). If it is allowed to return personal or passenger data then it will not generate any error, but if it is not allowed then it will return a 500 error and information about the leaked data. You can see logs and charts about the data leaked. Follow the gif:
Logo

  1. In page Users, you can add users of type "user" or "admin". Both can access dashboard page, apis pages, configuration page and my account page. But, only the admin can access the users page(Can add, update and delete). Just for control, the admin user is created in the first run of the backend. So if you updated password and forgot, you can delete the user admin and run the backend container again. Follow the gif:
Logo

How to run locally

Requirements

You need to have installed the following tools:

  • NodeJS
  • Yarn
  • MongoDB(can be installed with Docker, local or cloud service)

Or you can use Docker and Docker Compose to run the project.

  • Docker and Docker Compose

Steps

Clone the repository of the project. git clone https://github.com/carloshssouza/data-controller-system.git

Backend(NodeJS+Express+MongoDB+gRPC)

  1. Access the backend-web-service folder and run the command yarn to install the dependencies.

  2. Create a .env file in the backend-web-service folder and add the following variables or you can use the .env.example file:

EXPRESS_PORT= # port
PREFIX= # prefix(optional)
GRPC_PORT= # grpc port
MONGO_URL= # mongo database url
SECRET_KEY= # secret key for jwt
  1. Run the command yarn start to run the backend

Frontend(React+Vite)

  1. Access the frontend-web-service folder and run the command yarn to install the dependencies(make sure the node version is 14.18+, 16+.), more information https://vitejs.dev/guide/.
  2. Create a .env file in the frontend-web-service folder and add the following variables or you can use the .env.example file:
VITE_BASE_URL= # baseurl backend with prefix(if you have)
VITE_BASE_URL_SOCKET= # baseurl backend without prefix
VITE_DEV_SERVER_PORT= # port to run the frontend
  1. Run the command yarn dev to run the frontend

Proxy server(NodeJS+gRPC)

  1. Access the proxy-server folder and run the command yarn to install the dependencies.
  2. Create a .env file in the proxy-server folder and add the following variables or you can use the .env.example file:
TARGET= # the target server(This will be your server)
PROXY_PORT= # the port to run the proxy server
GRPC_HOST= # the grpc host (this needs to be the same as the backend grpc host format, for default is 0.0.0.0:port)

  1. Run the command yarn start to run the proxy server

Now you can access the frontend and backend in the ports you provided in the .env file and the proxy server will wait for requests. Don't forget to run your backend(target) and register the routes in the frontend before make requests.


About

This project was created for final paper at the Federal University of Itajubá (UNIFEI). REST API data analysis system

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages