Skip to content

FTP-like interface over S3 to allow auth users (OAuth, SAML) manage files under some buckets and folders, with normal users and admins.

Notifications You must be signed in to change notification settings

davidayalas/aws-ftp-s3-serverless

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

82 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PoC AWS FTP S3 Serverless

The goal of this project is to provide a space for users under AWS S3 buckets (shared S3 buckets with a "folder" for every user), with a web interface to upload, browse, download and remove files.

This PoC has been done between asamo7 and davidayalas.

Features:

  • It's possible to setup "users" and "admins".

  • A user adds its username (email from saml response) to each key (uploading, browsing, deleting...) . An admin can see everything under a bucket or a folder. See permissions.csv.

  • A user can upload folders (drag and drop) and the structure is recreated in S3

  • Files can be downloaded (not directories)

  • Folders can be deleted (included not empty)

  • A user can create a folder

Architecture

architecture

Quick deploy

Deployment details

  • Install all dependencies

      $ npm install serverless-s3-sync
      $ npm --prefix ./backend/custom-auth install ./backend/custom-auth
      $ npm --prefix ./backend/login install ./backend/login
      $ npm --prefix ./frontend install ./frontend
    
  • Update "serviceName" with your own in setup.demo.json

  • Basic env variables for SAML (your own, because demo is setup):

    • SAML_CERT: you idp saml certificate as string
    • IDP_HOST: your idp
    • JWT_SECRET: to sign JWT from SAML and validate from custom authorizer
  • Generate frontend distribution

      $ cd frontend
      $ npm run build
    
  • Deploy demo

      $ sls deploy
      $ sls info | grep GET -m 1 | awk -F[/:] '{printf "const endpoint={get(){return '\''https://"$4"/demo/'\'';}};export default endpoint;"}' > frontend/src/assets/js/endpoint.js
      $ sls s3sync
    
  • Update your sp-metadata.xml with:

    • your issuer
    • your api gw endpoint
    • your api gw stage
  • Upload this file to https://samltest.id/upload.php

Sample interface

screen screen while uploading

Backend

  1. Upload: this lambda generates the signature for valid uploads.

  2. Browsing: this lambda retrieves the objects in a path

  3. Delete: this lambda deletes recursively all objects in a path

  4. Download: this lambda generate presigned urls for objects to be downloaded safely

  5. Login: in this case SAML based to generate a JWT token.

    • Default setup points to samltest.id. Metadata for samltest.id is generated with https://www.samltool.com/sp_metadata.php:

    • In "Attribute Consume Service Endpoint (HTTP-POST)" you have to put your api endpoint:

        https://${api gateway id}.execute-api.${region}.amazonaws.com/${stage}/login/callback
      
    • In "entityID" update your issuer (same in environment variable for login handler in serverless.yml)

  6. Custom authorizer: validates JWT token and adds extra permissions from CSV

Frontend

  • New frontend is Vue based. Then follow the Getting Started

  • Create file "frontend/src/assets/js/endpoint.js" with the following content replacing values with aproppiate

    const endpoint={
        get(){
            return 'https://${your api id}/${your stage}/';
        }
    };
        
    export default endpoint;
  • Or you can generate it from sls info output:

      $  sls info | grep GET -m 1 | awk -F[/:] '{printf "const endpoint={get(){return '\''https://"$4"/demo/'\'';}};export default endpoint;"}' > frontend/src/assets/js/endpoint.js
    

TODO

  • Quotas
  • Sharing

About

FTP-like interface over S3 to allow auth users (OAuth, SAML) manage files under some buckets and folders, with normal users and admins.

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages