S3 file upload cloud formation setup

[emettely]

Context

We want to upload directly to S3 from the client using the signed URL.

Acceptance Criteria

• Set up S3 in Cloudformation and deploy in Cosmos (infrastructure repository)

-Make sure upload of files are only done through signed URLs, retrieved from the API-
(moved to a different ticket)

Some useful references

• S3 docs

[emettely]

I'm setting up the S3 as part of the API.

[emettely]

The S3 is setup for int and test bundled with the API. The permissions from the API is:

                                "s3:ListBucket",
                                "s3:*Object"

https://github.com/bbc/digital-paper-edit-infrastructure/blob/master/dpe-api/infrastructure/src/main.py

^ This is the bit that has that logic. @jamesdools and @pietrop , we might want to consider what the life-cycle (expire in 1 year or half a year...) of items in the bucket should be. I have it currently commented out - but once we figure out the subdirectory names (e.g. audio, original, preview etc.) we can uncomment + implement the lifecycle policy.

[pietrop]

@emettely can you add some more details on how this works?

I get there's something added to the cloud formation to give the EC2 instance for the API serve the capability to access the bucket(?)

Do we know more about how do you then access (add or get files from the S3 bucket) from within the express server inside the API? (do we use the AWS Node SDK?)

Also I see the link goes straight into master, was there a PR for this?

And is this still in progress ready for review or done?

[pietrop]

narrowing down this card to be only covering first part of acceptance criteria to do with

• Set up S3 in Cloudformation and deploy in Cosmos (infrastructure repository)

moving the rest to a separate card bbc/digital-paper-edit-api#4

[pietrop]

To review this (draft)

1. Use python script to generates cloud formation file
2. Use Jenkins to upload to AWS (stack job)
3. see if it gets picked up in int, test etc.. in cosmos
4. in AWS console you should see the resources with right policies

[emettely]

bbc/digital-paper-edit-infrastructure@b04120e...c33727b