Client-side encryption with presigned URLs

[ThisGuyCodes]

Describe the feature

Identical to this request in the Java SDK: aws/aws-sdk-java-v2#464

I would like to be able to pass a presigned URL to the s3 encryption client and it will use the configured encryption materials to encrypt/decrypt the object.

Use Case

Want to use client-side encryption when giving presigned URLs to a frontend (CLI frontend in my case) so that my infrastructure has no ability to read the contents of the upload (the goal is only the uploader has this information).

Proposed Solution

Allow the encryption client to take a presigned URL in the put_object and get_object methods.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

3.201.1

Environment details (OS name and version, etc.)

macOS Sonoma 14.5

[alextwoods]

I'm trying to understand this use case a bit more - so you are creating S3 presigned urls for PutObject and GetObject and providing those to a user who then performs their own client side encryption, and uses the presigned put object request to store it to your bucket.
The customer later uses a presigned GetObject URL to get the object and applies their own client side decryption to that object?

I'm not sure that requires the S3 Encryption client - users are free to encrypt/decrypt their objects using whatever libraries/keys they want and nothing need change in how you create the s3 get/put object presigned urls.

[ThisGuyCodes]

That is exactly the use case. However I'm operating under an assumption that s3 does some sort of enveloping (an assumption informed by documentation), and I'd rather not have to reverse-engineer / re-impliment this enveloping.

[alextwoods]

S3 doesn't do any of the enveloping, its all client side. But the envelop requires either a second get/put object request OR additional metadata headers - both of these I think would not work with presigned urls (eg: it would require you to know the details of the users envelop so that you could correctly presign either 1 or 2 urls to send them), so it may be better for users to BYO encryption (thus requiring you to know nothing about it).

[ThisGuyCodes]

ah, I see. Is there a way to encrypt-while-uploading with a KMS key as you describe? One of the advantages to the encryption client is that I don't have to encrypt the whole thing in memory / on disk before upload, and it just does the encryption while it uploads.

Right now I'm doing my own chunk-based loop with async, which is pretty detail oriented and manual.

[mullermp]

Presigned URLs are typically for GET but they can be used for presigned POST or PUT, but to do so, you have to supply the body in a separate request. That's why Alex was suggesting you bring your own encryption. I don't think this is possible to support. The only thing I could maybe think of is creating an encryption client with the :endpoint as your presigned url, setting :sigv4_signer to nil (undocumented and private option that we have) and maybe try doing a put object? Otherwise the logic you have is what is recommended. There are also service side encryption features you can use.

[ThisGuyCodes]

@mullermp what you describe with :endpoint sounds like it would do exactly what I need. Is :endpoint also undocumented, or do I just not know where to look for it?

Service side encryption would not fulfill my purpose.

[mullermp]

:endpoint is a public option but :sigv4_signer is not. Since the url is already signed, that's why I am suggesting to set that to nil to disable signing and see what happens.

[alextwoods]

Yeah, its definitely worth trying the :endpoint and sigv4_signer options, but I'm skeptical that it will work - the envelope encoding requires either an additional get/put request or additional headers to be added to the request, which breaks pre-signing I think.

If it doesn't work, you could consider using the Aws::S3::EncryptionV2::IOEncrypter and Aws::S3::EncryptionV2::IODecrypter as the bodies of your http request/response. Both of these wrap an underlying IO and provide the encryption/decryption, but do require you to construct the cipher yourself (and won't handle envelope).

[ThisGuyCodes]

@mullermp I cannot locate :endpoint in any of the documentation, can you link me to something?

@alextwoods I'll try it. the IOEncrypter and IODecrypter look great! I'm actually moving my implementation to use those. I can't seem to find them in the official documentation but I found third party docs.

Ideally though I could also support using KMS keys (from a different AWS account than the s3 bucket, for clarity).

Now that I know about the IOEncrypter option, this doesn't feel as clearly like a s3 SDK request; I initially made the request here because the encryption client already supported client-side encryption with your own cipher, so "just allow using presigned URLs" seemed like the most minimal change. Now it seems the answer is "home roll it, except if you have a non-presigned use case you can use this utility"?

[mullermp]

:endpoint is an option on the s3 client which would be passed into the encryption client. You can try that approach. You can also choose which headers to sign or not sign with the presigner, so you may have to use that option. I agree with you that "home roll it" is the best option in this case. I'm going to close this for now, but feel free to re-open if there's still issue or respond back with a solution for anyone else who comes across this.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.