error: Failed after retrying 5 times: {"message": "Endpoint request timed out"}

[stevebanik]

Occurs when attempting to upload ~200 files

also:

error: Failed after retrying 5 times: {"message": "Internal server error"}

I'm thinking a potential issue is API rate limit exceeded, or perhaps something with Lambda function concurrency

[guregu]

Thanks, I've seen this happen sporadically. I think you're right about the causes. I'm thinking I will tweak the way uploads are processed, putting them in a real queue (SQS) so they can be handled a bit better.

[stevebanik]

I like the SQS approach - seems like a good one. I'm thinking CloudWatch Logs should show if API rate limits are exceeded, if Lambda is handling the uploads and is writing to CloudWatch. Interesting that HTTP error codes are not being passed, which could be helpful in troubleshooting.