Uploading files to S3 bucket is inconsistent

[rahulks993]

Describe the bug

I'm attempting to upload a zip file to an S3 bucket, but the process seems inconsistent. Sometimes the file uploads successfully, while other times it fails to do so. Despite checking the logs, which indicate that the upload method is being executed, the file isn't consistently being uploaded. Is there a potential explanation for this behavior?

Code & Dependency :

 protected static void uploadHeapDumpFileToS3Bucket(S3Client s3Client, String bucketName, Path zipFilePath, String zipFileName, Instant retentionDays) {
        try (s3Client) {
            logger.info("At the starting of uploadHeapDumpFileToS3Bucket method ");
            byte[] contentBytes = Files.readAllBytes(zipFilePath);
            String contentMD5 = calculateMD5(contentBytes);
            PutObjectRequest request = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(zipFileName)
                    .contentMD5(contentMD5)
                    .expires(retentionDays)
                    .build();
            PutObjectResponse response = s3Client.putObject(request, zipFilePath);
            logger.info("Uploaded Successfully! {} and the Response: {}", zipFileName, response);
            logger.info("Retention policy set for object '{}' in bucket '{}' until '{}'", zipFileName, bucketName, retentionDays);
        } catch (IOException | NoSuchAlgorithmException | S3Exception e) {
            logger.error("Something's broken: {}", e.getMessage());
        } finally {
            logger.info("Inside the finally block of uploadHeapDumpFileToS3Bucket method");
        }
    }

<dependency>
         <groupId>software.amazon.awssdk</groupId>
         <artifactId>s3</artifactId>
         <version>2.25.0</version>
</dependency>

Expected Behavior

 * This method uploads a heap dump file to an S3 bucket.
 *
 * @param s3Client     The S3 client used to interact with AWS S3.
 * @param bucketName   The name of the S3 bucket where the file will be uploaded.
 * @param zipFilePath  The path of the file to be uploaded.
 * @param zipFileName  The name of the file to be uploaded.
 * @param retentionDays The expiration date of the uploaded file.
 *
 * The method first reads all bytes from the file located at the provided path and calculates its MD5 hash.
 * Then, it creates a PutObjectRequest to define the parameters of the upload operation.
 * The file is uploaded to the S3 bucket using the putObject method of the S3 client.
 * If the upload is successful, it logs the success message and the response from the S3 service.
 * If any exception occurs during the process, it logs the error message.
 * The S3 client is automatically closed at the end of the operation thanks to the try-with-resources statement.
 

Current Behavior

The upload process seems to be inconsistent, sometimes it uploads file and sometimes it's not able to upload files to a S3 bucket.

We are actually trying to achieve this:

• Fetching the Task ID from the ECS metadata endpoint, which is done by another method
• Then we create a zip file in the same ECS container and then we are moving the zip file created to S3 bucket
• Uploading the archive to the S3 bucket with the constructed name.
• Sets the S3 object to expire in 30 days.

Reproduction Steps

protected static void uploadHeapDumpFileToS3Bucket(S3Client s3Client, String bucketName, Path zipFilePath, String zipFileName, Instant retentionDays) {
        try (s3Client) {
            logger.info("At the starting of uploadHeapDumpFileToS3Bucket method ");
            byte[] contentBytes = Files.readAllBytes(zipFilePath);
            String contentMD5 = calculateMD5(contentBytes);
            PutObjectRequest request = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(zipFileName)
                    .contentMD5(contentMD5)
                    .expires(retentionDays)
                    .build();
            PutObjectResponse response = s3Client.putObject(request, zipFilePath);
            logger.info("Uploaded Successfully! {} and the Response: {}", zipFileName, response);
            logger.info("Retention policy set for object '{}' in bucket '{}' until '{}'", zipFileName, bucketName, retentionDays);
        } catch (IOException | NoSuchAlgorithmException | S3Exception e) {
            logger.error("Something's broken: {}", e.getMessage());
        } finally {
            logger.info("Inside the finally block of uploadHeapDumpFileToS3Bucket method");
        }
    }

This method is used to upload file to s3 bucket, however after checking logs also it seems, that the behavior is quite inconsistent.

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.25.0

JDK version used

amazon-corretto-11

Operating System and version

Windows 11

[rahulks993]

@debora-ito @zoewangg Could you please help/suggest on this issue?

[debora-ito]

Is there a potential explanation for this behavior?

Logged errors are the best indicators of the possible cause. Can you share a stacktrace of the error?

[rahulks993]

Hi @debora-ito, I've just activated the aws-sdk logs at DEBUG level to investigate a peculiar issue. Whenever the code attempts to execute PutObjectResponse response = s3Client.putObject(request, zipFilePath);, it abruptly fails without any subsequent logs, even skipping the finally block.

Upon scrutinizing the DEBUG level logs, I observed it throws a SdkClientException related to credentials. Interestingly, this occurs regardless of whether the operation succeeds or fails. Notably, in case of failure, there's a lack of RequestId or ExtendedRequestId in the response.

Could you provide any insights on this? If you need more detailed logs, I can furnish them after a short delay.

[debora-ito]

Can you provide the full stacktrace of the error?

[github-actions]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.