Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CancellationException on cancelling PartUploadRequest #733

Closed
sdhuka opened this issue Oct 24, 2022 · 6 comments · Fixed by smithy-lang/smithy-kotlin#723
Closed

CancellationException on cancelling PartUploadRequest #733

sdhuka opened this issue Oct 24, 2022 · 6 comments · Fixed by smithy-lang/smithy-kotlin#723
Assignees
@aajtodd
Labels
bug This issue is a bug.

Comments

@sdhuka
Copy link

sdhuka commented Oct 24, 2022

Describe the bug

App occasionally crashes on cancelling the CouroutineScope used to upload a part for multi-part upload to S3.

Expected behavior

Cancelling the request should not result in CancellationException crash.

Current behavior

App occasionally crashes on cancelling the coroutine used to upload part for multi-part upload to S3. This issue is similar to #704

com.amplifyframework.storage.s3.test
10-24 16:38:34.828 19894 19998 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver disabled
10-24 16:38:34.828 19894 19998 D WM-Processor: Processor abd8f6ed-2f27-48c9-a5d0-a31817194d27 executed; reschedule = false
10-24 16:38:34.828 19894 19998 D WM-Processor: WorkerWrapper interrupted for abd8f6ed-2f27-48c9-a5d0-a31817194d27
10-24 16:38:34.830 19894 19998 D WM-Processor: Processor stopping background work 96d795cd-9448-46ac-8927-5c34c5a84311
10-24 16:38:34.830 19894 19998 D WM-Processor: WorkerWrapper could not be found for 96d795cd-9448-46ac-8927-5c34c5a84311
10-24 16:38:34.830 19894 19998 D WM-StopWorkRunnable: StopWorkRunnable for 96d795cd-9448-46ac-8927-5c34c5a84311; Processor.stopWork = false
10-24 16:38:34.831 19894 19925 I System.out: ERROR/amplify:aws-s3-storage:PartUploadTransferWorker: TransferWorker failed with exception: kotlinx.coroutines.JobCancellationException: Job was cancelled; job=JobImpl{Cancelling}@3dc7703
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: Work [ id=9d5f35b1-5b66-488c-adde-7ab5709b3b8b, tags={ 27, UPLOAD, awsS3StoragePlugin, PartUploadRequest, com.amplifyframework.storage.s3.transfer.worker.RouterWorker, MULTIPART_UPLOAD } ] was cancelled
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: java.util.concurrent.CancellationException: Task was cancelled.
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.futures.AbstractFuture.cancellationExceptionWithCause(AbstractFuture.java:1184)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.futures.AbstractFuture.getDoneValue(AbstractFuture.java:514)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.futures.AbstractFuture.get(AbstractFuture.java:475)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.WorkerWrapper$2.run(WorkerWrapper.java:311)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:91)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.execute(SerialExecutor.java:46)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.testing.TestWorkManagerImpl$1.executeOnBackgroundThread(TestWorkManagerImpl.java:80)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at androidx.work.impl.WorkManagerImpl.cancelUniqueWork(WorkManagerImpl.java:465)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.TransferOperations.pause$aws_storage_s3_debug(TransferOperations.kt:85)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.transfer.TransferManager.pause(TransferManager.kt:191)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.service.AWSS3StorageService.pauseTransfer(AWSS3StorageService.kt:157)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.pause$lambda-3(AWSS3StorageUploadFileOperation.kt:125)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.lambda$9rNG1iyetA60n9AvEWdNwKGJoz4(Unknown Source:0)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.operation.-$$Lambda$AWSS3StorageUploadFileOperation$9rNG1iyetA60n9AvEWdNwKGJoz4.run(Unknown Source:2)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
10-24 16:38:34.832 19894 19998 I WM-WorkerWrapper: 	at java.lang.Thread.run(Thread.java:919)
10-24 16:38:34.832 19894 19998 D WM-WorkerWrapper: Work interrupted for Work [ id=9d5f35b1-5b66-488c-adde-7ab5709b3b8b, tags={ 27, UPLOAD, awsS3StoragePlugin, PartUploadRequest, com.amplifyframework.storage.s3.transfer.worker.RouterWorker, MULTIPART_UPLOAD } ]
--------- beginning of crash
10-24 16:38:34.833 19894 20002 E AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher
10-24 16:38:34.833 19894 20002 E AndroidRuntime: Process: com.amplifyframework.storage.s3.test, PID: 19894
10-24 16:38:34.833 19894 20002 E AndroidRuntime: java.util.concurrent.CancellationException: Job was cancelled
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.ExceptionsKt.CancellationException(Exceptions.kt:22)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at aws.smithy.kotlin.runtime.http.engine.CoroutineUtilsKt$attachToOuterJob$cleanupHandler$1.invoke(CoroutineUtils.kt:39)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at aws.smithy.kotlin.runtime.http.engine.CoroutineUtilsKt$attachToOuterJob$cleanupHandler$1.invoke(CoroutineUtils.kt:37)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.InvokeOnCancelling.invoke(JobSupport.kt:1457)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1499)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.tryMakeCancelling(JobSupport.kt:795)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.makeCancelling(JobSupport.kt:755)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:671)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.parentCancelled(JobSupport.kt:637)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.ChildHandleNode.invoke(JobSupport.kt:1465)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1499)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath(JobSupport.kt:900)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:863)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelMakeCompleting(JobSupport.kt:696)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:667)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelInternal(JobSupport.kt:632)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancel(JobSupport.kt:617)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at kotlinx.coroutines.Job$DefaultImpls.cancel$default(Job.kt:183)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.CoroutineWorker$1.run(CoroutineWorker.kt:50)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:91)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.execute(SerialExecutor.java:46)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.testing.TestWorkManagerImpl$1.executeOnBackgroundThread(TestWorkManagerImpl.java:80)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at androidx.work.impl.WorkManagerImpl.cancelUniqueWork(WorkManagerImpl.java:465)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at com.amplifyframework.storage.s3.TransferOperations.pause$aws_storage_s3_debug(TransferOperations.kt:85)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at com.amplifyframework.storage.s3.transfer.TransferManager.pause(TransferManager.kt:191)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at com.amplifyframework.storage.s3.service.AWSS3StorageService.pauseTransfer(AWSS3StorageService.kt:157)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.pause$lambda-3(AWSS3StorageUploadFileOperation.kt:125)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.lambda$9rNG1iyetA60n9AvEWdNwKGJoz4(Unknown Source:0)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at com.amplifyframework.storage.s3.operation.-$$Lambda$AWSS3StorageUploadFileOperation$9rNG1iyetA60n9AvEWdNwKGJoz4.run(Unknown Source:2)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: 	at java.lang.Thread.run(Thread.java:919)
10-24 16:38:34.833 19894 20002 E AndroidRuntime: Caused by: kotlinx.coroutines.JobCancellationException: Job was cancelled; job=JobImpl{Cancelled}@3dc7703

Steps to Reproduce

  • Start multi-part upload
  • Canceling the coroutine will cause a crash.

Possible Solution

Catch the Exception and not bubble it up to the caller.

Context

No response

AWS Kotlin SDK version used

0.17.7-beta

Platform (JVM/JS/Native)

Native

Operating System and version

Android 10.
@sdhuka sdhuka added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Oct 24, 2022
@aajtodd aajtodd self-assigned this Oct 25, 2022
@aajtodd
Copy link
Collaborator

aajtodd commented Oct 25, 2022

I haven't been able to reproduce this but I can trace it through the OkHTTP source to get a picture of what is happening:

The OkHttp based engine adapts SdkByteReadChannel into an okhttp3.RequestBody. The method that we have to override is writeTo.

OkHttp invokes the writeTo method internally here when writing out the request body contents to the underlying connection. The entire block only looks for IOException.

Exchange.createRequestBody(...) is defined here which implements a write method that also only looks for IOException (and on catching one will mark the call failed).

There are no documentation or issues related to this to confirm but the underlying assumption seems to be that RequestBody.writeTo is only expected to fail with IOException.

I can see that the CallServerInterceptor implements Interceptor which does state:

For asynchronous calls made with Call.enqueue, an IOException is propagated to the caller indicating that the call was canceled. The interceptor’s exception is delivered to the current thread’s uncaught exception handler. By default this crashes the application on Android and prints a stacktrace on the JVM. (Crash reporting libraries may customize this behavior.)

I did find this comment interesting which wraps CancellationException in an IOException.

Cancellation is already wired up in executeAsync() which will cancel the underlying okhttp3.Call when the coroutine is cancelled. This isn't getting a chance to run because writeTo is tearing down the thread. When a call is cancelled it would close the underlying socket and normally propagate up as an IOException if the caller was still writing to the underlying Sink in RequestBody.writeTo(sink: Sink).

The internals of enqueue leads us to AsyncCall which is the outermost call site that wraps the entire call chain and deals with notifying via callback of any failures.

On cancellation this should result in the continuation being resumed with an exception. When the underlying coroutine is already cancelled though this would just be ignored.

TL;DR

I think what we want todo is either (1) ignore any CancellationException coming from our implementation of RequestBody.writeTo OR (2) wrap them in IOException which would result in exception being routed to the failure callback.

(1) would result in reaching this codepath first and then ultimately reaching the failure callback which would be ignored when the coroutine is already cancelling. (2) would result in the failure callback being invoked first which would also be ignored when the coroutine is already being cancelled.

I think the safest option would be (2) wrap all of our exceptions in writeTo as IOException because these are provided to the onFailure callback whereas other exceptions are provided to the callback and then thrown in the dispatcher thread

@sdhuka
Copy link
Author

sdhuka commented Oct 25, 2022

@aajtodd Thanks for diving deep into this issue. I agree with the second suggestion to convert the CancellationException to IOException to prevent okhttp crash the application.

@aajtodd aajtodd removed the needs-triage This issue or PR still needs to be triaged. label Oct 25, 2022
@aajtodd aajtodd mentioned this issue Oct 25, 2022
Merged
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@sdhuka sdhuka mentioned this issue Oct 31, 2022
Merged
1 task
@aajtodd aajtodd reopened this Nov 2, 2022
@sdhuka
Copy link
Author

sdhuka commented Nov 2, 2022

We are still observing this crash after updating to latest build; I have attached the new logs.

11-02 12:31:57.109 18503 18503 D WM-WorkerWrapper: Starting work for com.amplifyframework.storage.s3.transfer.worker.RouterWorker
11-02 12:31:57.127 18503 18503 D WorkmanagerTestUtils: failed to find task, error:Work with id a518453f-901f-4c3b-a33b-902a9fce9195 is not enqueued!
11-02 12:31:57.128 18503 18503 D WM-Processor: Processor: processing f7929fb1-eeb6-4d41-a2d8-0cb4657439f2
11-02 12:31:57.133 18503 18503 D WM-WorkerWrapper: Starting work for com.amplifyframework.storage.s3.transfer.worker.RouterWorker
11-02 12:31:57.139 18503 18503 D WM-Processor: Work 2ce65e1d-9d80-4615-9a0b-a07f5740615f is already enqueued for processing
11-02 12:31:57.139 18503 18503 D WorkmanagerTestUtils: failed to find task, error:Work with id a518453f-901f-4c3b-a33b-902a9fce9195 is not enqueued!
11-02 12:31:57.139 18503 18503 D WM-Processor: Work f7929fb1-eeb6-4d41-a2d8-0cb4657439f2 is already enqueued for processing
11-02 12:31:57.155 18503 18611 D WM-Processor: Processor cancelling 2ce65e1d-9d80-4615-9a0b-a07f5740615f
11-02 12:31:57.155 18503 18611 D WM-WorkerWrapper: Work interrupted for Work [ id=2ce65e1d-9d80-4615-9a0b-a07f5740615f, tags={ 27, UPLOAD, awsS3StoragePlugin, PartUploadRequest, com.amplifyframework.storage.s3.transfer.worker.RouterWorker, MULTIPART_UPLOAD } ]
11-02 12:31:57.156 18503 18611 D WM-Processor: Processor 2ce65e1d-9d80-4615-9a0b-a07f5740615f executed; reschedule = false
11-02 12:31:57.156 18503 18611 D WM-Processor: WorkerWrapper interrupted for 2ce65e1d-9d80-4615-9a0b-a07f5740615f
11-02 12:31:57.158 18503 18611 D WM-Processor: Processor cancelling a518453f-901f-4c3b-a33b-902a9fce9195
11-02 12:31:57.158 18503 18611 D WM-Processor: WorkerWrapper could not be found for a518453f-901f-4c3b-a33b-902a9fce9195
11-02 12:31:57.160 18503 18611 D WM-Processor: Processor cancelling f7929fb1-eeb6-4d41-a2d8-0cb4657439f2
11-02 12:31:57.160 18503 18611 D WM-WorkerWrapper: Work interrupted for Work [ id=f7929fb1-eeb6-4d41-a2d8-0cb4657439f2, tags={ 27, UPLOAD, awsS3StoragePlugin, PartUploadRequest, com.amplifyframework.storage.s3.transfer.worker.RouterWorker, MULTIPART_UPLOAD } ]
11-02 12:31:57.161  1180  1287 D PackageManager: setEnabledSetting : userId = 0 packageName = com.amplifyframework.storage.s3.test cmp = androidx.work.impl.background.systemalarm.RescheduleReceiver newState = 2 callingPackage = 17118/com.amplifyframework.storage.s3.test
11-02 12:31:57.163 18503 18611 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver disabled
11-02 12:31:57.163 18503 18611 D WM-Processor: Processor f7929fb1-eeb6-4d41-a2d8-0cb4657439f2 executed; reschedule = false
11-02 12:31:57.163 18503 18611 D WM-Processor: WorkerWrapper interrupted for f7929fb1-eeb6-4d41-a2d8-0cb4657439f2
11-02 12:31:57.167 18503 18618 I System.out: ERROR/amplify:aws-s3-storage:PartUploadTransferWorker: PartUploadTransferWorker failed with exception: kotlinx.coroutines.JobCancellationException: Job was cancelled; job=JobImpl{Cancelling}@2993cf0
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: Work [ id=2ce65e1d-9d80-4615-9a0b-a07f5740615f, tags={ 27, UPLOAD, awsS3StoragePlugin, PartUploadRequest, com.amplifyframework.storage.s3.transfer.worker.RouterWorker, MULTIPART_UPLOAD } ] was cancelled
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: java.util.concurrent.CancellationException: Task was cancelled.
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.futures.AbstractFuture.cancellationExceptionWithCause(AbstractFuture.java:1184)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.futures.AbstractFuture.getDoneValue(AbstractFuture.java:514)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.futures.AbstractFuture.get(AbstractFuture.java:475)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.WorkerWrapper$2.run(WorkerWrapper.java:311)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:91)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.utils.SerialExecutor.execute(SerialExecutor.java:46)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.testing.TestWorkManagerImpl$1.executeOnBackgroundThread(TestWorkManagerImpl.java:80)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at androidx.work.impl.WorkManagerImpl.cancelUniqueWork(WorkManagerImpl.java:465)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.TransferOperations.pause$aws_storage_s3_debug(TransferOperations.kt:85)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.transfer.TransferManager.pause(TransferManager.kt:191)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.service.AWSS3StorageService.pauseTransfer(AWSS3StorageService.kt:157)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.pause$lambda-3(AWSS3StorageUploadFileOperation.kt:125)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.lambda$9rNG1iyetA60n9AvEWdNwKGJoz4(Unknown Source:0)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at com.amplifyframework.storage.s3.operation.-$$Lambda$AWSS3StorageUploadFileOperation$9rNG1iyetA60n9AvEWdNwKGJoz4.run(Unknown Source:2)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
11-02 12:31:57.167 18503 18611 I WM-WorkerWrapper: 	at java.lang.Thread.run(Thread.java:919)
11-02 12:31:57.167 18503 18611 D WM-WorkerWrapper: Work interrupted for Work [ id=2ce65e1d-9d80-4615-9a0b-a07f5740615f, tags={ 27, UPLOAD, awsS3StoragePlugin, PartUploadRequest, com.amplifyframework.storage.s3.transfer.worker.RouterWorker, MULTIPART_UPLOAD } ]
11-02 12:31:57.169  1180  1287 D PackageManager: setEnabledSetting : userId = 0 packageName = com.amplifyframework.storage.s3.test cmp = androidx.work.impl.background.systemalarm.RescheduleReceiver newState = 2 callingPackage = 17118/com.amplifyframework.storage.s3.test
--------- beginning of crash
11-02 12:31:57.172 18503 18613 E AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher
11-02 12:31:57.172 18503 18613 E AndroidRuntime: Process: com.amplifyframework.storage.s3.test, PID: 18503
11-02 12:31:57.172 18503 18613 E AndroidRuntime: java.util.concurrent.CancellationException: Job was cancelled
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.ExceptionsKt.CancellationException(Exceptions.kt:22)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at aws.smithy.kotlin.runtime.http.engine.CoroutineUtilsKt$attachToOuterJob$cleanupHandler$1.invoke(CoroutineUtils.kt:39)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at aws.smithy.kotlin.runtime.http.engine.CoroutineUtilsKt$attachToOuterJob$cleanupHandler$1.invoke(CoroutineUtils.kt:37)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.InvokeOnCancelling.invoke(JobSupport.kt:1457)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1499)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.tryMakeCancelling(JobSupport.kt:795)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.makeCancelling(JobSupport.kt:755)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:671)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.parentCancelled(JobSupport.kt:637)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.ChildHandleNode.invoke(JobSupport.kt:1465)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1499)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath(JobSupport.kt:900)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:863)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelMakeCompleting(JobSupport.kt:696)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:667)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancelInternal(JobSupport.kt:632)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.JobSupport.cancel(JobSupport.kt:617)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at kotlinx.coroutines.Job$DefaultImpls.cancel$default(Job.kt:183)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.CoroutineWorker$1.run(CoroutineWorker.kt:50)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:91)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:93)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.testing.SynchronousExecutor.execute(SynchronousExecutor.java:30)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.scheduleNext(SerialExecutor.java:55)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.impl.utils.SerialExecutor.execute(SerialExecutor.java:46)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.testing.TestWorkManagerImpl$1.executeOnBackgroundThread(TestWorkManagerImpl.java:80)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at androidx.work.impl.WorkManagerImpl.cancelUniqueWork(WorkManagerImpl.java:465)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at com.amplifyframework.storage.s3.TransferOperations.pause$aws_storage_s3_debug(TransferOperations.kt:85)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at com.amplifyframework.storage.s3.transfer.TransferManager.pause(TransferManager.kt:191)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at com.amplifyframework.storage.s3.service.AWSS3StorageService.pauseTransfer(AWSS3StorageService.kt:157)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.pause$lambda-3(AWSS3StorageUploadFileOperation.kt:125)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at com.amplifyframework.storage.s3.operation.AWSS3StorageUploadFileOperation.lambda$9rNG1iyetA60n9AvEWdNwKGJoz4(Unknown Source:0)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at com.amplifyframework.storage.s3.operation.-$$Lambda$AWSS3StorageUploadFileOperation$9rNG1iyetA60n9AvEWdNwKGJoz4.run(Unknown Source:2)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: 	at java.lang.Thread.run(Thread.java:919)
11-02 12:31:57.172 18503 18613 E AndroidRuntime: Caused by: kotlinx.coroutines.JobCancellationException: Job was cancelled; job=JobImpl{Cancelled}@2993cf0
11-02 12:31:57.174 18503 18611 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver disabled
11-02 12:31:57.175 18503 18611 D WM-Processor: Processor stopping background work 2ce65e1d-9d80-4615-9a0b-a07f5740615f
11-02 12:31:57.175 18503 18611 D WM-Processor: WorkerWrapper could not be found for 2ce65e1d-9d80-4615-9a0b-a07f5740615f
11-02 12:31:57.175 18503 18611 D WM-StopWorkRunnable: StopWorkRunnable for 2ce65e1d-9d80-4615-9a0b-a07f5740615f; Processor.stopWork = false

@aajtodd aajtodd mentioned this issue Nov 3, 2022
Merged
@aajtodd
Copy link
Collaborator

aajtodd commented Nov 8, 2022

Closing as fixed now. Open a new ticket as necessary

@aajtodd aajtodd closed this as completed Nov 8, 2022
@github-actions
Copy link

github-actions bot commented Nov 8, 2022

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@trOnk12 trOnk12 mentioned this issue Dec 20, 2022
Closed
8 tasks
@trOnk12 trOnk12 mentioned this issue Dec 29, 2022
Closed
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aajtodd aajtodd

Labels
bug This issue is a bug.
Projects
None yet
Milestone
No milestone
2 participants
@aajtodd @sdhuka