Repository navigation
Conversation
3594960 to
37f9e49
Compare
…fails or the putObject future is cancelled
37f9e49 to
c0ae095
Compare
| @@ -191,6 +208,10 @@ public void onNext(CloseableAsyncRequestBody asyncRequestBody) { | |||
| CompletableFutureUtils.forwardExceptionTo(returnFuture, createMultipartUploadFuture); | |||
There was a problem hiding this comment.
This forwarding (and the same one in MultipartUploadHelper.createMultipartUpload) defeats abortWhenUploadIdKnown() when the CreateMultipartUpload response hasn't arrived yet:
- The body fails, or the caller cancels.
returnFuturecompletes exceptionally. - That failure is forwarded here, so
createMultipartUploadFuturefails too. BecauseBaseAsyncClientHandlerforwards it to the API call, the HTTP request is also cancelled. - The callback takes the
throwable != nullbranch and callsuploadIdFuture.completeExceptionally(...). abortWhenUploadIdKnown()usesthenAccept, so it never runs.
If S3 had already handled the request, the upload stays open with no parts. The SDK never learns its id, so the caller can only find it by listing the bucket's uploads. Zero parts means no storage cost, but this contradicts the description: "If the upload id is not known yet, the abort waits for the CreateMultipartUpload callback to deliver it."
Suggestion: don't forward a returnFuture failure into the CreateMultipartUpload future on this path. Let the call finish, and let the existing success branch (state != RUNNING -> abort through uploadIdFuture) clean up. The cost is that the client waits for one extra response after a cancel.
| if (t != null) { | ||
| s.cancel(); | ||
| MultipartUploadHelper.cancelingOtherOngoingRequests(futures, t); | ||
| if (tryMarkFailing()) { |
There was a problem hiding this comment.
The known-length subscriber also aborts when CompleteMultipartUpload is in flight:
if (tryMarkFailing() || state.get() == State.COMPLETING) {Here, a cancel during COMPLETING loses the CAS, so no abort is sent. CompleteMultipartUpload keeps going and can store the object at the key while the caller gets a CancellationException. The abort would race the complete, but it gives the caller the same best-effort outcome as the known-length path. Can these two match? If the difference is intended, a short comment saying why would help.
| /** | ||
| * Models a CreateMultipartUpload whose response has already arrived, so a failure forwarded to it is ignored. | ||
| */ | ||
| private static final class RespondedCreateMultipartUploadFuture extends CompletableFuture<CreateMultipartUploadResponse> { |
There was a problem hiding this comment.
RespondedCreateMultipartUploadFuture ignores completeExceptionally. That means CMPU_RESPONSE_PENDING_IN_CALLBACK (line 66) really models a response that has already arrived. No test covers a CreateMultipartUpload that is still pending when the body fails or the caller cancels, which is the case in my comment on line 208 of UnknownContentLengthAsyncRequestBodySubscriber. Could you add FailurePoint and CancelPoint cases that use a plain CompletableFuture, and assert that one abort is sent once the response arrives? Those cases fail on the current head.
Motivation and Context
This PR fixes two bugs in multipart
putObject(multipartEnabled(true)) when the content length is unknown.1. The request body fails, and the SDK still completes the upload
CreateMultipartUploadorUploadPartresponse, the SDK fails theputObjectfuture. It still sendsCompleteMultipartUploadwith the parts uploaded so far. When that request succeeds, S3 stores a partial object at the key. The SDK must abort the upload instead and leave the object at the key unchanged.onErrorsetsisDone, the same flagonCompletesets. The check that sendsCompleteMultipartUploadnever reads the failure flag. TheCreateMultipartUploadcallback also sends parts 1 and 2 without checking for a failure. A failure before the upload id is known sends no abort.2. The caller cancels, and the SDK leaves the parts in S3
putObjectfuture between part uploads, the SDK sends neitherCompleteMultipartUploadnorAbortMultipartUpload. S3 keeps the parts, and bills their storage, until a lifecycle rule or a manual abort removes them. The SDK must abort the upload instead.Modifications
UnknownContentLengthAsyncRequestBodySubscribernow uses the state model ofKnownContentLengthAsyncRequestBodySubscriber(Fix S3TransferManager upload pause hanging and racing completion #7516). OneStatefield (RUNNING,COMPLETING,FAILING) replaces thefailureActionInitiatedandcompletedMultipartInitiatedflags.RUNNINGtoFAILING.CompleteMultipartUploadneeds a move fromRUNNINGtoCOMPLETING, so a failed upload never completes. When the state moves toFAILING, the SDK sendsAbortMultipartUpload. If the upload id is not known yet, the abort waits for theCreateMultipartUploadcallback to deliver it. After a failure, the SDK starts no newUploadPartand closes the part bodies it does not upload.onSubscribenow rejects a null subscription and cancels a second subscription instead of the first (as Reactive Streams rules 2.13 and 2.5 require).Testing
Added unit tests
AI tools
License