feat(s3): add disableExpect100ContinueForPuts to S3Configuration#6774
Open
feat(s3): add disableExpect100ContinueForPuts to S3Configuration#6774
Conversation
61e752a to
2f98f89
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Motivation and Context
Reported in #6459 (comment) by the Apache Hadoop S3A team — they configure
ApacheHttpClient.builder().expectContinueEnabled(false)to disableExpect: 100-continuefor their use case, but the SDK'sStreamingRequestInterceptorre-adds the header anyway, making their setting ineffective. They requested a way to turn it off from the SDK side for rare emergencies where the header causes issues with proxies or S3-compatible stores.We considered adding this control at the
SdkHttpClientlevel, but since this behavior is specific to S3 (only S3'sStreamingRequestInterceptoradds the header), exposing it onSdkHttpClient.Builderwould mean adding public API surface across all HTTP clients and piping values from the HTTP layer up to S3 — when only Apache even has its own independentexpectContinueEnabledsetting. Putting it onS3Configurationkeeps the control scoped to where the behavior lives.Modifications
disableExpect100ContinueForPuts(Boolean)toS3Configuration(defaults tofalse, preserving existing behavior).StreamingRequestInterceptorto read the config fromSdkExecutionAttribute.SERVICE_CONFIGand skip adding theExpect: 100-continueheader when disabled.Expect: 100-continuevia its ownexpectContinueEnabledsetting (which defaults totrue), so disabling onS3Configurationalone won't suppress the header on the wire when using Apache.Example — fully disabling
Expect: 100-continuewith Apache:For all other HTTP clients (URL Connection, Netty, CRT), setting
disableExpect100ContinueForPuts(true)onS3Configurationis sufficient.Testing
StreamingRequestInterceptorTest): Parameterized tests coveringdisableExpect100ContinueForPutswithtrue,false,null, no service config, non-streaming requests (GetObject), and non-zero content-length — 7 scenarios via@MethodSource. Also covers RFC 9110 compliance (zero vs non-zero content length forPutObject/UploadPart).Expect100ContinueHeaderTest): WireMock-based end-to-end tests across 5 HTTP client types (Apache, Apache withexpectContinueEnabled=false, Apache withexpectContinueEnabled=true, URL Connection, CRT sync, CRT async, Netty) × 14 config combinations × 2 operations (PutObject+UploadPart). Verifies the actualExpectheader presence/absence on the wire.Log.LogLevel.Trace— confirmed the genericAwsCrtHttpClientpasses theExpectheader through (does not strip it).Types of changes
License