Skip to content

feat: S3 Express support #916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jun 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e45be08
feat: Add support for S3 Express customization
jbelkins Mar 14, 2025
0defc56
Fix codegen tests
jbelkins Mar 14, 2025
fa8c6d2
Fix swiftlint
jbelkins Mar 16, 2025
ac0fc16
Merge branch 'main' into jbe/s3_express
jbelkins Mar 21, 2025
159ff51
Merge branch 'main' into jbe/s3_express
jbelkins Apr 7, 2025
3fb1fdd
Merge branch 'main' into jbe/s3_express
jbelkins Apr 22, 2025
adcf54d
Merge branch 'main' into jbe/s3_express
jbelkins Apr 30, 2025
34ab12b
Merge remote-tracking branch 'origin/main' into jbe/s3_express
jbelkins May 2, 2025
12e4048
Merge branch 'main' into jbe/s3_express
jbelkins May 2, 2025
9a34126
Merge branch 'main' into jbe/s3_express
jbelkins May 8, 2025
22fe049
Merge branch 'main' into jbe/s3_express
jbelkins May 23, 2025
3d9bdc9
Merge branch 'main' into jbe/s3_express
jbelkins Jun 2, 2025
6f8daba
Merge branch 'main' into jbe/s3_express
jbelkins Jun 5, 2025
81f95eb
Mod auth scheme for rules-based signing name
jbelkins Jun 5, 2025
d6bd2bd
Fix codegen tests
jbelkins Jun 5, 2025
fcf7558
Merge branch 'main' into jbe/s3_express
jbelkins Jun 10, 2025
6cdaf0e
Remove Sendable from ClientConfiguration
jbelkins Jun 10, 2025
224ba4d
Merge branch 'main' into jbe/s3_express
jbelkins Jun 12, 2025
76d16da
Merge branch 'main' into jbe/s3_express
jbelkins Jun 12, 2025
3cd687d
Fix Swift6 issue with client config
jbelkins Jun 13, 2025
f3351f1
Correct build failure
jbelkins Jun 13, 2025
d2d8e79
Merge branch 'main' into jbe/s3_express
jbelkins Jun 18, 2025
1592869
Merge branch 'main' into jbe/s3_express
jbelkins Jun 20, 2025
4457293
Merge branch 'main' into jbe/s3_express
jbelkins Jun 23, 2025
03a9123
Merge branch 'main' into jbe/s3_express
jbelkins Jun 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions Sources/ClientRuntime/Config/Context+Config.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
//
// Copyright Amazon.com Inc. or its affiliates.
// All Rights Reserved.
//
// SPDX-License-Identifier: Apache-2.0
//

import class Smithy.Context
import class Smithy.ContextBuilder
import struct Smithy.AttributeKey

public extension Context {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allow the client config object to be placed into the context. This allows the client config to be passed to the S3 Express identity resolver, so that it can use the S3 client to obtain S3 Express credentials.


var clientConfig: DefaultClientConfiguration? {
get { get(key: clientConfigKey)?.clientConfig }
set { set(key: clientConfigKey, value: ClientConfigurationWrapper(clientConfig: newValue)) }
}
}

public extension ContextBuilder {

func withClientConfig(value: DefaultClientConfiguration?) -> Self {
let wrapped = ClientConfigurationWrapper(clientConfig: value)
attributes.set(key: clientConfigKey, value: wrapped)
return self
}
}

private let clientConfigKey = AttributeKey<ClientConfigurationWrapper>(name: "SmithySwiftClientConfigWrapper")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client config is wrapped in a @unchecked Sendable wrapper before adding it to the context, since Context requires that anything added to its storage be Sendable.

See comment on code immediately below for more details.

/// A wrapper used to allow a client configuration object to be placed in Context, since client config is not Sendable.
///
/// Placing the client config into Context is safe because the client config is not modified after being placed into Context.
/// Client config is unwrapped, then may be used to create a service client and make calls as part of performing an operation.
///
/// This type is public so that it may be accessed in other runtime modules. It is protected as SPI because it is a cross-module
/// implementation detail that does not affect customers.
///
/// `@unchecked Sendable` is used to make the wrapper Sendable even though it is technically not, due to the non-Sendable
/// client config stored within.
@_spi(ClientConfigWrapper)
public final class ClientConfigurationWrapper: @unchecked Sendable {
public let clientConfig: DefaultClientConfiguration

init?(clientConfig: DefaultClientConfiguration?) {
guard let clientConfig else { return nil }
self.clientConfig = clientConfig
}
}
19 changes: 19 additions & 0 deletions Sources/ClientRuntime/Config/IdentityPropertyKeys.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
//
// Copyright Amazon.com Inc. or its affiliates.
// All Rights Reserved.
//
// SPDX-License-Identifier: Apache-2.0
//

import struct Smithy.AttributeKey

public enum IdentityPropertyKeys {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client config is also wrapped before storing it as an identity property. The wrapper defined immediately above is used.


/// The service client config to be used in credential resolution.
///
/// Used only in conjunction with the `awsv4-s3express` auth scheme, which generates bucket-specific credentials
/// for use with the S3 Express service.
@_spi(ClientConfigWrapper)
public static let clientConfigWrapper =
AttributeKey<ClientConfigurationWrapper>(name: "ClientConfigurationWrapperIdentityKey")
}
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ extension EndpointResolverMiddleware: ApplyEndpoint {
signingName = param.signingName
case .sigV4A(let param):
signingName = param.signingName
case .sigV4S3Express(let param):
signingName = param.signingName
case .none:
break
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@ import enum SmithyHTTPAPI.EndpointPropertyValue
public enum EndpointsAuthScheme: Equatable {
case sigV4(SigV4Parameters)
case sigV4A(SigV4AParameters)
case sigV4S3Express(SigV4Parameters)
case none

/// The name of the auth scheme
public var name: String {
switch self {
case .sigV4: return "sigv4"
case .sigV4A: return "sigv4a"
case .sigV4S3Express: return "sigv4-s3express"
case .none: return "none"
}
}
Expand All @@ -36,6 +38,8 @@ extension EndpointsAuthScheme {
self = .sigV4(try SigV4Parameters(from: dictionary))
case "sigv4a":
self = .sigV4A(try SigV4AParameters(from: dictionary))
case "sigv4-s3express":
self = .sigV4S3Express(try SigV4Parameters(from: dictionary))
case "none":
self = .none
default:
Expand Down Expand Up @@ -156,7 +160,7 @@ public struct DefaultEndpointsAuthSchemeResolver: EndpointsAuthSchemeResolver {
/// Supported auth schemes by the SDK
let supportedAuthSchemes: Set<String>

public init(supportedAuthSchemes: Set<String> = ["sigv4", "sigv4a", "none"]) {
public init(supportedAuthSchemes: Set<String> = ["sigv4", "sigv4a", "sigv4-s3express", "none"]) {
self.supportedAuthSchemes = supportedAuthSchemes
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,14 @@ extension AuthSchemeMiddleware: SelectAuthScheme {
context: attributes
)
// Resolve identity using the resolver from auth scheme
let identity = try await identityResolver.getIdentity(identityProperties: option.identityProperties)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client config is wrapped, then added to identity properties (if any client config was stored to the context when the operation was started.)

The modified identity properties are then passed into the identity resolver. This is used by the S3 Express identity resolver to call S3.

var modifiedIdentityProperties = option.identityProperties
modifiedIdentityProperties.set(
key: IdentityPropertyKeys.clientConfigWrapper,
value: ClientConfigurationWrapper(clientConfig: attributes.clientConfig)
)
let identity = try await identityResolver.getIdentity(
identityProperties: modifiedIdentityProperties
)
// Save selected auth scheme
selectedAuthScheme = SelectedAuthScheme(
schemeID: option.schemeID,
Expand Down
1 change: 1 addition & 0 deletions Sources/SmithyHTTPAuth/CRTAdapters.swift
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ extension SigningAlgorithm {
switch self {
case .sigv4: return .signingV4
case .sigv4a: return .signingV4Asymmetric
case .sigv4s3express: return .signingV4S3Express
}
}
}
Expand Down
5 changes: 3 additions & 2 deletions Sources/SmithyHTTPAuth/SigV4AuthScheme.swift
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ public struct SigV4AuthScheme: AuthScheme {
value: context.isBidirectionalStreamingEnabled
)

// Set signing name and signing region flags
updatedSigningProperties.set(key: SigningPropertyKeys.signingName, value: context.signingName)
// Set resolved signing name and signing region flags
let signingName = updatedSigningProperties.get(key: SigningPropertyKeys.signingName) ?? context.signingName
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If signing properties include a signing name, use it, else fall back to the default signing name defined in the model's sigv4 trait.

updatedSigningProperties.set(key: SigningPropertyKeys.signingName, value: signingName)
updatedSigningProperties.set(key: SigningPropertyKeys.signingRegion, value: context.signingRegion)

// Set expiration flag
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,6 @@ public enum SigningAlgorithm: String, Sendable {
case sigv4
/// Signature Version 4 Asymmetric
case sigv4a
/// Signature Version 4 for S3 Express
case sigv4s3express = "sigv4-s3express"
}
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,9 @@ open class OperationEndpointResolverMiddleware(
// Write code that saves endpoint params to middleware context for use in auth scheme middleware when using rules-based auth scheme resolvers
if (AuthSchemeResolverGenerator.usesRulesBasedAuthResolver(ctx)) {
writer.write(
"context.set(key: \$N<EndpointParams>(name: \"EndpointParams\"), value: endpointParamsBlock(context))",
"context.set(key: \$N<EndpointParams>(name: \$S), value: endpointParamsBlock(context))",
SmithyTypes.AttributeKey,
"EndpointParams",
)
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ extension EventStreamTestClientTypes.TestStream {
val context = setupTests("eventstream.smithy", "aws.protocoltests.restjson#TestService")
println(context.manifest.files)
val contents = getFileContents(context.manifest, "Sources/Example/EventStreamTestClient.swift")
var expected = """
val expected = """
public func testStreamOp(input: TestStreamOpInput) async throws -> TestStreamOpOutput {
let context = Smithy.ContextBuilder()
.withMethod(value: .post)
Expand Down
Loading