aws-sigv4 failing to calculate the right signature when using "content-type: multipart/form-data" #13351
Description
I did this
NOTE: Changing the actual paths and hashes to some redacted values.
curl -vvv -X POST \
"https://my-api-gateway-endpoint.com/my-resource?my_query=param" \
--user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
--aws-sigv4 "aws:amz:${AWS_REGION}:execute-api" \
--header "x-amz-security-token: $AWS_SESSION_TOKEN" \
-H "accept: application/json" \
-H "Content-Type: multipart/form-data" \
-F "content=@${FILE};type=application/x-zip-compressed" \
-F "extra_metadata=@${JSON_META};type=application/json"Which gave me the following output:
Note: Unnecessary use of -X or --request, POST is already inferred.
* Host my-api-gateway-endpoint.com:443 was resolved.
* IPv6: (none)
* IPv4: x.x.x.x, x.x.x.x
* Trying x.x.x.x:443...
* Connected to my-api-gateway-endpoint.com (x.x.x.x) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / x25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
* subject: CN=my-api-gateway-endpoint.com
* start date: Apr 4 00:00:00 2024 GMT
* expire date: May 3 23:59:59 2025 GMT
* subjectAltName: host "my-api-gateway-endpoint.com" matched cert's "my-api-gateway-endpoint.com"
* issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M03
* SSL certificate verify ok.
* Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* Server auth using AWS_SIGV4 with user 'ASIA*******'
* [HTTP/2] [1] OPENED stream for https://my-api-gateway-endpoint.com/my-resource?my_query=param
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: my-api-gateway-endpoint.com]
* [HTTP/2] [1] [:path: /my-resource?my_query=param]
* [HTTP/2] [1] [authorization: AWS4-HMAC-SHA256 Credential=ASIA*******/20240411/eu-central-1/execute-api/aws4_request, SignedHeaders=accept;host;x-amz-date;x-amz-security-token, Signature=2fee53*****]
* [HTTP/2] [1] [x-amz-date: 20240411T130311Z]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [x-amz-security-token: IQoJb3JpZ********+la3yaTN6lr9AH+3JJCPquZB2VoKT]
* [HTTP/2] [1] [accept: application/json]
* [HTTP/2] [1] [content-length: 655]
* [HTTP/2] [1] [content-type: multipart/form-data; boundary=------------------------GlkWoiCdSW4YuTzdI6B8TJ]
> POST /my-resource?my_query=param HTTP/2
> Host: my-api-gateway-endpoint.com
> Authorization: AWS4-HMAC-SHA256 Credential=ASIA*******/20240411/eu-central-1/execute-api/aws4_request, SignedHeaders=accept;host;x-amz-date;x-amz-security-token, Signature=2fee53******
> X-Amz-Date: 20240411T130311Z
> User-Agent: curl/8.7.1
> x-amz-security-token: IQoJb3JpZ************+oSscHqSd+la3yaTN6lr9AH+3JJCPquZB2VoKT
> accept: application/json
> Content-Length: 655
> Content-Type: multipart/form-data; boundary=------------------------GlkWoiCdSW4YuTzdI6B8TJ
>
* upload completely sent off: 655 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 403
< date: Thu, 11 Apr 2024 13:03:11 GMT
< content-type: application/json
< content-length: 1701
< x-amzn-requestid: 050dbcba-4d82-46df-9c16-5b96fbeeb2b4
< x-amzn-errortype: InvalidSignatureException
< x-amz-apigw-id: WD_****Eaqg=
<
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'POST\n/my-resource\nmy_query=param\naccept:application/json\nhost:my-api-gateway-endpoint.com\nx-amz-date:20240411T130311Z\nx-amz-security-token:IQoJb3JpZ*******+oSscHqSd+la3yaTN6lr9AH+3JJCPquZB2VoKT\n\naccept;host;x-amz-date;x-amz-security-token\n2fa2cc887bff*******c9d22cce'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20240411T130311Z\n20240411/eu-central-1/execute-api/aws4_request\n6b6f17fc5e653b********c952640aa3d66'\n"}
* Connection #0 to host my-api-gateway-endpoint.com left intact
When using the curl command with only a single file and without Content-Type: multipart/form-data everything works as expected, but fails to sign properly when given Content-Type: multipart/form-data.
I expected the following
I expect for the signing to work and hit the logic behind my endpoint.
curl/libcurl version
curl 8.7.1 (aarch64-apple-darwin23.4.0) libcurl/8.7.1 (SecureTransport) OpenSSL/3.2.1 zlib/1.2.12 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libssh2/1.11.0 nghttp2/1.60.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
operating system
Darwin M-01619 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:49 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6020 arm64