Geek Answers Handbook

Download from Amazon AWS S3 using POST -

I am creating a web application that includes a file upload function. My goal is to initiate a download from users directly into the S3 bucket. The strategy is to pre-sign the POST request, which will be submitted as a form.

The road block is a SignatureDoesNotMatch error - as far as I can tell, I followed the documentation and studied a lot of options, but still could not solve it. I can generate assigned download links.

Referencing:

AWS POST Documentation

Example

boto3 generate_presigned_post link

Create a signed request:

 def s3_upload_creds(name, user): s3 = boto3.client('s3') key = '${filename}' region = 'us-east-1' date_short = datetime.datetime.utcnow().strftime('%Y%m%d') date_long = datetime.datetime.utcnow().strftime('%Y%m%dT000000Z') fields = { 'acl': 'private', 'date': date_short, 'region': region, 'x-amz-algorithm': 'AWS4-HMAC-SHA256', 'x-amz-date': date_long } return s3.generate_presigned_post( Bucket = 'leasy', Fields = fields, Key = key, Conditions = [ {'acl': 'private'}, {'x-amz-algorithm': 'AWS4-HMAC-SHA256'}, {'x-amz-credential': '/'.join(['AKI--snip--', date_short, region, 's3', 'aws4_request'])}, {'x-amz-date': date_long} ] )

Download the form (filled in the fields above):

 <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> </head> <body> {{ creds }} <form action="{{ creds.url }}" method="post" enctype="multipart/form-data"> Key to upload: <input type="input" name="key" value="${filename}" /><br /> <input type="input" name="acl" value="{{ creds.fields.acl }}" /> <input type="hidden" name="Policy" value="{{ creds.fields.policy }}" /> <input type="text" name="X-Amz-Algorithm" value="{{ creds.fields['x-amz-algorithm'] }}" /> <input type="input" name="X-Amz-Credential" value="{{ creds.fields.AWSAccessKeyId }}/{{ creds.fields.date }}/us-east-1/s3/aws4_request" /> <input type="input" name="X-Amz-Date" value="{{ creds.fields['x-amz-date'] }}" /> <input type="input" name="X-Amz-Signature" value="{{ creds.fields.signature }}" /> File: <input type="file" name="file" /> <br /> <!-- The elements after this will be ignored --> <input type="submit" name="submit" value="Upload to Amazon S3" /> </form> </html>

Relevant part of the answer:

 <Error> <Code>SignatureDoesNotMatch</Code> <Message> The request signature we calculated does not match the signature you provided. Check your key and signing method. </Message> <AWSAccessKeyId>AKI--snip--</AWSAccessKeyId> <StringToSign> eyJjb25kaXRpb25zIjogW3siYWNsIjogInByaXZhdGUifSwgeyJ4LWFtei1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LCB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlDVjRNVlBUUlFHU1lLV1EvMjAxNTEyMTgvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LCB7IngtYW16LWRhdGUiOiAiMjAxNTEyMThUMDAwMDAwWiJ9LCB7ImJ1Y2tldCI6ICJsZWFzeSJ9LCBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAiIl1dLCAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTE4VDA1OjEwOjU2WiJ9 </StringToSign> <SignatureProvided>wDOjsBRc0iIW7JNtz/4GHgfvKaU=</SignatureProvided>

Base64 decoding of StringToSign in the above error:

 {u'conditions': [{u'acl': u'private'}, {u'x-amz-algorithm': u'AWS4-HMAC-SHA256'}, {u'x-amz-credential': u'AKI--snip--/20151218/us-east-1/s3/aws4_request'}, {u'x-amz-date': u'20151218T000000Z'}, {u'bucket': u'leasy'}, [u'starts-with', u'$key', u'']], u'expiration': u'2015-12-18T04:59:32Z'}
+9
source share
3 answers

Found a solution: I had to explicitly configure the s3 client to use the new Amazon v4 signature. The error occurs because the old version is used by default, which causes a mismatch. A little face - at a time when it was not written in the boto3 docs, although people at Amazon say it should be soon.

The method is simplified since it now returns exactly the required fields:

 def s3_upload_creds(name): BUCKET = 'mybucket' REGION = 'us-west-1' s3 = boto3.client('s3', region_name=REGION, config=Config(signature_version='s3v4')) key = '${filename}' return s3.generate_presigned_post( Bucket = BUCKET, Key = key )

This means that the form can be easily generated:

 <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> </head> <body> {{ creds }} <form action="https://mybucket.s3.amazonaws.com" method="post" enctype="multipart/form-data"> {% for key, value in creds.fields.items() %} <input type="hidden" name="{{ key }}" value="{{ value }}" /> {% endfor %} File: <input type="file" name="file" /> <br /> <input type="submit" name="submit" value="Upload to Amazon S3" /> </form> </html>

Greetings

+5
source

Several years have passed since the last answer, but I was stuck with this for the last one or two days, so I will share my experience with everyone who can help it.

I get the error: "403: The AWS passkey code that you specified does not exist in our records" when I try to load into the s3 bucket through my pre-assigned URL.

I managed to generate a pre-assigned URL, similar to the above, using server-side code:

 signed_url_dict = self.s3_client.generate_presigned_post( self.bucket_name, object_name, ExpiresIn=300

This returned a dictionary with the structure:

 { url: "https://___", fields: { key: "___", AWSAccesKeyId: "___", x-amz-security-token: "___", policy: "___", signature: "___" } }

This led to the fact that in 2019 everything was a bit different with javascript on the browser side, where the required form input changed. Instead of setting up the form as an OP, I had to create my form as shown below:

 <form action="https://pipeline-poc-ed.s3.amazonaws.com/" method="post" enctype="multipart/form-data" name="upload_form"> <!-- Copy ALL of the 'fields' key:values returned by S3Client.generate_presigned_post() --> <input type="hidden" name="key" value="___" /> <input type="hidden" name="AWSAccessKeyId" value="___" /> <input type="hidden" name="policy" value="___"/> <input type="hidden" name="signature" value="___" /> <input type="hidden" name="x-amz-security-token" value="___" /> File: <input type="file" name="file" /> <br /> <input type="submit" name="submit" value="Upload to Amazon S3" /> </form>

My mistake was that I followed the example in the documentation for boto3 1.9.138 and did not specify "x-amz-security-token" in the form, which turned out to be absolutely necessary. A thoughtless oversight may part, but I hope this helps someone else.

EDIT: My results above were based on N. Virginia's lambda function. When I ran generate_presigned_post(...) in Ohio (the region where my cart is located), I got results similar to OP:

 { "url": "https://__", "fields": { "key": "___", "x-amz-algorithm": "___", "x-amz-credential": "___", "x-amz-date": "___", "x-amz-security-token": "___", "policy": "___", "x-amz-signature": "___" } }

Perhaps function results vary by region?

+1
source

In my case, I generated a Base64 encoded form.

The problem was because Firefox inherently encodes the Base64 policy and security token values ​​encoded on top of it.

Thus, there was double coding, and therefore the signature did not meet the requirements.

0
source

More articles:


All Articles