Access Denied s3cmd from EC2 machine

Question

I am trying to use the log rotation configuration for my nginx server, which I use as a reverse proxy machine located on an EC2 Ubuntu instance.

I want to save these logs in the S3 bucket after rotation, but I get a “denial of access, are you sure you have keys with ListAllMyBuckets authority” when I try to configure the s3cmd tools.

I'm sure my credentials are configured correctly in IAM, have tried at least five different credentials (even the root one) with the same result. It works great to list all my buckets from my local computer using aws-cli tools with the same credentials, which puzzles me that I don't have access only to my EC2 instance.

this is what i run:

which s3cmd /usr/local/bin/s3cmd s3cmd --configure --debug Access Key: ************** Secret Key: ******************************* Encryption password: Path to GPG program: /usr/bin/gpg Use HTTPS protocol: False HTTP Proxy server name: HTTP Proxy server port: 0 

and this is the result

 ... DEBUG: ConnMan.put(): connection put back to pool (http://s3.amazonaws.com#1) DEBUG: S3Error: 403 (Forbidden) DEBUG: HttpHeader: x-amz-id-2: nMI8DF+............ DEBUG: HttpHeader: server: AmazonS3 DEBUG: HttpHeader: transfer-encoding: chunked DEBUG: HttpHeader: x-amz-request-id: 5912737605BB776C DEBUG: HttpHeader: date: Wed, 23 Apr 2014 13:16:53 GMT DEBUG: HttpHeader: content-type: application/xml DEBUG: ErrorXML: Code: 'AccessDenied' DEBUG: ErrorXML: Message: 'Access Denied' DEBUG: ErrorXML: RequestId: '5912737605BB776C' DEBUG: ErrorXML: HostId: 'nMI8DF+............ ERROR: Test failed: 403 (AccessDenied): Access Denied ERROR: Are you sure your keys have ListAllMyBuckets permissions? 

The only thing facing my nginx server is load balancing, but I don’t understand why this might interfere with my request. Maybe this is something else that I missed?

Answer 1

I found a solution to my problems by removing all s3cmd installations. Then make sure that apt-get is updated and installed it again from apt-get. After my setup (same as before) it turned out just fine!

Answer 2

Please check that the user permission is the IAM whose keys you are using

Steps would be

• AWS console, go to the IAM panel.
• IAM user > Select User > in the bottom menu. Second tab Resolution

• add user policy

   { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:ListAllMyBuckets"], "Resource": "arn:aws:s3:::*" }, { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": "arn:aws:s3:::YOU-Bucket-Name" }, { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::YOU-Bucket-Name/*" } ] } 

Let me know how it works.

Answer 3

Please do not trust the -configure switch:

i ran into the same problem. it showed 403 in -configure, but in the end I saved the settings and then tried:

ERROR: test failed: 403 (AccessDenied): access denied
Repeat configuration? [Y / n] n
Save settings? [y / N] y
Configuration saved in '/root/.s3cfg'

 # s3cmd put MyFile s3://MyBucket/ 

& he worked.

Answer 4

s3cmd creates a file called .s3cfg in your home directory when you configure it. I would make sure that you put this file somewhere where your logrotate script can read this, and use the -c flag.

For example, to load the logfile.txt file into the logbucket bucket:

/ usr / local / bin / s3cmd -c / home / ubuntu / .s3cfg put logfile.txt s3: // logbucket

Answer 5

What is the version of s3cmd you are using?

I tried it with s3cmd 1.1, it seems that s3cmd 1.1 does not work with IAM roles.

But someone says s3cmd 1.5 alpha2 has IAM role support. ( http://t1983.file-systems-s3-s3tools.file-systemstalk.info/s3cmd-1-5-0-alpha2-iam-roles-supportincluded-t1983.html )

I tried s3cmd 1.5 beta1 ( https://github.com/s3tools/s3cmd/archive/v1.5.0-beta1.tar.gz ), it works fine with IAM roles.

So, there are two ways to access the s3 s3cmd substance:

• Using the passkey and private key `

  you need to install the configuration file in /root/.s3cfg (detection path), as shown below:

  access_key = XXXXXXXX secret_key = xxxxxxxxxxxxxxxxxxxxx

  Please note that just set the above two key values ​​in .s3cfg, there is no need for other keys.

  `
• Using IAM adds an s3 policy with s3cmd> 1.5 alph2. `

  you need to add an IAM instance for ec2, this role may contain the policy below:

  {"Effect": "Allow", "Action": ["S3:"], "Resource": ""} `

Answer 6

I also had a similar problem. Even after matching my EC2 instance with the IAM role with the s3 full access policy, my s3cmd failed because it did not have a .s3cfg file. I fixed the s3cmd version update.

sudo pip install s3cmd == 1.6.1

Did the trick!