Geek Answers Handbook

Overwrite S3 endpoint with Boto3 configuration file

OVERVIEW:

I am trying to overwrite specific variables in boto3 using the configuration file ( ~/aws/confg ). In my utility, I want to use the fakes3 service and send S3 requests to the local host.

Example:

In boto (not boto3 ), I can create a configuration in ~/.boto like this:

 [s3] host = localhost calling_format = boto.s3.connection.OrdinaryCallingFormat [Boto] is_secure = False

And the client can successfully select the necessary changes and instead of sending traffic to the real S3 service, he will send it to the local host.

 >>> import boto >>> boto.connect_s3() S3Connection:localhost >>>

WHAT DID I SAY:

Im trying to achieve a similar result using the boto3 library. After examining the source code, I found that I could use the location ~/aws/config . I also found an example configuration in the unittests botocore folder.

I tried to reconfigure to achieve the desired behavior. But, unfortunately, this does not work.

Here is the configuration:

 [default] aws_access_key_id = XXXXXXXXX aws_secret_access_key = YYYYYYYYYYYYYY region = us-east-1 is_secure = False s3 = host = localhost

Question:

  • How to overwrite clients variables using configuration file?
  • Where can I find a complete list of valid configuration variables?
+6
source share
4 answers

You cannot set the host in the configuration file, however you can cancel it from your code using boto3.

 import boto3 session = boto3.session.Session() s3_client = session.client( service_name='s3', aws_access_key_id='aaa', aws_secret_access_key='bbb', endpoint_url='http://localhost', )

Then you can interact as usual.

 print(s3_client.list_buckets())
+9
source

boto3 only reads the signature version for s3 from this configuration file. You might want to open a function request, but now you can access the user endpoint:

 import boto3 from botocore.utils import fix_s3_host resource = boto3.resource(service_name='s3', endpoint_url='http://localhost') resource.meta.client.meta.events.unregister('before-sign.s3', fix_s3_host)

This meta bit is important because boto3 automatically changes the endpoint to your_bucket_name.s3.amazonaws.com when it sees the desired 1 . If you work with both your own host and s3, you can refuse functionality, and not completely remove it.

+2
source

Another way:

 import boto3 s3client = boto3.client('s3', endpoint_url='http://XXxX:8080/', aws_access_key_id = 'XXXXXXX', aws_secret_access_key = 'XXXXXXXX') bucket_name = 'aaaaa' s3client.create_bucket(Bucket=bucket_name)
+1
source

using the boto3 resource:

 import boto3 # use third party object storage s3 = boto3.resource('s3', endpoint_url='https://URL:443', aws_access_key_id = 'AccessKey', aws_secret_access_key = 'SecertKey') # Print out bucket names for bucket in s3.buckets.all(): print(bucket.name)
0
source

More articles:


All Articles