Geek Answers Handbook

Creating a signed HTTP request for AWS Elasticsearch in Python

I am trying to make a simple Lambda Python that takes snapshots of our Elasticsearch database. This is done through the Elasticsearch REST API using simple HTTP requests.

However, for AWS, I have to sign these requests. I feel this can be achieved with boto3 low-level clients, probably with generate_presigned_url , but I can’t figure out for the rest of my life how to properly call this function. For example, what are valid ClientMethod s? I tried ESHttpGet , but to no avail.

Can someone point me in the right direction?

+9
source share
4 answers

There are several Python extensions in the request library that will execute the SigV4 signature for you. I used this one and it works well.

+8
source

For some time I struggled to do this. Currently, the boto3 library does not support the creation of signed es requests, although since I raised the question with them, it has become a function request.

Here is what I did in the meantime using the DavidMuller library mentioned above and boto3 to get my STS session credentials:

 import boto3 from aws_requests_auth.aws_auth import AWSRequestsAuth from elasticsearch import Elasticsearch, RequestsHttpConnection session = boto3.session.Session() credentials = session.get_credentials().get_frozen_credentials() es_host = 'search-my-es-domain.eu-west-1.es.amazonaws.com' awsauth = AWSRequestsAuth( aws_access_key=credentials.access_key, aws_secret_access_key=credentials.secret_key, aws_token=credentials.token, aws_host=es_host, aws_region=session.region_name, aws_service='es' ) # use the requests connection_class and pass in our custom auth class es = Elasticsearch( hosts=[{'host': es_host, 'port': 443}], http_auth=awsauth, use_ssl=True, verify_certs=True, connection_class=RequestsHttpConnection ) print(es.info())

Hope this saves you a little time.

+12
source

I recently posted requests-aws-sign , which provides an AWS V4 request subscription for the Python request library.

If you look at this code , you will see how you can use Botocore to generate a subscription to a V4 request.

+4
source

Although the other answers are fine, I wanted to exclude the use of external packages. Obviously, the TG40 itself has all the necessary functionality for signing requests, it was just a matter of viewing the source code. This is what I used to send AWS API requests directly (all of this is hard-coded for demo purposes):

  import boto3 import botocore.credentials from botocore.awsrequest import AWSRequest from botocore.endpoint import URLLib3Session from botocore.auth import SigV4Auth params = '{"name": "hello"}' headers = { 'Host': 'ram.ap-southeast-2.amazonaws.com', } request = AWSRequest(method="POST", url="https://ram.ap-southeast-2.amazonaws.com/createresourceshare", data=params, headers=headers) SigV4Auth(boto3.Session().get_credentials(), "ram", "ap-southeast-2").add_auth(request) session = URLLib3Session() r = session.send(request.prepare())
+3
source

More articles:


All Articles