Need advice to add experimental snooze logic to AWS API in python

Question

I have several lambda functions that create mutliple AWS Elastic beanstalk API Call, written in python. It worked fine. but from the last few days we get the Throttling error. After discussing with AWS, they said adding exponential return logic to the code. Therefore, if it is throttled, it will repeat the same API call for an incremental interval. I got what they say and how it works, but I don’t understand how to add to my code. They have documentation for the CLI, but they do not have for the API, as follows, http://docs.aws.amazon.com/general/latest/gr/api-retries.html

can someone please give me a simple example of how we can match the response of an API call and try again if it throttles like my one api call, which I use in my code as shown below,

import boto3 conn = boto3.client('elasticbeanstalk') response = conn.describe_environments(EnvironmentNames=["xyz"]) return response 

I know a simple way to do this, if the condition is by checking the answer "Speeding", while I think I can do it. but I want to check, as shown in the CLI example, how can I do this for the API?

Any help would be appreciated!

Answer 1

You can use the proxy object to wrap any AWS client object and add repeated logic for the proxy object:

 import retrying import wrapt class RetriedClient(wrapt.ObjectProxy): """Add retry logic to a boto3 client. Wait 2^x * 1000 milliseconds between each retry, up to 10 seconds, then 10 seconds afterwards. """ @retry(wait_exponential_multiplier=1000, wait_exponential_max=10000) def __getattr__(self, name): return getattr(self.__wrapped__, name) # Create a boto3 client to Cloudformation cf_client = boto3.client('cloudformation') # Add exponential backoff retries to all client methods wrapped_cf_client = RetryClient(cf_client) 

Then you can just use wrapped_cf_client , since you usually use the built-in boto3 client:

 resp = wrapped_cf_client.describe_stacks()