How to handle request request error (408) in boto?

Question

We use the domain.select () method that boto provides to query SimpleDB. For small queries (queries containing a couple of hours of data) this method works fine. But when I start using multiple threads and longer queries (24 hours of data), it starts a timeout, giving the following error in standard mode:

-------------------------
         4 0 8
...
<?xml version="1.0"?>
<Response><Errors><Error><Code>QueryTimeout</Code><Message>A timeout occurred when attempting to query domain 'd110824' with query expression 'select * from `d110824` where `timestamp` &gt;= '2011-08-24T10:45:56' and `timestamp` &lt; '2011-08-25T10:45:56' and `identifier` = '00063F052C49' order by `timestamp` asc </Message><BoxUsage>0.0055590278</BoxUsage></Error></Errors><RequestID>....</RequestID></Response>

I want to implement a retry mechanism (exponential shutdown) when this error occurs. Boto makes no exceptions for this error and simply prints it. To implement the retry mechanism, I need some kind of error code or exception to know that an error has occurred.

Any thoughts on how to achieve this in boto?

Answer 1

Boto will try again at 503, but not at 408.

There are several things that will make boto try again, including 503 (the service is not available) and some types of HTTP errors when trying to connect. It will use exponential rollback and by default will work up to 5 times. You can change the number of attempts by setting the num_retries.boto configuration file:

[Boto]
num_retries = 3

I don’t know why he doesn’t try again at 408. The AWS documents that I saw recommend doing this.

Answer 2

I had the same problem with boto and I finished deploying it and adding exponential rollback directly. See https://github.com/datacratic/boto/blob/develop/boto/sdb/queryresultset.py#L83 .