Python requests sometimes freeze

Question

I have a Python program that sends several requests (about 5-6) of poll requests in parallel, using different threads for each poll through a packet of requests. And I realized that some of my threads sometimes just freeze. When this happens, the server to which I am sending the request does not receive the request. I also set a timeout in the request, and it does not work.

try: print("This line prints") response = requests.head(poll_request_url, timeout=180) print("This line does not print when freeze occurs") except ReadTimeout: print("Request exception.") except RequestException as e: print("Request exception.") except Exception: print("Unknown exception.") print("This line does not print either when freeze occurs.") 

I do this on a Raspberry Pi 2 machine with the Raspbian operating system.

I used the same program without problems when I used Python 2.7. I recently switched to Python 3.5. I tested using both versions of queries from 2.8.1 and 2.9.1.

This problem does not occur often, but occurs 2-3 times a day on different threads.

What could be the problem? How can I debug this?

Edit: The problem was resolved by updating the Linux kernel.

Answer 1

According to the docs:

http://docs.python-requests.org/en/master/user/quickstart/#timeouts

A Timeout exception should be thrown when a timeout occurs. This would mean that the line:

 print("This line does not print when freeze occurs") 

Will not be called its timeout in fact.

Do you catch an exception? Or any other exception? It may be that it’s for sure, but you just don’t see it. Maybe try something like this:

 try: response = requests.head(poll_request_url, timeout=180) except requests.exceptions.Timeout: print("Timeout occurred") 

So you can see what is happening.

EDIT: perhaps this is a “connect” step that does not sync correctly. This can be a big timeout value for the "connect" step - it somehow messes it up. Perhaps try shortening the wait time (as indicated here):

http://docs.python-requests.org/en/master/user/advanced/#timeouts

eg.

 response = requests.head(poll_request_url, timeout=(3, 180)) 

Otherwise, it could be some kind of DNS lookup problem? Perhaps see if the hard encoding of the IP address causes the same problem?

Answer 2

The problem with using timers (from streaming import Timer) has been fixed. If the result is not the next 10 seconds - repeat, if the result is not the next 10 seconds - type "Error" and continue. You cannot control the status of the timer using the if statement if the request freezes, but you can do it through the while loop, adding time if the result is ok ( Python: run the code every n seconds and restart the timer provided ).