Why does Python queue.Queue.get () allow me to return to the beginning of a timeout?

Question

UPDATE: this question is based on the erroneous mental behavior model of Queue.get (), which was triggered by some slightly ambiguous documentation, but mainly because of the erroneous, manual implementation of timedelta.total_seconds() . I found this error while trying to prove that the original answers are incorrect. Now that timedelta.total_seconds() provided by Python (since version 2.7), I will use it.

Sorry for the confusion.

---

This is not "Why is my code not starting?" question, but "What is the motivation for this design decision?"

Starting in 2.3, the Python queue module contains a Queue class with a get method that takes a timeout parameter. Here is the section from the manual:

Queue.get([block[, timeout]])
Remove and return the item from the queue. If the optional argument args is true and timeout is None (the default), block, if necessary, until the item is available. If the timeout is a positive number, it blocks no more than a timeout of seconds and throws an Empty exception if no items were available during this time. [...]

(Emphasis mine)

Note that this may raise an Empty exception even if it does not reach a timeout. In fact, I see this behavior on Ubuntu (but not on Windows). This is a guarantee a little earlier, and I had minor consequences for my code - I can encode it, though.

Most lock timeouts take a minimum timeout, which makes sense in real-time operating systems, including Windows and Linux. There is no guarantee that the OS will switch the context to your process or thread at any given time.

However, this requires a maximum timeout. Can someone explain how this constructive solution can make sense?

Answer 1

I think you are misinterpreting the documentation. He does not say that he can throw an Empty exception after less than timeout seconds, he says that he will block no more than timeout seconds. It can block less than if it can satisfy get .

I understand that you say that you see that it raises Empty earlier, but to be honest, it sounds like a mistake or that you rely on more accuracy than the system can provide. (It seems that, in order to obey the exact wording of the specification, the implementation should round the timeout until its timer is resolved, and not up, as you think.)

Answer 2

If you do not program in real time, the correct program should assume that any timeout value is a guide anyway, so it doesn’t matter which way you are wrong. Note that the queue does wait at least a timeout if the queue remains empty (confirmed by viewing the code, which basically is a loop that exits if remaining (time) is less than 0). It returns earlier if the item is queued in the meantime.

You expect this behavior:

 [0] (Thread) A calls Queue.get(True, 10) [0] B waits 9 seconds [9] B does something on X [11] A destroys X 

But a valid schedule would also be:

 [0] (Thread) A calls Queue.get(True, 10) [0] B waits 9 seconds [9] Operating system is busy for 2 seconds [11] A gets picked by the OS, destroys X [12] B does something on X 

You need to fix the program to work regardless of the actual time during which the thread is selected.

Answer 3

Queue.get has a specific bug, at least in python 2.6.6. In posix, a queue.get (timeout = 1) seems to exit almost immediately (raising an empty exception), while queue.get (timeout = 2) works fine.

I used one queue with parallel * get * ing threads on it ...