Who stops my threads?

Question

I have some threads that queue for work, something like this:

class Worker(Thread): [...] def run(self): while not self.terminated: job = myQueue.get_nowait() job.dosomething() sleep(0.5) 

Now self.terminated is just the bool value that I use to exit the loop, but this is a problem, they stop working several times a day without my intervention. All of them, except for one: the application starts with, say, 5 workflows and at any time I check them, and one only works. All others have both _Thread__initialized and _Thread__stopped fields true. Threads and tasks do not interact with each other. What should I look for?

PS: I understand that it is very difficult to try to figure out the problem without the actual code, but it is huge.

UPDATE: in fact, Queue.Empty is the only exception to the trap - I think I believed that all internal job errors propagate without destroying eheh threads - so I'm going to block all exceptions and see ...

Answer 1

As an example, an exception inside the loop will stop the thread.

Why are you using get_nowait() and not get() ? What if the queue is empty?

Answer 2

If this is actual code, this is pretty obvious: myQueue.get_nowait() throws an exception ( Empty ) when the queue is empty!

Answer 3

Stackoverflow? :)

Answer 4

I have two suggestions.

1) get_nowait () will throw a Queue.Empty exception if there are no elements available. Make sure exceptions do not kill your threads.

2) Use get () instead. Put a None in turn to signal the exit of the stream instead of the Boolean flag. Then you do not need half a second, and you will process items faster.

 def run(self): while True: job = queue.get() if job: try: job.do_something() except Exception as e: print e else: # exit thread when job is None break 

Answer 5

Gil at one time, only the interpreter performs the task, if you want true parallelism you use multiprocessing, see http://docs.python.org/library/multiprocessing.html