I see RunTime errors on exit when using two multiprocessing.Pool each with a different merge queue. The following is a simple code example showing the problem.
import multiprocessing
import time
q_1 = multiprocessing.JoinableQueue()
q_2 = multiprocessing.JoinableQueue()
def process():
pool_1 = multiprocessing.Pool(1, worker_1,(q_1,))
pool_2 = multiprocessing.Pool(1, worker_2,(q_2,))
for i in range(3):
q_1.put(i)
q_1.join()
q_2.join()
pool_1.close()
pool_2.close()
def worker_1(input_q):
while True:
val = input_q.get(True)
print("1:"+str(val))
q_2.put(val)
input_q.task_done()
def worker_2(input_q):
while True:
val = input_q.get(True)
time.sleep(2)
print("2:"+str(val))
input_q.task_done()
When this happens, I get the following error
Exception RuntimeError: RuntimeError('cannot join current thread',) in <Finalize object, dead> ignored
I have two environments available to me on
Ubuntu 12.04.3 LTS, python 2.7.3 - I get an error.
RHEL 6.3, python 2.6.6 - I do not get an error.
Search for most offers related to joining the pool after closing (), but if I do, the process will never exit. Also, all the examples I found have only one pool, and not two or more (using one pool / queue does not lead to an error).
Any ideas on what caused this error?