RuntimeError from python multiprocessing pools with related queues

I see RunTime errors on exit when using two multiprocessing.Pool each with a different merge queue. The following is a simple code example showing the problem.

import multiprocessing
import time

q_1 = multiprocessing.JoinableQueue()
q_2 = multiprocessing.JoinableQueue()

def process():
    pool_1 = multiprocessing.Pool(1, worker_1,(q_1,))
    pool_2 = multiprocessing.Pool(1, worker_2,(q_2,))
    for i in range(3):
        q_1.put(i)
    q_1.join()
    q_2.join()
    pool_1.close()
    pool_2.close()

def worker_1(input_q):
    while True:
        val = input_q.get(True)
        print("1:"+str(val))
        q_2.put(val)
        input_q.task_done()

def worker_2(input_q):
    while True:
        val = input_q.get(True)
        time.sleep(2)
        print("2:"+str(val))
        input_q.task_done()

When this happens, I get the following error

Exception RuntimeError: RuntimeError('cannot join current thread',) in <Finalize object, dead> ignored

I have two environments available to me on

Ubuntu 12.04.3 LTS, python 2.7.3 - I get an error.

RHEL 6.3, python 2.6.6 - I do not get an error.

Search for most offers related to joining the pool after closing (), but if I do, the process will never exit. Also, all the examples I found have only one pool, and not two or more (using one pool / queue does not lead to an error).

Any ideas on what caused this error?

+4

All Articles