Python multiprocessing with fortran library

I use the multiprocessing module in Python to map a set of jobs to as many cores as I have. The work that I wrap is mostly done with some wrapped fortran code (wrapped in f2py). When I only call tasks, they go through all the kernels. However, if I try to call the fortran library first (and not through a function Pool().map()) and then call the card, the whole program freezes (if you want to know, it freezes exactly on line 339 threading.py on which is the only code waiter.acquire()).

For the program I am writing, I need to call the library before going and displaying it. I went around this by calling it, using Pool().map()it with only one process at first , but it’s a hack, and I would like to know why I need to do this!

To be more explicit, this one does not work :

def call_camb_transfer(H0): #wraps the call to define the keyword parameter to pass
    return pycamb.transfers(H0=H0)[1][6, :, 0]

b = call_camb_transfer(70)
pool = multiprocessing.Pool(2)
H0 = [60, 70]

results = pool.map(call_camb_transfer, H0)

but it does:

def call_camb_transfer(H0): #wraps the call to define the keyword parameter to pass
    return pycamb.transfers(H0=H0)[1][6, :, 0]

initial_pool = multiprocessing.Pool(1)
b = initial_pool.map(call_camb_transfer, [65.0])

pool = multiprocessing.Pool(2)
H0 = [60, 70]

results = pool.map(call_camb_transfer, H0)

where pycambis the python CAMB shell, which is a fortran library.

+4
source share

All Articles