Geek Answers Handbook

Python multiprocessing testing: low speed due to overhead?

I am trying to learn about multiprocessingin python (2.7). My processor has 4 cores. In the following code, I test the speed of parallel Vs sequential execution of the same basic instruction.

I find that the time spent on 4 cores is only 0.67, if you take only one core, and it would be naive to expect ~ 0.25.

Overhead reason? Where is it from? Are 4 processes independent?

I also tried pool.mapand pool.map_async, with very similar results in terms of speed.

from multiprocessing import Process
import time

def my_process(a):
    for i in range(0,a[1]):
        j=0
        while j<10000:
            j = j+1
    print(a,j)

if __name__ == '__main__':
    # arguments to pass:
    a = ((0,2000),(1,2000),(2,2000),(3,2000))

    # --- 1) parallel processes:
    # 4 cores go up to 100% each here
    t0 = time.time()
    proc1 = Process(target=my_process, args=(a[0],))
    proc2 = Process(target=my_process, args=(a[1],))
    proc3 = Process(target=my_process, args=(a[2],))
    proc4 = Process(target=my_process, args=(a[3],))
    proc1.start(); proc2.start(); proc3.start(); proc4.start()
    proc1.join() ; proc2.join() ; proc3.join() ; proc4.join()
    dt_parallel = time.time()-t0
    print("parallel : " + str(dt_parallel))

    # --- 2) serial process :
    # 1 core only goes up to 100%
    t0 = time.time()
    for k in a:
        my_process(k)
    dt_serial = time.time()-t0
    print("serial : " + str(dt_serial))

    print("t_par / t_ser = " + str(dt_parallel/dt_serial))

2 (2 = 2 * 1 , lscpu [thanks @goncalopp]). script 2 , 0,62, , , 3 4 . , .

lscpu: CPU (s): 32, Thread (s) : 2, ​​() : 8, Socket (s): 2, 0,34, @dano.

+4
1

, , :

4 ( 2 ), , , chepner. 2 , < 0.5

+3

More articles:


All Articles