How to resize shared memory in Python

Question

I want to use an array for shared memory. The problem is that the program is structured so that the child processes are spawned before I know the size of the shared array. If I send a message to expand the array, nothing happens, and if I try to send the shared array itself, I get an error message. Below is a small script to demonstrate my problem.

import multiprocessing as mp import numpy as np def f(a,pipe): while True: message, data = pipe.recv() if message == 'extend': a = np.zeros(data) print a elif message == 'exit': break if __name__ == '__main__': unshared_arr = np.zeros(1) a = mp.Array('d', unshared_arr) p1,p2 = mp.Pipe() p = mp.Process(target=f, args=(a,p2)) p.start() p1.send(('extend', 10)) p1.send(('exit', None)) p.join() b = np.frombuffer(a.get_obj()) 

Answer 1

try:

 unshared_Arr = mp.Array(ctypes.c_uint8,SIZE_NEEDED) #should be size #and not the array itself np_shared = np.frombuffer(ushared_Arr.get_obj(),dtype=ctypes.c_uint8) np_shared.reshape(SIZE_NEEDED/2,SIZE_NEEDED/2) #or (,SIZE_NEEDED) ie. any shape #you want as long as the allocated size #does not change 

now use np_shared, like any numpy array. You must keep it global if it needs several processes.