Geek Answers Handbook

Import and use a module that uses multiprocessing without causing an infinite loop in Windows

I have a module called multi.py . If I just wanted to execute multi.py as a script, then a workaround to avoid a Windows crash (spawning an infinite number of processes) is to put the multiprocessing code:

 if __name__ == '__main__':

However, I am trying to import it as a module from another script and call multi.start() . How can I do that?

 # multi.py import multiprocessing def test(x): x**=2 def start(): pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()-2) pool.map(test, (i for i in range(1000*1000))) pool.terminate() print('done.') if __name__ == '__main__': print('runs as a script,',__name__) else: print('runs as imported module,',__name__)

This is my test.py I run:

 # test.py import multi multi.start()
+4
source share
2 answers

I do not quite understand what you are asking. You do not need to do anything so that it does not spawn infinitely many processes. I just ran it in Windows XP --- imported the file and ran multi.start() --- and it completed perfectly after a couple of seconds.

The reason you should protect if __name__=="__main__" is because on Windows, multiprocessing has to import the main script to run the target function, which means that the top-level module code in this file will be executed. The problem occurs only if the code of the top-level module itself tries to create a new process. In your example, the top-level module code does not use multiprocessing, so there is no endless chain of processes.

Edit: Now I get what you ask. You do not need to protect multi.py . You need to protect your main script, whatever that is. If you get a failure, it is because basically the script you do multi.start() in the code of the top-level module. Your script should look like this:

 import multi if __name__=="__main__": multi.start()

"Protection" is always required in the main script.

+5
source 
 if __name__ == '__main__': print('runs as a script,',__name__) else: print('runs as imported module,',__name__)
0
source

More articles:


All Articles