Ipython concurrent scripts work and return local variables

Question

For example, I run 4 scripts sequentially:

%run -i script1.py %run -i script2.py %run -i script3.py %run -i script4.py 

Each run time is quite long. Is there a way in iPython laptop to run scripts in parallel and return local variables from all of them (2 or 3 variables are important)? When executed sequentially, it works fine, but for a long time. Thank you in advance.

I tried to apply the code from this thread , but got stuck in the first part:

 def my_func(my_file): !python pgm.py my_file 

or in my case:

  def my_func(my_file): %run -i $my_file 

I see that the code is executing, but after that I do not see local variables from these scripts.

Answer 1

Suppose you run 4 engines and you send your 4 scripts to each.

After doing

 rc = parallel.Client() view = rc.load_balanced_view() r = view.map_async(my_func, ["script1.py", "script2.py", "script3.py", "script4.py"]) 

After that, you can access, say, the variable a and b using pull :

 var = rc[:].pull(["a","b"]) var.result # or var.r or var.result_dict [[10, 12020.2], [11, 14], [1, 0], [1, 14425]] 

Which corresponds to the value of a and b after each run of each script.

So, in script1.py , at the end you have a==10 and b==12020.2 .

Hope this helps.

---

By the way, I edited the link a little