Geek Answers Handbook

How to write a parallel loop in julia?

I have the following Julia code, and I would like to parallelize it.

using DistributedArrays function f(x) return x^2; end y = DArray[] @parallel for i in 1:100 y[i] = f(i) end println(y)

The output of DistributedArrays.DArray[] . I would like to have a y value like this: y=[1,4,9,16,...,10000]

+8
parallel-processing julia-lang
source share
2 answers

Hi Kira

I am new to Julia but am facing the same problem. Try this approach and make sure it fits your needs.

 function f(x) return x^2; end y=@parallel vcat for i= 1:100 f(i); end; println(y)

Regards, RN

+3
source share

You can use n-dimensional methods of a distributed array:

First you need to add some more processes, local or remote:

 julia> addprocs(CPU_CORES - 1);

Then you should use DistributedArrays for each of the created processes:

 julia> @everywhere using DistributedArrays

Finally, you can use the @DArray macro, for example:

 julia> x = @DArray [@show x^2 for x = 1:10]; From worker 2: x ^ 2 = 1 From worker 2: x ^ 2 = 4 From worker 4: x ^ 2 = 64 From worker 2: x ^ 2 = 9 From worker 4: x ^ 2 = 81 From worker 4: x ^ 2 = 100 From worker 3: x ^ 2 = 16 From worker 3: x ^ 2 = 25 From worker 3: x ^ 2 = 36 From worker 3: x ^ 2 = 49

You can see that it does what you expect:

 julia> x 10-element DistributedArrays.DArray{Int64,1,Array{Int64,1}}: 1 4 9 16 25 36 49 64 81 100

Remember that it works with an arbitrary number of dimensions:

 julia> y = @DArray [@show i + j for i = 1:3, j = 4:6]; From worker 4: i + j = 7 From worker 4: i + j = 8 From worker 4: i + j = 9 From worker 2: i + j = 5 From worker 2: i + j = 6 From worker 2: i + j = 7 From worker 3: i + j = 6 From worker 3: i + j = 7 From worker 3: i + j = 8 julia> y 3x3 DistributedArrays.DArray{Int64,2,Array{Int64,2}}: 5 6 7 6 7 8 7 8 9 julia>

This is the most Julian way to do what you intended IMHO.

We can look at macroexpand output to see what happens:

Note: this result has been slightly edited for readability, T means:

 DistributedArrays.Tuple{DistributedArrays.Vararg{DistributedArrays.UnitRange{DistributedArrays.Int}}} 
 julia> macroexpand(:(@DArray [i^2 for i = 1:10])) :( DistributedArrays.DArray( ( #231#I::T -> begin [i ^ 2 for i = (1:10)[#231#I[1]]] end ), DistributedArrays.tuple(DistributedArrays.length(1:10)) ) )

This is basically the same as when entering manually:

 julia> n = 10; dims = (n,); julia> DArray(x -> [i^2 for i = (1:n)[x[1]]], dims) 10-element DistributedArrays.DArray{Any,1,Array{Any,1}}: 1 4 9 16 25 36 49 64 81 100 julia>
+9
source share

More articles:


All Articles