Using Nested Parallel.For

Question

Consider the following example:

var x = 0; for (var i = 0; i < 100; i++ ) { for (var a = i+1; a < 100; a++) x += 1; } 

When printing x, we always get 4950. What if I want to parallelize this?

This is what I came up with

 Parallel.For(0, 100, i => Parallel.For(i + 1, 100, a => { x += 1; })); 

However, this does not print 4950 every time I run it. Why?

Answer 1

Parallel extensions help you with task creation, distribution, launch, and rendezvous in an almost imperative syntax. What it does not do is take care of every kind of thread safety (one of the pitfalls ). You are trying to create parallel threads at the same time, updating one common variable. To do something like this, you must enter, for example. lock.

I'm not sure what you are trying to do. I assume your code is just a placeholder or an experiment. Parallelization is only suitable when you can isolate your different parts of the work; not when you constantly have to deal with general data.

Answer 2

There will be a “right” way to do this, it will not require you to lock your final shared object and will only require you to perform blocking operations at the end of each local stream loop.

 int x = 0; Parallel.For(0, 100, () => 0, //LocalInit (i, loopstate, outerlocal) => { Parallel.For(i + 1, 100, () => 0, //LocalInit (a, loopState, innerLocal) => { return innerLocal + 1; }, (innerLocal) => Interlocked.Add(ref outerlocal, innerLocal)); //Local Final return outerlocal; }, (outerLocal) => Interlocked.Add(ref x, outerLocal)); //Local Final 

However, having two nested Parallel statements doing this little work is probably a bad idea. The overhead that you need to consider if you are doing such a little work would be much better to do just one Parallel statement or not at all.

I highly recommend that you download and read Parallel Programming Patterns , which details why small nested parallel loops like this are not a good idea.

Answer 3

As an alternative to locking, each time you can use local thread variables in combination with locking:

 Object thisLock = new Object(); var globalSum = 0; System.Threading.Tasks.Parallel.For(0, 100, i => { System.Threading.Tasks.Parallel.For(i + 1, 100, () => 0, (num, loopState, subSum) => ++subSum, subSum => { lock(thisLock) { globalSum += subSum; } }); }); Console.WriteLine(globalSum);