Nested concurrent performance issue

Question

I have a question.

Are there any advantages to using Parallel.Invoke inside another Parallel.ForEach?

Here is my code:

Parallel.ForEach(yearMonths, () => new List<DJVSStatsCo>(), (yearMonth, loopState, localDjvsStatsCo) => { var coVintageCounter = 0; var coExitsCounter = 0; var coExtant = 0; Parallel.Invoke(() => coVintageCounter = globalData.ValuationEventsPit. Where(x => x.FirstRoundYearMonth <= yearMonth). Select(x => x.CompanyId).Distinct().Count(), () => coExitsCounter = globalData.ValuationEventsPit. Where(x => x.ExitDate != null && x.ExitDateYearMonth == yearMonth). Select(x => x.CompanyId).Distinct().Count(), () => coExtant = globalData.ValuationEventsPit. Where(x => x.FirstRoundYearMonth <= yearMonth && (x.ExitDate == null || x.ExitDateYearMonth > yearMonth)). Select(x => x.CompanyId).Distinct().Count() ); localDjvsStatsCo.Add(new DJVSStatsCo(yearMonth, coVintageCounter, coExtant, coExitsCounter)); return localDjvsStatsCo; }, x => { lock (locker) { djvsStatsCos.AddRange(x); } }); 

I have about 50 thousand records, and my machine has 2 main processors and calculates the calculation time. I get almost the same result. So my question is, are there any advantages to using Parallel inside Parallel? What is the best practice for this?

Thank you very much.

Regards, Vlad.

Answer 1

In this case, there is probably no benefit. This can be useful when you have relatively few "external" tasks, but potentially many "internal" tasks.

On the other hand, it also depends on what these three tasks do. If they are essentially asynchronous tasks (for example, in a database) that can be executed in parallel, then it is necessary ... but if they are local tasks with heavy CPU usage, then you are probably just going to provide additional work to the scheduler without real benefits .

Given the look of your code, it seems to me that you could benefit from doing one request (or maybe three) and grouping yearMonth , though ...

Answer 2

Since parallelism of the outer loop already keeps your processors busy (50k elements), it’s of little use to introduce parallelism in the loop. In the interest of readability, I would remove the Parallel.Invoke call to simplify your code.