C # Parallel and Serial

Question

I have a large list to go through (1,500,000 points), each item should do a very small check. In just 30 seconds.

CPU usage when using Sequential is around 10%, so a lot of resources are not used.

The first thought was to use Parallel, but because of the limited length of time for each Parallel element it lasts longer than the serial Foreach, this is due to “ Why was the parallel version slower than the serial version in this example? ”, Which explains that creating each task will be worth the time.

So, I had another thought, and this is to divide the list into 4 (or more) equal faces and create a stream to iterate over the elements in order to get it faster.

Before creating my own class, is this a good approach? Or any other thoughts on how to speed up the process? Or you know the best way to handle this.

the code

Code I created for another parallel approach: (used in my own static class)

public static void ForEach<T>(IEnumerable<T> list, Action<T> body, int listDevide) { // Number of items int items = list.Count(); // Divided (in int, so floored) int listPart = items / listDevide; // Get numbers extra for last run int rest = items % listDevide; // List to save the actions var actions = new List<Action>(); for(var x = 0; x < listDevide; x++) { // Create the actions actions.Add(delegate { foreach(var item in list.Skip(x * listPart).Take(listPart)) { body.Invoke(item); } }); } // Run the actions parallel Parallel.Invoke(actions.ToArray()); } 

Note: the variable "rest" to execute the last elements is not currently used in this example.

The solution is below, more information: http://msdn.microsoft.com/en-us/library/dd997411.aspx

Answer 1

Yes, splitting the input array is a good approach.

In fact, Microsoft provides the Partitioner class to help in this way.

Here is an example showing how to do this:

 using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Diagnostics; using System.Threading.Tasks; namespace Demo { class Program { private void run() { double sum = 0; Func<double, double> func = x => Math.Sqrt(Math.Sin(x)); object locker = new object(); double[] data = testData(); // For each double in data[] we are going to calculate Math.Sqrt(Math.Sin(x)) and // add all the results together. // // To do this, we use class Partitioner to split the input array into just a few partitions, // (the Partitioner will use knowledge about the number of processor cores to optimize this) // and then add up all the values using a separate thread for each partition. // // We use threadLocalState to compute the total for each partition, and then we have to // add all these together to get the final sum. We must lock the additon because it isn't // threadsafe, and several threads could be doing it at the same time. Parallel.ForEach ( Partitioner.Create(0, data.Length), () => 0.0, (subRange, loopState, threadLocalState) => { for (int i = subRange.Item1; i < subRange.Item2; i++) { threadLocalState += func(data[i]); } return threadLocalState; }, finalThreadLocalState => { lock (locker) { sum += finalThreadLocalState; } } ); Console.WriteLine("Sum = " + sum); } private static double[] testData() { double[] array = new double[1000003]; // Test with an odd number of values. Random rng = new Random(12345); for (int i = 0; i < array.Length; ++i) array[i] = rng.Next() & 3; // Don't want large values for this simple test. return array; } static void Main() { new Program().run(); } } }