Parallel performance

Question

I am using the .NET Chart Control library that comes with .NET 4.0 Beta 2 to create and save images to disk in the background stream. However, I do not display the diagram on the screen, just creating a diagram, saving it to disk and destroying it. Something like that:

public void GeneratePlot(IList<DataPoint> series, Stream outputStream) { using (var ch = new Chart()) { ch.ChartAreas.Add(new ChartArea()); var s = new Series(); foreach (var pnt in series) s.Points.Add(pnt); ch.Series.Add(s); ch.SaveImage(outputStream, ChartImageFormat.Png); } } 

It took about 300-400 ms to create and save each chart. I have potentially hundreds of diagrams to create, so I decided to use Parallel.For() to parallelize these tasks. I have an 8-core machine, however, when I try to create 4 charts at a time, my chart creates / saves time, increasing anywhere from 800 to 1400 ms, almost all of which are consumed by Chart.SaveImage .

I thought this might be a disk I / O limitation, therefore, to check that I changed the last line to:

 ch.SaveImage(Stream.Null, ChartImageFormat.Png); 

Even when writing to zero stream, the performance is still approximately the same (800-1400 ms).

Should I not create images in the background thread in parallel with this library, or am I doing something wrong?

thanks

EDIT: Added full code example

Just change the flag passed to CreateCharts() to check concurrency and seriality.

 using System; using System.Collections.Generic; using System.IO; using System.Threading; using System.Threading.Tasks; using System.Windows.Forms.DataVisualization.Charting; namespace ConsoleChartTest { class Program { public static void GeneratePlot(IEnumerable<DataPoint> series, Stream outputStream) { long beginTime = Environment.TickCount; using (var ch = new Chart()) { ch.ChartAreas.Add(new ChartArea()); var s = new Series(); foreach (var pnt in series) s.Points.Add(pnt); ch.Series.Add(s); long endTime = Environment.TickCount; long createTime = endTime - beginTime; beginTime = Environment.TickCount; ch.SaveImage(outputStream, ChartImageFormat.Png); endTime = Environment.TickCount; long saveTime = endTime - beginTime; Console.WriteLine("Thread Id: {0,2} Create Time: {1,3} Save Time: {2,3}", Thread.CurrentThread.ManagedThreadId, createTime, saveTime); } } public static void CreateCharts(bool parallel) { var data = new DataPoint[20000]; for (int i = 0; i < data.Length; i++) { data[i] = new DataPoint(i, i); } if (parallel) { Parallel.For(0, 10, (i) => GeneratePlot(data, Stream.Null)); } else { for (int i = 0; i < 10; i++) GeneratePlot(data, Stream.Null); } } static void Main(string[] args) { Console.WriteLine("Main Thread Id: {0,2}", Thread.CurrentThread.ManagedThreadId); long beginTime = Environment.TickCount; CreateCharts(false); long endTime = Environment.TickCount; Console.WriteLine("Total Time: {0}", endTime - beginTime); } } } 

Answer 1

You have problems with the System.Drawing namespace. There is some powerful thread that will serialize certain tasks. Until you name Chart.SaveImage() , does it really render the image, that is all your time.

If you change your test program a bit, you will see that parallelization is occurring, but it is very difficult to lock inside the graphic drawing code.

Play around with count = 50 in the main method here ... seeing both outputs at the same time, I think you can see that Parallel is one sequentially faster, although it does not scale linearly due to a lock in the drawing namespace:

 using System; using System.Collections.Generic; using System.Diagnostics; using System.Drawing; using System.Drawing.Imaging; using System.IO; using System.Threading; using System.Threading.Tasks; using System.Windows.Forms.DataVisualization.Charting; namespace ConsoleChartTest { class Program { static void Main(string[] args) { var count = 50; Console.WriteLine("Serial Test Start, Count: {0}"); Console.WriteLine("Main Thread Id: {0,2}", Thread.CurrentThread.ManagedThreadId); var sw = new Stopwatch(); sw.Start(); CreateCharts(count, false); sw.Stop(); Console.WriteLine("Total Serial Time: {0}ms", sw.ElapsedMilliseconds); Console.WriteLine("Parallel Test Start"); Console.WriteLine("Main Thread Id: {0,2}", Thread.CurrentThread.ManagedThreadId); sw.Restart(); CreateCharts(count, true); sw.Stop(); Console.WriteLine("Total Parallel Time: {0}ms", sw.ElapsedMilliseconds); } public static void GeneratePlot(IEnumerable<DataPoint> series, Stream outputStream) { var sw = new Stopwatch(); sw.Start(); var ch = new Chart(); ch.ChartAreas.Add(new ChartArea()); var s = new Series(); foreach(var pnt in series) s.Points.Add(pnt); ch.Series.Add(s); sw.Stop(); long createTime = sw.ElapsedMilliseconds; sw.Restart(); ch.SaveImage(outputStream, ChartImageFormat.Png); sw.Stop(); Console.WriteLine("Thread Id: {0,2} Create Time: {1,3}ms Save Time: {2,3}ms", Thread.CurrentThread.ManagedThreadId, createTime, sw.ElapsedMilliseconds); } public static void CreateCharts(int count, bool parallel) { var data = new DataPoint[20000]; if (parallel) { Parallel.For(0, data.Length, (i) => data[i] = new DataPoint(i, i)); Parallel.For(0, count, (i) => GeneratePlot(data, Stream.Null)); } else { for (int i = 0; i < data.Length; i++) data[i] = new DataPoint(i, i); for (int i = 0; i < count; i++) GeneratePlot(data, Stream.Null); } } } } 

What locks Chart.SaveImage() → ChartImage.GetImage() → ChartPicture.Paint()

Answer 2

Perhaps you could save the image as BMP, which would require more disk space, but reduce the amount of processing that it would need to do. You should try different image formats to see if compression causes problems. If so, you may want to unscrew some other threads to perform compression after.

Answer 3

Remember that you have Hyper-threading and indeed a kernel. You have to be careful, hyperthreading does not have the same performance as the kernel.

Another thing is to work well with parallel execution, while you create a pool thread, you must set the maximum number of threads, for example

MaxThreads = Cores - 2;

When I talk about kernels, read kernels, not hyper-threading kernels.

1 - OS 1 - The main application of X - Cores for processing.

If you create too many threads, you will lose performance due to a match in the processor.

Creating Jpeg or Png images is another good point, so you'll spend less time on HD when saving images. Take care of the quality of jpeg and png too, because if it is 100%, it can be great.

Another point that matters. You will have consent to HD, because it will have many streams that create archives on it. What can you do about it? This is really harder to solve because we do not have parallel hds. This way you can create a place to send image buffers, like another thread that does not process anything, just get the image buffer and store them in an internal list, for example. And after 5 em 5 seconds (or some conditions that you think is better), he starts writing images on hd. This way you will have a stream that works only on HD without, and other streams just process the images.

in.

Answer 4

If I had to guess, Id says that the code inside SaveImage is protected by CriticalSection or Lock, which only allows one thread to run in portions of this code.

So, you may be right in that you are not allowed to use background threads, but I think it is more likely that you are not allowed to have more than 1 thread running inside SaveImage at a time. The documentation for this feature is pretty rare, but the timings are very suggestive. 4 charts occupy about 4 times longer than 1 chart.

If you save only 1 chart using a background thread - does this happen at full speed?