Speed ​​Up ListLinePlot

Question

I am trying to use Mathematica to analyze some raw data. I would like to be able to dynamically display the range of data that interests me using Manipulate and ListLinePlot , but plot rendering is extremely slow. How can I speed it up?

Here are some additional details. An external text file stores raw data: the first column is a timestamp, the second, third and fourth columns are data, for example:

 1309555993069, -2.369941, 6.129157, 6.823794 1309555993122, -2.260978, 6.170018, 7.014479 1309555993183, -2.070293, 6.129157, 6.823794 1309555993242, -1.988571, 6.238119, 7.123442 

One data file contains up to 2 and middle, 10 ⁶ lines. To display, for example, the second column, I use:

 x = Import["path/to/datafile"]; ListLinePlot[x[[All, {1, 2}]]] 

The execution time for this operation is unbearably long. To display a variable data range, I tried using Manipulate :

 Manipulate[ListLinePlot[Take[x, numrows][[All, {1, 2}]]], {numrows, 1, Length[x]}] 

This instruction works, but it scans quickly when I try to display more than a few thousand lines. How can I speed it up?

Additional information:

• MATLAB displays the same amount of data on one computer almost instantly, so raw data size should not be a problem.
• I already tried to disable graphical anti-aliasing, but this did not affect the rendering speed at all.
• Using DataRange to avoid Take does not help.
• Using MaxPlotPoints distorts the chart too much to be useful.
• Not using Take in Manipulate does not help.
• Rendering seems to take a huge amount of time. Doing Timing[ListLinePlot[Take[x,100000][[All, {1, 2}]]]] returns 0.33 : this means that the Take score itself is almost instantaneous, it's a plot rendering that slows everything down.
• I am running Mathematica on Ubuntu Linux 11.10 using fglrx drivers. Forcing Mathematica to use mesa drivers did not help.

Any clues?

Answer 1

I have not tested this much on my machine (I have a Mac, so I cannot rule out Linux related issues). but I get a few points. For me it was pretty fast, but obviously slower than if the dataset was smaller. You draw hundreds of thousands of data points.

 data = Accumulate@RandomVariate[NormalDistribution[], 200000]; Manipulate[ListLinePlot[Take[data, n]], {n, 1, Length[data]}] 

• In Manipulate you can arbitrarily change the amount of data displayed with Take . Try only increasing numrows every 100 or so, so the rendering is less.
• Try using the parameter ContinuousAction->False (see the documentation ) (I see that @Szabolcs has the same idea as when entering text.
• I was going to suggest MaxPlotPoints , but instead try the PerformanceGoal ->"Speed" option. (see documentation )

Answer 2

If your goal is to simply visualize your data quickly, but correctly, you can use the following trick, which I constantly use.

I divide the data into several blocks corresponding approximately to the resolution of my screen (usually 1000 or less), more detailed information cannot be displayed in any case. Then I define the Min and Max of each block and draw a zigzag line from min to max to min max ... The result will look exactly the same as the original data. However, you cannot “zoom in” because then you will see a zigzag line (for example, when exporting to a pdf file with high resolution). Then you need to use more blocks.

 rv = RandomVariate[ExponentialDistribution[2], 100000]; ListLinePlot[rv, PlotRange -> All] (* original, slow *) ListLinePlot[rv, PlotRange -> All, MaxPlotPoints -> 1000] (* fast but distorted *) numberOfBlocks = 1000; ListLinePlot[Riffle @@ Through[{Min /@ # &, Max /@ # &}[ Partition[rv,Floor[Length[rv]/numberOfBlocks]]]], PlotRange -> All] 

You can add the option DataRange->{...} to mark the x axis accordingly.

Hope this helps!

EDIT: See also this similar question about Mathematica Stackexchange: https://mathematica.stackexchange.com/q/140/58

Answer 3

I also noticed that occasionally Mathematica would take too much time to render graphics. In fact, this should be some step in moving from a Mathematica Graphics expression to some other representation, which takes a lot of time, because after rendering, resizing (and, therefore, re-rendering) the graph is much faster. Presentation of version 6 was faster for many examples (but also does not have a lot of functionality that 6+ have).

Some ideas on what you could do:

• Use the MaxPlotPoints ListLinePlot parameter to reduce data before plotting. This may not affect the appearance if you lower it. The Method option should select the downsample algorithm, but I can not find any documents for it (anyone?)

• Use ContinuousAction -> False in Manipulate so that it doesn't reprogram everything in real time while dragging the sliders.

Answer 4

Another idea here is to use the Ramer-Douglas-Peucker algorithm to reduce the number of data points before plotting. This is likely to save the data form better. I do not know if you still need this, so I will not offer an implementation.