How to accurately draw large data vectors at all zoom levels in real time?

Question

I have large data sets (data 10 Hz, so 864 thousand points in 24 hours), which I need to build in real time. The idea is that the user can scale and pan to very detailed scatterplots.

Data is not very continuous and there are spikes. Since the data set is so large, I can’t build every point every time the plot is updated.

But I also can’t just speak every fifth point, otherwise I’ll miss the basic functions, such as big but short bursts.

Matlab does it right. You can give it an 864k vector full of zeros, and just set any point to 1 and it will draw correctly in real time using zooming and panning.

How does Matlab do this?

My target system is Java, so I will generate views for this plot in Swing / Java2D.

Answer 1

You should try the file from MATLAB Central:

https://mathworks.com/matlabcentral/fileexchange/15850-dsplot-downsampled-plot

From the author:

This version of the "chart" will allow you to visualize data with a very large number of elements. Building a large dataset makes your graphics sluggish, but in most cases you don't need all the information displayed on the graphics. There will be so many pixels on your screen, and your eyes will not be able to detect any information that does not appear on the screen.

This function will reduce data and display only a subset of the data, thereby improving memory requirements. When the graph is enlarged, more information is displayed. Some work is being done to ensure that emissions are captured.

Syntax:

dsplot(x, y) dsplot(y) dsplot(x, y, numpoints) 

Example:

 x =linspace(0, 2*pi, 1000000); y1=sin(x)+.02*cos(200*x)+0.001*sin(2000*x)+0.0001*cos(20000*x); dsplot(x,y1); 

Answer 2

I don't know how Matlab does it, but I started with Quadtrees .

Drop all your data points in quadrants, then to display at a given zoom level, you go down the quadrant (starting from areas that overlap what you are viewing) until you reach areas that are comparable to pixel sizes. Attach a pixel in the middle of this area.

added: Performing a drawing using OpenGL / JOGL will also help you draw faster. Especially if you can predict the panning and create points that will be displayed in the display list, or something else so that you do not need to work with the processor for new frames.

Answer 3

10 Hz data means you only need 10 frames per second. This should be easy as many games reach> 100 frames per second with much more complex graphics.

If you create 10 pixels per second for each possible data point, you can display minute data using a widget with a width of 600 pixels. If you keep the index from 600 to the last sample, it will be easy for you to draw only the latest data.

If you do not have a new data point every 10 seconds, you need to come up with a way to insert an interpolated data point. Three options come to mind:

• Repeat the last data point.
• Insert an "empty" data point. This will cause gaps in the chart.
• Do not update the graph until the next data point appears. Then insert all the pixels that you did not draw at a time, with linear interpolation between the data points.

To make the animation smooth, use double buffering . If your target language supports a canvas widget, it probably supports double buffering.

When scaling, you have the same three options as above, since the enlarged data points are not continuous, even if the original data points were.

This can help implement it in Java.