Matplotlib Agg agglomerate rendering complexity error

Question

I am trying to print a graph with a resolution of 600 dpi using Python matplotlib. However, Python built 2 out of 8 graphs and threw an error:

OverflowError: Agg rendering complexity exceeded. Consider downsampling or decimating your data. 

I draw a huge chunk of data (7,500,000 data per column), so I assume this will be some kind of overload problem or I need to set a large cell_block_limit element.

I tried to find solutions to change cell_block_limit on Google, but to no avail. What would be a good approach?

The code is as follows: -

  import matplotlib.pyplot as plt from matplotlib.ticker import MultipleLocator, FormatStrFormatter majorLocator = MultipleLocator(200) majorFormatter = FormatStrFormatter('%d') minorLocator = MultipleLocator(20) fig = plt.figure() ax = fig.add_subplot(111) ax.xaxis.set_major_locator(majorLocator) ax.xaxis.set_major_formatter(majorFormatter) ax.xaxis.set_minor_locator(minorLocator) ax.xaxis.set_ticks_position('bottom') ax.xaxis.grid(True,which='minor') ax.yaxis.grid(True) plt.plot(timemat,fildata) plt.xlabel(plotxlabel,fontsize=14) plt.ylabel(plotylabel,fontsize=14) plt.title(plottitle,fontsize=16) fig.savefig(plotsavetitle,dpi=600) 

Answer 1

In addition to @Lennart, indicate that there is no need for full resolution, you can also consider a schedule similar to the following.

Calculating the max / mean / min "chunked" version is very simple and efficient if you use the 2D representation of the source array and the axis keyword arg for x.min() , x.max() , etc.

Even with filtering, plotting is much faster than plotting a full array.

(Note: in order to build this many points, you will have to change the noise level a little. Otherwise, you will get the mentioned OverflowError . If you want to compare the construction of a “full” data set, change the line y += 0.3 * y.max() np.random... more like 0.1 or remove it completely.)

 import matplotlib.pyplot as plt import numpy as np np.random.seed(1977) # Generate some very noisy but interesting data... num = 1e7 x = np.linspace(0, 10, num) y = np.random.random(num) - 0.5 y.cumsum(out=y) y += 0.3 * y.max() * np.random.random(num) fig, ax = plt.subplots() # Wrap the array into a 2D array of chunks, truncating the last chunk if # chunksize isn't an even divisor of the total size. # (This part won't use _any_ additional memory) chunksize = 10000 numchunks = y.size // chunksize ychunks = y[:chunksize*numchunks].reshape((-1, chunksize)) xchunks = x[:chunksize*numchunks].reshape((-1, chunksize)) # Calculate the max, min, and means of chunksize-element chunks... max_env = ychunks.max(axis=1) min_env = ychunks.min(axis=1) ycenters = ychunks.mean(axis=1) xcenters = xchunks.mean(axis=1) # Now plot the bounds and the mean... ax.fill_between(xcenters, min_env, max_env, color='gray', edgecolor='none', alpha=0.5) ax.plot(xcenters, ycenters) fig.savefig('temp.png', dpi=600) 

Answer 2

With a resolution of 600 dpi, you will need to make a 13-meter-wide plot to draw this data without dropping it. :-)

I would suggest breaking the data into pieces into a couple of hundred, or maybe even into a thousand samples, and extract the maximum value from this.

Something like that:

 def chunkmax(data, chunk_size): source = iter(data) chunk = [] while True: for i in range(chunk_size): chunk.append(next(source)) yield max(chunk) 

Then it would be if a chunk_size of 1000 gives you 7,500 points for the chart, where you can easily see where the shock occurs in the data. (If the data is not so noisy, you will have to average it to see if there is a pallet or not, but this is also easily fixed).