Matplotlib runs out of memory when plotting in a loop

Question

I have a fairly simple charting procedure that looks like this:

from __future__ import division import datetime import matplotlib matplotlib.use('Agg') from matplotlib.pyplot import figure, plot, show, legend, close, savefig, rcParams import numpy from globalconstants import * def plotColumns(columnNumbers, t, out, showFig=False, filenamePrefix=None, saveFig=True, saveThumb=True): lineProps = ['b', 'r', 'g', 'c', 'm', 'y', 'k', 'b--', 'r--', 'g--', 'c--', 'm--', 'y--', 'k--', 'g--', 'b.-', 'r.-', 'g.-', 'c.-', 'm.-', 'y.-', 'k.-'] rcParams['figure.figsize'] = (13,11) for i in columnNumbers: plot(t, out[:,i], lineProps[i]) legendStrings = list(numpy.zeros(NUMCOMPONENTS)) legendStrings[GLUCOSE] = 'GLUCOSE' legendStrings[CELLULOSE] = 'CELLULOSE' legendStrings[STARCH] = 'STARCH' legendStrings[ACETATE] = 'ACETATE' legendStrings[BUTYRATE] = 'BUTYRATE' legendStrings[SUCCINATE] = 'SUCCINATE' legendStrings[HYDROGEN] = 'HYDROGEN' legendStrings[PROPIONATE] = 'PROPIONATE' legendStrings[METHANE] = "METHANE" legendStrings[RUMINOCOCCUS] = 'RUMINOCOCCUS' legendStrings[METHANOBACTERIUM] = "METHANOBACTERIUM" legendStrings[BACTEROIDES] = 'BACTEROIDES' legendStrings[SELENOMONAS] = 'SELENOMONAS' legendStrings[CLOSTRIDIUM] = 'CLOSTRIDIUM' legendStrings = [legendStrings[i] for i in columnNumbers] legend(legendStrings, loc='best') dt = datetime.datetime.now() dtAsString = dt.strftime('%d-%m-%Y_%H-%M-%S') if filenamePrefix is None: filenamePrefix = '' if filenamePrefix != '' and filenamePrefix[-1] != '_': filenamePrefix += '_' if saveFig: savefig(filenamePrefix+dtAsString+'.eps') if saveThumb: savefig(filenamePrefix+dtAsString+'.png', dpi=300) if showFig: f.show() close('all') 

When I draw this in single iterations, it works fine. However, as soon as I put it in the loop, matplotlib throws the spike ...

 Traceback (most recent call last): File "c4hm_param_variation_h2_conc.py", line 148, in <module> plotColumns(columnNumbers, timeVector, out, showFig=False, filenamePrefix='c 4hm_param_variation_h2_conc_'+str(hydrogen_conc), saveFig=False, saveThumb=True) File "D:\phdproject\alexander paper\python\v3\plotcolumns.py", line 48, in plo tColumns savefig(filenamePrefix+dtAsString+'.png', dpi=300) File "C:\Python25\lib\site-packages\matplotlib\pyplot.py", line 356, in savefi g return fig.savefig(*args, **kwargs) File "C:\Python25\lib\site-packages\matplotlib\figure.py", line 1032, in savef ig self.canvas.print_figure(*args, **kwargs) File "C:\Python25\lib\site-packages\matplotlib\backend_bases.py", line 1476, i n print_figure **kwargs) File "C:\Python25\lib\site-packages\matplotlib\backends\backend_agg.py", line 358, in print_png FigureCanvasAgg.draw(self) File "C:\Python25\lib\site-packages\matplotlib\backends\backend_agg.py", line 314, in draw self.figure.draw(self.renderer) File "C:\Python25\lib\site-packages\matplotlib\artist.py", line 46, in draw_wr apper draw(artist, renderer, *kl) File "C:\Python25\lib\site-packages\matplotlib\figure.py", line 773, in draw for a in self.axes: a.draw(renderer) File "C:\Python25\lib\site-packages\matplotlib\artist.py", line 46, in draw_wr apper draw(artist, renderer, *kl) File "C:\Python25\lib\site-packages\matplotlib\axes.py", line 1735, in draw a.draw(renderer) File "C:\Python25\lib\site-packages\matplotlib\artist.py", line 46, in draw_wr apper draw(artist, renderer, *kl) File "C:\Python25\lib\site-packages\matplotlib\legend.py", line 374, in draw bbox = self._legend_box.get_window_extent(renderer) File "C:\Python25\lib\site-packages\matplotlib\offsetbox.py", line 209, in get _window_extent px, py = self.get_offset(w, h, xd, yd) File "C:\Python25\lib\site-packages\matplotlib\offsetbox.py", line 162, in get _offset return self._offset(width, height, xdescent, ydescent) File "C:\Python25\lib\site-packages\matplotlib\legend.py", line 360, in findof fset return _findoffset(width, height, xdescent, ydescent, renderer) File "C:\Python25\lib\site-packages\matplotlib\legend.py", line 325, in _findo ffset_best ox, oy = self._find_best_position(width, height, renderer) File "C:\Python25\lib\site-packages\matplotlib\legend.py", line 817, in _find_ best_position verts, bboxes, lines = self._auto_legend_data() File "C:\Python25\lib\site-packages\matplotlib\legend.py", line 669, in _auto_ legend_data tpath = trans.transform_path(path) File "C:\Python25\lib\site-packages\matplotlib\transforms.py", line 1911, in t ransform_path self._a.transform_path(path)) File "C:\Python25\lib\site-packages\matplotlib\transforms.py", line 1122, in t ransform_path return Path(self.transform(path.vertices), path.codes, File "C:\Python25\lib\site-packages\matplotlib\transforms.py", line 1402, in t ransform return affine_transform(points, mtx) MemoryError: Could not allocate memory for path 

This happens at iteration 2 (counting from 1), if that matters. The code runs on a 32-bit version of Windows XP with python 2.5 and matplotlib 0.99.1, numpy 1.3.0 and scipy 0.7.1.

EDIT: The code is now updated to reflect the fact that the crash actually occurs when calling legend() . Commenting on this call, it solves the problem, although, obviously, I still would like to place a legend on my graphs ...

Answer 1

Is each loop supposed to generate a new digit? I don’t see you close it or create a new instance of the figure from loop to loop.

This call will clear the current digit after saving at the end of the loop:

pyplot.clf ()

I would refactor and make your code more OO and create a new instance of the shape in each loop:

 from matplotlib import pyplot while True: fig = pyplot.figure() ax = fig.add_subplot(111) ax.plot(x,y) ax.legend(legendStrings, loc = 'best') fig.savefig('himom.png') # etc.... 

Answer 2

I also ran into this error. which is apparently fixed.

 while True: fig = pyplot.figure() ax = fig.add_subplot(111) ax.plot(x,y) ax.legend(legendStrings, loc = 'best') fig.savefig('himom.png') #new bit here pylab.close(fig) #where f is the figure 

Now my loop works stably with fluctuating memory, but there is no constant increase

Answer 3

The answer from ninjasmith also worked for me - pyplot.close() let my loops work.

From the pyplot tutorial Working with multiple shapes and axes :

You can clear the current digit with clf() and the current axis with cla() . If you find this condition annoying, not desperate, it's just a thin wrapper for the object around the object oriented API, which you can use instead (see the Guide for Artists )

If you are making a long sequence of numbers, you need to know one more thing: the memory required for the figure is not completely until the figure is explicitly closed with close() . Removing all links to the shape and / or using the window manager to destroy the window in which the shape appears on the screen is not enough, because pyplot maintains internal links until close() called.