Why matplotlib will replace the right parenthesis with "!" in latex expression?

Question

I am in a situation where I need to translate a python expression into a Latex Bitmap for enduser (which feels confident enough to write python functions on its own, but prefers to watch the result in latex).

I am using Matplotlib.mathtext to do the job (from the translated latex string) with the following code.

import wx import wx.lib.scrolledpanel as scrolled import matplotlib as mpl from matplotlib import cm from matplotlib import mathtext class LatexBitmapFactory(): """ Latex Expression to Bitmap """ mpl.rc('image', origin='upper') parser = mathtext.MathTextParser("Bitmap") mpl.rc('text', usetex=True) DefaultProps = mpl.font_manager.FontProperties(family = "sans-serif",\ style = "normal",\ weight = "medium",\ size = 6) # size is changed from 6 to 7 #------------------------------------------------------------------------------- def SetBitmap(self, _parent, _line, dpi = 150, prop = DefaultProps): bmp = self.mathtext_to_wxbitmap(_line, dpi, prop = prop) w,h = bmp.GetWidth(), bmp.GetHeight() return wx.StaticBitmap(_parent, -1, bmp, (80, 50), (w, h)) #------------------------------------------------------------------------------- def mathtext_to_wxbitmap(self, _s, dpi = 150, prop = DefaultProps): ftimage, depth = self.parser.parse(_s, dpi, prop) w,h = ftimage.get_width(), ftimage.get_height() return wx.BitmapFromBufferRGBA(w, h, ftimage.as_rgba_str()) EXP = r'$\left(\frac{A \cdot \left(vds \cdot rs + \operatorname{Vdp}\left(ri, Rn, Hr, Hd\right) \cdot rh\right) \cdot \left(rSurf + \left(1.0 - rSurf\right) \cdot ft\right) \cdot \left(1.0 - e^{- \left(\left(lr + \frac{\operatorname{Log}\left(2\right)}{tem \cdot 86400.0}\right)\right) \cdot tFr \cdot 3600.0}\right)}{rc \cdot \left(lr + \frac{\operatorname{Log}\left(2\right)}{tem \cdot 86400.0}\right) \cdot tFr \cdot 3600.0} + 1\right)$' class aFrame(wx.Frame): def __init__(self, title="Edition"): wx.Frame.__init__(self, None, wx.ID_ANY, title=title, size=(600,400), style=wx.DEFAULT_DIALOG_STYLE|wx.RESIZE_BORDER) self.SetBackgroundColour(wx.Colour(255,255,255)) sizer = wx.FlexGridSizer(cols=25, vgap=4, hgap=4) panel = scrolled.ScrolledPanel(self) image_latex = LatexBitmapFactory().SetBitmap(panel, EXP) sizer.Add(image_latex, flag=wx.EXPAND|wx.ALL) panel.SetSizer(sizer) panel.SetAutoLayout(1) panel.SetupScrolling() app = wx.App(redirect=True, filename="latexlog.txt") frame = aFrame() frame.Show() app.MainLoop() 

with size = 6, the following image is displayed

with size = 7, I have this

The latex expression is correct, the second image is correct. I have no error message, only the right bracket is replaced by "!".

If I keep writing the expression, I still have a "!" with size 6.

If the expression is simpler, the correct bracket is displayed correctly.

Any idea to solve it?

Answer 1

TL; DR In the next line of mathtext.py line 727 an error appears. It associates a valid Bigg bracket with the index '\x21' , but it is the index for the exclamation point. A line with little context reads

 _size_alternatives = { '(' : [('rm', '('), ('ex', '\xa1'), ('ex', '\xb3'), ('ex', '\xb5'), ('ex', '\xc3')], ')' : [('rm', ')'), ('ex', '\xa2'), ('ex', '\xb4'), ('ex', '\xb6'), ('ex', '\x21')], 

I'm not quite sure which index needs to be changed, but I suggest you change the local copy of mathtext.py as follows:

 _size_alternatives = { '(' : [('rm', '('), ('ex', '\xa1'), ('ex', '\xb3'), ('ex', '\xb5'), ('ex', '\x28')], ')' : [('rm', ')'), ('ex', '\xa2'), ('ex', '\xb4'), ('ex', '\xb6'), ('ex', '\x29')] 

The brackets it produces are a little rounded as they are the main brackets, but they work. Similarly, you can replace the Bigg sizes with '\xb5' and 'xb6'

Matplotlib github Issue 5210 Reported

---

I can reproduce this problem with size=6 , using the code as provided (made the constant a bit bigger if it was a width problem). I can’t reproduce the “fix” by setting size = 7 , but I can, if I go up to size = 8 or higher, suppose that it can be an unpleasant error in the edge and, possibly, depending on the system ...

I have done quite a lot of research / diagnoses (see below), but it seems that there is a mistake - reported on matplotlib github here .

However, casting only an example to matplotlib gives a very good one as shown below. Note. I set my matplotlib to use latex rendering by default - but you can explicitly set the parameters for the same results.

The code

  import matplotlib.pyplot as plt import matplotlib as mpl mpl.rc('image', origin='upper') mpl.rc('text', usetex=True) DefaultProps = mpl.font_manager.FontProperties(family = "sans-serif",\ style = "normal",\ weight = "medium",\ size = 6) EXP = r'$\left(\frac{A \cdot \left(vds \cdot rs + \operatorname{Vdp}\left(ri, Rn, Hr, Hd\right) \cdot rh\right) \cdot \left(rSurf + \left(1.0 - rSurf\right) \cdot ft\right) \cdot \left(1.0 - e^{- \left(\left(lr + \frac{\operatorname{Log}\left(2\right)}{tem \cdot 86400.0}\right)\right) \cdot tFr \cdot 3600.0}\right)}{rc \cdot \left(lr + \frac{\operatorname{Log}\left(2\right)}{tem \cdot 86400.0}\right) \cdot tFr \cdot 3600.0} + 10589 \right)$' plt.title(EXP, fontsize=6) plt.gca().set_axis_off() # Get rid of the plotting axis for clarity plt.show() 

Exit

The output window is cropped and slightly enlarged for clarity, but you can see the bracket displays OK

This suggests that the problem is either in how matplotlib uses the rendering engine, output to a bitmap, or interact with wxPython

From an experiment, I noticed that if you increase the dpi to 300, the code works fine in size = 6 , but it starts crashing in size = 3 again. This means that the problem is that one of the libraries does not believe that it can display an element in a certain number of pixels.

Root cause

Diagnosing which bit does this is hard (IMO)

First, I added

  self.parser.to_png('D:\\math_out.png', _s, color=u'black', dpi=150) 

as the first line of mathtext_to_wxbitmap(self, _s, dpi = 150, prop = DefaultProps) . This gave a nice png result, making me think that apparently it was not a matplotlib parser ... EDIT based on @Baptiste's helpful answer , I checked this a bit more. In fact - if I explicitly pass fontsize this call, I can reproduce the look of the exclamation mark. In addition, the dpi passed to this call is ignored - so in my tests, I actually deal with a resolution of 300 dpi. Thus, the focus should be based on MathTextParser , and this may be a problem with dpi resolution.

---

Further research

A bit more research - I monkey paid to install matplotlib - adding print(result) immediately after calling parseString() here , with a working expression that works great and prints a text view. With a listened script, I see:

 Traceback (most recent call last): File "D:\baptiste_test.py", line 9, in <module> parser.to_png(filename, s, fontsize=size) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 3101, in to_ png rgba, depth = self.to_rgba(texstr, color=color, dpi=dpi, fontsize=fontsize) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 3066, in to_ rgba x, depth = self.to_mask(texstr, dpi=dpi, fontsize=fontsize) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 3039, in to_ mask ftimage, depth = self.parse(texstr, dpi=dpi, prop=prop) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 3012, in par se box = self._parser.parse(s, font_output, fontsize, dpi) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 2339, in par se print(result[0]) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 1403, in __r epr__ ' '.join([repr(x) for x in self.children])) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 1403, in __r epr__ ' '.join([repr(x) for x in self.children])) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 1403, in __r epr__ ' '.join([repr(x) for x in self.children])) File "C:\Python27\lib\site-packages\matplotlib\mathtext.py", line 1403, in __r epr__ ' '.join([repr(x) for x in self.children])) UnicodeEncodeError: 'ascii' codec can't encode character u'\xb3' in position 1: ordinal not in range(128) 

This indicates that an error may occur in an incorrectly translated character - perhaps missing code in the font?

I also noted that you can play without the letter N in the minimal Baptiste example.

---

Further research

Holding debugging prints in _get_glyph in the BakomaFonts class. In the unsuccessful case, the code seems to be looking for an exclamation mark (u '!'), When you expect it to look for u '\ xc4' and return parenrightBigg (i.e. where the corresponding left bracket is looking up u '\ xc3 'and returns parenleftBigg). In situations where he uses only parenrightbigg, there is no problem (this happens for fontsize = 5 in this example, but not for others). The debug line that I installed in _get_glyph was:

 print('Getting glyph for symbol',repr(unicode(sym))) print('Returning',cached_font, num, symbol_name, fontsize, slanted) 

I guess if he needs a bigg or Bigg version based on a combination of fonts and dpi

OK - I think the problem is in this line: https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/mathtext.py#L727

this reads (with a little context):

 _size_alternatives = { '(' : [('rm', '('), ('ex', '\xa1'), ('ex', '\xb3'), ('ex', '\xb5'), ('ex', '\xc3')], ')' : [('rm', ')'), ('ex', '\xa2'), ('ex', '\xb4'), ('ex', '\xb6'), ('ex', '\x21')], ## <---- Incorrect line. 

'\x21' incorrect, but I can’t understand that the correct '\x29' is close, but "too curved." I assumed that '\xc4' follows the pattern, but it is a down arrow. We hope that one of the main developers can easily find this number (195 decimal) against the glyph that it displays and corrects.

Answer 2

Here is the shortest piece of code I could write to get this unexpected behavior:

 import matplotlib.mathtext as mt s = r'$\left(\frac{\frac{\frac{M}{I}}{N}}' \ r'{\frac{\frac{B}{U}}{G}}\right)$' parser = mt.MathTextParser("Bitmap") for size in range(1, 30): filename = "figure{0}.png".format(size) parser.to_png(filename, s, fontsize=size) 

The outputs are as follows (choice 6-12):

This post was more to share minimal code to reproduce the error, not the answer to the question

Answer 3

I still have a "!" with dpi = '300' at a size of 9, but about 9 and besides 3, this is normal.

I created a new frame for playback with both dpi resolution and size and (')' and '[]'. Just use "LatexBitmapFactory" to play.

I am on Windows XP using python 2.7. I have the same error on Windows 7.

 class aFrame(wx.Frame): def __init__(self, title="Edition"): wx.Frame.__init__(self, None, wx.ID_ANY, title=title, size=(1300,600), style=wx.DEFAULT_DIALOG_STYLE) self.SetBackgroundColour(wx.Colour(255,255,255)) self.sizer = wx.BoxSizer(wx.VERTICAL) self.panel = scrolled.ScrolledPanel(self) init_dpi = 300 min_dpi, max_dpi = 100, 500 self.dpi_slider = wx.Slider(self.panel, wx.ID_ANY, init_dpi, min_dpi, max_dpi, (30, 60), (250, -1), wx.SL_HORIZONTAL | wx.SL_AUTOTICKS | wx.SL_LABELS) init_size = 9 min_size, max_size = 3, 40 self.size_slider = wx.Slider( self.panel, wx.ID_ANY, init_size, min_size, max_size, (30, 60), (250, -1), wx.SL_HORIZONTAL | wx.SL_AUTOTICKS | wx.SL_LABELS) self.log = wx.StaticText(self.panel, wx.ID_ANY, "DPI:%d SIZE:%d" %(init_dpi, init_size)) image_latex1 = self.create_lateximage(init_dpi, init_size) image_latex2 = self.create_lateximage(init_dpi, init_size, replace_parenthesis=True) self.sizer.Add(image_latex1, 1, flag=wx.ALIGN_CENTER|wx.EXPAND|wx.ALL) self.sizer.Add(image_latex2, 1, flag=wx.ALIGN_CENTER|wx.EXPAND|wx.ALL) self.sizer.Add(self.dpi_slider, 0, wx.ALIGN_CENTER) self.sizer.Add(self.size_slider, 0, wx.ALIGN_CENTER) self.sizer.Add(self.log, 0, wx.ALIGN_CENTER) self.dpi_slider.Bind(wx.EVT_SCROLL_CHANGED, self.OnSliderChanged) self.size_slider.Bind(wx.EVT_SCROLL_CHANGED, self.OnSliderChanged) self.panel.SetSizer(self.sizer) self.panel.SetAutoLayout(1) self.panel.SetupScrolling() def create_lateximage(self, dpi, size, replace_parenthesis=False): updatedProps = mpl.font_manager.FontProperties(family = "sans-serif",\ style = "normal",\ weight = "medium",\ size = size) if replace_parenthesis: tp_exp = EXP.replace("right)", "right]") tp_exp = tp_exp.replace("left(", "left[") return LatexBitmapFactory().SetBitmap(self.panel, tp_exp, dpi=dpi, prop=updatedProps) return LatexBitmapFactory().SetBitmap(self.panel, EXP, dpi=dpi, prop=updatedProps) def OnSliderChanged(self, evt): dpi = int(self.dpi_slider.GetValue()) size = int(self.size_slider.GetValue()) self.log.SetLabel("DPI:%d SIZE:%d" %(dpi, size)) self.Freeze() new_image_latex1 = self.create_lateximage(dpi, size) new_image_latex2 = self.create_lateximage(dpi, size, True) prev_image_latex1 = self.sizer.Remove(0) prev_image_latex2 = self.sizer.Remove(0) del prev_image_latex1 del prev_image_latex2 self.sizer.Insert(0, new_image_latex2, 1, flag=wx.ALIGN_CENTER|wx.EXPAND|wx.ALL) self.sizer.Insert(0, new_image_latex1, 1, flag=wx.ALIGN_CENTER|wx.EXPAND|wx.ALL) self.sizer.Layout() assert len(self.sizer.GetChildren()) == 5,\ " must have len 5, now %d"%len(self.sizer.GetChildren()) self.panel.SetupScrolling() self.panel.SetScrollRate(100,100) self.Thaw() app = wx.App(redirect=True, filename="latexlog.txt") frame = aFrame() frame.Show() app.MainLoop()