Why can't Python contain string.format with "\ x00"?

Question

I need to fill the string with null characters ("\ x00"). I know many ways to do this, so please do not respond with alternatives. I want to know: why does Python's string.format() function not allow filling with zeros?

Test cases:

 >>> "{0:\x01<10}".format("bbb") 'bbb\x01\x01\x01\x01\x01\x01\x01' 

This shows that hex escaped characters work in general.

 >>> "{0:\x00<10}".format("bbb") 'bbb ' 

But "\ x00" turns into space ("\ x20").

 >>> "{0:{1}<10}".format("bbb","\x00") 'bbb ' >>> "{0:{1}<10}".format("bbb",chr(0)) 'bbb ' 

Even try a couple of other ways to do this.

 >>> "bbb" + "\x00" * 7 'bbb\x00\x00\x00\x00\x00\x00\x00' 

This works, but does not use string.format

 >>> spaces = "{0: <10}".format("bbb") >>> nulls = "{0:\x00<10}".format("bbb") >>> spaces == nulls True 

Python explicitly replaces spaces ( chr(0x20) ) instead of zeros ( chr(0x00) ).

Answer 1

Since the string.format method in Python2.7 is the return port from Python3 string.format . Python2.7 unicode is a Python 3 string, where the Python2.7 string is Python3 bytes. A string is the wrong type for expressing binary data in Python3. You must use bytes that do not have a formatting method. So really, you should ask why the format method in the string is not present at all in 2.7, when it was supposed to be valid only for the Unicode type, since this is what became the string in Python3.

I think this answer is that it is too convenient to have.

As a related question, why is there no format in bytes yet

Answer 2

Delving into the source code for Python 2.7, I found that the problem in this section is from ./Objects/stringlib/formatter.h , lines 718-722 (in version 2.7.3):

 /* Write into that space. First the padding. */ p = fill_padding(STRINGLIB_STR(result), len, format->fill_char=='\0'?' ':format->fill_char, lpad, rpad); 

The problem is that the zero / zero character ( '\0' ) is used by default if the padding character is not specified. This should provide this behavior:

 >>> "{0:<10}".format("foo") 'foo ' 

Perhaps you can set format->fill_char = ' '; as the default value in parse_internal_render_format_spec() in ./Objects/stringlib/formatter.h:186 , but there is a bit about backward compatibility, which later checks for '\0' . In any case, my curiosity is satisfied. I will admit that someone else will answer if he has more history or a more detailed explanation of the reasons for this.

Answer 3

The answer to the original question is that it was a bug in python.

It was documented as authorized, but was not. It was fixed in 2014. For python 2, the fix first appeared in either 2.7.7 or 2.7.8 (I'm not sure how to say that)

Original track issue .