More Pythonic to Binary Conversion?

Question

Here is a contrived example of how many of our classes return binary representations (for reading C ++).

def to_binary(self): 'Return the binary representation as a string.' data = [] # Binary version number. data.append(struct.pack('<I', [2])) # Image size. data.append(struct.pack('<II', *self.image.size)) # Attribute count. data.append(struct.pack('<I', len(self.attributes))) # Attributes. for attribute in self.attributes: # Id. data.append(struct.pack('<I', attribute.id)) # Type. data.append(struct.pack('<H', attribute.type)) # Extra Type. if attribute.type == 0: data.append(struct.pack('<I', attribute.typeEx)) return ''.join(data) 

What I do not like:

• Each line begins with data.append(struct.pack( , distracting from the unique part of the line.
• Byte order ( '<' ) is repeated over and over.
• You must remember to return the template ''.join(data) .

What I like:

• Format specifiers are displayed next to the attribute name. For example, it is easy to see that self.image.size written as two unsigned ints.
• The lines are (mostly) independent. For example, to remove an identifier field from an attribute, you do not need to touch more than one line of code.

Is there a more readable / pythonic way to do this?

Answer 1

 from StringIO import StringIO import struct class BinaryIO(StringIO): def writepack(self, fmt, *values): self.write(struct.pack('<' + fmt, *values)) def to_binary_example(): data = BinaryIO() data.writepack('I', 42) data.writepack('II', 1, 2) return data.getvalue() 

Answer 2

You can try to implement declarative syntax for your data.

This could lead to something like:

 class Image(SomeClassWithMetamagic): type = PackedValue(2) attribute = PackedValue('attributes') # accessed via self.__dict__ #or using decorators @pack("<II") def get_size(): pass #and a generic function in the Superclass def get_packed(): stuff 

etc...

Other examples would be SQLAlchemy declarative_base, ToscaWidgets, and sprox

Answer 3

If you want a stronger syntax, you can use generators / decorators:

 from functools import wraps def packed(g): '''a decorator that packs the list data items that is generated by the decorated function ''' @wraps(g) def wrapper(*p, **kw): data = [] for params in g(*p, **kw): fmt = params[0] fields = params[1:] data.append(struct.pack('<'+fmt, *fields)) return ''.join(data) return wrapper @packed def as_binary(self): '''just |yield|s the data items that should be packed by the decorator ''' yield 'I', [2] yield 'II', self.image.size[0], self.image.size[1] yield 'I', len(self.attributes) for attribute in self.attributes: yield 'I', attribute.id yield 'H', attribute.type if attribute.type == 0: yield 'I', attribute.typeEx 

It basically uses a generator to implement the “monad,” an abstraction commonly found in functional languages ​​such as Haskell. It separates the generation of some values ​​from the code, which decides to combine these values ​​together. This is a more functional programming approach than pythonic, but I think it improves readability.

Answer 4

How about protocol buffers is an extended google cross language format and data exchange protocol.

Answer 5

 def to_binary(self): struct_i_pack = struct.Struct('<I').pack struct_ii_pack = struct.Struct('<II').pack struct_h_pack = struct.Struct('<H').pack struct_ih_pack = struct.Struct('<IH').pack struct_ihi_pack = struct.Struct('<IHI').pack return ''.join([ struct_i_pack(2), struct_ii_pack(*self.image.size), struct_i_pack(len(self.attributes)), ''.join([ struct_ih_pack(a.id, a.type) if a.type else struct_ihi_pack(a.id, a.type, a.typeEx) for a in attributes ]) ]) 

Answer 6

You can reorganize your code to wrap a template in a class. Something like:

 def to_binary(self): 'Return the binary representation as a string.' binary = BinaryWrapper() # Binary version number. binary.pack('<I', [2]) # alternatively, you can pass an array stuff = [ ('<II', *self.image.size), # Image size. ('<I', len(self.attributes)), # Attribute count ] binary.pack_all(stuff) return binary.get_packed() 

Answer 7

The worst problem is that you need the appropriate C ++ code to read the output. Can you intelligently arrange for code to be read or written mechanically from the general specification or used by it? How to get around this depends on your needs in C ++, as well as Python.

Answer 8

You can get rid of repetition while it is easy to read as follows:

 def to_binary(self): output = struct.pack( '<IIII', 2, self.image.size[0], self.image.size[1], len(self.attributes) ) return output + ''.join( struct.pack('<IHI', attribute.id, attribute.type, attribute.typeEx) for attribute in self.attributes )