Python / PySerial bit operator in string

Question

I am using PySerial (Python 2.7) to read information from a device like this:

buffer + = ser.read (3)

Now I have three bytes in the buffer (i.e. 0xAE0259), which is of type str. Since I'm new to Python, I'm looking for a “Python” way to cut off the left side (0xAE) of three bytes, and then interpret the other two as int. At first I thought of a little mask: buffer & = 0xFFFF but python will not allow me to use bitwise operators on str. Any attempt to convert the buffer to int also failed. Then I read about the bitstring module, which allows me to truncate bit ranges from BitArray, but I assume that using this for this would be a bit above the top?

Answer 1

You need to know whether multibyte types are large or small, and whether they are signed or not. Assuming the two bytes are unsigned, short-endian short, I would do the following:

>>> buf = '\xAE\x02\x59' >>> from struct import unpack >>> unpack('>BH', buf) (174, 601) 

'>' means the big end. "B" is the first unsigned byte that you do not need. "H" is an unsigned character.

Answer 2

There are two easy ways to do this. One way would be to convert your buffer to a hexadecimal integer and use a bit mask to get the last 32 bits. Another is to use the slice operator to get the last 4 characters and interpret this remainder as a hexadecimal integer.

 >>> buffer = 'AE0259' >>> print int(buffer, 16) & 0xFFFF 601 >>> print int(buffer[-4:], 16) 601 

---

EDIT - eryksun has the correct answer, but I wanted to update my example for actual use.

 >>> buffer = '\xAE\x02\x59' >>> # print the integer value of the last two binary "characters" >>> print sum((ord(x) << i*8 for i,x in enumerate(buffer[:-2-1:-1]))) 601 >>> # print the integer value all binary "characters" >>> # with a bitmask of the lower 32 digits >>> print sum((ord(x) << i*8 for i,x in enumerate(buffer[::-1]))) & 0xFFFF 601 

Answer 3

If you only need to perform byte decompression, then the struct module is your friend (see eryksun answer), as well as bytearray Type:

 >>> ba = bytearray('\xae\x02\x59') 

This allows indexing and slicing at byte level.

 >>> hex(ba[0]) '0xae' >>> ba[1:3] bytearray(b'\x02Y') 

In terms of converting multiple bytes to int, this is very useful, but you are unlikely to get much more struct if you don't have unusual byte lengths. Converting two bytes to int will be:

 >>> (ba[1] << 8) + ba[2] 601 

In the commentary, you say that you need a general way to perform a bitwise slice. I'm afraid there is nobody - your best place to start is switching and disguising with bytearray. This is why useful modules such as bitstring are useful (I wrote this by the way) - you make someone make all the tedious mistakes! / p>

 >>> b = bitstring.Bits(ba) >>> b[8:].uint 601 >>> b.unpack('hex:8, uint:16') ['ae', 601] 

Answer 4

if buffer is a string, you can trim the remaining characters by trimming the string as follows:

 newstr = buffer[1:] 

Answer 5

@ironchefpython suggested what you should use, except that your comment indicates that your buffer actually consists of binary data. Assuming this is so, this solution should work, although it is not very elegant:

 from struct import unpack def strmask(buffer, mask): #calculate the number of bytes to extract mask_length = mask.bit_length() / 8 + (1 if mask.bit_length() % 8 > 0 else 0) #extract those bytes and put them into a Python int, then perform the mask return mask & reduce(lambda l,r: (l<<8)+r, unpack("B" * mask_length, buffer[-1 * mask_length:])) 

This gives the result you want - for example:

 >>> print strmask('\xAE\x02\x59', 0xFFFF) 601