How to parse / encode binary message formats?

Question

I need to parse and encode into legacy binary message format in Java. I started by using a DataOutputStream to read / write primitive types, but the problem I am encountering is that the message format does not align with the offset bytes and includes bit flags.

For example, I have to deal with such messages:

+----------+---+---+----------+---------+--------------+ +uint32 +b +b + uint32 +4bit enum+32 byte string+ +----------+---+---+----------+---------+--------------+ 

Where (b) is one bit. The problem is that java primitive types do not align with byte boundaries, so I would not be able to use a DataOutputStream to encode this, since the lowest level type I can write is a byte.

Are there libraries, standard or third-party, for handling message formats of arbitrary bit level?

Edit: Thanks @Software Monkey for making me take a closer look at my spec. The spectrum that I use actually aligns the byte boundaries, so DataOutputStream is suitable. Given my initial question, although I would go with the solution suggested by @emboss.

Edit: Although the message format for this question was found at byte boundaries, I found a different message format that is applicable to the original question. This format defines a 6-bit character mapping, where each character really only takes 6 bits, not a full byte, so character strings are not aligned by byte boundaries. I found several binary output streams that solve this problem. Like this one: http://introcs.cs.princeton.edu/java/stdlib/BinaryOut.java.html

Answer 1

There is a built-in byte type in Java, and you can only read byte[] buffers using InputStream # read (byte []) and write to OutputStream using OutputStream # write (byte [], int, int) , so there is no problem .

As for your messages - as you noted correctly, the minimum bit of information that you receive at one time is a byte, so you will first have to decompose your message format into 8 bits:

Suppose your message is in bytes [] named data. I also assume a little.

A uint32 has a length of 32 bits → four bytes. (Be careful when analyzing this in Java, Java integers and longs are signed, you need to handle it. An easy way to avoid trouble is to take a lot of time. Data [0] fills bits 31 - 24, data [1] 23 - 16, data [2] bits 15 to 8 and data [3] bits 7 to 0. Therefore, you need to accordingly move them to the left and glue them together with a logical OR:

 long uint32 = ((data[0]&0xFF) << 24) | ((data[1]&0xFF) << 16) | ((data[2]&0xFF) << 8) | (data[3]&0xFF); 

Further, there are two separate bits. I suppose you need to check if they are "on" (1) or "off" (0). To do this, you use bit masks and compare your byte with logical I.

First bit: (binary mask | 1 0 0 0 0 0 0 0 0 | = 128 = 0x80)

 if ( (data[4] & 0x80 ) == 0x80 ) // on 

The second bit: (binary mask | 0 1 0 0 0 0 0 0 | = 64 = 0x40)

 if ( (data[4] & 0x40 ) == 0x40 ) // on 

To compose the next uint32, you have to compose bytes above the byte boundaries of the underlying data. For example. for the first byte, take the remaining 6 bits of data [4], move them two to the left (they will be bits 8 to 2 of uint32) and "add" the first (highest) two data [5], moving them 6 bits to the right (they will take the remaining 1 and 0 uint32 slots). "Add" means logically OR'ing:

 byte uint32Byte1 = (byte)( (data[4]&0xFF) << 2 | (data[5]&&0xFF) >> 6); 

Building your uint32 will be the same as in the first example. And so on and so forth.

Answer 2

I heard good things about Preon .

Answer 3

To add an answer to pholser, I think the Preon version will be something like this:

 class DataStructure { @BoundNumber(size="32") long first; // uint32 @Bound boolean second; // boolean @Bound boolean third; // boolean @BoundNumber(size="32") long fourth; // uint32 @BoundNumber(size="4") int fifth; // enum @BoundString(size="32") String sixth; // string } 

... but in fact, you can make your life even easier by using Preon support to directly handle listings .

Creating a Codec for it and using it to decode some data would be something like this:

 Codec<DataStructure> codec = Codecs.create(DataStructure.class) DataStructure data = Codecs.decode(codec, ....) 

Answer 4

with Java Binary Block Parser script to parse the message will be

  class Parsed { @Bin int field1; @Bin (type = BinType.BIT) boolean field2; @Bin(type = BinType.BIT) boolean field3; @Bin int field4; @Bin(type = BinType.BIT) int enums; @Bin(type = BinType.UBYTE_ARRAY) String str; } Parsed parsed = JBBPParser.prepare("int field1; bit field2; bit field3; int field4; bit:4 enums; ubyte [32] str;").parse(STREAM).mapTo(Parsed.class); 

Answer 5

You need to apply bit arithmetic (AND, OR, AND NOT) to change or read individual bits in a byte in Java. Arithmetic Operators - &, | and ~