C #, how can I convert an array of bytes of binary data to a user type that models the data?

Question

Scenario: I received raw binary data via HTTP and saved the data in an array of bytes. I have documentation describing various fields that can represent binary data, but the actual value of the data must be determined at runtime. For example, if the byte representing the occurrence of the error = 1, the value of the next byte is changed.

Using C # with .NET 4, I want to create one or more classes that reflect the fields described in the documentation, and then somehow initialize the classes using an array of bytes of binary data. I would like the solution to minimize code duplication, and also be modular and elegant.

I studied the creation of Serializable classes, but I don’t see how this can work, since I start with an array of bytes that was not created (and therefore not serialized) by me.

I also tried using generics and reflection to get the sizes and types of fields contained in custom classes. Then I used this information to dynamically slice data from an array of bytes and assign them the appropriate fields. However, this method has led to a lot of ugly, unmanaged code.

Any advice or pointers on developing an extensible, untied solution to this problem would be greatly appreciated.

Edit: example classes containing fields that reflect fields in a specification

public class PriceHistoryResponse : BinaryResponse { public List<Quote> quotes { get; set; } private CountData countData { get; set; } private EndingDelimiterSection endingDelimiterSection { get; set; } /* This code performs the logic needed to check for optional fields and to find the number of times that certain fields are repeated */ public PriceHistoryResponse(byte[] responseBytes) : base(responseBytes) { countData = new CountData(); ParseResponseSection(countData); quotes = new List<Quote>(); for (int i = 0; i < countData.quoteCount; i++) { quotes.Add(new Quote()); quotes[i].symbolData = new SymbolData(); ParseResponseSection(quotes[i].symbolData); if (quotes[i].symbolData.errorCode == 1) { quotes[i].errorData = new ErrorData(); ParseResponseSection(quotes[i].errorData); break; } quotes[i].chartBarData = new ChartBarData(); ParseResponseSection(quotes[i].chartBarData); quotes[i].chartBars = new List<ChartBar>(); for (int j = 0; j < quotes[i].chartBarData.chartBarCount; j++) { quotes[i].chartBars.Add(new ChartBar()); ParseResponseSection(quotes[i].chartBars[j]); } } endingDelimiterSection = new EndingDelimiterSection(); ParseResponseSection(endingDelimiterSection); } } class CountData : IResponseSection { public int quoteCount { get; set; } } public class Quote { public SymbolData symbolData { get; set; } public ErrorData errorData { get; set; } public ChartBarData chartBarData { get; set; } public List<ChartBar> chartBars { get; set; } } public class SymbolData : IResponseSection { public string symbol { get; set; } public byte errorCode { get; set; } } public class ErrorData : IResponseSection { public string errorText { get; set; } } public class ChartBarData : IResponseSection { public int chartBarCount { get; set; } } public class ChartBar : IResponseSection { public float close { get; set; } public float high { get; set; } public float low { get; set; } public float open { get; set; } public float volume { get; set; } public long timestamp { get; set; } } 

Answer 1

I pasted your code into VS, clicked the "Generate Method Stub" several times, and moved it around. I think it will be a trick.

The code you provide is pretty smart, it is a bit like a visitor template where the overload switches to the correct method.

  public class BinaryResponse { private BinaryReader _rdr; public BinaryResponse(byte[] responseBytes) { _rdr = new BinaryReader(new MemoryStream(responseBytes)); // wrap the byte[] in a BinaryReader to be able to pop the bytes off the top } protected void ParseResponseSection(CountData countData) { countData.quoteCount = _rdr.ReadInt16(); // guessing 64.000 quotes should be enough in one response, the documentation will have the type } protected void ParseResponseSection(SymbolData symbolData) { symbolData.errorCode = _rdr.ReadByte(); // depending on your format, where is the ErrorCOde in the byte[]? the symbol might be first int symbolLength = _rdr.ReadInt16(); // if it not written by a .Net WriteString on the other end better to read this count yourelf symbolData.symbol = new string(_rdr.ReadChars(symbolLength)); // read the chars and put into string } protected void ParseResponseSection(ErrorData errorData) { int errorLenth = _rdr.ReadInt16(); errorData.errorText = new string(_rdr.ReadChars(errorLenth)); } protected void ParseResponseSection(ChartBarData chartBarData) { chartBarData.chartBarCount = _rdr.ReadInt16(); } protected void ParseResponseSection(ChartBar chartBar) { // check the order with the documentation, also maybe some casting is needed because other types are in the byte[] chartBar.close = _rdr.ReadSingle(); chartBar.high = _rdr.ReadSingle(); chartBar.low = _rdr.ReadSingle(); chartBar.open = _rdr.ReadSingle(); chartBar.timestamp = _rdr.ReadInt64(); } protected void ParseResponseSection(EndingDelimiterSection endingDelimiterSection) { int checkValue = _rdr.ReadInt16(); if (checkValue != 12345) throw new InvalidDataException("Corrupt Response! Expecting End Delimiter"); // assert that the end delimiter is some value } } 

Is this what you are looking for? You didn't say anything about encoding, you may need to consider this when reading bytes, etc.

Regards Geert-Yang

Answer 2

You can try to do something like this:

 public object ByteArrayToObject(byte[] byteArray) { try { // convert byte array to memory stream System.IO.MemoryStream memoryStream = new System.IO.MemoryStream(byteArray); // create new BinaryFormatter System.Runtime.Serialization.Formatters.Binary.BinaryFormatter binaryFormatter = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter(); // set memory stream position to starting point memoryStream.Position = 0; // Deserializes a stream into an object graph and return as a object. return binaryFormatter.Deserialize(memoryStream); } catch (Exception ex) { Console.WriteLine("Exception caught in process: {0}", ex.ToString()); } return null; } 

EDIT

Take a look at protobuf-net. I think this is what you need:

http://code.google.com/p/protobuf-net/

 protected bool EvaluateBuffer(byte[] buffer, int length) { if (length < 8) { return false; } MessageType messageType = (MessageType)BitConverter.ToInt32(buffer, 0); int size = BitConverter.ToInt32(buffer, 4); if (length < size + 8) { return false; } using (MemoryStream memoryStream = new MemoryStream(buffer)) { memoryStream.Seek(8, SeekOrigin.Begin); if (messageType == MessageType.MyMessage) { MyMessage message = ProtoBuf.Serializer.Deserialize<MyMessage>(memoryStream); } } } 

Answer 3

There is a lot of project that will allow you to (de) serialize classes, but if the format is set and does not match some standard protocol buffers, then I think they will not help you. Are you sure this is not a standard format?

If you need to read a byte to find out what the following bytes mean, I assume that you need to read them byte by byte using BinaryReader , and then pass the rest of the byte array to another function based on the result.

If you want to avoid manually coding these “parsers”, you can do a little DSL or something that will generate dataclasses and accompanying code to read one of the byte array. But it sounds like a lot of work to save a little.

How many different classes do you need to read these byte arrays? My first guess would be to just write the code manually. You must have many different classes or complex logic to restore your development efforts.

Some more random ideas: look at the source code of the protocol buffer implementation, maybe you can choose ideas from them. http://code.google.com/p/protobuf-net/ or http://code.google.com/p/protobuf-csharp-port/

Rgds gert-jan