Protocol Buffer Registration

Question

In our business, we require to register every request / response received on our server. We are currently using xml as a standard implementation. Log files are used if we need to debug / track some kind of error.

I am curious if we switch to protocol buffers, since this is a binary file, what would be the best way to write requests / responses to a file?

For example:

FileOutputStream output = new FileOutputStream("\\files\log.txt"); request.build().writeTo(outout); 

How do you record your request / response for those who used protocol buffers in your application, in case we need it for debugging purposes?

Answer 1

TL; DR: write debug logs to text, write long-term logs in binary format.

There are at least two ways to maintain this log (and perhaps you should do both):

• Writing your magazines in text format. This is good for debugging and quickly checking for problems with your eyes.
• Writing your logs in binary format will make future analysis much faster since you can load data using the same protocol buffer code and do all kinds of things with them.

Honestly, this is more or less how it is done in the place where this technology came from.

Answer 2

We use the ShortDebugString() method on a C ++ object to write a public version of all incoming and outgoing messages to a text file. ShortDebugString() returns a single-line version of the same string returned by the toString () method in Java. Not sure how easy it is to do the same thing in Java.

Answer 3

If you have competing needs for logging and performance, I suppose you could upload your binary data to a file as is, and perhaps each entry should be preceded by a tag containing a timestamp and a length value so that you know where it is a particular data bit ends. But I hasten to admit that it is very ugly. You will need to write a utility to read and analyze this file and will be helpless without this utility.

A smarter solution would be to dump your binary data in text form. I think of the “lines” of text, again, starting with any tag information that you think is relevant, it is followed by a certain length of information in decimal or hexadecimal form, followed by as many hexes as needed to flush your buffer — such So you can have some pretty long lines. But since the file is structured line by line, you can use text tools (an editor in the simplest case) to work with it. Hex dumping essentially means that you use two bytes in the log to represent one byte of data (plus a bit of overhead). Heh, disk space is cheap these days.

If these binary buffers have a fairly consistent structure, you can even break out and tag fields (or something like that) to make your data more readable and, more importantly, more searchable. Of course, you don't care how much effort you want to immerse yourself in making your journal entries look beautiful; but the time spent here may well pay off a little later in the analysis.