General PB analysis in java

Question

Is it possible to parse protobuf in general form in Java?

I looked at GeneratedMessage and could not find a way to parse any PB byte buffer in GeneratedMessage.

Essentially, I'm trying to parse the PB byte buffer in GeneratedMessage, and then I will use reflection to detect the fields inside it.

Answer 1

First of all, you cannot analyze PB data without knowing the patterns. Initially, this scheme comes from the ".proto" file and is usually embedded in the code generated by protoc . However, you can also tell protoc to save the schema in a format that the Protobuf Java library can use:

 protoc --descriptor_set_out=mymessages.desc mymessages.proto 

Then load it into Java code:

 FileInputStream fin = new FileInputStream("mymessages.desc"); Descriptors.FileDescriptorSet set = Descriptors.FileDescriptorSet.parseFrom(fin); Descriptors.Descriptor md = set.getFile(0).getMessageType(0); 

Once you have a schema for the message ( Descriptor.Descriptor ), parsing the message is easy:

 byte[] data = ...; DynamicMessage m = DynamicMessage.parseFrom(md, data); 

DynamicMessage has a reflective API that allows you to view fields.

The unused part calls the protoc tool to convert the .proto file to a convenient format. The C ++ Protobuf library has the ability to directly upload the ".proto" files, but unfortunately there is no Protobuf Java library.

Answer 2

I have a working solution tested with the latest protobuf v.3.1.0

This is an improved solution raised in previous answers. Thanks to both authors.

 import com.example.address.AddressBookManager; import com.example.address.AddressBookProtos.AddressBook; import com.google.protobuf.DescriptorProtos.FileDescriptorSet; import com.google.protobuf.Descriptors; import com.google.protobuf.Descriptors.Descriptor; import com.google.protobuf.Descriptors.FileDescriptor; import com.google.protobuf.DynamicMessage; import java.io.File; import java.io.InputStream; public class DynamicMessageDemo { private static final String ADDRESS_BOOK_SOURCE_FILENAME = "test.ab"; private static final String ADDRESS_BOOK_DESC_FILENAME = File.separator + "address.desc"; public static void main(String[] args) throws Exception { InputStream is = DynamicMessageDemo.class .getResourceAsStream(ADDRESS_BOOK_DESC_FILENAME); FileDescriptorSet set = FileDescriptorSet.parseFrom(is); FileDescriptor fd = Descriptors.FileDescriptor.buildFrom( set.getFile(0), new Descriptors.FileDescriptor[]{} ); // "AddressBook" is the second message in my *.proto // so index must be '1' Descriptor messageType = fd.getMessageTypes().get(1); // for testing purpose AddressBook book = AddressBookManager.readFromFile(ADDRESS_BOOK_SOURCE_FILENAME); byte[] data = book.toByteArray(); DynamicMessage message = DynamicMessage.parseFrom(messageType, data); System.out.println("\n Dynamic message:\n" + message); } } 

Answer 3

This is a valid example:

 private static DynamicMessage parseData(byte[] data) throws IOException, DescriptorValidationException { FileInputStream fin = new FileInputStream("test.desc"); DescriptorProtos.FileDescriptorSet set = DescriptorProtos.FileDescriptorSet.parseFrom(fin); Descriptor md = Descriptors.FileDescriptor.buildFrom(set.getFile(0), new Descriptors.FileDescriptor[] {}).findMessageTypeByName("Person"); return DynamicMessage.parseFrom(md, data); }