Geek Answers Handbook

Java: serializing a huge amount of data into a single file

I need to serialize a huge amount of data (about 2gigs) of small objects into a single file, in order to be subsequently processed by another Java process. Performance is important. Can anyone suggest a good method to achieve this?

+5
source share
9 answers

Have you looked at google protocol buffers ? Sounds like a use case.

+4
source

I do not know why Java Serialization was voted, it is a completely viable mechanism.

This is not clear from the original message, but still 2G of data on the heap at the same time? Or are you dumping something else?

"" , Externalizable , . - , . Externalizable, , , .

- , . , , , , . ObjectStream / Java.

, ObjectOutputStream.writeObject(yourBigDatastructure) , Externalizable.

writeObject .

"objectToFile", , . , Externalizable, .

, , .. , .

+3

Java-? ObjectOutputStream ObjectInputStream. , Serializable. , , , .

0

, , - NIO (java.nio.MappedByteBuffer). (), , . .

0

: . : http://code.google.com/apis/protocolbuffers/docs/javatutorial.html

(, ). , , , . , , . .proto :

optimize_for = SPEED;

, , .

0

, . . 2 - . , XML , 2 .

, " " .

- Java, Java , .

0

JOAFIP .

0

Apache Avro . .

.

0

, - , Hibernate, ( , , , )

-1

More articles:


All Articles