Speed ​​up serialization of multiple objects

Question

A slightly related issue with this and this .

Basically, I would like to serialize objects as they become available, like a log file, except that I want to non-sterilize them later. This means that I do not have all the objects initially.

From the previous answers, it seems that if one archive is open, you can continue to add more and more objects to the archive.

But how do I get them? Do I have to look to the future and see if the effect is achieved before each extraction? Do I have to put a dashed line in the save procedure so that later I can read the input line by line (this will probably only work with binary archives (and possibly with text), since xml uses linear breaks, and maybe even there if the binary can use linebreak sometimes)? Perhaps the operation → throws an exception if the end of the file is reached, and I can wrap it in an infinite loop using try catch around it?

And how would I do it if I wanted to do it for different objects? Maybe there is an enumeration for all objects and serialization of the enumeration just before that, and when non-serializing, is there a switch based on the enumeration?

thanks

Answer 1

Here is what I did at the end. Since Nicol is right, and it’s really impractical, you must first turn off pointer tracking. Otherwise, you get false shared objects. Therefore, for starters, boot

BOOST_CLASS_TRACKING(yourDerivedClass,boost::serialization::track_never) 

I also focused on simply registering objects that come from the same base object (for this you could just create an empty virtual database). Once this is done, it is important to make sure that it is considered as abstract (I definitely had virtual destructors and still added

 BOOST_SERIALIZATION_ASSUME_ABSTRACT(yourBaseClass) 

Register all derived classes with the archive after it is created (both for writing and reading)

 arMsgs.template register_type<yourDerivedClass>(); 

Write only the final non-abstract class (if A is taken from B, comes from C, B is not registered). At least any registered class should disable tracking.

Finally add them to the archive as they become available.

To reload them, instead of using a special token for the end of the file, which will require verification, I went to

 try { for(;;) { yourBaseClass obj; arObjs >> boost::serialization::make_nvp("Obj",obj); //your logic } } catch(boost::archive::archive_exception const& e) { } 

It may be too much, so additional verification may be required, but it works for me. The advantage is that proper closure is not so important, for example. if your application crashes halfway, you can still process the last message you read.

Answer 2

What you are talking about is not really a serialization issue. Serialization is designed to work when the order of writing and reading is determined by compilation time. Even version control is compilation time; you either serialize the value or not based on the version of the execution. The order is preserved.

Now you can add tokens to the output, some kind of integer value that maps to class types. That is, before writing a class to a stream, you write an integer representing what the class is. You will then read it later and decide which type to serialize the next based on this.

But the problem that you will encounter is that in the end you will run out of content. And in the serialization archives there really is no way to say "everything." It is expected that they will be determined at compile time, so reading beyond the end of input is considered a user error. Therefore, the processing of arbitrary amounts of serialized data is not always supported.

Again, you can write a special token for output, something that says "this is the end."

Answer 3

I am involved in training, and I think you can use boost serialization as a log file and continue to add values ​​using your own logic. I ran into the same problem, and if I'm not mistaken, your code will be something like this:

 #include <iostream> #include <fstream> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> int main() { int two=2; for(int i=0;i<10;i++) { std::ofstream ofs("table.txt"); boost::archive::text_oarchive om(ofs); om << two; two = two+30; std::cout<<"\n"<<two; } return 0; } 

Here, when you close the curly braces (loop brackets), the serialization file closes. And you can see only one value written in the table.txt file, if you want to save several values, your code should look something like this:

 #include <iostream> #include <fstream> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> int main() { int two=2; { std::ofstream ofs("table.txt"); boost::archive::text_oarchive om(ofs); for(int i=0;i<10;i++) { om << two; two = two+30; std::cout<<"\n"<<two; } } return 0; } 

Here you can see that the brackets containing boost :: serialization :: text_oarchive only close when I have finished serializing the result of my logic.