What is a good memory serializer to replace BinaryFormatter?

Question

I am working with a large application using .Net v3.5 with C # (Visual Studio 2008) that uses BinaryFormatter to create a data file.

 Stream stream = File.Open(filePath, FileMode.Create, FileAccess.Write, FileShare.None); BinaryFormatter formatter = new BinaryFormatter(null, (new StreamingContext(StreamingContextStates.All, false))); formatter.Serialize(stream, data); stream.Flush(); stream.Close(); 

Unfortunately, I often get an OutOfMemoryException from this implementation. I am looking for some alternative to BinaryFormatter with which I can quickly switch.

It is worth noting that this application mainly relies on ISerializable attributes, not [Serializable] to save versions (views). In addition, the data that we serialize has several variables that point to the same object. Finally, we also serialize lists and dictionaries so that the data contains a fairly deep hierarchy of ISerializable s.

So, I would prefer an alternative that uses ISerializable.GetObjectData , capable of handling duplicate pointers to the same object.

Edit: in response to dbc you will ask a very good question. After replicating the problem, an error occurs:

 System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at System.Runtime.Serialization.ObjectIDGenerator.Rehash() at System.Runtime.Serialization.ObjectIDGenerator.GetId(Object obj, Boolean& firstTime) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.InternalGetId(Object obj, Boolean assignUniqueIdToValueType, Type type, Boolean& isNew) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteString(NameInfo memberNameInfo, NameInfo typeNameInfo, Object stringObject) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteKnownValueClass(NameInfo memberNameInfo, NameInfo typeNameInfo, Object data) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteMembers(NameInfo memberNameInfo, NameInfo memberTypeNameInfo, Object memberData, WriteObjectInfo objectInfo, NameInfo typeNameInfo, WriteObjectInfo memberObjectInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteMemberSetup(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo, String memberName, Type memberType, Object memberData, WriteObjectInfo memberObjectInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo, String[] memberNames, Type[] memberTypes, Object[] memberData, WriteObjectInfo[] memberObjectInfos) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph) 

Quick Googling tells me that this has to do with serializing an oversized object. I'm pretty sure that I don't store binary information like images or audio, so that seems strange to me. I assume that perhaps the list is too large for serialization.

By the way, I tried the code below using the same procedure and they do not throw exceptions.

 SerializationInfo info = new SerializationInfo(typeof(Data), new FormatterConverter()); StreamingContext context = new StreamingContext(StreamingContextStates.All, false); data.GetObjectData(info, context); foreach (SerializationEntry e in info) { Debug.WriteLine("Name: " + e.Name); Debug.WriteLine("Type: " + e.ObjectType.ToString()); Debug.WriteLine("Value: " + e.Value.ToString()); } 

Answer 1

This particular exception is not thrown due to the fact that you are serializing an oversized object. It is called because you serialize an object graph with so many objects that the ObjectIDGenerator inside the BinaryFormatter cannot allocate a sufficiently large hash table to assign unique identifiers to each pointer. The serializer uses an ObjectIDGenerator to generate a unique runtime identifier for each reference class serialized in such a way as to correctly serialize multiple references to the same instance of the class as an id-based indirect reference. Any graph serializer selected should do something like this.

Instead of using the new graph serialization technology, which will be quite onerous, is it possible to reduce the number of class instances that you serialize sequentially? For example, can you split an object graph into disconnected segments and serialize each of them sequentially into a stream? (One tutorial on how to do this is here: Serializing many different objects into a single file .) Or you have several classes that contain several small sheet classes that were never split and could be replaced by a single proxy for all of them?