Geek Answers Handbook

Binary file format with 1000s entries in C #

I would like array model objects to be serialized into a binary stream. The model class will mainly have string and integer properties.

I believe that I can mark the class as [Serializable] and use the binary formattter, however, I would be interested to know if you think this is the best way, given that my priority is to have a smaller file, such as possible for transferring via low bandwidth connection (I can zip / unzip the file).

There can be 1000 records in a file, so ideally I would like to be able to add to a disk and read from a disk record by record, without having at the same time the whole file in memory.

So, my priorities are: small file size and efficient memory usage.

Maybe there is a preliminary basis for this? This is easy to do with XML and CSV files! Hope this is also with binary format.

thank

+5
source share
6 answers

I suggest protobuf.net which is very effective.

Having said that, it will not be able to handle serializing / deserting individual objects in your collection. You need to implement this part yourself.

  • One solution: Store objects as separate files in a folder. The file name will contain a link so that based on the name you can find the object you need.

  • , , . , , , , , , b-tree .

+6

- ZIP . , MemoryMappedFile .

+2

BinaryFormatter. , , . , .

, , - DataSet. () DataSet, , , , BinaryFormatter.

public static TDataSet LoadBinary<TDataSet>(Stream stream) where TDataSet : DataSet
{
    var formatter = new BinaryFormatter();
    return (TDataSet)formatter.Deserialize(stream);
}

public static void WriteBinary<TDataSet>(this TDataSet dataSet, Stream stream) where TDataSet : DataSet
{
    dataSet.RemotingFormat = SerializationFormat.Binary;
    var formatter = new BinaryFormatter();
    formatter.Serialize(stream, dataSet);
}

DataContractSerializer, .NET- ( # 4.0 , ). "Best Practices: Data Contract Versioning" . , () XML JSON, ( ). .

/// <summary>
/// Converts this instance to XML using the <see cref="DataContractSerializer"/>.
/// </summary>
/// <typeparam name="TSerializable">
/// A type that is serializable using the <see cref="DataContractSerializer"/>.
/// </typeparam>
/// <param name="value">
/// The object to be serialized to XML.
/// </param>
/// <returns>
/// Formatted XML representing this instance. Does not include the XML declaration.
/// </returns>
public static string ToXml<TSerializable>(this TSerializable value)
{
    var serializer = new DataContractSerializer(typeof(TSerializable));
    var output = new StringWriter();
    using (var writer = new XmlTextWriter(output) { Formatting = Formatting.Indented })
    {
        serializer.WriteObject(writer, value);
    }
    return output.GetStringBuilder().ToString();
}

/// <summary>
/// Converts this instance to XML using the <see cref="DataContractSerializer"/> and writes it to the specified file.
/// </summary>
/// <typeparam name="TSerializable">
/// A type that is serializable using the <see cref="DataContractSerializer"/>.
/// </typeparam>
/// <param name="value">
/// The object to be serialized to XML.
/// </param>
/// <param name="filePath">Path of the file to write to.</param>
public static void WriteXml<TSerializable>(this TSerializable value, string filePath)
{
    var serializer = new DataContractSerializer(typeof(TSerializable));
    using (var writer = XmlWriter.Create(filePath, new XmlWriterSettings { Indent = true }))
    {
        serializer.WriteObject(writer, value);
    }
}

/// <summary>
/// Creates from an instance of the specified class from XML.
/// </summary>
/// <typeparam name="TSerializable">The type of the serializable object.</typeparam>
/// <param name="xml">The XML representation of the instance.</param>
/// <returns>An instance created from the XML input.</returns>
public static TSerializable CreateFromXml<TSerializable>(string xml)
{
    var serializer = new DataContractSerializer(typeof(TSerializable));

    using (var stringReader = new StringReader(xml))
    using (var reader = XmlReader.Create(stringReader))
    {
        return (TSerializable)serializer.ReadObject(reader);
    }
}

/// <summary>
/// Creates from an instance of the specified class from the specified XML file.
/// </summary>
/// <param name="filePath">
/// Path to the XML file.
/// </param>
/// <typeparam name="TSerializable">
/// The type of the serializable object.
/// </typeparam>
/// <returns>
/// An instance created from the XML input.
/// </returns>
public static TSerializable CreateFromXmlFile<TSerializable>(string filePath)
{
    var serializer = new DataContractSerializer(typeof(TSerializable));

    using (var reader = XmlReader.Create(filePath))
    {
        return (TSerializable)serializer.ReadObject(reader);
    }
}

public static T LoadJson<T>(Stream stream) where T : class
{
    var serializer = new DataContractJsonSerializer(typeof(T));
    object readObject = serializer.ReadObject(stream);
    return (T)readObject;
}

public static void WriteJson<T>(this T value, Stream stream) where T : class
{
    var serializer = new DataContractJsonSerializer(typeof(T));
    serializer.WriteObject(stream, value);
}
+1

Sql Server Compact , , , .

(), , , . , , , .

, XML- zip-compression, . XML , , .

+1

, , . , . , 255 , .

http://msdn.microsoft.com/en-us/library/system.bitconverter.aspx

, ,

id (ushort)

data_size (uint)

data_size

, , , . , , .

0

BinaryFormatter , , protobuf.net, .

If the aspect of random access to this is very important (reading and adding records by record), you can look at creating a zip file (or similar) containing an index file, and each object serialized into its own file in a zip file (or, possibly, in small collections).

Thus, you can effectively have a mini file system that is compressed and gives you access to your records individually.

0
source

More articles:


All Articles