Geek Answers Handbook

What data structure is good for temporary binary large data storage?

Plan a data structure for storing temporary binary data in memory for analysis.

The maximum data size will be about 10MB .

data will be added at the end of 408 bytes at a time.

do not search, retrieve operations with this temporary binary data.

the data will be destroyed and the storage will be reused for the next analysis.

questions:

  • What structure is good for this purpose? byte[10MB], List<bytes>(10MB), List<MyStruct>(24000), or ...?
  • how to quickly erase data (not List.Clear (), just set it to 0) for a list or array?
  • If I say List.Clear(), will the memory for this List decrease or is the capacity (memory) of the list still there and no memory is allocated when calling List.AddRange() after Clear()?
  • Will List.Insert () make the list larger or just replace an existing item?
+4
source share
3 answers

If your data is usually the same size and always under a certain size, use an array of bytes.

Create byte [] and int, which allows you to find out where the end of the "full" part of this buffer ends, and the "free" part begins. You never need to clean it; just rewrite what was there. The only problem is that sometimes your data is 100 kb, sometimes 10 MB, and sometimes a little more than previously planned.

The list will be slower to use and more in memory, although they process different sizes of data out of the box.

0
source

You will need to describe that you are doing more to give better answers, but it seems that you are concerned about the effectiveness of / perf, so

  • byte []
  • No need to clear the array, just keep track of where the "end" of your current loop is
  • n / a
  • n / a
+1
source 
 using (System.IO.MemoryStream memStream = new System.IO.MemoryStream()) { Do stuff } // the using ensures proper simple disposal occurs here so you don't have to worry about cleaning up.
0
source

More articles:


All Articles