Need to delete objects: implement Dispose or create objects in a function?

Question

I have some objects that read a file, store data in arrays and perform some operations. The sequence is to create object A, work with object A. Create object B, work with object B ... The data read by each object can be about 10 MB. Thus, the best option would be to delete each object after working with each of them. Let's say I want my program to allocate about 10 MB in memory, and not 10 MB * 1000 objects = 1 GB

Objects are similar to:

class MyClass { List<string[]> data; public MyClass(string datafile) { using (CsvReader csv = new CsvReader(new StreamReader(datafile), true)) { data = csv.ToList<string[]>(); } } public List<string> Operate() { ... } } 

My question is: should I implement a utility? And do something like:

 List<string> results = new List<results>(); using (MyClass m = new MyClass("fileM.txt")) { results.AddRange(m.Operate()); } using (MyClass d = new MyClass("fileD.txt")) { results.AddRange(d.Operate()); } ... 

I read that to implement one-time use, it is recommended to use unmanaged resources (sockets, streams, ...), but in my class I have only large data arrays.

Another way would be to create functions for each object (I suppose GC will automatically delete the object created in the function):

 List<string> results = new List<results>(); results.AddRange(myFunction("fileM.txt")); results.AddRange(myFunction("fileD.txt")); public List<string> myFunction(string file) { MyClass c = new MyClass(file); return results.AddRange(c.Operate()); } 

Answer 1

IDisposable etc. they will not help you, since it does not cause to collect anything. In this type of scenario, perhaps the best approach is to use a pool to reduce allocations - essentially becoming your own memory manager. For example, if your List<string> large, you can avoid many arrays by reusing lists - after clearing them, obviously. If you call .Clear() , the lookup array is not reset - it just sets a logical marker, considering it empty. In your particular case, many of your objects will be individual string s; it is more complicated, but at least they are small and should be collected in the zero generation.

Answer 2

In your case, I would allocate one buffer array. For example, select an array of 10 MB in size and fill it with the necessary data. Then, when you move on to the next object, just reuse the array. If you need a more massive array, you can simply allocate a new, larger array and use it instead. The garbage collector will eventually delete your smaller one.

You can also use List<T> , it will internally do the same (select the array, keep it until it gets too small, select the new one). Just Clear before creating the next object.

Note that you cannot force garbage collector ¹ to collect an object. IDisposable really only used to clean unmanaged resources, as the garbage collector does not know about them or closes it (file). Calling Dispose does not guarantee (or imply) that the object is deleted from memory.

However, if you do not change anything, your code will still be correct and will work correctly. The garbage collector is responsible for removing unused objects whenever it looks like this, and this ensures that a lot of memory is available at any time. The only thing you need to do for the collector to do his job is to release any links to old objects (by rewriting them or setting them to null or allowing them to exit the scope).

¹ ) You can force the garbage collector to collect your data by calling GC.Collect() . However, this is not recommended. Let the garbage collector think for himself.

Answer 3

If you are using .NET 4.0 or higher, check out the BlockingCollection class. The constructor that accepts the Int32 parameter allows you to specify the upper bound on the size of the collection. Add and use working methods as chokes. Add will be successful only if the upper limit has not been reached. If he is, he will be blocked. Execution will be performed only if the item exists. If an item does not exist, it is locked until it is available. Of course, the class has some variations of these methods, so study the documentation completely to see which one makes sense.