How can I speed up instantiation of a large collection of objects?

Question

The code below is extremely slow for tables of any significant size. (100, 1000, etc.). The criminal copies my objects using new T() . Please note that this is not my final code, I just broke parts of it to get a simpler profile. Ignoring and initializing happen together as soon as I reorganize the code back into the form.

Is there any way to speed this up? I probably forget something really simple, or maybe I'm the backbone. I hope the first one.

 public static IList<T> ToList<T>(this DataTable table) where T : Model, new() { T[] entities = new T[table.Rows.Count]; // THIS LOOP IS VERY VERY SLOW for (int i = 0; i < table.Rows.Count; i++) entities[i] = new T(); // THIS LOOP IS FAST for (int i = 0; i < table.Rows.Count; i++) entities[i].Init(table, table.Rows[i]); return new List<T>(entities); } 

for more information:

The constructor of any given ModelType will look like this:

 public ModelType() { _modelInfo = new ModelTypeInfo(); } 

The constructor of any given ModelTypeInfo will simply set some string and string [] values, and this is the only class task to provide the set values.

edit for more information:

Since this seems to be a hot topic, here is what my method looks like for realities, before deducing the object construction and initialization:

 public static IList<T> ToList<T>(this DataTable table, ModelInfo modelInfo) where T : Model, new() { var tempRepository = new Repository<T>(modelInfo); var list = new List<T>(); foreach (DataRow row in table.Rows) list.Add(tempRepository.FromData(table, row)); return list; } 

Answer 1

Under the covers, new T() generates a call to System.Activator.CreateInstance<T>() , which is (reflexively) slow:

 L_0012: ldc.i4.0 L_0013: stloc.1 L_0014: br.s L_0026 L_0016: ldloc.0 L_0017: ldloc.1 L_0018: call !!0 [mscorlib]System.Activator::CreateInstance<!!T>() L_001d: stelem.any !!T L_0022: ldloc.1 L_0023: ldc.i4.1 L_0024: add L_0025: stloc.1 

Instead, you might consider handing over a construction delegate.

Answer 2

The name of your question suggests that this is because the method is generic. Does the same number of objects without generics emit? If not, this should be related to any work done in your constructor. Can you post the constructor code?

EDITED Here is what I wrote some time ago to create a cache in DynamicMethod, which is very fast:

In your class:

 delegate T ConstructorDelegate(); 

Method body:

 DynamicMethod method = new DynamicMethod(string.Empty, typeof(T), null, MethodBase.GetCurrentMethod().DeclaringType.Module); ILGenerator il = method.GetILGenerator(); il.Emit(OpCodes.Newobj, type.GetConstructor(Type.EmptyTypes)); il.Emit(OpCodes.Ret); var constructor = (ConstructorDelegate)method.CreateDelegate(typeof(ConstructorDelegate)); 

Answer 3

The problem is that the expression new T() actually uses reflection behind the scenes. (It calls Activator.CreateInstance ) Therefore, every call will take some time.

---

One solution would be to limit T to implement ICloneable. Then you can write new T() once and clone it in a loop. Obviously, you can only do this if you have full control over the model.

---

Another option would be to force the method to be accepted by the creator delegate, for example:

 public static IList<T> ToList<T>(this DataTable table, Func<T> creator) where T : Model { T[] entities = new T[table.Rows.Count]; for (int i = 0; i < table.Rows.Count; i++) entities[i] = creator(); //... } 

Then you call it like this:

 table.ToList(() => new MyModelType()); 

Since it is used in a parameter, you will not need to specify a generic type explicitly when calling a method.

---

The least intrusive method would be to use LINQ expressions to create your own creator methods.

EDIT : like this:

 static class CreatorFactory<T> where T : new() { public static readonly Func<T> Method = Expression.Lambda<Func<T>>(Expression.New(typeof(T)).Compile(); } public static IList<T> ToList<T>(this DataTable table) where T : Model { var entities = table.Rows.Select(r => CreatorFactory<T>.Method()).ToList(); for (int i = 0; i < table.Rows.Count; i++) entities[i].Init(table, table.Rows[i]); return entities; } 

Answer 4

Do you really need a list, or is IEnumerable good enough? If so, you can do lazy / delayed creation of your objects:

 public static IEnumerable<T> ToEnumerable<T>(this DataTable table) where T : Model, new() { foreach (DataRow row in table.Rows) { T entity = new T(); entity.Init(table, row); yield return entity; } } 

Unfortunately, this is likely to be slow, because most of the time, most likely, was spent on the construction of the object, but it can allow you to postpone this load long enough so that the application looks faster or until you can filter some from objects completely.

Alternatively, you can think about this using the Factory pattern:

 public static IEnumerable<T> ToEnumerable<T>(this DataTable table, Func<DataRow, T> TFactory) { foreach (DataRow row in table.Rows) { yield return TFactory(row); } } 

Answer 5

Are you testing release builds?
Is tables.loop.count a simple property, and can you compare it to the output from the loop?
What is the cost of creating an instance of T?
Does T create a selection of many small objects so that you run into multiple garbage collections?

Answer 6

To show an example, this method in C #:

 public T Method<T>() where T : new() { return new T(); } 

compiled into this MSIL code (from Reflector):

 .method public hidebysig instance !!T Method<.ctor T>() cil managed { .maxstack 2 .locals init ( [0] !!T CS$1$0000, [1] !!T CS$0$0001) L_0000: nop L_0001: ldloca.s CS$0$0001 L_0003: initobj !!T L_0009: ldloc.1 L_000a: box !!T L_000f: brfalse.s L_001c L_0011: ldloca.s CS$0$0001 L_0013: initobj !!T L_0019: ldloc.1 L_001a: br.s L_0021 L_001c: call !!0 [mscorlib]System.Activator::CreateInstance<!!T>() L_0021: stloc.0 L_0022: br.s L_0024 L_0024: ldloc.0 L_0025: ret } 

In order not to go into too many internal components, there are several steps, several conditions are checked, the need to initialize data fields, etc., and finally, an activator call may be required. All this to create an instance of an object of a general type. And yes, this is used instead of directly calling the type constructor.

Answer 7

Even if you need to use a list, why create an array first?

 public static IList<T> ToList<T>(this DataTable table) where T : Model, new() { var list = new List<T>(); foreach (DataRow dr in table.Rows) { T entity = new T(); entity.Init(table, dr); list.Add(entity); } return list; } 

Answer 8

For those who work on this issue later, this blog post was very helpful to me: http://blogs.msdn.com/haibo_luo/archive/2005/11/17/494009.aspx

Here are the changes I made to my factory method. (not quite the right factory, but serves the purpose)

 public class Repository<T> : IRepository<T> where T : Model, new() { // ... private delegate T CtorDelegate(); private CtorDelegate _constructor = null; private CtorDelegate Constructor { get { if (_constructor == null) { Type type = typeof(T); DynamicMethod dm = new DynamicMethod(type.Name + "Constructor", type, new Type[] { }, typeof(Repository<T>).Module); ILGenerator ilgen = dm.GetILGenerator(); ilgen.Emit(OpCodes.Nop); ilgen.Emit(OpCodes.Newobj, type.GetConstructor(Type.EmptyTypes)); ilgen.Emit(OpCodes.Ret); _constructor = (CtorDelegate)dm.CreateDelegate(typeof(CtorDelegate)); } return _constructor; } } public T FromData(DataTable table, DataRow row) { T model = Constructor(); // was previously = new T(); model.Init(table, row); return model; } }