Is there a better way to use Lambda with N groups?

Question

I have a Process(IEnumerable<Record> records) method Process(IEnumerable<Record> records) that can use UP, but no more than 3 records at a time. I have hundreds of entries, so I need to go through groups. I'm doing it:

 var _Records = Enumerable.Range(1, 16).ToArray(); for (int i = 0; i < int.MaxValue; i += 3) { var _ShortList = _Records.Skip(i).Take(3); if (!_ShortList.Any()) break; Process(_ShortList); } // TODO: finish 

It works, but ... is there a better way?

Answer 1

You can use this extension method:

 public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> source, int chunkSize) { return source .Select((value, i) => new { Index = i, Value = value }) .GroupBy(item => item.Index % chunkSize) .Select(chunk => chunk.Select(item => item.Value)); } 

It splits the original set of elements into several pieces with a given size. So your code will look like this:

 foreach (var chunk in Enumerable.Range(1, 16).Split(3)) { Process(chunk); } 

Answer 2

you can use MoreLinq Batch

 var result=Enumerable.Range(1, 16).Batch(3); 

or

 var arrayOfArrays = Enumerable.Range(1, 16).Batch(3).Select(x => x.ToArray()).ToArray(); 

And here is the source if you want to take a look at it.

Answer 3

Here is another LINQ-y way:

 var batchSize = 3; Enumerable.Range(0, (_Records.Length - 1)/batchSize + 1) .ToList() .ForEach(i => Process(_Records.Skip(i * batchSize).Take(batchSize))); 

Answer 4

If you need to “pagination” several times in your solution, you can use the extension method.

Hacked in LINQPad, it will do the trick.

 public static class MyExtensions { public static IEnumerable<IEnumerable<T>> Paginate<T>(this IEnumerable<T> source, int pageSize) { T[] buffer = new T[pageSize]; int index = 0; foreach (var item in source) { buffer[index++] = item; if (index >= pageSize) { yield return buffer.Take(pageSize); index = 0; } } if (index > 0) { yield return buffer.Take(index); } } } 

Basically, it fills the buffer with pageSize size and gives it when it is pageSize . If there are < pageSize elements left, we also give them. Thus,

 Enumerable.Range(1, 10).Paginate(3).Dump(); // Dump is a LINQPad extension 

will give

 {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}, {10}} 

Answer 5

 var _Records = Enumerable.Range(1, 16).ToArray(); int index = 0; foreach (var group in _Records.GroupBy(element => index++ / 3)) Process(group); 

---

NOTE. The above code is short and relatively efficient, but still not as efficient as it can be (it will essentially build a hash table behind the scenes). A bit more cumbersome but faster way:

 var _Records = Enumerable.Range(1, 16).ToArray(); var buff = new int[3]; int index = 0; foreach (var element in _Records) { if (index == buff.Length) { Process(buff); index = 0; } buff[index++] = element; } if (index > 0) Process(buff.Take(index)); 

Or, pack it in a more reusable form:

 public static class EnumerableEx { public static void Paginate<T>(this IEnumerable<T> elements, int page_size, Action<IEnumerable<T>> process_page) { var buff = new T[3]; int index = 0; foreach (var element in elements) { if (index == buff.Length) { process_page(buff); index = 0; } buff[index++] = element; } if (index > 0) process_page(buff.Take(index)); } } // ... var _Records = Enumerable.Range(1, 16).ToArray(); _Records.Paginate(3, Process); 

Answer 6

You can create your own extension method:

 static class Extensions { public static IEnumerable<IEnumerable<T>> ToBlocks<T>(this IEnumerable<T> source, int blockSize) { var count = 0; T[] block = null; foreach (var item in source) { if (block == null) block = new T[blockSize]; block[count++] = item; if (count == blockSize) { yield return block; block = null; count = 0; } } if (count > 0) yield return block.Take(count); } } 

Answer 7

 public static void ChunkProcess<T>(IEnumerable<T> source, int size, Action<IEnumerable<T>> action) { var chunk = source.Take(size); while (chunk.Any()) { action(chunk); source = source.Skip(size); chunk = source.Take(size); } } 

and your code will be

 ChunkProcess(_Records, 3, Process); 

Answer 8

This extension method works correctly.

 public static class EnumerableExtentions { public static IEnumerable<IEnumerable<T>> Chunks<T>(this IEnumerable<T> items, int size) { return items.Select((member, index) => new { Index = index, Value = member }) .GroupBy(item => (int)item.Index / size) .Select(chunk => chunk.Select(item => item.Value)); } }