Is there a general Java utility for breaking down a list into packages?

Question

I wrote myself a utility to break the list into batches of a given size. I just wanted to know if there are already any Apache Commons utilities for this.

public static <T> List<List<T>> getBatches(List<T> collection,int batchSize){ int i = 0; List<List<T>> batches = new ArrayList<List<T>>(); while(i<collection.size()){ int nextInc = Math.min(collection.size()-i,batchSize); List<T> batch = collection.subList(i,i+nextInc); batches.add(batch); i = i + nextInc; } return batches; } 

Please let me know if there is already an existing utility for the same.

Answer 1

Check out the Lists.partition(java.util.List, int) from Google Guava :

Returns consecutive list sublists, each of which has the same size (the last list may be smaller). For example, splitting a list containing [a, b, c, d, e] with the size of section 3 gives [[a, b, c] , [d, e]] - an external list containing two internal lists of three and two elements, all in the original order.

Answer 2

If you want to create a Java-8 batch package, you can try the following code:

 public static <T> Stream<List<T>> batches(List<T> source, int length) { if (length <= 0) throw new IllegalArgumentException("length = " + length); int size = source.size(); if (size <= 0) return Stream.empty(); int fullChunks = (size - 1) / length; return IntStream.range(0, fullChunks + 1).mapToObj( n -> source.subList(n * length, n == fullChunks ? size : (n + 1) * length)); } public static void main(String[] args) { List<Integer> list = Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14); System.out.println("By 3:"); batches(list, 3).forEach(System.out::println); System.out.println("By 4:"); batches(list, 4).forEach(System.out::println); } 

Output:

 By 3: [1, 2, 3] [4, 5, 6] [7, 8, 9] [10, 11, 12] [13, 14] By 4: [1, 2, 3, 4] [5, 6, 7, 8] [9, 10, 11, 12] [13, 14] 

Answer 3

Another approach is to use Collectors.groupingBy indexes, and then map the grouped indexes to the actual elements:

  final List<Integer> numbers = range(1, 12) .boxed() .collect(toList()); System.out.println(numbers); final List<List<Integer>> groups = range(0, numbers.size()) .boxed() .collect(groupingBy(index -> index / 4)) .values() .stream() .map(indices -> indices .stream() .map(numbers::get) .collect(toList())) .collect(toList()); System.out.println(groups); 

Output:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11]]

Answer 4

I came up with this:

 private static <T> List<List<T>> partition(Collection<T> members, int maxSize) { List<List<T>> res = new ArrayList<>(); List<T> internal = new ArrayList<>(); for (T member : members) { internal.add(member); if (internal.size() == maxSize) { res.add(internal); internal = new ArrayList<>(); } } if (internal.isEmpty() == false) { res.add(internal); } return res; } 

Answer 5

The following example shows a split list:

 package de.thomasdarimont.labs; import java.util.ArrayList; import java.util.Arrays; import java.util.HashMap; import java.util.List; import java.util.Map; public class SplitIntoChunks { public static void main(String[] args) { List<Integer> ints = Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11); List<List<Integer>> chunks = chunk(ints, 4); System.out.printf("Ints: %s%n", ints); System.out.printf("Chunks: %s%n", chunks); } public static <T> List<List<T>> chunk(List<T> input, int chunkSize) { int inputSize = input.size(); int chunkCount = (int) Math.ceil(inputSize / (double) chunkSize); Map<Integer, List<T>> map = new HashMap<>(chunkCount); List<List<T>> chunks = new ArrayList<>(chunkCount); for (int i = 0; i < inputSize; i++) { map.computeIfAbsent(i / chunkSize, (ignore) -> { List<T> chunk = new ArrayList<>(); chunks.add(chunk); return chunk; }).add(input.get(i)); } return chunks; } } 

Output:

 Ints: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] Chunks: [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11]] 

Answer 6

In Java 9, you can use IntStream.iterate() with a hasNext . This way you can simplify your method code to this:

 public static <T> List<List<T>> getBatches(List<T> collection, int batchSize) { return IntStream.iterate(0, i -> i < collection.size(), i -> i + batchSize) .mapToObj(i -> collection.subList(i, Math.min(i + batchSize, collection.size()))) .collect(Collectors.toList()); } 

Using {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} , the result of getBatches(numbers, 4) will be:

 [[0, 1, 2, 3], [4, 5, 6, 7], [8, 9]] 

Answer 7

Using various cheats from the Internet, I came to this solution:

 int[] count = new int[1]; final int CHUNK_SIZE = 500; Map<Integer, List<Long>> chunkedUsers = users.stream().collect( Collectors.groupingBy( user -> { count[0]++; return Math.floorDiv( count[0], CHUNK_SIZE ); } ) ); 

We use count to simulate a normal collection index.
Then we group the elements of the collection in buckets, using the algebraic relation as the number of buckets. The final card contains the bucket number key , and the bucket is the value .

Then you can easily perform an operation on each of the buckets with:

 chunkedUsers.values().forEach( ... ); 

Answer 8

There was another question that was closed as a duplicate of this, but if you read it carefully, it is a little different. Therefore, if someone (for example, me) really wants to divide the list into a given number of almost identical sub-lists in size, then read on.

I just ported the algorithm described here to Java.

 @Test public void shouldPartitionListIntoAlmostEquallySizedSublists() { List<String> list = Arrays.asList("a", "b", "c", "d", "e", "f", "g"); int numberOfPartitions = 3; List<List<String>> split = IntStream.range(0, numberOfPartitions).boxed() .map(i -> list.subList( partitionOffset(list.size(), numberOfPartitions, i), partitionOffset(list.size(), numberOfPartitions, i + 1))) .collect(toList()); assertThat(split, hasSize(numberOfPartitions)); assertEquals(list.size(), split.stream().flatMap(Collection::stream).count()); assertThat(split, hasItems(Arrays.asList("a", "b", "c"), Arrays.asList("d", "e"), Arrays.asList("f", "g"))); } private static int partitionOffset(int length, int numberOfPartitions, int partitionIndex) { return partitionIndex * (length / numberOfPartitions) + Math.min(partitionIndex, length % numberOfPartitions); } 

Answer 9

 List<T> batch = collection.subList(i,i+nextInc); -> List<T> batch = collection.subList(i, i = i + nextInc); 

Answer 10

Use Apache Commons ListUtils.partition .

Answer 11

Another approach to solving this issue:

 public class CollectionUtils { /** * Splits the collection into lists with given batch size * @param collection to split in to batches * @param batchsize size of the batch * @param <T> it maintains the input type to output type * @return nested list */ public static <T> List<List<T>> makeBatch(Collection<T> collection, int batchsize) { List<List<T>> totalArrayList = new ArrayList<>(); List<T> tempItems = new ArrayList<>(); Iterator<T> iterator = collection.iterator(); for (int i = 0; i < collection.size(); i++) { tempItems.add(iterator.next()); if ((i+1) % batchsize == 0) { totalArrayList.add(tempItems); tempItems = new ArrayList<>(); } } if (tempItems.size() > 0) { totalArrayList.add(tempItems); } return totalArrayList; } } 

Answer 12

import com.google.common.collect.Lists;

List<List<T>> batches = Lists.partition(List<T>,batchSize)

Use Lists.partition (List, batchSize). You need to import Lists from the general Google package ( com.google.common.collect.Lists )

It will return a List of List<T> with the size of each element equal to your batchSize .

Answer 13

Single line in Java 8 will be:

 import static java.util.function.Function.identity; import static java.util.stream.Collectors.*; private static <T> Collection<List<T>> partition(List<T> xs, int size) { return IntStream.range(0, xs.size()) .boxed() .collect(collectingAndThen(toMap(identity(), xs::get), Map::entrySet)) .stream() .collect(groupingBy(x -> x.getKey() / size, mapping(Map.Entry::getValue, toList()))) .values(); }