Passing a list iterator to multiple threads in Java

Question

I have a list containing approximately 200K items.

Can I pass this list to the iterator for several threads and iterate over the entire batch without any of them, referring to the same elements?

This is what I am thinking about right now.

Main:

public static void main(String[] args) { // Imagine this list has the 200,000 elements. ArrayList<Integer> list = new ArrayList<Integer>(); // Get the iterator for the list. Iterator<Integer> i = list.iterator(); // Create MyThread, passing in the iterator for the list. MyThread threadOne = new MyThread(i); MyThread threadTwo = new MyThread(i); MyThread threadThree = new MyThread(i); // Start the threads. threadOne.start(); threadTwo.start(); threadThree.start(); } 

MyThread:

 public class MyThread extends Thread { Iterator<Integer> i; public MyThread(Iterator<Integer> i) { this.i = i; } public void run() { while (this.i.hasNext()) { Integer num = this.i.next(); // Do something with num here. } } } 

My desired result is that each thread processes approximately 66,000 elements each, without picking up an iterator too much, and also without any threads accessing the same element.

Does this sound sound?

Answer 1

Do you really need to manually manipulate threads and iterators? You can use Java 8 Stream and let parallel() do the job.

By default, it will use one less thread since you have processors.

Example:

 list.stream() .parallel() .forEach(this::doSomething) ; //For example, display the current integer and the current thread number. public void doSomething(Integer i) { System.out.println(String.format("%d, %d", i, Thread.currentThread().getId())); } 

Result:

 49748, 13 49749, 13 49750, 13 192710, 14 105734, 17 105735, 17 105736, 17 [...] 

Edit: if you are using maven, you will need to add this part of the configuration to pom.xml in order to use Java 8:

 <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.3</version> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> </plugins> </build> 

Answer 2

You cannot make this a safe thread with a single iterator. I suggest using a sublist:

 List sub1 = list.subList(0, 100); List sub2 = list.subList(100, 200); 

ArrayList#subList() method simply transfers the given list without copying the elements. You can then iterate through each sublist in a different thread.

Answer 3

Since the next() method of the class that implements the Iterator interface manipulates data, using the next() method simultaneously requires synchronization. Synchronization can be performed using a synchronized block on an iterator object as follows:

 synchronized(i) { i.next(); } 

Although I recommend using the Stream API, as in the answer above, if you need only parallel list processing.

Answer 4

Hi, so that your threads from dreadlocks or hunger you can use the ExecutorService from the thread pool class. These words are better for me than using synchronized, locks or re-locks. You can also try using Fork / join, but I haven't used it before. This is sample code, but I hope you get the idea

 public static void main(String[] args){ ExecutorService executor = Executors.newFixedThreadPool(200000); List<Future<Integer>> futureList = new ArrayList<>(); //iteration code goes here executor.shutdown(); } Public class MyThread implements Callable<ArrayList<Integer>>{ @Override public Iterator<Integer> call() throws Exception { //code goes here! } } 

Answer 5

If you use a parallel thread, you will execute your code in many threads, with the elements distributed evenly between the threads:

 list.parallelStream().forEach(this::processInteger); 

This approach makes it very easy to encode; all heavy lifting is done by jre.

Also, as far as your code is concerned, this is a bad style for Thread extension. Instead, run Runnable and pass the instance to the Thread constructor - see Live