Geek Answers Handbook

The best way to summarize at the same time

I am trying to calculate some big numbers. To speed up the calculation, I would like to use multithreading. Each thread must calculate a number, and at the end the sum is calculated.

I once saw something that worked with SumThread and Collector , which looked like this:

 public BigInteger compute(int p) { Collector c = new Collector(p); for(T element : Collection<T> bigCollection) { new SumThread(c) { @Override protected void doTheJob() { long big = someVeryComplexCalculation(element, ...); //n! receive(BigInteger.valueOf(big)); } } } if(collector.isReady()) return collector.getResult(); return null; } public class Collector { private int numberOfProcesses; private int numberOfAllowedProcesses; private BigInteger result; public Collector(int n) { numberOfAllowedProcesses = n; numberOfProcesses = 0; result = BigInteger.ZERO; } synchronized public void enter() throws InterruptedException { if (numberOfProcesses == numberOfAllowedProcesses) wait(); numberOfProcesses++; } synchronized public void leave() { numberOfProcesses--; notify(); } synchronized public void register(BigInteger v) { result = result.add(v); } synchronized public boolean isReady() throws InterruptedException { while (numberOfProcesses > 0) wait(); return true; } ... } public abstract class SumThread extends Thread { private Collector collector; public SumThread(Collector c) throws InterruptedException { collector = c; collector.enter(); } abstract protected void doTheJob(); //complex calculations can be done in here public void receive(BigInteger t) { collector.register(t); } public void run() { doTheJob(); collector.leave(); } }

I thought I could easily surpass this by using ExecutorService instead of constantly adding a new Thread :

 public BigInteger compute(int p) { ExecutorService pool = Executors.newFixedThreadPool(p); Future<BigInteger>[] futures = new Future<BigInteger>[bigCollection.size()]; int i = 0; for(T element : Collection<T> bigCollection) { futures[i++] = p.submit(new Callable<BigInteger>() { @Override public BigInteger call() { long big = someVeryComplexCalculation(element, ...); //n! return BigInteger.valueOf(big); } } } // or with ExecutorCompletionService, but the loop remains I guess BigInteger res = BigInteger.ZERO for(Future<BigInteger> f : futures) res = res.add(f.get()); return res; }

However, this code could not surpass the SumThread - Collector solution. I also saw things about LongAdder , for example, but I need some kind of adder for BigInteger s ...

So my question is: how can I calculate the amount at the same time? Is this one of the above or is there a completely different (but better) way?

+6
source share
2 answers

As already mentioned, LongAdder , which has been added to Java-8 and uses efficient final variables, I assume you are using Java-8. In this version, the best way to solve your problem is to use the Stream API :

 BigInteger result = bigCollection.parallelStream() .map(e -> BigInteger.valueOf(someVeryComplexCalculation(e, ...))) .reduce(BigInteger.ZERO, BigInteger::add);

Your task is a classic task with a reduction in size, in which you must transform each element of a collection, and then combine the results of individual transformations into the final result. Stream API is able to efficiently parallelize such tasks without any manual work. In Oracle JDK, tasks are performed in the ForkJoinPool shared pool , which by default creates as many threads as you have processor cores.

+7
source

You have two solutions:

First, I would suggest using the Fork-Join Framework from JDK7 for this task:

You will need to implement a recursive task

A second solution (JDK8) would be to use a pararell stream, just as @ tagir-valeev suggested.

In both cases, it depends on what you need and what version of Java you are using.

+2
source

More articles:


All Articles