High performance buffering for rands stream

Question

I have code that consumes a large number (millions at the present time, ultimately billions) of relatively short (5-100 elements) arrays of random numbers and does not very stressful mathematics with them. The random numbers were, well, random, ideally I would like to generate them on several cores, since the generation of random numbers is> 50% of my runtime in profiling. However, it is difficult for me to distribute a large number of small tasks in such a way that it is no slower than a single-threaded approach.

My code currently looks something like this:

for(int i=0;i<1000000;i++){ for(RealVector d:data){ while(!converged){ double[] shortVec = new double[5]; for(int i=0;i<5;i++) shortVec[i]=rng.nextGaussian(); double[] longerVec = new double[50]; for(int i=0;i<50;i++) longerVec[i]=rng.nextGaussian(); /*Do some relatively fast math*/ } } } 

The approaches I used that did not :

• 1+ topics populating ArrayBlockingQueue, and my main loop consuming and populating an array (boxing / unpacking was a killer here)
• Generating vectors with Callable (giving the future) when doing independent parts of mathematics (it seems that the overhead of indirect preponderance outweighs all the parallelism that I got)
• Uses 2 ArrayBlockingQueue, each of which is filled with a stream, one for a short one and one for a long array (still about two times slower than a direct single-threaded case).

I am not looking for a “solution” for my specific problem, just as I can handle the general case of generating large flows of small independent independent elements in parallel and consuming them from a single stream.

Answer 1

This is more efficient than using a queue because:

• the payload is an array of double[] , meaning that the background thread can generate more data before transferring it.
• all objects are recycled.

.

 public class RandomGenerator { private final ExecutorService generator = Executors.newSingleThreadExecutor(new ThreadFactory() { @Override public Thread newThread(Runnable r) { Thread t = new Thread(r, "generator"); t.setDaemon(true); return t; } }); private final Exchanger<double[][]> exchanger = new Exchanger<>(); private double[][] buffer; private int nextRow = Integer.MAX_VALUE; public RandomGenerator(final int rows, final int columns) { buffer = new double[rows][columns]; generator.submit(new Callable<Void>() { @Override public Void call() throws Exception { Random random = new Random(); double[][] buffer2 = new double[rows][columns]; while (!Thread.interrupted()) { for (int r = 0; r < rows; r++) for (int c = 0; c < columns; c++) buffer2[r][c] = random.nextGaussian(); buffer2 = exchanger.exchange(buffer2); } return null; } }); } public double[] nextArray() throws InterruptedException { if (nextRow >= buffer.length) { buffer = exchanger.exchange(buffer); nextRow = 0; } return buffer[nextRow++]; } } 

---

Random is thread safe and synchronized. This means that each thread needs it to execute the Random function to execute at the same time.

how to handle the general case of generating large flows of small independent independent elements and use them from a single stream.

I would use Exchanger<double[][]> to fill in the values ​​in the background, how to transfer them efficiently (without much overhead on the GC)

Answer 2

The problem with your performance, apparently, is that the individual tasks are too small, so most of the time is spent on synchronizing and queuing the tasks themselves. One thing to consider is not to generate a large stream of small tasks, but to deliver to each work stream an average set of tasks that it will comment on with the answer.

For example, instead of repeating in your loop with the first thread iterating # 0, the next thread iterating # 1, ... I will have the first thread iterating from # 0 to # 999 or some. They should work independently and annotate the Job class with an answer to their calculations. Then at the end they can return the entire collection of assignments that were completed as Future .

Your Job class might look something like this:

 public class Job { Collection<RealVector> dataCollection; Collection<SomeAnswer> answerCollection = new ArrayList<SomeAnswer>(); public void run() { for (RealVector d : dataCollection) { // do the magic work on the vector while(!converged){ ... } // put the associated "answer" in another collection answerCollection.add(someAnswer); } } }