I have a huge array of bytes that need to be processed. Theoretically, it would be possible to cut the work into even parts and assign them to different threads to increase productivity on a multi-core machine.
I allocated ByteBufferfor each stream and processed parts of the data each. Final performance is slower than with a single thread, although I have 8 logical processors. And this is very inconsistent. Sometimes the same input doubles as a slow process, or more. Why is this? First, the data is loaded into memory, so operations are not performed IO.
I allocate my byte buffers using MappedByteBuffer, because it is faster ByteBuffer.wrap():
public ByteBuffer getByteBuffer() throws IOException
{
File binaryFile = new File("...");
FileChannel binaryFileChannel = new RandomAccessFile(binaryFile, "r").getChannel();
return binaryFileChannel.map(FileChannel.MapMode.READ_ONLY, 0, binaryFileChannel.size());
}
I am doing parallel processing using Executors:
int threadsCount = Runtime.getRuntime().availableProcessors();
ExecutorService executorService = Executors.newFixedThreadPool(threadsCount);
ExecutorCompletionService<String> completionService = new ExecutorCompletionService<>(executorService);
for (ByteBufferRange byteBufferRange : byteBufferRanges)
{
Callable<String> task = () ->
{
performTask(byteBufferRange);
return null;
};
completionService.submit(task);
}
// Wait for all tasks to finish
for (ByteBufferRange ignored : byteBufferRanges)
{
completionService.take().get();
}
executorService.shutdown();
Parallel tasks performTask()use their own instances ByteBufferto read memory from a buffer, perform calculations, etc. They do not synchronize, do not write and do not affect each other. Any ideas on what is going wrong, or is this not a good case of parallelization?
The same problem exists with ByteBuffer.wrap()and MappedByteBuffer.
source
share