Geek Answers Handbook

HashMap Parallel Iterator: How Safe Is It For Threading?

I used a parallel hashmap to create the matrix. Its indices range up to 100 thousand. I created 40 threads. Each of the flows refers to these elements of the matrices and modifies them and writes them back to the matrix as:

ConcurrentHashMap<Integer, ArrayList<Double>> matrix = new ConcurrentHashMap<Integer, ArrayList<Double>>(25); for (Entry(Integer,ArrayList<Double>)) entry: matrix.entrySet()) upDateEntriesOfValue(entry.getValue());

I did not find it thread safe. Values ​​are often returned as null, and my program crashes. Is there any other way to make it thread safe. Or is it thread safe and I have a bug in some other places. One thing - my program does not crash in single-threaded mode.

+4
source share
3 answers

iterator really is thread safe for ConcurrentHashMap .

But what is not thread safe in your code is an ArrayList<Double> , which you seem to be updating! Your problems may arise from this data structure.

You can use a parallel data structure tailored to your needs.

+11
source

Using a card for a matrix is ​​really inefficient, and the way you used it, it won’t even support rare arrays especially well.

I suggest you use double [] [], where you block every row (or column, if that's better). If the matrix is ​​small enough, you might be better off using only one processor, as this can save you a bit of overhead.

I would suggest you create more threads than you have kernels. For tasks with an intensive processor, using a larger thread may be slower rather than faster.

The matrix is ​​100k * 50 at maximum

EDIT: Depending on the operation being performed, I would try to make sure that you have a shorter size so that you can handle each long size in a different thread efficiently.

eg

 double[][] matrix = new double[50][100*1000]; for(int i=0;i<matrix.length;i++) { final double[] line = matrix[i]; executorService.submit(new Runnable() { public void run() { synchronized(line) { processOneLine(line); } } }); }

This allows everyone that you perform to execute at the same time, since they do not share any data structures. They can also effectively access each double effect because they are continuous in memory and stored as efficiently as possible. that is, 100K doubles uses about 800 KB, but List<Double> uses 2800 KB, and each value can be randomly ordered in memory, which means that your cache should work a lot harder.

thanks, but actually I have only 80 cores

To use 80 cores efficiently, you may need to break longer strings into two or four so that you can keep all cores busy or find a way to perform more than one operation at a time.

+2
source

ConcurrentHashMap will be thread safe for accessing the Map, but Lists should be thread-safe if multiple threads can work in the same list instance, so use thread-safe list when changing.

In your case, working on ConcurrentHashMap is protection against the tread, but when a stream goes to ArrayList , it is not synchronized , and therefore several threads can access it at the same time, which makes it unsafe. or you can use synchronized block , where you do the modification in the list

+1
source

More articles:


All Articles