Geek Answers Handbook

The effect of cleavage and state on parallel flow processing

I work with parallel processing of Stream and find out that if I use a stream of flat arrays, it is processed very quickly. But if I use ArrayList, processing becomes a little slower. But if I use LinkedListeither some kind of binary tree, processing becomes even slower.

All that sounds like more is flow stall, the faster the processing will be. This means that the list of arrays and arrays is most effective in the case of parallel threads. It's true? If so, will we always use ArrayListeither Array if we want to process the stream in parallel? If yes, how to use LinkedListand BlockingQueuein the case of parallel flow?

Another thing is the completeness of the selected intermediate functions. If I perform the operation without conditions, such as filter(), map(), productivity is high, but if you do the full operation conditions, such as distinct(), sorted(), limit(), skip(), this takes time. Thus, parallel flow becomes slower. Does this mean that we should not go for full intermediate state functions in a parallel flow? If so, what does it work for?

+6
source share
2 answers

Good IMO questions.

, array ArrayList , LinkedList a Tree. , Spliterators . batch size (1024 ) . LinkedList Files.lines, . , ArrayList .

LinkedList, - , StreamEx Files.lines, ... .

, - , above, - state ... :

  IntStream.of(1, 3, 5, 2, 6)
            .filter(x -> {
                System.out.println("Filtering : " + x);
                return x > 2;
            })
            .sorted()
            .peek(x -> System.out.println("Peek : " + x))
            .boxed()
            .collect(Collectors.toList());

:

Filtering : 1
Filtering : 3
Filtering : 5
Filtering : 2
Filtering : 6
Peek : 3
Peek : 5
Peek : 6

sorted filter , , sorted .

, sorted:

  IntStream.of(1, 3, 5, 2, 6)
            .filter(x -> {
                System.out.println("Filtering : " + x);
                return x > 2;
            })
            // .sorted()
            .peek(x -> System.out.println("Peek : " + x))
            .boxed()
            .collect(Collectors.toList());

:

 Filtering : 1
 Filtering : 3
 Peek : 3
 Filtering : 5
 Peek : 5
 Filtering : 2
 Filtering : 6
 Peek : 6

, , ( ) - , , sorted - , TreeSet... .. - - , , .

, ; , , .

- , , , .

+4

, , - LinkedList . , .

, . -, , , , , AbstractSpliterator, -, .

, . , , . , , Collection spliterator(), Spliterator, default. . TreeSet . , , , , .

, LinkedList BlockingQueue , . , ( LinkedList ), , , ArrayList, , -, . . , .

, JRE , Java 9, , String.chars(), Files.lines() RandomAccess List s, LinkedList, BlockingQueue .

, , - , , .

, , distinct(), sorted(), limit(), skip(), , . , , , , , . , , , .

+5

More articles:


All Articles