Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #33367

Re: parallel stream ordering

Show all headers | View raw

Am 17.12.2015 22:38, schrieb Lew:
> On Wednesday, December 16, 2015 at 11:47:43 PM UTC-8, Philipp Kraus wrote:
>> Hello,
>>
>> I'm using calls like
>>
>> m_annotationexecution.parallelStream().flatMap( i -> i.execute( p_context, null ).stream() ).collect( Collectors.toList() )
>>
>> execute returns a List, so the flatMap enrolls the list elements to collect them into a new list, but I would like to fix the order. For example m_annotation is a list like
>>
>> [ "a", "b", "c" ]
>>
>> execute transform it to
>>
>> a -> ["a", "1", ...]
>> b -> ["b", "2",  ...]
>> c -> ["c", "3", ....]
>>
>> the flatMap enrolls it to
>>
>> ["a", "1", ..., "b", "2", ..., "c", "3", ....]
>>
>> but on a parallel stream the order is not fixed, so I can get also
>>
>> ["b", "2", ..., "a", "1", ..., "c", "3", ....] or ["c", "3", ..., "a", "1", ..., "b", "2", ....]
>>
>> So how can I fix the order, so that the first element in m_annotation is also the first element within the result list? execute and flatMap can be run in parallel, but only the collect must create the correct ordering.
>> If Iuse this input [ "a", "b", "c" ] the output should be ["a", "1", ..., "b", "2", ..., "c", "3", ....] during execution the temporal order can be changed
>>
>> Thanks
>>
>> Phil
>
> Quite aside from your question:
> * 'm_annotationexecution' isn't consistent with the Java naming conventions. It should be something like 'annotationExecution'.
> * s/enrolls/unrolls/ - different words with different meanings
>
> You can fix the order by using a 'for' loop instead of streams. Parallelization by its nature wants to disregard order.
>
You have at least two more options here. For one thing, you could use a 
library like RxJava, which has a concatMap method on Observables which 
as the name suggests concats the lists, while flatMap does a merge.

If you can provide a Comparator on the result elements to define the 
desired order, you can simply collect to a sorted list, like
...collect(toSortedList()), where toSortedList may be defined as

private static <T> Collector<T, ?, List<T>> toSortedList(Comparator<T> 
order) {
	return collectingAndThen(toCollection(ArrayList::new),
           (List<T> l) -> {
		l.sort(order);
		return l;
	});
}

The trick is to be sure to collect to a list that can be sorted in situ 
(ArrayList in this case). This involves no additional copying, in 
contrast to ...sorted().collect(toList()) which would copy the list once 
more.

collectingAndThen() is in java.util.stream.Collectors. Know thine API!

-- Sebastian

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

parallel stream ordering Philipp Kraus <philipp.kraus.flashpixx@gmail.com> - 2015-12-16 23:47 -0800
  Re: parallel stream ordering Lew <lewbloch@gmail.com> - 2015-12-17 13:38 -0800
    Re: parallel stream ordering Sebastian <news@seyweiler.dyndns.org> - 2015-12-18 23:11 +0100

csiph-web