Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #33360 > unrolled thread

parallel stream ordering

Back to article view | Back to comp.lang.java.programmer

Contents

  parallel stream ordering Philipp Kraus <philipp.kraus.flashpixx@gmail.com> - 2015-12-16 23:47 -0800
    Re: parallel stream ordering Lew <lewbloch@gmail.com> - 2015-12-17 13:38 -0800
      Re: parallel stream ordering Sebastian <news@seyweiler.dyndns.org> - 2015-12-18 23:11 +0100

#33360 — parallel stream ordering

Hello,

I'm using calls like

m_annotationexecution.parallelStream().flatMap( i -> i.execute( p_context, null ).stream() ).collect( Collectors.toList() )

execute returns a List, so the flatMap enrolls the list elements to collect them into a new list, but I would like to fix the order. For example m_annotation is a list like

[ "a", "b", "c" ]

execute transform it to

a -> ["a", "1", ...]
b -> ["b", "2",  ...]
c -> ["c", "3", ....]

the flatMap enrolls it to

["a", "1", ..., "b", "2", ..., "c", "3", ....]

but on a parallel stream the order is not fixed, so I can get also

["b", "2", ..., "a", "1", ..., "c", "3", ....] or ["c", "3", ..., "a", "1", ..., "b", "2", ....]

So how can I fix the order, so that the first element in m_annotation is also the first element within the result list? execute and flatMap can be run in parallel, but only the collect must create the correct ordering. 
If Iuse this input [ "a", "b", "c" ] the output should be ["a", "1", ..., "b", "2", ..., "c", "3", ....] during execution the temporal order can be changed

Thanks

Phil

[toc] | [next] | [standalone]

#33365

On Wednesday, December 16, 2015 at 11:47:43 PM UTC-8, Philipp Kraus wrote:
> Hello,
> 
> I'm using calls like
> 
> m_annotationexecution.parallelStream().flatMap( i -> i.execute( p_context, null ).stream() ).collect( Collectors.toList() )
> 
> execute returns a List, so the flatMap enrolls the list elements to collect them into a new list, but I would like to fix the order. For example m_annotation is a list like
> 
> [ "a", "b", "c" ]
> 
> execute transform it to
> 
> a -> ["a", "1", ...]
> b -> ["b", "2",  ...]
> c -> ["c", "3", ....]
> 
> the flatMap enrolls it to
> 
> ["a", "1", ..., "b", "2", ..., "c", "3", ....]
> 
> but on a parallel stream the order is not fixed, so I can get also
> 
> ["b", "2", ..., "a", "1", ..., "c", "3", ....] or ["c", "3", ..., "a", "1", ..., "b", "2", ....]
> 
> So how can I fix the order, so that the first element in m_annotation is also the first element within the result list? execute and flatMap can be run in parallel, but only the collect must create the correct ordering. 
> If Iuse this input [ "a", "b", "c" ] the output should be ["a", "1", ..., "b", "2", ..., "c", "3", ....] during execution the temporal order can be changed
> 
> Thanks
> 
> Phil

Quite aside from your question: 
* 'm_annotationexecution' isn't consistent with the Java naming conventions. It should be something like 'annotationExecution'. 
* s/enrolls/unrolls/ - different words with different meanings

You can fix the order by using a 'for' loop instead of streams. Parallelization by its nature wants to disregard order.

-- 
Lew

[toc] | [prev] | [next] | [standalone]

#33367

Am 17.12.2015 22:38, schrieb Lew:
> On Wednesday, December 16, 2015 at 11:47:43 PM UTC-8, Philipp Kraus wrote:
>> Hello,
>>
>> I'm using calls like
>>
>> m_annotationexecution.parallelStream().flatMap( i -> i.execute( p_context, null ).stream() ).collect( Collectors.toList() )
>>
>> execute returns a List, so the flatMap enrolls the list elements to collect them into a new list, but I would like to fix the order. For example m_annotation is a list like
>>
>> [ "a", "b", "c" ]
>>
>> execute transform it to
>>
>> a -> ["a", "1", ...]
>> b -> ["b", "2",  ...]
>> c -> ["c", "3", ....]
>>
>> the flatMap enrolls it to
>>
>> ["a", "1", ..., "b", "2", ..., "c", "3", ....]
>>
>> but on a parallel stream the order is not fixed, so I can get also
>>
>> ["b", "2", ..., "a", "1", ..., "c", "3", ....] or ["c", "3", ..., "a", "1", ..., "b", "2", ....]
>>
>> So how can I fix the order, so that the first element in m_annotation is also the first element within the result list? execute and flatMap can be run in parallel, but only the collect must create the correct ordering.
>> If Iuse this input [ "a", "b", "c" ] the output should be ["a", "1", ..., "b", "2", ..., "c", "3", ....] during execution the temporal order can be changed
>>
>> Thanks
>>
>> Phil
>
> Quite aside from your question:
> * 'm_annotationexecution' isn't consistent with the Java naming conventions. It should be something like 'annotationExecution'.
> * s/enrolls/unrolls/ - different words with different meanings
>
> You can fix the order by using a 'for' loop instead of streams. Parallelization by its nature wants to disregard order.
>
You have at least two more options here. For one thing, you could use a 
library like RxJava, which has a concatMap method on Observables which 
as the name suggests concats the lists, while flatMap does a merge.

If you can provide a Comparator on the result elements to define the 
desired order, you can simply collect to a sorted list, like
...collect(toSortedList()), where toSortedList may be defined as

private static <T> Collector<T, ?, List<T>> toSortedList(Comparator<T> 
order) {
	return collectingAndThen(toCollection(ArrayList::new),
           (List<T> l) -> {
		l.sort(order);
		return l;
	});
}

The trick is to be sure to collect to a list that can be sorted in situ 
(ArrayList in this case). This involves no additional copying, in 
contrast to ...sorted().collect(toList()) which would copy the list once 
more.

collectingAndThen() is in java.util.stream.Collectors. Know thine API!

-- Sebastian

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web