Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #32735 > unrolled thread

parallel stream with return

Back to article view | Back to comp.lang.java.programmer

Contents

  parallel stream with return Philipp Kraus <philipp.kraus.flashpixx@gmail.com> - 2015-09-29 23:35 -0700
    Re: parallel stream with return Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-09-30 08:01 -0400
      Re: parallel stream with return Philipp Kraus <philipp.kraus.flashpixx@gmail.com> - 2015-09-30 08:02 -0700
        Re: parallel stream with return Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-09-30 13:22 -0400
          Re: parallel stream with return Steven Simpson <ss@my.domain.invalid> - 2015-09-30 22:25 +0100
    Re: parallel stream with return Jeff Higgins <jeff@invalid.invalid> - 2015-09-30 16:41 -0400
      Re: parallel stream with return Jeff Higgins <jeff@invalid.invalid> - 2015-09-30 16:53 -0400
        Re: parallel stream with return Philipp Kraus <philipp.kraus.flashpixx@gmail.com> - 2015-09-30 23:36 -0700
          Re: parallel stream with return Eric Sosman <esosman@comcast-dot-net.invalid> - 2015-10-01 08:22 -0400
          Re: parallel stream with return Jeff Higgins <jeff@invalid.invalid> - 2015-10-01 08:54 -0400

#32735 — parallel stream with return

Hello,

I'm new with the Java 1.8 parallel stream, so I have got a problem:

I have the sequential for loop:

final Set<myClass> x = new HashSet<>();
for(int i=0; i < value; ++i)
    x.add( this.generate() );

And I would like to do this with a parallel stream which collects each generate return value into the set.
My first try was to use a synchronized HashSet and

IntStream.range(0, value).parallel().forEach( i-> { x.add(this.generate()) } );

After that I try to swap forEach to map, but at this I must return an int value and not a myClass value, so how can I do n generate calls in parallel and collect the results into a hashset?

Thanks

Phil

[toc] | [next] | [standalone]

#32736

On 9/30/2015 2:35 AM, Philipp Kraus wrote:
> Hello,
>
> I'm new with the Java 1.8 parallel stream, so I have got a problem:
>
> I have the sequential for loop:
>
> final Set<myClass> x = new HashSet<>();
> for(int i=0; i < value; ++i)
>      x.add( this.generate() );
>
> And I would like to do this with a parallel stream which collects each generate return value into the set.
> My first try was to use a synchronized HashSet and
>
> IntStream.range(0, value).parallel().forEach( i-> { x.add(this.generate()) } );
>
> After that I try to swap forEach to map, but at this I must return an int value and not a myClass value, so how can I do n generate calls in parallel and collect the results into a hashset?

     I think you want java.util.stream.Collectors, either toSet()
or toCollection().

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32738

Am Mittwoch, 30. September 2015 14:01:37 UTC+2 schrieb Eric Sosman:
> On 9/30/2015 2:35 AM, Philipp Kraus wrote:
> > Hello,
> >
> > I'm new with the Java 1.8 parallel stream, so I have got a problem:
> >
> > I have the sequential for loop:
> >
> > final Set<myClass> x = new HashSet<>();
> > for(int i=0; i < value; ++i)
> >      x.add( this.generate() );
> >
> > And I would like to do this with a parallel stream which collects each generate return value into the set.
> > My first try was to use a synchronized HashSet and
> >
> > IntStream.range(0, value).parallel().forEach( i-> { x.add(this.generate()) } );
> >
> > After that I try to swap forEach to map, but at this I must return an int value and not a myClass value, so how can I do n generate calls in parallel and collect the results into a hashset?
> 
>      I think you want java.util.stream.Collectors, either toSet()
> or toCollection().

but how can I iterate within the range [0,n) in parallel? Can you show me the code of the sequential for loop?

[toc] | [prev] | [next] | [standalone]

#32739

On 9/30/2015 11:02 AM, Philipp Kraus wrote:
> Am Mittwoch, 30. September 2015 14:01:37 UTC+2 schrieb Eric Sosman:
>> On 9/30/2015 2:35 AM, Philipp Kraus wrote:
>>> Hello,
>>>
>>> I'm new with the Java 1.8 parallel stream, so I have got a problem:
>>>
>>> I have the sequential for loop:
>>>
>>> final Set<myClass> x = new HashSet<>();
>>> for(int i=0; i < value; ++i)
>>>       x.add( this.generate() );
>>>
>>> And I would like to do this with a parallel stream which collects each generate return value into the set.
>>> My first try was to use a synchronized HashSet and
>>>
>>> IntStream.range(0, value).parallel().forEach( i-> { x.add(this.generate()) } );
>>>
>>> After that I try to swap forEach to map, but at this I must return an int value and not a myClass value, so how can I do n generate calls in parallel and collect the results into a hashset?
>>
>>       I think you want java.util.stream.Collectors, either toSet()
>> or toCollection().
>
> but how can I iterate within the range [0,n) in parallel? Can you show me the code of the sequential for loop?

     Untested:

	Set<myClass> x =
	    IntStream.range(0, value)
	    .parallel()
	    .mapToObj(i -> this.generate())
	    .collect(Collectors.toSet());

Note that this won't necessarily produce a HashSet, but is more
likely to produce some other kind of Set.  If you really need an
actual HashSet, you could

     - Use the toCollection() method of Collectors (but I'm not
       100% sure about what kind of synchronization you'll need), or

     - Copy the unknown Set's content into a HashSet after the
       pipeline finishes:

	Set<myClass> x = new HashSet<>(
	    IntStream.range ...

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32742

On 30/09/15 18:22, Eric Sosman wrote:
> Untested:
>
>     Set<myClass> x =
>         IntStream.range(0, value)
>         .parallel()
>         .mapToObj(i -> this.generate())
>         .collect(Collectors.toSet());
>
> Note that this won't necessarily produce a HashSet, but is more
> likely to produce some other kind of Set.  If you really need an
> actual HashSet, you could
>
>     - Use the toCollection() method of Collectors

import java.util.*;
import java.util.function.*;
import java.util.stream.*;

public class Slop {
     private static Object generate() {
         return new Object();
     }

     public static void main(String[] args) throws Exception {
         Collection<Object> foo =
             IntStream.range(0, 4096)
             .parallel()
             .mapToObj(i -> generate())
             .collect(Collectors.toCollection(() -> new HashSet<Object>()));
         System.out.println(foo);
     }
}



> (but I'm not
>       100% sure about what kind of synchronization you'll need), or

<http://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.stream.Collector->
> When executed in parallel, multiple intermediate results may be 
> instantiated, populated, and merged so as to maintain isolation of 
> mutable data structures. Therefore, even when executed in parallel 
> with non-thread-safe data structures (such as |ArrayList|), no 
> additional synchronization is needed for a parallel reduction.

Perhaps it would be interesting to see whether the lambda provided to 
toCollection is invoked multiple times...

import java.util.*;
import java.util.concurrent.atomic.*;
import java.util.function.*;
import java.util.stream.*;

public class Slop {
     private static Object generate() {
         return new Object();
     }

     private static AtomicInteger counter = new AtomicInteger();

     private static class MySet extends HashSet<Object> {
         public MySet() {
             counter.getAndIncrement();
         }
     }

     public static void main(String[] args) throws Exception {
         Collection<Object> foo =
             IntStream.range(0, 4096)
             .parallel()
             .mapToObj(i -> generate())
             .collect(Collectors.toCollection(() -> new MySet()));
         System.out.println(foo);
         System.out.println(counter);
     }
}

I got '16'.


-- 
ss at comp dot lancs dot ac dot uk

[toc] | [prev] | [next] | [standalone]

#32740

On 09/30/2015 02:35 AM, Philipp Kraus wrote:
> Hello,
>
> I'm new with the Java 1.8 parallel stream, so I have got a problem:
>
> I have the sequential for loop:
>
> final Set<myClass> x = new HashSet<>();
> for(int i=0; i < value; ++i)
>      x.add( this.generate() );
>
> And I would like to do this with a parallel stream which collects each generate return value into the set.
> My first try was to use a synchronized HashSet and
>
> IntStream.range(0, value).parallel().forEach( i-> { x.add(this.generate()) } );
>
> After that I try to swap forEach to map, but at this I must return an int value and not a myClass value, so how can I do n generate calls in parallel and collect the results into a hashset?
>
What's wrong with what you had to start?
Have I misunderstood?

import java.util.HashSet;
import java.util.stream.IntStream;

public class Scratch {

   private final int count;
   private final String name;

   private Scratch(int i, String t) { count = i; name = t;}

   public static void main(String[] args) {
     HashSet<Scratch> x = new HashSet<>();
     IntStream.range(0, 4096).parallel()
       .forEach(i -> { x.add(generate(i)); });
     for (Scratch s : x) {
       System.out.println(s.count + " : " + s.name);
     }
   }

   private static Scratch generate(int i) {
     return new Scratch(i, Thread.currentThread().getName());
   }

}

[toc] | [prev] | [next] | [standalone]

#32741

On 09/30/2015 04:41 PM, Jeff Higgins wrote:
> On 09/30/2015 02:35 AM, Philipp Kraus wrote:
>> Hello,
>>
>> I'm new with the Java 1.8 parallel stream, so I have got a problem:
>>
>> I have the sequential for loop:
>>
>> final Set<myClass> x = new HashSet<>();
>> for(int i=0; i < value; ++i)
>>      x.add( this.generate() );
>>
>> And I would like to do this with a parallel stream which collects each
>> generate return value into the set.
>> My first try was to use a synchronized HashSet and
>>
>> IntStream.range(0, value).parallel().forEach( i-> {
>> x.add(this.generate()) } );
>>
>> After that I try to swap forEach to map, but at this I must return an
>> int value and not a myClass value, so how can I do n generate calls in
>> parallel and collect the results into a hashset?
>>
> What's wrong with what you had to start?
> Have I misunderstood?
>
> import java.util.HashSet;
> import java.util.stream.IntStream;
>
> public class Scratch {
>
>    private final int count;
>    private final String name;
>
>    private Scratch(int i, String t) { count = i; name = t;}
>
>    public static void main(String[] args) {
>      HashSet<Scratch> x = new HashSet<>();
>      IntStream.range(0, 4096).parallel()
>        .forEach(i -> { x.add(generate(i)); });
>      for (Scratch s : x) {
>        System.out.println(s.count + " : " + s.name);
>      }
>    }
>
>    private static Scratch generate(int i) {
>      return new Scratch(i, Thread.currentThread().getName());
>    }
>
> }
>
>
>

Or:

import java.util.ArrayList;
import java.util.List;
import java.util.stream.IntStream;

public class Scratch {

   private final int count;
   private final String name;

   private Scratch(int i, String t) { count = i; name = t;}

   public static void main(String[] args) {
     List<Scratch> x = new ArrayList<>();
     IntStream.range(0, 4096).parallel()
       .forEachOrdered(i -> { x.add(generate(i)); });
     for (Scratch s : x) {
       System.out.println(s.count + " : " + s.name);
     }
   }

   private static Scratch generate(int i) {
     return new Scratch(i, Thread.currentThread().getName());
   }

}

[toc] | [prev] | [next] | [standalone]

#32743

Am Mittwoch, 30. September 2015 22:54:04 UTC+2 schrieb Jeff Higgins:

>      IntStream.range(0, 4096).parallel()
>        .forEachOrdered(i -> { x.add(generate(i)); });

Did I understand It correct, that I need the forEachOrdered to create the equal order like in the intstream range? So if I need only 4k objects and the order is irrelevant I can do this with forEach.

Phil

[toc] | [prev] | [next] | [standalone]

#32745

On 10/1/2015 2:36 AM, Philipp Kraus wrote:
> Am Mittwoch, 30. September 2015 22:54:04 UTC+2 schrieb Jeff Higgins:
>
>>       IntStream.range(0, 4096).parallel()
>>         .forEachOrdered(i -> { x.add(generate(i)); });
>
> Did I understand It correct, that I need the forEachOrdered to create the equal order like in the intstream range? So if I need only 4k objects and the order is irrelevant I can do this with forEach.

     If the calls to generate() must occur in a particular order,
you'd better not use parallel() ...

-- 
esosman@comcast-dot-net.invalid
"Don't be afraid of work. Make work afraid of you." -- TLM

[toc] | [prev] | [next] | [standalone]

#32747

On 10/01/2015 02:36 AM, Philipp Kraus wrote:
> Am Mittwoch, 30. September 2015 22:54:04 UTC+2 schrieb Jeff Higgins:
>
>>       IntStream.range(0, 4096).parallel()
>>         .forEachOrdered(i -> { x.add(generate(i)); });
>
> Did I understand It correct, that I need the forEachOrdered to create the equal order like in the intstream range? So if I need only 4k objects and the order is irrelevant I can do this with forEach.
>
> Phil
>
You can look at the documentation for Stream.forEach and
Stream.ForEachOrdered and package java.util.stream.

And you can try these experiments with the class below.

Run it as is.
Run with change : List<Scratch> list = new ArrayList<>(4096);
Run with change : private static Scratch generate(int i) {


import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.stream.IntStream;

public class Scratch {

   private final int count;
   private final String name;

   private Scratch(int i, String t) { count = i; name = t;}

   public static void main(String[] args) {
     List<Scratch> list = new ArrayList<>();
     List<Scratch> x = Collections.synchronizedList(list);
     IntStream.range(0, 4096).parallel()
       .forEach(i -> { x.add(generate(i)); });
     for (Scratch s : x) {
       System.out.println(s.count + " : " + s.name);
     }
   }

   private static synchronized Scratch generate(int i) {
     return new Scratch(i, Thread.currentThread().getName());
   }

}

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web