Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #88380 > unrolled thread

Re: generator/coroutine terminology

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-03-31 13:18 +0000
    Re: generator/coroutine terminology Dave Angel <davea@davea.name> - 2015-03-31 09:38 -0400
      Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-03-31 15:03 +0000
        Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-04-01 02:36 +1100
    Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-04-03 17:02 +1100
      Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-04-18 17:52 +0000
    Re: generator/coroutine terminology Paul Rubin <no.email@nospam.invalid> - 2015-04-02 23:46 -0700

#88380 — Re: generator/coroutine terminology

In article <55062bda$0$12998$c3e8da3$5496439d@news.astraweb.com>,
Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:
>Marko Rauhamaa wrote:
>
>> Chris Angelico <rosuav@gmail.com>:
>>
>>> On Sun, Mar 15, 2015 at 9:15 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>>> Is it necessary/useful for a Python application programmer to be
>>>> conscious of the different types of iterator? What mistaken usage
>>>> could arise if the application just treated all iterators as, well,
>>>> iterators?
>>>
>>> If you treat them all as iterators, then you're safe, because that's
>>> the lowest common denominator. But there are a number of other
>>> iterators that have more features, including files, generators, etc.
>>
>> What features do generator iterators provide on top of generic
>> iterators?
>
>The biggest difference is syntactic. Here's an iterator which returns a
>never-ending sequence of squared numbers 1, 4, 9, 16, ...
>
>class Squares:
>    def __init__(self):
>        self.i = 0
>    def __next__(self):
>        self.i += 1
>        return self.i**2
>    def __iter__(self):
>        return self

You should give an example of usage. As a newby I'm not up to
figuring out the specification from source for
something built of the mysterious __ internal
thingies.
(I did experiment with Squares interactively. But I didn't get
further than creating a Squares object.)


>
>
>Here's the same thing written as a generator:
>
>def squares():
>    i = 1
>    while True:
>        yield i**2
>        i += 1
>
>
>Four lines, versus eight. The iterator version has a lot of boilerplate
>(although some of it, the two-line __iter__ method, could be eliminated if
>there was a standard Iterator builtin to inherit from).
>
>Here's a more complicated example:
>
>class Randoms:
>    def __init__(self):
>        self.finished = False
>    def __next__(self):
>        x = random.random()
>        if x > 0.5:
>            self.finished = True
>        if self.finished:
>            raise StopIteration
>        else:
>            return x
>    def __iter__(self):
>        return self
>
>
>def randoms():
>    x = random.random()
>    while x < 0.5:
>        yield x
>        x = random.random()
>
>
>Generators, as a rule, are significantly easier to write, understand, and
>debug. There's nothing they can do that can't be done with an iterator
>class, but the fiddly, unexciting bits related to halting and resuming and
>saving state are all handled for you, allowing you to concentrate on the
>behaviour you want, not the boilerplate.

This is illuminating. Thanks.

>
>--
>Steven

Groetjes Albert
>
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [next] | [standalone]

#88383

On 03/31/2015 09:18 AM, Albert van der Horst wrote:
> In article <55062bda$0$12998$c3e8da3$5496439d@news.astraweb.com>,
> Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:

>>
>> The biggest difference is syntactic. Here's an iterator which returns a
>> never-ending sequence of squared numbers 1, 4, 9, 16, ...
>>
>> class Squares:
>>     def __init__(self):
>>         self.i = 0
>>     def __next__(self):
>>         self.i += 1
>>         return self.i**2
>>     def __iter__(self):
>>         return self
>
> You should give an example of usage. As a newby I'm not up to
> figuring out the specification from source for
> something built of the mysterious __ internal
> thingies.
> (I did experiment with Squares interactively. But I didn't get
> further than creating a Squares object.)
>

He did say it was an iterator.  So for a first try, write a for loop:

class Squares:
    def __init__(self):
        self.i = 0
    def __next__(self):
        self.i += 1
        return self.i**2
    def __iter__(self):
        return self

for i in Squares():
     print(i)
     if i > 50:
         break

print("done")


>
>>
>>
>> Here's the same thing written as a generator:
>>
>> def squares():
>>     i = 1
>>     while True:
>>         yield i**2
>>         i += 1
>>
>>
>> Four lines, versus eight. The iterator version has a lot of boilerplate
>> (although some of it, the two-line __iter__ method, could be eliminated if
>> there was a standard Iterator builtin to inherit from).
>>
-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#88387

In article <mailman.372.1427809109.10327.python-list@python.org>,
Dave Angel  <davea@davea.name> wrote:
>On 03/31/2015 09:18 AM, Albert van der Horst wrote:
>> In article <55062bda$0$12998$c3e8da3$5496439d@news.astraweb.com>,
>> Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:
>
>>>
>>> The biggest difference is syntactic. Here's an iterator which returns a
>>> never-ending sequence of squared numbers 1, 4, 9, 16, ...
>>>
>>> class Squares:
>>>     def __init__(self):
>>>         self.i = 0
>>>     def __next__(self):
>>>         self.i += 1
>>>         return self.i**2
>>>     def __iter__(self):
>>>         return self
>>
>> You should give an example of usage. As a newby I'm not up to
>> figuring out the specification from source for
>> something built of the mysterious __ internal
>> thingies.
>> (I did experiment with Squares interactively. But I didn't get
>> further than creating a Squares object.)
>>
>
>He did say it was an iterator.  So for a first try, write a for loop:
>
>class Squares:
>    def __init__(self):
>        self.i = 0
>    def __next__(self):
>        self.i += 1
>        return self.i**2
>    def __iter__(self):
>        return self
>
>for i in Squares():
>     print(i)
>     if i > 50:
>         break
>

This is what I get:
/ --------------------------
albert@cherry:/tmp$ more aap.py
class Squares:
    def __init__(self):
        self.i = 0
    def __next__(self):
        self.i += 1
        return self.i**2
    def __iter__(self):
        return self

albert@cherry:/tmp$ python
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from aap import *
>>> for i in Squares():
...     print i
...     if i>50: break
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: instance has no next() method
>>>

/ --------------------------

Probably not what is intended.

Last minute note:
   renaming __next__() into next() did the job.

>--
>DaveA

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#88391

On Wed, Apr 1, 2015 at 2:03 AM, Albert van der Horst
<albert@spenarnc.xs4all.nl> wrote:
> class Squares:
>     def __init__(self):
>         self.i = 0
>     def __next__(self):
>         self.i += 1
>         return self.i**2
>     def __iter__(self):
>         return self
>
> albert@cherry:/tmp$ python
> Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> [GCC 4.4.5] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from aap import *
>>>> for i in Squares():
> ...     print i
> ...     if i>50: break
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: instance has no next() method
>>>>
>
> / --------------------------
>
> Probably not what is intended.
>
> Last minute note:
>    renaming __next__() into next() did the job.

That class was written for Python 3, not Python 2. In Py2, you need to
rename __next__ to next, as you noted, and you probably also want to
explicitly subclass object. Or just run it under Python 3. :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#88472

On Wednesday 01 April 2015 00:18, Albert van der Horst wrote:

> In article <55062bda$0$12998$c3e8da3$5496439d@news.astraweb.com>,
> Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:

>>The biggest difference is syntactic. Here's an iterator which returns a
>>never-ending sequence of squared numbers 1, 4, 9, 16, ...
>>
>>class Squares:
>>    def __init__(self):
>>        self.i = 0
>>    def __next__(self):
>>        self.i += 1
>>        return self.i**2
>>    def __iter__(self):
>>        return self
> 
> You should give an example of usage. As a newby I'm not up to
> figuring out the specification from source for
> something built of the mysterious __ internal
> thingies.
> (I did experiment with Squares interactively. But I didn't get
> further than creating a Squares object.)


Ah, sorry about that!

Usage is:

it = Squares()  # create an iterator
print(next(it))  # print the first value
x = next(it)  # extract the second
while x < 100:
    print(x)
    x = next(it)


Beware of doing this:

for x in Squares():
    print(x)

since Squares is an *infinite* generator, it will continue for ever if you 
let it. Fortunately you can hit Ctrl-C to interrupt the for loop at any 
point.

In Python 2, you will need to rename __next__ to just next without the 
double-leading-and-trailing underscores.


>>Here's the same thing written as a generator:
>>
>>def squares():
>>    i = 1
>>    while True:
>>        yield i**2
>>        i += 1

And for this one:

it = squares()  # create the iterator
print(next(it))  # print the first value
x = next(it)  # extract the second
while x < 100:
    print(x)
    x = next(it)


Usage is pretty much exactly the same.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#89129

In article <551e2cfd$0$11123$c3e8da3@news.astraweb.com>,
Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:
>On Wednesday 01 April 2015 00:18, Albert van der Horst wrote:
>
>> In article <55062bda$0$12998$c3e8da3$5496439d@news.astraweb.com>,
>> Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:
>
>>>The biggest difference is syntactic. Here's an iterator which returns a
>>>never-ending sequence of squared numbers 1, 4, 9, 16, ...
>>>
>>>class Squares:
>>>    def __init__(self):
>>>        self.i = 0
>>>    def __next__(self):
>>>        self.i += 1
>>>        return self.i**2
>>>    def __iter__(self):
>>>        return self
>>
>> You should give an example of usage. As a newby I'm not up to
>> figuring out the specification from source for
>> something built of the mysterious __ internal
>> thingies.
>> (I did experiment with Squares interactively. But I didn't get
>> further than creating a Squares object.)
>
>
>Ah, sorry about that!
>
>Usage is:
>
>it = Squares()  # create an iterator
>print(next(it))  # print the first value
>x = next(it)  # extract the second
>while x < 100:
>    print(x)
>    x = next(it)
>
>
>Beware of doing this:
>
>for x in Squares():
>    print(x)
>
>since Squares is an *infinite* generator, it will continue for ever if you
>let it. Fortunately you can hit Ctrl-C to interrupt the for loop at any
>point.
>
>In Python 2, you will need to rename __next__ to just next without the
>double-leading-and-trailing underscores.
>
>
>>>Here's the same thing written as a generator:
>>>
>>>def squares():
>>>    i = 1
>>>    while True:
>>>        yield i**2
>>>        i += 1
>
>And for this one:
>
>it = squares()  # create the iterator
>print(next(it))  # print the first value
>x = next(it)  # extract the second
>while x < 100:
>    print(x)
>    x = next(it)
>
>
>Usage is pretty much exactly the same.

Thanks, I get it now. next and yield are more or less
switching between coroutines.

>
>
>--
>Steve
>
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#88474

albert@spenarnc.xs4all.nl (Albert van der Horst) writes:
> You should give an example of usage. As a newby I'm not up to
> figuring out the specification from source for
> something built of the mysterious __ internal
> thingies.

In reality because of generator expressions, the yield statement, and
some useful built-in generators in the itertools module, you rarely
have to use those special methods.  

I'd advise reading through the itertools module documentation from
beginning to end, trying to understand what everything does.  Even if
some are not that useful, it will help convey the mode of thinking that
went into these features.

At a somewhat deeper level you might like the SICP book: 

  http://mitpress.mit.edu/sicp/

It's somewhat old now and it's about Scheme rather than Python, but it
builds up the relevant concepts quite nicely.

Regarding the squares example, consider this even simpler generator:

  def count(n):
   while True:
     yield n
     n += 1

so count(1) yields 1, 2, 3, 4 ...

This is a very useful generator but you don't need to write it since
it's included in the itertools module as itertools.count.  Its initial
value defaults to 0.  So the  squares generator can be written:

   def squares():
      return (i*i for i in itertools.count(0))

Now if you want all the squares less than 100 (i.e. 0, 1, 4, 9, ..., 81):

  wanted = itertools.takewhile(lambda x: x<100, squares())

You can print that out as a list:

   print(list(wanted))

The built-in sum function consumes an iterator, so you can add up the
squares less than 100:

  print(sum(itertools.takewhile(lambda x: x<100, squares())))

this prints 205 which is 1+4+9+16+25+36+49+64+81.

These features fit together quite elegantly and code like this flows off
the fingertips naturally once you've used to it.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web