Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #87319 > unrolled thread

generator/coroutine terminology

Back to article view | Back to comp.lang.python

Contents

  generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-12 06:35 -0700
    Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-13 00:55 +1100
    Re: generator/coroutine terminology breamoreboy@gmail.com - 2015-03-12 06:57 -0700
    Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-13 03:27 +1100
      Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-12 09:52 -0700
        Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-12 19:55 +0200
          Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-12 19:23 -0700
            Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-13 14:30 +1100
              Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-12 22:28 -0700
                Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-13 19:23 +1100
                  Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-13 02:12 -0700
                    Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-13 11:36 +0200
                      Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-14 17:04 +1100
                        Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-14 09:54 +0200
                          Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-14 08:04 +0000
                            Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-14 10:30 +0200
                          Re: generator/coroutine terminology Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-14 14:14 -0600
                            Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-14 21:15 -0700
                          Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-14 20:31 +0000
                        Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-14 08:29 -0700
                          Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-15 02:56 +1100
                          Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-14 08:59 -0700
                            Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-15 03:14 +1100
                              Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-14 09:33 -0700
                                Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-15 03:51 +1100
                                  Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-14 10:17 -0700
                                Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-14 16:56 +0000
                                Re: generator/coroutine terminology Dave Angel <davea@davea.name> - 2015-03-14 13:07 -0400
                            Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-03-31 12:57 +0000
                          Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-15 19:37 +1100
                      Re: generator/coroutine terminology CHIN Dihedral <dihedral88888@gmail.com> - 2015-04-18 11:07 -0700
                    Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-13 22:32 +1100
        Re: generator/coroutine terminology Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-03-14 22:02 +0000
          Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-15 00:15 +0200
            Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-15 09:24 +1100
              Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-15 02:15 +0200
                Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-15 11:22 +1100
                  Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-15 02:48 +0200
                    Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-15 13:02 +1100
                Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 12:03 +1100
                  Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 09:12 +0200
                    Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-16 18:21 +1100
                      Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 09:40 +0200
                        Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 22:59 +1100
                    Re: generator/coroutine terminology Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-16 01:37 -0600
                      Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 09:52 +0200
                        Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 23:02 +1100
                          Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 14:42 +0200
                          Re: generator/coroutine terminology Jonas Wielicki <jonas@wielicki.name> - 2015-03-16 13:39 +0100
                    Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 19:36 +1100
                      Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-16 19:58 +1100
                        Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-16 22:51 +1100
                          Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-03-17 00:16 +1100
                      Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 14:32 +0200
                        Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 05:51 -0700
                          Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 15:13 +0200
                            Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 01:32 +1100
                              Re: generator/coroutine terminology Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-16 08:45 -0600
                          Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 00:39 +1100
                            Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 07:19 -0700
                              Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-16 14:26 +0000
                                Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 07:37 -0700
                                  Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 07:55 -0700
                                  Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-16 18:19 +0000
                                    Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 19:52 -0700
                                      Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 03:07 +0000
                                        Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 20:18 -0700
                                          Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 03:25 +0000
                                            Re: generator/coroutine terminology Rustom Mody <rustompmody@gmail.com> - 2015-03-16 20:33 -0700
                                              Re: generator/coroutine terminology Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-03-17 03:55 +0000
                                      Re: generator/coroutine terminology Mario Figueiredo <marfig@gmail.com> - 2015-03-17 04:22 +0100
                              Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 01:35 +1100
                                Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-03-17 01:36 +1100
                            Re: generator/coroutine terminology Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-16 08:52 -0600
                              Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 17:09 +0200
                                Re: generator/coroutine terminology Ian Kelly <ian.g.kelly@gmail.com> - 2015-03-16 09:26 -0600
                                  Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-16 18:05 +0200
                  Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-03-31 13:18 +0000
                    Re: generator/coroutine terminology Dave Angel <davea@davea.name> - 2015-03-31 09:38 -0400
                      Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-03-31 15:03 +0000
                        Re: generator/coroutine terminology Chris Angelico <rosuav@gmail.com> - 2015-04-01 02:36 +1100
                    Re: generator/coroutine terminology Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-04-03 17:02 +1100
                      Re: generator/coroutine terminology albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-04-18 17:52 +0000
                    Re: generator/coroutine terminology Paul Rubin <no.email@nospam.invalid> - 2015-04-02 23:46 -0700
    Re: generator/coroutine terminology Terry Reedy <tjreedy@udel.edu> - 2015-03-12 16:11 -0400
      Re: generator/coroutine terminology Marko Rauhamaa <marko@pacujo.net> - 2015-03-12 22:22 +0200

Page 1 of 5  [1] 2 3 4 5  Next page →

#87319 — generator/coroutine terminology

This is more a question about standard terminology/conventions than about semantics - of course assuming I understand :-)

Say I have a simple yielding function:

def foo(x):
   yield x+1
   yield x+2

And I have

g = foo(2)

If I look at type, g's type is 'generator' whereas foo is just plain-ol 'function.'

Whereas in informal usage we say foo is a generator.

So the question:
What should we call foo and what should we call g?

Same applies when foo is a 'coroutine' ie
something having yield used in an rhs and used with '.send' from outside:
What to call foo and what to call foo(x)?

[toc] | [next] | [standalone]

#87320

On Fri, Mar 13, 2015 at 12:35 AM, Rustom Mody <rustompmody@gmail.com> wrote:
> If I look at type, g's type is 'generator' whereas foo is just plain-ol 'function.'
>
> Whereas in informal usage we say foo is a generator.
>
> So the question:
> What should we call foo and what should we call g?

g is a generator object; foo is a generator function - a function
which returns generator objects. Usually, calling both of them
"generators" isn't confusing, as there's not often any context in
which they're ambiguous.

ChrisA

[toc] | [prev] | [next] | [standalone]

#87321

On Thursday, March 12, 2015 at 1:35:48 PM UTC, Rustom Mody wrote:
> This is more a question about standard terminology/conventions than about semantics - of course assuming I understand :-)
> 
> Say I have a simple yielding function:
> 
> def foo(x):
>    yield x+1
>    yield x+2
> 
> And I have
> 
> g = foo(2)
> 
> If I look at type, g's type is 'generator' whereas foo is just plain-ol 'function.'
> 
> Whereas in informal usage we say foo is a generator.
> 
> So the question:
> What should we call foo and what should we call g?
> 
> Same applies when foo is a 'coroutine' ie
> something having yield used in an rhs and used with '.send' from outside:
> What to call foo and what to call foo(x)?

Try the glossary https://docs.python.org/3/glossary.html

If this comes out badly please free to shout as I'm on gg :)

[toc] | [prev] | [next] | [standalone]

#87322

Rustom Mody wrote:

> This is more a question about standard terminology/conventions than about
> semantics - of course assuming I understand :-)
> 
> Say I have a simple yielding function:
> 
> def foo(x):
>    yield x+1
>    yield x+2
> 
> And I have
> 
> g = foo(2)
> 
> If I look at type, g's type is 'generator' whereas foo is just plain-ol
> 'function.'
> 
> Whereas in informal usage we say foo is a generator.

Hopefully it is clear from context what we actually mean. When in doubt, we
should be explicit.


> So the question:
> What should we call foo and what should we call g?

foo is a generator function, i.e. a function which returns a generator. I'd
also allow generator factory, naming it by intent rather than type.

The result of calling foo, namely foo(x), which you have named g, is a
generator object, which is a kind of iterator.


> Same applies when foo is a 'coroutine' ie
> something having yield used in an rhs and used with '.send' from outside:
> What to call foo and what to call foo(x)?

foo is a generator function, and foo(x) is a generator object. In this case
we can call foo(x) by usage rather than type, and call it a coroutine.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87323

Guess I should be pleased that I am doing as good as you (and Chris) describe.
For some reason or not I am not...

On Thursday, March 12, 2015 at 9:58:07 PM UTC+5:30, Steven D'Aprano wrote:
> Rustom Mody wrote:
> 
> > This is more a question about standard terminology/conventions than about
> > semantics - of course assuming I understand :-)
> > 
> > Say I have a simple yielding function:
> > 
> > def foo(x):
> >    yield x+1
> >    yield x+2
> > 
> > And I have
> > 
> > g = foo(2)
> > 
> > If I look at type, g's type is 'generator' whereas foo is just plain-ol
> > 'function.'
> > 
> > Whereas in informal usage we say foo is a generator.
> 
> Hopefully it is clear from context what we actually mean. When in doubt, we
> should be explicit.

There is a very important 'context' where both have to exist together -- teaching beginners.

foo's are written to produce g's. g's come from foo-like.

Better naming would help clarify -- your 'factory' is the best Ive seen so far.
But the docs?!?! Hoo Boy My head spins trying to grok this
https://docs.python.org/3/reference/expressions.html#generator-expressions
And thats after being familiar with the origins of the idea in
scheme/simula/CLU etc.

Noobs could be forgiven for doing worse dont you think??

I guess we need
1. A clear ontology of the base concepts (which is a buzzword for nailed-down terminology)
2. Some neat pictures would sure help (me!)

[toc] | [prev] | [next] | [standalone]

#87325

Rustom Mody <rustompmody@gmail.com>:

> I guess we need
> 1. A clear ontology of the base concepts (which is a buzzword for
> nailed-down terminology)

According to the documentation, a function whose definition contains a
yield statement is a generator:

   Using a yield expression in a function’s body causes that function to
   be a generator.

   <URL: https://docs.python.org/3/reference/expressions.html#yield-exp
   ressions>

   generator
      A function which returns an iterator.

   <URL: https://docs.python.org/3/glossary.html#term-generator>

Apparently, what we have here is a discrepancy between the documentation
and the CPython implementation.

In a word, it's a bug (somewhere).


Marko

[toc] | [prev] | [next] | [standalone]

#87345

On Thursday, March 12, 2015 at 11:25:32 PM UTC+5:30, Marko Rauhamaa wrote:
> Rustom Mody :
> 
> > I guess we need
> > 1. A clear ontology of the base concepts (which is a buzzword for
> > nailed-down terminology)
> 
> According to the documentation, a function whose definition contains a
> yield statement is a generator:
> 
>    Using a yield expression in a function's body causes that function to
>    be a generator.
> 
>    <URL: https://docs.python.org/3/reference/expressions.html#yield-exp
>    ressions>
> 
>    generator
>       A function which returns an iterator.
> 
>    <URL: https://docs.python.org/3/glossary.html#term-generator>
> 
> Apparently, what we have here is a discrepancy between the documentation
> and the CPython implementation.
> 
> In a word, it's a bug (somewhere).

Throwing out some thought for better terminology.
[Yeah needs to be chewed on a bit...]

With respect to my example above:

def foo(x):
   yield x+1
   yield x+2


g = foo(2) 

Generator-instance: g
Generator-factory: foo
Generator: Concept in python (ontology) elaborated in terms of the above

Situation with coroutines is worse
[Or I cant think of good terms...]

[toc] | [prev] | [next] | [standalone]

#87349

Rustom Mody wrote:

> On Thursday, March 12, 2015 at 11:25:32 PM UTC+5:30, Marko Rauhamaa wrote:
>> Rustom Mody :
>> 
>> > I guess we need
>> > 1. A clear ontology of the base concepts (which is a buzzword for
>> > nailed-down terminology)
>> 
>> According to the documentation, a function whose definition contains a
>> yield statement is a generator:
>> 
>>    Using a yield expression in a function's body causes that function to
>>    be a generator.
>> 
>>    <URL: https://docs.python.org/3/reference/expressions.html#yield-exp
>>    ressions>
>> 
>>    generator
>>       A function which returns an iterator.
>> 
>>    <URL: https://docs.python.org/3/glossary.html#term-generator>
>> 
>> Apparently, what we have here is a discrepancy between the documentation
>> and the CPython implementation.
>> 
>> In a word, it's a bug (somewhere).
> 
> Throwing out some thought for better terminology.
> [Yeah needs to be chewed on a bit...]
> 
> With respect to my example above:
> 
> def foo(x):
>    yield x+1
>    yield x+2
> 
> 
> g = foo(2)
> 
> Generator-instance: g
> Generator-factory: foo
> Generator: Concept in python (ontology) elaborated in terms of the above

I think that is is reasonable. I would include "generator function" as a
synonym for generator factory. (Factories are more inclusive than
functions -- a factory could include a class with a "make_generator"
method, for example.)

Where the distinction is clear, or not important, than calling both g and
foo "generators" is acceptable shorthand. I realise that can be confusing
to beginners, but jargon often is confusing to beginners until they have
learned enough to be able to correctly interpret things in context.


> Situation with coroutines is worse
> [Or I cant think of good terms...]

Not really. The difference between a coroutine and a generator is mostly in
usage, that is, intent. They both use the same type, they both have send
methods, they both are generated the same way. If you are foolish, you can
even mix generator-behaviour and coroutine-behaviour in the same function:


py> def g():
...     yield 1
...     yield 2
...     x = yield 3
...     yield x + 1000
...
py> a = g()
py> next(a)
1
py> next(a)
2
py> next(a)
3
py> a.send(99)
1099

But don't do that.

So as far as Python is concerned, a coroutine is just a generator instance
which you use in a particular way, not a different kind of object. So I
would use similar terminology:

A generator function/factory returns a generator instance, which we use as a
coroutine, so we can call it a coroutine.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87355

On Friday, March 13, 2015 at 9:00:17 AM UTC+5:30, Steven D'Aprano wrote:
> Rustom Mody wrote:
> 
> > On Thursday, March 12, 2015 at 11:25:32 PM UTC+5:30, Marko Rauhamaa wrote:
> >> Rustom Mody :
> >> 
> >> > I guess we need
> >> > 1. A clear ontology of the base concepts (which is a buzzword for
> >> > nailed-down terminology)
> >> 
> >> According to the documentation, a function whose definition contains a
> >> yield statement is a generator:
> >> 
> >>    Using a yield expression in a function's body causes that function to
> >>    be a generator.
> >> 
> >>    <URL: https://docs.python.org/3/reference/expressions.html#yield-exp
> >>    ressions>
> >> 
> >>    generator
> >>       A function which returns an iterator.
> >> 
> >>    <URL: https://docs.python.org/3/glossary.html#term-generator>
> >> 
> >> Apparently, what we have here is a discrepancy between the documentation
> >> and the CPython implementation.
> >> 
> >> In a word, it's a bug (somewhere).
> > 
> > Throwing out some thought for better terminology.
> > [Yeah needs to be chewed on a bit...]
> > 
> > With respect to my example above:
> > 
> > def foo(x):
> >    yield x+1
> >    yield x+2
> > 
> > 
> > g = foo(2)
> > 
> > Generator-instance: g
> > Generator-factory: foo
> > Generator: Concept in python (ontology) elaborated in terms of the above
> 
> I think that is is reasonable.

Good that we agree!

> Where the distinction is clear, or not important, than calling both g and
> foo "generators" is acceptable shorthand. I realise that can be confusing
> to beginners, but jargon often is confusing to beginners until they have
> learned enough to be able to correctly interpret things in context.

I would prefer -- when shortforming is required:
generator-instance -> instance
generator-factory -> factory

rather than

generator-instance -> generator
generator-factory -> generator

Yeah that can quarrel with the more usual OO notion of instance, but  I find that ambiguity more remote
> 
> 
> > Situation with coroutines is worse
> > [Or I cant think of good terms...]
> 
> Not really. The difference between a coroutine and a generator is mostly in
> usage, that is, intent. They both use the same type, they both have send
> methods, they both are generated the same way. If you are foolish, you can
> even mix generator-behaviour and coroutine-behaviour in the same function:
> 
> 
> py> def g():
> ...     yield 1
> ...     yield 2
> ...     x = yield 3
> ...     yield x + 1000
> ...
> py> a = g()
> py> next(a)
> 1
> py> next(a)
> 2
> py> next(a)
> 3
> py> a.send(99)
> 1099
> 
> But don't do that.
> 
> So as far as Python is concerned, a coroutine is just a generator instance
> which you use in a particular way, not a different kind of object. So I
> would use similar terminology:
> 
> A generator function/factory returns a generator instance, which we use as a
> coroutine, so we can call it a coroutine.

I think coroutines are intended to be symmetric in a way that generators are not.

The nut-n-bolts of both may involve using yield but conceptually I find them different.

And even there I would expect generators to close with StopIteration
Whereas I would expect coroutines to close (on close method) with GeneratorExit
[Ive not thought all this through properly so may be off the mark]

[toc] | [prev] | [next] | [standalone]

#87358

On Fri, Mar 13, 2015 at 4:28 PM, Rustom Mody <rustompmody@gmail.com> wrote:
> And even there I would expect generators to close with StopIteration
> Whereas I would expect coroutines to close (on close method) with GeneratorExit
> [Ive not thought all this through properly so may be off the mark]

I expect both of them to close with "return". You can throw
GeneratorExit into a generator, but that's an implementation detail
more than anything else (same as the throwing of SystemExist is, and
for the same reason). When any iterator is exhausted, next() will
raise StopIteration rather than returning something; if you're simply
iterating over a generator, you can treat StopIteration as an
implementation detail too:

def squares_up_to(top):
    for n in range(top):
        if n*n > top: return
        yield n*n

for sq in squares_up_to(50):
    print(sq)

You don't need to be concerned about all those little details of
resuming and stack frames and so on. You definitely don't need to
worry about strange interactions around a "with" block inside the
generator - you can confidently trust that everything will behave
correctly. As it happens, this notion of "behaving correctly" is
largely implemented using exceptions, but if it were implemented with
a bunch of special cases in the CPython source code, it wouldn't
matter.

Write idiomatic source code and don't worry about the details. Learn
how things work if you're curious, but it's seldom going to matter.

ChrisA

[toc] | [prev] | [next] | [standalone]

#87364

On Friday, March 13, 2015 at 1:53:50 PM UTC+5:30, Chris Angelico wrote:
> On Fri, Mar 13, 2015 at 4:28 PM, Rustom Mody  wrote:
> > And even there I would expect generators to close with StopIteration
> > Whereas I would expect coroutines to close (on close method) with GeneratorExit
> > [Ive not thought all this through properly so may be off the mark]
> 
> I expect both of them to close with "return".

Nice demo of the same confusing terminology we are talking about.

When I say "expect generators to close" I mean 'generator-instances' ie at runtime

When you say "expect both to close with return" you are talking of program-texts
ie the 'factory'

[Or I am guilty of the same I am accusing python of]

[toc] | [prev] | [next] | [standalone]

#87365

Rustom Mody <rustompmody@gmail.com>:

> Nice demo of the same confusing terminology we are talking about.

Why don't you just stick with the terminology of the language
specification? I think your students are going to be more confused if
you try to come up with something better.

> When I say "expect generators to close" I mean 'generator-instances'
> ie at runtime
>
> When you say "expect both to close with return" you are talking of
> program-texts ie the 'factory'
>
> [Or I am guilty of the same I am accusing python of]

Your 'factory' is a:

    generator
        A function which returns an iterator.
    <URL: https://docs.python.org/3/glossary.html>

Your 'generator-instance' is an:

    iterator
        An object representing a stream of data.
    <URL: https://docs.python.org/3/glossary.html>

I don't think you should read much into what str(obj) returns. It's not
much more than a random printable some CPython coder has cooked up in
the heat of the moment.


Marko

[toc] | [prev] | [next] | [standalone]

#87409

Marko Rauhamaa wrote:

> Your 'factory' is a:
> 
>     generator
>         A function which returns an iterator.
>     <URL: https://docs.python.org/3/glossary.html>

That glossary entry is misleading, or at least incomplete, and it fails to
match the way "generator" is used by actual Python programmers.

Here is a function which returns an iterator. According to the docs, it's a
generator, but that's simply wrong. In no way, shape or form should it be
considered to be a generator:

def not_a_generator():
    return iter([1, 2, 3])


A generator (function) may be a function which returns an iterator, but not
all functions that return iterators are generators, and in some ways
returning an iterator is the *least* interesting part of what makes a
generator a generator.

What distinguishes a generator from a regular function is the use of
`yield`. Any definition which fails to mention that fact is useless.

"Generator" is used to describe both functions with the `yield` statement,
and the return result of calling such functions. Where it is necessary to
distinguish the two, we call the function-with-yield a "generator
function". This terminology comes straight from the PEP introducing
generators:

https://www.python.org/dev/peps/pep-0255/


Such functions-with-yield *are* functions (or methods):

py> def gen():
...     yield 1
...
py> type(gen)
<type 'function'>


but they're a special kind of function, distinguished by a flag on the
__code__ object (also known as func_code in Python 2). The inspect module
has a function to check that flag, which looks like this:

def isgeneratorfunction(object):
    return bool((isfunction(object) or ismethod(object)) and
                object.func_code.co_flags & CO_GENERATOR)


The result of calling `gen` is an instance of the generator type, that is to
say, an instance of a type which considers its own name to be "generator":

py> x = gen()
py> type(x)
<type 'generator'>

Although this type is built-in, it is not available in the builtins
namespace, but it is bound to the name "GeneratorType" in the types module.
The inspect module has a function for this too:

def isgenerator(object):
    return isinstance(object, types.GeneratorType)


For brevity, I've deleted the docstring, but it is very informative to read
it. Run `import inspect; help(inspect.isgenerator)` for more details.


Like many English words, we have two meanings for "generator":

(1) A function containing the `yield` keyword, or "generator-function".

(2) The result of calling such a function, an instance of
types.GeneratorType. PEP 255 calls that a "generator-iterator", but that
name doesn't appear to have caught on anywhere.

If people can cope with the difference between a TV program and a computer
program, they can cope with "generator" having two meanings, especially
since we have ways to disambiguate between the two when needed.

 
> Your 'generator-instance' is an:
> 
>     iterator
>         An object representing a stream of data.
>     <URL: https://docs.python.org/3/glossary.html>

Generator instances are iterators, but not all iterators are generator
instances. Generator instances are special: they are subroutines which can
be suspended and resumed, with multiple exit points (each yield is an exit
point). 

Generators are a subset of coroutines, which are a generalization of
generators. Coroutines have multiple entry points and exit points (each
yield is both an entry point and exit point). CPython uses the same
internal mechanism for both, and as far as I know, there is no programmatic
way to distinguish a coroutine from a generator. Or at least no obvious
way -- there's no `inspect.iscoroutine` function that I know of.


> I don't think you should read much into what str(obj) returns. It's not
> much more than a random printable some CPython coder has cooked up in
> the heat of the moment.

I think that is completely wrong. The repr() and str() of generator
instances is hardly "random", it reflects the consensus of the PEP authors
and the Python core developers, in particular the BDFL Guido who approved
the PEP, that the type of object it is should be called "generator".



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#87411

Steven D'Aprano <steve+comp.lang.python@pearwood.info>:

> Marko Rauhamaa wrote:
>
>> Your 'factory' is a:
>> 
>>     generator
>>         A function which returns an iterator.
>>     <URL: https://docs.python.org/3/glossary.html>
>
> That glossary entry is misleading, or at least incomplete, and it
> fails to match the way "generator" is used by actual Python
> programmers.

I am an actual Python programmer (I develop Python programs) and my
definitive source for Python is the documentation. If there is an error
in the documentation, I would very much like it to be corrected.

> A generator (function) may be a function which returns an iterator,
> but not all functions that return iterators are generators, and in
> some ways returning an iterator is the *least* interesting part of
> what makes a generator a generator.
>
> What distinguishes a generator from a regular function is the use of
> `yield`. Any definition which fails to mention that fact is useless.

The language reference had better use more precise language. It *is* the
definitive source, after all.

> Like many English words, we have two meanings for "generator":
>
> (1) A function containing the `yield` keyword, or "generator-function".
>
> (2) The result of calling such a function, an instance of
> types.GeneratorType. PEP 255 calls that a "generator-iterator", but
> that name doesn't appear to have caught on anywhere.
>
> If people can cope with the difference between a TV program and a
> computer program, they can cope with "generator" having two meanings,
> especially since we have ways to disambiguate between the two when
> needed.

I think that is untenable when you talk about a programming language.

>> I don't think you should read much into what str(obj) returns. It's
>> not much more than a random printable some CPython coder has cooked
>> up in the heat of the moment.
>
> I think that is completely wrong. The repr() and str() of generator
> instances is hardly "random", it reflects the consensus of the PEP
> authors and the Python core developers, in particular the BDFL Guido
> who approved the PEP, that the type of object it is should be called
> "generator".

Be that as it may,

   <URL: https://docs.python.org/3/reference/>

and

   <URL: https://docs.python.org/3/library/>

are the definitive sources for Python application programmers.

Or is there a better reference you would recommend?


Marko

[toc] | [prev] | [next] | [standalone]

#87413

On 14/03/2015 07:54, Marko Rauhamaa wrote:
> Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
>
>> Marko Rauhamaa wrote:
>>
>>> Your 'factory' is a:
>>>
>>>      generator
>>>          A function which returns an iterator.
>>>      <URL: https://docs.python.org/3/glossary.html>
>>
>> That glossary entry is misleading, or at least incomplete, and it
>> fails to match the way "generator" is used by actual Python
>> programmers.
>
> I am an actual Python programmer (I develop Python programs) and my
> definitive source for Python is the documentation. If there is an error
> in the documentation, I would very much like it to be corrected.
>

Five minutes work for you (singular) on the bug tracker.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#87414

Mark Lawrence <breamoreboy@yahoo.co.uk>:

> On 14/03/2015 07:54, Marko Rauhamaa wrote:
>> I am an actual Python programmer (I develop Python programs) and my
>> definitive source for Python is the documentation. If there is an
>> error in the documentation, I would very much like it to be
>> corrected.
>
> Five minutes work for you (singular) on the bug tracker.

I haven't determined that there is an error. I couldn't because to me,
the documentation defines right and wrong.

The only sign of discord so far has been between the return value of
str() and the language of the documentation. The return values of str()
and repr() are not defined very precisely:

 object.__repr__(self)
    Called by the repr() built-in function to compute the “official”
    string representation of an object. If at all possible, this should
    look like a valid Python expression that could be used to recreate
    an object with the same value (given an appropriate environment). If
    this is not possible, a string of the form <...some useful
    description...> should be returned. The return value must be a
    string object. [...]

 object.__str__(self)
    Called by str(object) and the built-in functions format() and
    print() to compute the “informal” or nicely printable string
    representation of an object. The return value must be a string
    object.

Thus one shouldn't read too much into what str() and repr() return. In
particular, there is no requirement that two valid Python
implementations ought to return similar-looking strings.


Marko

[toc] | [prev] | [next] | [standalone]

#87445

On Sat, Mar 14, 2015 at 1:54 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Steven D'Aprano <steve+comp.lang.python@pearwood.info>:
>
>> Marko Rauhamaa wrote:
>>
>>> Your 'factory' is a:
>>>
>>>     generator
>>>         A function which returns an iterator.
>>>     <URL: https://docs.python.org/3/glossary.html>
>>
>> That glossary entry is misleading, or at least incomplete, and it
>> fails to match the way "generator" is used by actual Python
>> programmers.
>
> I am an actual Python programmer (I develop Python programs) and my
> definitive source for Python is the documentation. If there is an error
> in the documentation, I would very much like it to be corrected.

I think that Steven was basing his statement on the definition you
cited. I don't think that he actually went and looked up the
definition. If he had, he would have seen that you omitted 90% of it.
Here's the full entry:

"""
A function which returns an iterator. It looks like a normal function
except that it contains yield statements for producing a series of
values usable in a for-loop or that can be retrieved one at a time
with the next() function. Each yield temporarily suspends processing,
remembering the location execution state (including local variables
and pending try-statements). When the generator resumes, it picks-up
where it left-off (in contrast to functions which start fresh on every
invocation).
"""

>> A generator (function) may be a function which returns an iterator,
>> but not all functions that return iterators are generators, and in
>> some ways returning an iterator is the *least* interesting part of
>> what makes a generator a generator.
>>
>> What distinguishes a generator from a regular function is the use of
>> `yield`. Any definition which fails to mention that fact is useless.

As can be seen above, the glossary definition *does* in fact discuss
the use of yield.

> The language reference had better use more precise language. It *is* the
> definitive source, after all.

Okay, but you cited the glossary entry, not the language reference.
The language reference says this [1]:

"When a generator function is called, it returns an iterator known as
a generator."

Which I think is quite clear, although to be fair the same section
also uses "generator" alone to refer to the function in the preceding
paragraph. That usage however is justified by PEP 255 [2]:

"""
When a generator function is called, the actual arguments are bound to
function-local formal argument names in the usual way, but no code in
the body of the function is executed.  Instead a generator-iterator
object is returned; this conforms to the iterator protocol, so in
particular can be used in for-loops in a natural way.  Note that when
the intent is clear from context, the unqualified name "generator" may
be used to refer either to a generator-function or a
generator-iterator.

"""

Now which should be considered definitive, the language reference or
the PEP? This question is not rhetorical; I don't know the answer.
Regardless of the answer though, the PEP at least illuminates the
design intent of the terminology.


[1] https://docs.python.org/3/reference/expressions.html#yield-expressions

[2] https://www.python.org/dev/peps/pep-0255/

[toc] | [prev] | [next] | [standalone]

#87463

On Sunday, March 15, 2015 at 1:45:37 AM UTC+5:30, Ian wrote:
> Now which should be considered definitive, the language reference or
> the PEP? This question is not rhetorical; I don't know the answer.
> Regardless of the answer though, the PEP at least illuminates the
> design intent of the terminology.
>  
>  
> [1] https://docs.python.org/3/reference/expressions.html#yield-expressions
>  
> [2] https://www.python.org/dev/peps/pep-0255/

Thanks

That PEP deserves careful reading, particularly the end...

| BDFL Pronouncements 
| Issue:  Introduce another new keyword (say, "gen" or "generator") in
| place of "def", or otherwise alter the syntax, to distinguish
| generator-functions from non-generator functions.
|  
| Con:  In practice (how you think about them), generators *are*
| functions, but with the twist that they're resumable.  The mechanics of
| how they're set up is a comparatively minor technical issue, and
| introducing a new keyword would unhelpfully overemphasize the
| mechanics of how generators get started (a vital but tiny part of a
| generator's life).
|  
| Pro:  In reality (how you think about them), generator-functions are
| actually factory functions that produce generator-iterators as if by
| magic.  In this respect they're radically different from non-generator
| functions, acting more like a constructor than a function, so reusing
| "def" is at best confusing.  A "yield" statement buried in the body is
| not enough warning that the semantics are so different.
|  
| BDFL:  "def" it stays.  No argument on either side is totally
| convincing, so I have consulted my language designer's intuition.  It
| tells me that the syntax proposed in the PEP is exactly right - not too
| hot, not too cold.  But, like the Oracle at Delphi in Greek mythology,
| it doesn't tell me why, so I don't have a rebuttal for the arguments
| against the PEP syntax.  The best I can come up with (apart from
| agreeing with the rebuttals ... already made) is "FUD".  If this had
| been part of the language from day one, I very much doubt it would have
| made Andrew Kuchling's "Python Warts" page.

So while I dont go to the extent of suggesting introducing a different keyword for
generators, the fact that the overloading of def is problematic and arbitrary
comes from the horse's (ie BDFL's) mouth.

[toc] | [prev] | [next] | [standalone]

#87446

On 14/03/2015 20:14, Ian Kelly wrote:
>
> Now which should be considered definitive, the language reference or
> the PEP? This question is not rhetorical; I don't know the answer.
> Regardless of the answer though, the PEP at least illuminates the
> design intent of the terminology.
>

The language reference should be the definitive guide.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#87422

On Saturday, March 14, 2015 at 11:34:27 AM UTC+5:30, Steven D'Aprano wrote:
> 
> A generator (function) may be a function which returns an iterator,...

I find "generator-function" misleading in the same way that "pineapple" 
misleadingly suggests "apple that grows on pines"

A builtin function is a function in the builtin (or builtins -- can never remember) module
A pure function is function that does not assign or mutate non-locals
A Steven-function is a function that presumably Steven wrote 

However a "generator function" is a weird sort of function (at best).
Not regarding it as a function is IMO more reasonable.

[toc] | [prev] | [next] | [standalone]

Page 1 of 5  [1] 2 3 4 5  Next page →

Back to top | Article view | comp.lang.python

csiph-web