Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #89968 > unrolled thread

asyncio: What is the difference between tasks, futures, and coroutines?

Back to article view | Back to comp.lang.python

Contents

  asyncio: What is the difference between tasks, futures, and coroutines? Paul  Moore <p.f.moore@gmail.com> - 2015-05-05 08:22 -0700
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Zachary Ware <zachary.ware+pylist@gmail.com> - 2015-05-05 11:11 -0500
      Re: asyncio: What is the difference between tasks, futures, and coroutines? Paul  Moore <p.f.moore@gmail.com> - 2015-05-05 10:55 -0700
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Terry Reedy <tjreedy@udel.edu> - 2015-05-05 13:15 -0400
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Marko Rauhamaa <marko@pacujo.net> - 2015-05-05 20:45 +0300
      Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-05 21:47 -0700
        Re: asyncio: What is the difference between tasks, futures, and coroutines? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-06 15:48 +1000
          Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:06 -0700
            Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 14:33 +1000
              Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:53 -0700
                Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:55 -0700
                Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 15:09 +1000
                  Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 23:36 -0700
                    Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 16:42 +1000
                    Re: asyncio: What is the difference between tasks, futures, and coroutines? Dave Angel <davea@davea.name> - 2015-05-08 07:53 -0400
                    Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 23:02 +1000
        Re: asyncio: What is the difference between tasks, futures, and coroutines? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-05-06 09:11 -0400
          Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:20 -0700
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-05 11:46 -0600
      Re: asyncio: What is the difference between tasks, futures, and coroutines? Paul  Moore <p.f.moore@gmail.com> - 2015-05-05 11:03 -0700
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Skip Montanaro <skip.montanaro@gmail.com> - 2015-05-05 12:55 -0500
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Terry Reedy <tjreedy@udel.edu> - 2015-05-05 18:38 -0400

Page 1 of 2  [1] 2  Next page →

#89968 — asyncio: What is the difference between tasks, futures, and coroutines?

I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.

I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".

But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?

Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

I concede that I've not read the rest of the asyncio documentation in much detail yet, and I'm skipping everything to do with IO (I want to understand the "async" bit for now, not so much the "IO" side). But I don't really want to dive into the details while I am this hazy on the basic concepts.

Can anyone clarify for me?

Thanks,
Paul

[toc] | [next] | [standalone]

#89971

On Tue, May 5, 2015 at 10:22 AM, Paul  Moore <p.f.moore@gmail.com> wrote:
> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.

I've been using (and, in part at least, understanding :)) asyncio for
work for the past few months, so I can at least give you my
impressions based on use.

> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".

That's basically how I understand it.

> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

My understanding is that Futures are somewhat like 'non-executable
coroutines', if that makes any sense whatsoever.  Futures are used as
something you can pass around when you need to start execution of the
"job" in one method and finish it in another.  For instance, talking
to a remote network entity, and you want to just do "yield from
server.get(something)".  Your 'get' method can make the request,
create a Future, stick it somewhere that the method monitoring
incoming traffic can find it (along with some identifying metadata),
and then do 'return (yield from future)' to wait for the Future to be
fulfilled (by the listener) and return its result.  The listener then
matches up incoming requests with Futures, and calls
Future.set_result() or Future.set_exception() to fulfill the Future.
Once one of those methods has been called on the Future (and control
has been passed back to the scheduler), your server.get method
unblocks from its '(yield from future)' call, and either raises the
exception or returns the result that was set on the Future.

> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?
>
> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

You kind of mixed things up in this paragraph (you said "Tasks somehow
wrap up coroutines", then "wrap a task in a coroutine"; the first is
more correct, I believe).  As I understand it, Tasks are
specializations of Futures that take care of the
set_result/set_exception based on the execution of the coroutine.  I'm
not clear on how that is all implemented (Are all coroutines wrapped
in Tasks?  Just those wrapped in asyncio.async()? No idea.), but I use
it something like this:

   # save a reference to this task so it can be cancelled elsewhere if need be
   self.current_task = asyncio.async(some_coroutine())
   # now the coroutine is scheduled, but we still have control
   # we can do anything else we need to here, including scheduling
other coroutines
   return (yield from self.current_task)

> I concede that I've not read the rest of the asyncio documentation in much detail yet, and I'm skipping everything to do with IO (I want to understand the "async" bit for now, not so much the "IO" side). But I don't really want to dive into the details while I am this hazy on the basic concepts.
>
> Can anyone clarify for me?

I hope I've done some good on that front, but I'm still a bit hazy
myself.  I found it worked best to just take examples or otherwise
working code, and poke and prod at it until it breaks and you can
figure out how you broke it.  Just try to avoid mixing asyncio and
threads for as long as you can :).

-- 
Zach

[toc] | [prev] | [next] | [standalone]

#89982

On Tuesday, 5 May 2015 17:11:39 UTC+1, Zachary Ware  wrote:
>On Tue, May 5, 2015 at 10:22 AM, Paul  Moore wrote:
>> I'm working my way through the asyncio documentation. I have got to the
>> "Tasks and coroutines" section, but I'm frankly confused as to the
>> difference between the various things described in that section:
>> coroutines, tasks, and futures.
> 
> I've been using (and, in part at least, understanding :)) asyncio for
> work for the past few months, so I can at least give you my
> impressions based on use.

Thanks for taking the time.
 
>> I think can understand a coroutine. Correct me if I'm wrong, but it's
>> roughly "something that you can run which can suspend itself".
> 
> That's basically how I understand it.

Cool, I got that right at least :-)

>> But I don't understand what a Future is.
[...]
> 
> My understanding is that Futures are somewhat like 'non-executable
> coroutines', if that makes any sense whatsoever.  Futures are used as
> something you can pass around when you need to start execution of the
> "job" in one method and finish it in another.  For instance, talking
> to a remote network entity, and you want to just do "yield from
> server.get(something)".  Your 'get' method can make the request,
> create a Future, stick it somewhere that the method monitoring
> incoming traffic can find it (along with some identifying metadata),
> and then do 'return (yield from future)' to wait for the Future to be
> fulfilled (by the listener) and return its result.  The listener then
> matches up incoming requests with Futures, and calls
> Future.set_result() or Future.set_exception() to fulfill the Future.
> Once one of those methods has been called on the Future (and control
> has been passed back to the scheduler), your server.get method
> unblocks from its '(yield from future)' call, and either raises the
> exception or returns the result that was set on the Future.

Hmm. My head hurts. I sort of think I see what you might mean, but I have no idea how that translates to "almost the same as a concurrent.futures.Future, which in my experience is nothing like that.

A c.f.Future is (in my naive view) a "promise" of a result, which you can later wait to be fulfilled. Creating one directly (as opposed to via submit) doesn't make sense because then there's nothing that's promised to provide a result.

What you described sounds more like a "container that can signal when it's been filled", which I can sort of see as related to the Future API, but not really as related to a c.f.Future.

>> Reading between the lines, it seems that the event loop schedules Tasks
>> (which makes sense) and that Tasks somehow wrap up coroutines - but I
>> don't see *why* you need to wrap a task in a coroutine rather than just
>> scheduling coroutines. And I don't see where Futures fit in - why not
>> just wrap a coroutine in a Future, if it needs to be wrapped up at all?
> 
> You kind of mixed things up in this paragraph (you said "Tasks somehow
> wrap up coroutines", then "wrap a task in a coroutine"; the first is
> more correct, I believe).

Whoops, sorry, my mistake - the first was what I meant, the second was a typo.

> As I understand it, Tasks are
> specializations of Futures that take care of the
> set_result/set_exception based on the execution of the coroutine.

That sounds far more like a concurrent.futures.Future. Except that (AIUI) tasks get scheduled, and you can list all tasks for an event loop, and things like that. So they seem like a little bit more than a c.f.Future, which would sort of match with the idea that an asyncio.Future was equivalent to a c.f.Future, except that it's not (as above).

There's also things like asyncio.async - which makes no sense to me at all (I can see how the examples *use* it, in much the same way as I can see how the examples work, in general, but I can't understand its role, so I'm not sure I'd be able to design code that used it from scratch :-()

>> Can anyone clarify for me?
> 
> I hope I've done some good on that front, but I'm still a bit hazy
> myself.  I found it worked best to just take examples or otherwise
> working code, and poke and prod at it until it breaks and you can
> figure out how you broke it.  Just try to avoid mixing asyncio and
> threads for as long as you can :).

Thanks for the insights. I think I've learned some things, but to be honest I'm still confused. Maybe it's because of the angle I'm coming at it from. I don't typically write network code directly[1], so the examples in the asyncio docs are mostly irrelevant at best for me. I do find that I use threads a reasonable amount, and I'd like to know if asyncio would be a worthwhile alternative. So I need to understand the concepts and try to apply them to my situations. Hence the comment above about designing asyncio code from scratch.

Paul

[1] When I do write network code, I typically want to use a high level library like requests. So in an asyncio context, I'm looking at how to use an existing, synchronous library, in async code. In twisted, I'd use defer_to_thread, and I think run_in_executor is the equivalent asyncio concept. The docs say that is a coroutine, so I probably need to wrap that in a task to schedule it, but that's where a proper understanding of the concepts becomes crucial...

[toc] | [prev] | [next] | [standalone]

#89979

On 5/5/2015 11:22 AM, Paul Moore wrote:
> I'm working my way through the asyncio documentation. I have got to
> the "Tasks and coroutines" section, but I'm frankly confused as to
> the difference between the various things described in that section:
> coroutines, tasks, and futures.
>
> I think can understand a coroutine. Correct me if I'm wrong, but it's
> roughly "something that you can run which can suspend itself".

The simple answer is that a coroutine within asyncio is a generator used 
as a (semi)coroutine rather than merely as an iterator.  IE, the send, 
throw, and close methods are used.  I believe PEP 492 will change that 
definition (for Python) to any object with with those methods.

However, if an asyncio coroutine is the result of the coroutine 
decorator, the situation is more complex, as the result is either a 
generator function, a coro function that can wrap any function, or a 
debug wrapper.  Details in asyncio/coroutine.py.

> I concede that I've not read the rest of the asyncio documentation in
> much detail yet, and I'm skipping everything to do with IO (I want to
> understand the "async" bit for now, not so much the "IO" side). But I
> don't really want to dive into the details while I am this hazy on
> the basic concepts.

You might try reading the Python code for the task and future classes.
asyncio/futures.py, asyncio/tasks.py

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#89980

Paul  Moore <p.f.moore@gmail.com>:

> But I don't understand what a Future is.

A future stands for a function that is scheduled to execute in the
background.

Personally, I have never found futures a very useful idiom in any
language (Scheme, Java, Python). Or more to the point, concurrency and
the notion of a function don't gel well in my mind.

See also: <URL: http://en.wikipedia.org/wiki/Futures_and_promises>

> A Task is a subclass of Future, but the documentation doesn't say what
> it *is*,

A task is almost synonymous with a future. It's something that needs to
get done.

--

I think asyncio is a worthwhile step up from the old asyncore module.
Its main advantage is portability between operating systems. Also, it
has one big advantage over threads: events can be multiplexed using
asyncio.wait(return_when=FIRST_COMPLETED).

However, I happen to have the privilege of working with linux so
select.epoll() gives me all the power I need to deal with asynchrony.
The asyncio model has the same basic flaw as threads: it makes the
developer think of concurrency in terms of a bunch of linear streaks of
execution. I much prefer to look at concurrency through states and
events (what happened? what next?).

My preferred model is known as "Callback Hell" (qv.). Asyncio seeks to
offer a salvation from it. However, I am at home in the Callback Hell;
equipped with Finite State Machines, the inherent complexities of the
Reality are managed in the most natural, flexible manner.


Marko

[toc] | [prev] | [next] | [standalone]

#90026

On Tuesday, May 5, 2015 at 11:15:42 PM UTC+5:30, Marko Rauhamaa wrote:
 
> Personally, I have never found futures a very useful idiom in any
> language (Scheme, Java, Python). Or more to the point, concurrency and
> the notion of a function don't gel well in my mind.

Interesting comment.

It strikes me that the FP crowd has stretched the notion of function beyond recognition
And the imperative/OO folks have distorted it beyond redemption.

And the middle road shown by Pascal has been overgrown with weeds for some 40 years...

If the classic Pascal (or Fortran or Basic) sibling balanced abstractions of function-for-value
procedure-for-effect were more in the collective consciousness rather than C's
travesty of function, things might not have been so messy.

Well... Dont feel right bashing C without some history...

C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
Nor the confusion of = for assignment; Fortran did that.

[toc] | [prev] | [next] | [standalone]

#90028

On Wednesday 06 May 2015 14:47, Rustom Mody wrote:

> It strikes me that the FP crowd has stretched the notion of function
> beyond recognition And the imperative/OO folks have distorted it beyond
> redemption.

In what way?


> And the middle road shown by Pascal has been overgrown with weeds for some
> 40 years...

As much as I like Pascal, and am pleased to see someone defending it, I'm 
afraid I have no idea what you mean by this.


> If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> of function-for-value procedure-for-effect were more in the collective
> consciousness rather than C's travesty of function, things might not have
> been so messy.

I'm not really sure that having distinct procedures, as opposed to functions 
that you just ignore their return result, makes *such* a big difference. Can 
you explain what is the difference between these?

sort(stuff)  # A procedure.
sort(stuff)  # ignore the function return result

And why the first is so much better than the second?



> Well... Dont feel right bashing C without some history...
> 
> C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> Nor the confusion of = for assignment; Fortran did that.

Pardon, but = has been used for "assignment" for centuries, long before 
George Boole and Ada Lovelace even started working on computer theory. Just 
ask mathematicians:

"let y = 23"

Is that not a form of assignment?




-- 
Steve

[toc] | [prev] | [next] | [standalone]

#90113

On Wednesday, May 6, 2015 at 11:19:07 AM UTC+5:30, Steven D'Aprano wrote:
> On Wednesday 06 May 2015 14:47, Rustom Mody wrote:
> 
> > It strikes me that the FP crowd has stretched the notion of function
> > beyond recognition And the imperative/OO folks have distorted it beyond
> > redemption.
> 
> In what way?
> 
> 
> > And the middle road shown by Pascal has been overgrown with weeds for some
> > 40 years...
> 
> As much as I like Pascal, and am pleased to see someone defending it, I'm 
> afraid I have no idea what you mean by this.

There are many hats...

As a programmer what you say is fine.
As a language-designer maybe even required -- one would expect a language designer to invoke Occm's razor and throw away needless distinctions (read syntax categories)

But there are other hats.
Even if you disregard the teacher-hat as irrelevant, the learner-hat is universal
-- whatever you are master of now is because at some past point you walked its
learning curve.

> 
> 
> > If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> > of function-for-value procedure-for-effect were more in the collective
> > consciousness rather than C's travesty of function, things might not have
> > been so messy.
> 
> I'm not really sure that having distinct procedures, as opposed to functions 
> that you just ignore their return result, makes *such* a big difference. Can 
> you explain what is the difference between these?
> 
> sort(stuff)  # A procedure.
> sort(stuff)  # ignore the function return result
> 
> And why the first is so much better than the second?

Here are 3 None-returning functions/methods in python.
ie semantically the returns are identical. Are they conceptually identical?

>>> x=print(1)
1
>>> x
>>> ["hello",None].__getitem__(1)
>>> {"a":1, "b":2}.get("c")
>>> 



> 
> 
> 
> > Well... Dont feel right bashing C without some history...
> > 
> > C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> > Nor the confusion of = for assignment; Fortran did that.
> 
> Pardon, but = has been used for "assignment" for centuries, long before 
> George Boole and Ada Lovelace even started working on computer theory. Just 
> ask mathematicians:
> 
> "let y = 23"
> 
> Is that not a form of assignment?

Truth?? Lets see...

Here is a set of assignments (as you call) that could occur in a school text
teaching business-math, viz how to calculate 'simple-interest'

amt = prin + si
si = prin * n * rate/100.0 
# for instance
prin = 1000.0 
n = 4.0
rate = 12.0

Put it into python and you get Name errors.

Put it into Haskell or some such; (after changing the comment char from '#' to '--')
and you'll get amt and si

Now you can view this operationally (which in some cases even the Haskell docs do)
and say a bunch of ='s in python is a sequence whereas in haskell its
'simultaneous' ie a set.

But this would miss the point viz that in Python
amt = prin + si 
denotes an *action*
whereas in Haskell it denotes a *fact* and a bunch of facts that modified by being
permuted would be a strange bunch!

Note I am not saying Haskell is right; particularly in its semantics of '='
there are serious issues:
http://blog.languager.org/2012/08/functional-programming-philosophical.html

Just that
x = y
in a functional/declarative/math framework means something quite different 
than in an imperative one

[toc] | [prev] | [next] | [standalone]

#90115

On Fri, May 8, 2015 at 2:06 PM, Rustom Mody <rustompmody@gmail.com> wrote:
>> > If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
>> > of function-for-value procedure-for-effect were more in the collective
>> > consciousness rather than C's travesty of function, things might not have
>> > been so messy.
>>
>> I'm not really sure that having distinct procedures, as opposed to functions
>> that you just ignore their return result, makes *such* a big difference. Can
>> you explain what is the difference between these?
>>
>> sort(stuff)  # A procedure.
>> sort(stuff)  # ignore the function return result
>>
>> And why the first is so much better than the second?
>
> Here are 3 None-returning functions/methods in python.
> ie semantically the returns are identical. Are they conceptually identical?
>
>>>> x=print(1)
> 1
>>>> x
>>>> ["hello",None].__getitem__(1)
>>>> {"a":1, "b":2}.get("c")
>>>>

Your second example is a poor one, as it involves calling a dunder
method. But all you've proven with that is that the empty return value
of Python is a real value, and can thus be stored and retrieved.

With print(), you have a conceptual procedure - it invariably returns
None, so it'll normally be called in contexts that don't care about
the return value. What about sys.stdout.write(), though? That's most
often going to be called procedurally - you just write your piece and
move on - but it has a meaningful return value. In normal usage, both
are procedures. The ability to upgrade a function from "always returns
None" to "returns some occasionally-useful information" is one of the
strengths of a unified function/procedure system.

With .get(), it's actually nothing to do with procedures vs functions,
but more to do with Python's use of None where C would most likely use
NULL. By default, .get() uses None as its default return value, so
something that isn't there will be treated as None. In the same way as
your second example, that's just a result of None being a real value
that you can pass around - nothing more special than that.

The other thing you're seeing here is that the *interactive
interpreter* treats None specially. In a program, an expression
statement simply discards its result, whether it's None or 42 or
[1,2,3] or anything else. You could write an interactive interpreter
that has some magic that recognizes that certain functions always
return None (maybe by checking their annotations), and omits printing
their return values, while still printing the return values of other
functions. I'm not sure what it'd gain you, but it wouldn't change the
concepts or semantics surrounding None returns.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90116

On Friday, May 8, 2015 at 10:04:02 AM UTC+5:30, Chris Angelico wrote:
> On Fri, May 8, 2015 at 2:06 PM, Rustom Mody wrote:
> >> > If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> >> > of function-for-value procedure-for-effect were more in the collective
> >> > consciousness rather than C's travesty of function, things might not have
> >> > been so messy.
> >>
> >> I'm not really sure that having distinct procedures, as opposed to functions
> >> that you just ignore their return result, makes *such* a big difference. Can
> >> you explain what is the difference between these?
> >>
> >> sort(stuff)  # A procedure.
> >> sort(stuff)  # ignore the function return result
> >>
> >> And why the first is so much better than the second?
> >
> > Here are 3 None-returning functions/methods in python.
> > ie semantically the returns are identical. Are they conceptually identical?
> >
> >>>> x=print(1)
> > 1
> >>>> x
> >>>> ["hello",None].__getitem__(1)
> >>>> {"a":1, "b":2}.get("c")
> >>>>
> 
> Your second example is a poor one, as it involves calling a dunder
> method. But all you've proven with that is that the empty return value
> of Python is a real value, and can thus be stored and retrieved.
> 
> With print(), you have a conceptual procedure - it invariably returns
> None, so it'll normally be called in contexts that don't care about
> the return value. What about sys.stdout.write(), though? That's most
> often going to be called procedurally - you just write your piece and
> move on - but it has a meaningful return value. In normal usage, both
> are procedures. The ability to upgrade a function from "always returns
> None" to "returns some occasionally-useful information" is one of the
> strengths of a unified function/procedure system.
> 
> With .get(), it's actually nothing to do with procedures vs functions,

That's backwards.

get is very much a function and the None return is semantically significant.
print is just round peg -- what you call conceptual function -- stuffed into
square hole -- function the only available syntax-category

> but more to do with Python's use of None where C would most likely use
> NULL. By default, .get() uses None as its default return value, so
> something that isn't there will be treated as None. In the same way as
> your second example, that's just a result of None being a real value
> that you can pass around - nothing more special than that.
> 
> The other thing you're seeing here is that the *interactive
> interpreter* treats None specially. 


Yeah I know
And if python did not try to be so clever, I'd save some time with
student-surprises

> In a program, an expression
> statement simply discards its result, whether it's None or 42 or
> [1,2,3] or anything else. You could write an interactive interpreter
> that has some magic that recognizes that certain functions always
> return None (maybe by checking their annotations), and omits printing
> their return values, while still printing the return values of other
> functions.


Hoo Boy!  You seem to be in the 'the-more-the-better' (of magic) camp


> I'm not sure what it'd gain you, but it wouldn't change the
> concepts or semantics surrounding None returns.

It would sure help teachers who get paid by the hour and would rather spend
time on technical irrelevantia than grapple with significant concepts

[toc] | [prev] | [next] | [standalone]

#90117

On Friday, May 8, 2015 at 10:24:06 AM UTC+5:30, Rustom Mody wrote:

> get is very much a function and the None return is semantically significant.
> print is just round peg -- what you call conceptual function -- stuffed into
> square hole -- function the only available syntax-category

Sorry "Conceptual procedure" of course

[toc] | [prev] | [next] | [standalone]

#90118

On Fri, May 8, 2015 at 2:53 PM, Rustom Mody <rustompmody@gmail.com> wrote:
> Yeah I know
> And if python did not try to be so clever, I'd save some time with
> student-surprises
>
>> In a program, an expression
>> statement simply discards its result, whether it's None or 42 or
>> [1,2,3] or anything else. You could write an interactive interpreter
>> that has some magic that recognizes that certain functions always
>> return None (maybe by checking their annotations), and omits printing
>> their return values, while still printing the return values of other
>> functions.
>
>
> Hoo Boy!  You seem to be in the 'the-more-the-better' (of magic) camp

No way! I wouldn't want the interactive interpreter to go too magical.
I'm just saying that it wouldn't break Python to have it do things
differently.

>> I'm not sure what it'd gain you, but it wouldn't change the
>> concepts or semantics surrounding None returns.
>
> It would sure help teachers who get paid by the hour and would rather spend
> time on technical irrelevantia than grapple with significant concepts

Why have the concept of a procedure? Python's rule is simple: Every
function either returns a value or raises an exception. (Even
generators. When you call a generator function, you get back a return
value which is the generator state object.) The procedure/function
distinction is irrelevant.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90120

On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:
> Why have the concept of a procedure? 

On Friday, Chris Angelico ALSO wrote:
> With print(), you have a conceptual procedure...

So which do you want to stand by?


Just to be clear I am not saying python should be any different on this front.

Gödel's (2nd) theorem guarantees that no formalism (aka programming language in our case)
can ever be complete and so informal borrowing is inevitable.
Its just that Pascal, Fortran, Basic, by ingesting this informal requirement into
the formal language make THIS aspect easier to learn/teach...
... at the cost of eleventeen others

[Regular expressions in Fortran anyone?]

[toc] | [prev] | [next] | [standalone]

#90121

On Fri, May 8, 2015 at 4:36 PM, Rustom Mody <rustompmody@gmail.com> wrote:
> On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:
>> Why have the concept of a procedure?
>
> On Friday, Chris Angelico ALSO wrote:
>> With print(), you have a conceptual procedure...
>
> So which do you want to stand by?

A procedure, in Python, is simply a function which returns None.
That's all. It's not any sort of special concept. It doesn't need to
be taught. If your students are getting confused by it, stop teaching
it!

ChrisA

[toc] | [prev] | [next] | [standalone]

#90151

On 05/08/2015 02:42 AM, Chris Angelico wrote:
> On Fri, May 8, 2015 at 4:36 PM, Rustom Mody <rustompmody@gmail.com> wrote:
>> On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:
>>> Why have the concept of a procedure?
>>
>> On Friday, Chris Angelico ALSO wrote:
>>> With print(), you have a conceptual procedure...
>>
>> So which do you want to stand by?
>
> A procedure, in Python, is simply a function which returns None.
> That's all. It's not any sort of special concept. It doesn't need to
> be taught. If your students are getting confused by it, stop teaching
> it!

One thing newbies get tripped up by is having some path through their 
code that doesn't explicitly return.  And in Python that path therefore 
returns None.  It's most commonly confusing when there are nested ifs, 
and one of the "inner ifs" doesn't have an else clause.

Anyway, it's marginally more useful to that newbie if the compiler would 
produce an error instead of later seeing a runtime error due to an 
unexpected None result.

I don't think Python would be improved by detecting such a condition and 
reporting on it.  That's a job for a linter, or a style guide program.

No different than the compile time checks for variable type that most 
languages impose.  They don't belong in Python.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#90165

On Fri, May 8, 2015 at 9:53 PM, Dave Angel <davea@davea.name> wrote:
> One thing newbies get tripped up by is having some path through their code
> that doesn't explicitly return.  And in Python that path therefore returns
> None.  It's most commonly confusing when there are nested ifs, and one of
> the "inner ifs" doesn't have an else clause.
>
> Anyway, it's marginally more useful to that newbie if the compiler would
> produce an error instead of later seeing a runtime error due to an
> unexpected None result.
>
> I don't think Python would be improved by detecting such a condition and
> reporting on it.  That's a job for a linter, or a style guide program.

They're not hard for linters to notice. After all, there's a clear
difference of intent between "return" (or just falling off the end of
the function) and "return None", even though they compile to the same
byte code. Though if you start enforcing that in your checkin policy,
you might have to deal with this kind of thing:

def raise_with_code(code):
    raise SpamError("Error %d in spamination: %s" % (code,
errortext.get(code, "<unknown>")))

def spaminate_text(f):
    rc = low_level_spamination_function(f)
    if not rc: return low_level_get_spamination_result()
    raise_with_code(rc)

Clearly one branch returns a value... but figuring out that the other
one always raises isn't easy. (Though I suppose in this case it could
be dealt with by having a function that constructs the error, and then
you "raise make_spam_error(rc)" instead.)

You're definitely right that Python shouldn't check for it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90052

On Tue, 5 May 2015 21:47:17 -0700 (PDT), Rustom Mody
<rustompmody@gmail.com> declaimed the following:

>If the classic Pascal (or Fortran or Basic) sibling balanced abstractions of function-for-value
>procedure-for-effect were more in the collective consciousness rather than C's
>travesty of function, things might not have been so messy.
>
	I suspect just the term "subprogram" (as in "function subprogram" and
"subroutine subprogram") would confuse a lot of cubs these days...

>C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
>Nor the confusion of = for assignment; Fortran did that.

	I don't think you can blame FORTRAN for that, given that it was one of
the first of the higher level languages, and had no confusion internally...
The comparison operator being .EQ. (and kin: .LT. .GT. .LE. .GE. .NE.); and
with assignment as a statement, not an expression in its own right, no
chance of even something like
	IF (X = Y .EQ. Z) GOTO 123
to be concerned about.

	Confusion over assignment vs equality I'd lay firmly on C
	if (x = y == z) break;

-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#90114

On Wednesday, May 6, 2015 at 6:41:38 PM UTC+5:30, Dennis Lee Bieber wrote:
> On Tue, 5 May 2015 21:47:17 -0700 (PDT), Rustom Mody declaimed the following:
> 
> >If the classic Pascal (or Fortran or Basic) sibling balanced abstractions of function-for-value
> >procedure-for-effect were more in the collective consciousness rather than C's
> >travesty of function, things might not have been so messy.
> >
> 	I suspect just the term "subprogram" (as in "function subprogram" and
> "subroutine subprogram") would confuse a lot of cubs these days...
> 
> >C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> >Nor the confusion of = for assignment; Fortran did that.
> 
> 	I don't think you can blame FORTRAN for that, given that it was one of
> the first of the higher level languages, and had no confusion internally...

BLAME?? Ha You are being funny Dennis!

There are fat regulation-books that pilots need to follow. Breaking them can
make one culpable all the way to homicide.
The Wright-brothers probably broke them all. Should we call them homicidal maniacs?

Shakespeare sometimes has funny spellings. I guess he's illiterate for not turning on the spell-checker in Word?

Or [my favorite] Abraham Lincoln used the word 'negro'. So he's a racist?

Speaking more conceptually, there are pioneers and us ordinary folks.¹
The world as we know it is largely a creation of these pioneers.
And if you take them and stuff them into the statistically ordinary mold then fit badly.

That puts people especially teachers into a bind.
If I expect my students to be 1/100 as pioneering as Backus, Thomson, Ritchie 
etc, I would be foolish
And if I dont spell out all their mistakes in minute detail and pull them up
for repeating them, I'd not be doing due diligence

--------------------
I guess this is nowadays called the 'romantic' view.
Ask the family-members of any of these greats for the 'other' view  :-)

[toc] | [prev] | [next] | [standalone]

#89981

On Tue, May 5, 2015 at 9:22 AM, Paul  Moore <p.f.moore@gmail.com> wrote:
> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.
>
> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".
>
> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

Fundamentally, a future is a placeholder for something that isn't
available yet. You can use it to set a callback to be called when that
thing is available, and once it's available you can get that thing
from it.

> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?

In asyncio, a Task is a a Future that serves as a placeholder for the
result of a coroutine. The event loop manages callbacks, and that's
all it does. An event that it's been told to listen for occurs, and
the event loop calls the callback associated with that event. The Task
manages a coroutine's interaction with the event loop; when the
coroutine yields a future, the Task instructs the event loop to listen
for the completion of that future, setting a callback that will resume
the coroutine.

> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

The coroutines themselves are not that interesting of an interface;
all you can do with them is resume them. The asynchronous execution
done by asyncio is all based on futures. Because a coroutine can
easily be wrapped in a Task, this allows for coroutines to be used
anywhere a future is expected.

I don't know if I've done a good job explaining, but I hope this helps.

[toc] | [prev] | [next] | [standalone]

#89984

On Tuesday, 5 May 2015 18:48:09 UTC+1, Ian  wrote:
> Fundamentally, a future is a placeholder for something that isn't
> available yet. You can use it to set a callback to be called when that
> thing is available, and once it's available you can get that thing
> from it.

OK, that makes a lot of sense. There seems to be two distinct strands within the asyncio world, the callback model and the task/coroutine model. AIUI, coroutines/tasks are supposed to let you avoid callbacks.

So I guess in that model, a Future isn't something you should use directly in task-based code, because it works via callbacks. And yet tasks are futures, so it's not that simple :-)

> In asyncio, a Task is a a Future that serves as a placeholder for the
> result of a coroutine. The event loop manages callbacks, and that's
> all it does.

Um, I thought it scheduled tasks?

> An event that it's been told to listen for occurs, and
> the event loop calls the callback associated with that event. The Task
> manages a coroutine's interaction with the event loop; when the
> coroutine yields a future, the Task instructs the event loop to listen
> for the completion of that future, setting a callback that will resume
> the coroutine.

OK, that (to an extent) explains how the callback and task worlds link together. That's useful.

> The coroutines themselves are not that interesting of an interface;
> all you can do with them is resume them. The asynchronous execution
> done by asyncio is all based on futures. Because a coroutine can
> easily be wrapped in a Task, this allows for coroutines to be used
> anywhere a future is expected.

And yet, futures are callback-based whereas tasks are suspension-point (yield from) based. I get a sense of what you're saying, but it's not very clear yet :-)

> I don't know if I've done a good job explaining, but I hope this helps.

It's all helping. It's a shame the docs don't do a better job of explaining the underlying model, to be honest, but I do feel like I'm slowly getting closer to an understanding.

It's no wonder terminology is such a hot topic in the discussion of the new async PEP on python-dev :-(

Paul

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web