Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #89971

Re: asyncio: What is the difference between tasks, futures, and coroutines?

Show key headers only | View raw

On Tue, May 5, 2015 at 10:22 AM, Paul  Moore <p.f.moore@gmail.com> wrote:
> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.

I've been using (and, in part at least, understanding :)) asyncio for
work for the past few months, so I can at least give you my
impressions based on use.

> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".

That's basically how I understand it.

> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

My understanding is that Futures are somewhat like 'non-executable
coroutines', if that makes any sense whatsoever.  Futures are used as
something you can pass around when you need to start execution of the
"job" in one method and finish it in another.  For instance, talking
to a remote network entity, and you want to just do "yield from
server.get(something)".  Your 'get' method can make the request,
create a Future, stick it somewhere that the method monitoring
incoming traffic can find it (along with some identifying metadata),
and then do 'return (yield from future)' to wait for the Future to be
fulfilled (by the listener) and return its result.  The listener then
matches up incoming requests with Futures, and calls
Future.set_result() or Future.set_exception() to fulfill the Future.
Once one of those methods has been called on the Future (and control
has been passed back to the scheduler), your server.get method
unblocks from its '(yield from future)' call, and either raises the
exception or returns the result that was set on the Future.

> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?
>
> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

You kind of mixed things up in this paragraph (you said "Tasks somehow
wrap up coroutines", then "wrap a task in a coroutine"; the first is
more correct, I believe).  As I understand it, Tasks are
specializations of Futures that take care of the
set_result/set_exception based on the execution of the coroutine.  I'm
not clear on how that is all implemented (Are all coroutines wrapped
in Tasks?  Just those wrapped in asyncio.async()? No idea.), but I use
it something like this:

   # save a reference to this task so it can be cancelled elsewhere if need be
   self.current_task = asyncio.async(some_coroutine())
   # now the coroutine is scheduled, but we still have control
   # we can do anything else we need to here, including scheduling
other coroutines
   return (yield from self.current_task)

> I concede that I've not read the rest of the asyncio documentation in much detail yet, and I'm skipping everything to do with IO (I want to understand the "async" bit for now, not so much the "IO" side). But I don't really want to dive into the details while I am this hazy on the basic concepts.
>
> Can anyone clarify for me?

I hope I've done some good on that front, but I'm still a bit hazy
myself.  I found it worked best to just take examples or otherwise
working code, and poke and prod at it until it breaks and you can
figure out how you broke it.  Just try to avoid mixing asyncio and
threads for as long as you can :).

-- 
Zach

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

asyncio: What is the difference between tasks, futures, and coroutines? Paul  Moore <p.f.moore@gmail.com> - 2015-05-05 08:22 -0700
  Re: asyncio: What is the difference between tasks, futures, and coroutines? Zachary Ware <zachary.ware+pylist@gmail.com> - 2015-05-05 11:11 -0500
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Paul  Moore <p.f.moore@gmail.com> - 2015-05-05 10:55 -0700
  Re: asyncio: What is the difference between tasks, futures, and coroutines? Terry Reedy <tjreedy@udel.edu> - 2015-05-05 13:15 -0400
  Re: asyncio: What is the difference between tasks, futures, and coroutines? Marko Rauhamaa <marko@pacujo.net> - 2015-05-05 20:45 +0300
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-05 21:47 -0700
      Re: asyncio: What is the difference between tasks, futures, and coroutines? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-06 15:48 +1000
        Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:06 -0700
          Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 14:33 +1000
            Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:53 -0700
              Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:55 -0700
              Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 15:09 +1000
                Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 23:36 -0700
                Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 16:42 +1000
                Re: asyncio: What is the difference between tasks, futures, and coroutines? Dave Angel <davea@davea.name> - 2015-05-08 07:53 -0400
                Re: asyncio: What is the difference between tasks, futures, and coroutines? Chris Angelico <rosuav@gmail.com> - 2015-05-08 23:02 +1000
      Re: asyncio: What is the difference between tasks, futures, and coroutines? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-05-06 09:11 -0400
        Re: asyncio: What is the difference between tasks, futures, and coroutines? Rustom Mody <rustompmody@gmail.com> - 2015-05-07 21:20 -0700
  Re: asyncio: What is the difference between tasks, futures, and coroutines? Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-05 11:46 -0600
    Re: asyncio: What is the difference between tasks, futures, and coroutines? Paul  Moore <p.f.moore@gmail.com> - 2015-05-05 11:03 -0700
  Re: asyncio: What is the difference between tasks, futures, and coroutines? Skip Montanaro <skip.montanaro@gmail.com> - 2015-05-05 12:55 -0500
  Re: asyncio: What is the difference between tasks, futures, and coroutines? Terry Reedy <tjreedy@udel.edu> - 2015-05-05 18:38 -0400

csiph-web