Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #103448

Re: [Python-ideas] How the heck does async/await work in Python 3.5

Show key headers only | View raw

On Wed, Feb 24, 2016 at 8:23 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Tem Pl <rtempl31@gmail.com>:
>
>> Here are some concurrency benchmarks for python vs other languages.
>>
>> https://github.com/atemerev/skynet/pull/53
>>
>> Is there something wrong with this implementation?
>
> It's a "fork bomb".

Isn't that the point of the benchmark?

But yeah, I tried playing with this and here are my results. It's
clearly pushing all those coroutines down into the event loop that
causes the problem. If you remove the asyncio.as_completed calls and
just await the descendant coroutines in order, then it's a lot faster
-- but this may be technically cheating because then the coroutines
are effectively just running recursively without involving the event
loop, so there's no exercise of actual concurrency.

# Straight-up recursion: 400 ms
def func(level=0, index=0):
    if level < LEVELS:
        sons = [func(level=level+1, index=index*SONS + x) for x in range(SONS)]
        sum_ = 0
        for son in sons:
            sum_ += son
        return sum_
    else:
        return index

import timeit
print(timeit.repeat('func()', 'from __main__ import func', number=10))
# [4.016848850995302, 4.1330014310078695, 4.149791953997919]

# Concurrent coroutines with event loop: 30 s
import asyncio
async def coroutine(level=0, index=0):
    if level < LEVELS:
        sons = [coroutine(level=level+1, index=index*SONS + x) for x
in range(SONS)]
        sum_ = 0
        for f in asyncio.as_completed(sons):
            got = await f
            sum_ += got
        return sum_
    else:
        return index

print(timeit.repeat('loop.run_until_complete(coroutine())', 'from
__main__ import coroutine; import asyncio; loop =
asyncio.get_event_loop()', number=1))
# [29.884846250002738, 30.26590966898948, 30.716448744002264]

# Recursion with coroutines: 600 ms
async def coro2(level=0, index=0):
    if level < LEVELS:
        sons = [coro2(level=level+1, index=index*SONS + x) for x in range(SONS)]
        sum_ = 0
        for son in sons:
            got = await son
            sum_ += got
        return sum_
    else:
        return index

print(timeit.repeat('loop.run_until_complete(coro2())', 'from __main__
import coro2; import asyncio; loop = asyncio.get_event_loop()',
number=1))
# [0.6264017040084582, 0.590631059021689, 0.5807875629980117]

# Recursive coroutines, no event loop: 600 ms
print(timeit.repeat('list(coro2().__await__())', 'from __main__ import
coro2', number=10))
# [6.028770218021236, 6.002665672975127, 5.987304503010819]

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

How the heck does async/await work in Python 3.5 Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-02-17 22:08 +0000
  Re: How the heck does async/await work in Python 3.5 Steven D'Aprano <steve@pearwood.info> - 2016-02-20 13:36 +1100
    Re: How the heck does async/await work in Python 3.5 Rustom Mody <rustompmody@gmail.com> - 2016-02-19 21:24 -0800
      Re: How the heck does async/await work in Python 3.5 Rustom Mody <rustompmody@gmail.com> - 2016-02-19 21:34 -0800
        Re: How the heck does async/await work in Python 3.5 Paul Rubin <no.email@nospam.invalid> - 2016-02-19 22:44 -0800
          Re: How the heck does async/await work in Python 3.5 Steven D'Aprano <steve@pearwood.info> - 2016-02-21 18:17 +1100
            Re: How the heck does async/await work in Python 3.5 Paul Rubin <no.email@nospam.invalid> - 2016-02-20 23:34 -0800
      Re: How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-20 00:48 -0700
      Re: How the heck does async/await work in Python 3.5 Chris Angelico <rosuav@gmail.com> - 2016-02-20 18:57 +1100
      Re: How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-20 01:14 -0700
      Re: How the heck does async/await work in Python 3.5 Chris Angelico <rosuav@gmail.com> - 2016-02-20 19:49 +1100
      Re: How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-20 02:11 -0700
      Re: How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-20 02:21 -0700
    Re: How the heck does async/await work in Python 3.5 Christian Gollwitzer <auriocus@gmx.de> - 2016-02-20 07:53 +0100
      Re: How the heck does async/await work in Python 3.5 "Sven R. Kunze" <srkunze@mail.de> - 2016-02-22 23:16 +0100
      Re: How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-22 17:48 -0700
      Re: How the heck does async/await work in Python 3.5 "Sven R. Kunze" <srkunze@mail.de> - 2016-02-23 17:50 +0100
      Re: How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-23 10:37 -0700
      Re: How the heck does async/await work in Python 3.5 "Sven R. Kunze" <srkunze@mail.de> - 2016-02-23 20:42 +0100
      Re: How the heck does async/await work in Python 3.5 "Sven R. Kunze" <srkunze@mail.de> - 2016-02-23 22:05 +0100
      Re: [Python-ideas] How the heck does async/await work in Python 3.5 "Joao S. O. Bueno" <jsbueno@python.org.br> - 2016-02-23 18:25 -0300
      Re: [Python-ideas] How the heck does async/await work in Python 3.5 Paul Moore <p.f.moore@gmail.com> - 2016-02-24 09:59 +0000
      Re: [Python-ideas] How the heck does async/await work in Python 3.5 Victor Stinner <victor.stinner@gmail.com> - 2016-02-24 11:01 +0100
      Re: [Python-ideas] How the heck does async/await work in Python 3.5 王珺 <wjun77@gmail.com> - 2016-02-24 18:40 +0800
        Re: [Python-ideas] How the heck does async/await work in Python 3.5 Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2016-02-25 10:00 +1300
          Re: [Python-ideas] How the heck does async/await work in Python 3.5 王珺 <wjun77@gmail.com> - 2016-02-25 08:40 +0800
          Re: [Python-ideas] How the heck does async/await work in Python 3.5 Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-02-24 20:37 -0500
      Re: [Python-ideas] How the heck does async/await work in Python 3.5 Tem Pl <rtempl31@gmail.com> - 2016-02-24 06:39 -0800
        Re: [Python-ideas] How the heck does async/await work in Python 3.5 Marko Rauhamaa <marko@pacujo.net> - 2016-02-24 17:23 +0200
          Re: [Python-ideas] How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-24 08:41 -0700
            Re: [Python-ideas] How the heck does async/await work in Python 3.5 Marko Rauhamaa <marko@pacujo.net> - 2016-02-24 18:13 +0200
              Re: [Python-ideas] How the heck does async/await work in Python 3.5 Ian Kelly <ian.g.kelly@gmail.com> - 2016-02-24 09:47 -0700

csiph-web