Path: csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: Ian Kelly <ian.g.kelly@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: [Python-ideas] How the heck does async/await work in Python 3.5
Date: Wed, 24 Feb 2016 08:41:25 -0700
Lines: 76
Message-ID: <mailman.97.1456328534.20994.python-list@python.org>
References: <mailman.224.1455746935.22075.python-list@python.org> <56c7d145$0$1597$c3e8da3$5496439d@news.astraweb.com> <na92c0$n65$1@dont-email.me> <56CCC98C.5060504@mail.de> <CAH0mxTQE9xn3OZJ8s-oj13LtQOYGGW2Y1uC9xM3YpzAtY8pqJQ@mail.gmail.com> <CAMpsgwbO1v7318u9VZCTrPHOERAAQiEAHJ9ChWCVqYP-7aqskw@mail.gmail.com> <mailman.95.1456326057.20994.python-list@python.org> <87vb5ejfnv.fsf@elektro.pacujo.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
In-Reply-To: <87vb5ejfnv.fsf@elektro.pacujo.net>
Precedence: list
Xref: csiph.com comp.lang.python:103448

On Wed, Feb 24, 2016 at 8:23 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Tem Pl <rtempl31@gmail.com>:
>
>> Here are some concurrency benchmarks for python vs other languages.
>>
>> https://github.com/atemerev/skynet/pull/53
>>
>> Is there something wrong with this implementation?
>
> It's a "fork bomb".

Isn't that the point of the benchmark?

But yeah, I tried playing with this and here are my results. It's
clearly pushing all those coroutines down into the event loop that
causes the problem. If you remove the asyncio.as_completed calls and
just await the descendant coroutines in order, then it's a lot faster
-- but this may be technically cheating because then the coroutines
are effectively just running recursively without involving the event
loop, so there's no exercise of actual concurrency.

# Straight-up recursion: 400 ms
def func(level=0, index=0):
    if level < LEVELS:
        sons = [func(level=level+1, index=index*SONS + x) for x in range(SONS)]
        sum_ = 0
        for son in sons:
            sum_ += son
        return sum_
    else:
        return index

import timeit
print(timeit.repeat('func()', 'from __main__ import func', number=10))
# [4.016848850995302, 4.1330014310078695, 4.149791953997919]

# Concurrent coroutines with event loop: 30 s
import asyncio
async def coroutine(level=0, index=0):
    if level < LEVELS:
        sons = [coroutine(level=level+1, index=index*SONS + x) for x
in range(SONS)]
        sum_ = 0
        for f in asyncio.as_completed(sons):
            got = await f
            sum_ += got
        return sum_
    else:
        return index

print(timeit.repeat('loop.run_until_complete(coroutine())', 'from
__main__ import coroutine; import asyncio; loop =
asyncio.get_event_loop()', number=1))
# [29.884846250002738, 30.26590966898948, 30.716448744002264]

# Recursion with coroutines: 600 ms
async def coro2(level=0, index=0):
    if level < LEVELS:
        sons = [coro2(level=level+1, index=index*SONS + x) for x in range(SONS)]
        sum_ = 0
        for son in sons:
            got = await son
            sum_ += got
        return sum_
    else:
        return index

print(timeit.repeat('loop.run_until_complete(coro2())', 'from __main__
import coro2; import asyncio; loop = asyncio.get_event_loop()',
number=1))
# [0.6264017040084582, 0.590631059021689, 0.5807875629980117]

# Recursive coroutines, no event loop: 600 ms
print(timeit.repeat('list(coro2().__await__())', 'from __main__ import
coro2', number=10))
# [6.028770218021236, 6.002665672975127, 5.987304503010819]