Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #74478

Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!weretis.net!feeder4.news.weretis.net!ecngs!feeder2.ecngs.de!novso.com!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <khamenya@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.093
X-Spam-Evidence '*H*': 0.82; '*S*': 0.00; 'generators': 0.09; 'subject: \n ': 0.16; 'subject:tasks': 0.16; 'tasks,': 0.16; 'accepted.': 0.22; 'task': 0.26; 'message-id:@mail.gmail.com': 0.30; 'once,': 0.31; 'lists': 0.32; 'running': 0.33; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'subject:?': 0.36; 'hi,': 0.36; 'similar': 0.36; 'tasks': 0.38; 'to:addr:python- list': 0.38; 'anything': 0.39; 'to:addr:python.org': 0.39; 'address': 0.63; 'pool?': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=GryAOpaAojfSNIYoIxajVsQ2xvVVKDlcA/YZ3G6ZxxI=; b=t4KVgyXmvJ1/Q30k0PNjN+mcQjc6q6GFbbPwOBp4zU/qrrNuXlz9Ct5cedkyWtoyRo LbsiMxbJ6Bhjr1cSPU6Ta0HjdohulVUvS4v+0IGzo0pwUbFYdrKgD2Acrc6bKwbg4FDt 5tvBCScNp4PJkVvPasC1X2TXzhIuo7aXHJjQStxuusIg+cgcIXu0O2zyRxG31i6gMEAY 5Cbmu9jJTlBRzl3yDjXxujvb2NIeo2iRvXPLJ0ilzZ37uqfMzZ30VycdzPVXc+XObRyt H2ZpsBeRlnGwD2hazUSTsVILbJH3lB0B2gLKwUG5LGkZoENr9NSwFqpjBfHMBHfVcF1R xm1w==
X-Received by 10.140.18.243 with SMTP id 106mr33091468qgf.105.1405426860434; Tue, 15 Jul 2014 05:21:00 -0700 (PDT)
MIME-Version 1.0
From Valery Khamenya <khamenya@gmail.com>
Date Tue, 15 Jul 2014 14:20:40 +0200
Subject Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks?
To python-list@python.org
Content-Type multipart/alternative; boundary=001a1135487ec6895804fe3a70ab
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.11834.1405426868.18130.python-list@python.org> (permalink)
Lines 36
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1405426868 news.xs4all.nl 2844 [2001:888:2000:d::a6]:47597
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:74478

Show key headers only | View raw


[Multipart message — attachments visible in raw view] - view raw

Hi,

both asyncio.as_completed() and asyncio.wait() work with lists only. No
generators are accepted. Are there anything similar to those functions that
pulls Tasks/Futures/coroutines one-by-one and processes them in a limited
task pool?

I have gazillion of Tasks, and do not want to instantiate them all at once,
but to instantiate and to address them one by one as the running tasks are
completed.

best regards
--
Valery

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks? Valery Khamenya <khamenya@gmail.com> - 2014-07-15 14:20 +0200

csiph-web