Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.048 X-Spam-Evidence: '*H*': 0.90; '*S*': 0.00; 'feasible.': 0.09; 'subject:skip:c 10': 0.09; 'thread': 0.14; '>>': 0.16; 'block.': 0.16; 'blocking': 0.16; 'caller.': 0.16; 'coroutines': 0.16; 'executor': 0.16; 'wrote:': 0.18; 'aug': 0.22; 'email addr:gmail.com>': 0.22; 'putting': 0.22; 'separate': 0.22; 'skip:e 30': 0.24; '>': 0.26; 'pass': 0.26; 'header:In-Reply- To:1': 0.27; 'function': 0.29; 'am,': 0.29; 'needed.': 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'critical': 0.32; 'run': 0.32; 'framework': 0.33; 'sense': 0.34; 'could': 0.34; "can't": 0.35; 'objects': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'done,': 0.36; 'yield': 0.36; 'tasks': 0.38; 'whatever': 0.38; 'to:addr:python-list': 0.38; 'pm,': 0.38; 'expect': 0.39; 'skip:& 20': 0.39; '12,': 0.39; 'to:addr:python.org': 0.39; 'ian': 0.60; 'course': 0.61; 'back': 0.62; 'holding': 0.65; 'apart': 0.72; 'protect': 0.79; 'asynchronous': 0.84; 'subject:Using': 0.84 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=/bif5981Sq+Bb0fgBtLY/nZMWHQfDCRrpucgyEVleyE=; b=XJqxXrdB19vtkmq0Nl0p8Q093s8BiJrTc1fDy1c+swbUPgGJ4FXbRVDdzVKcyMu8pW ejMkMmMcewviW6u7eHTCY/s/dCpEfNEUhU+wskTBfjW4ZCM2mh6oWwsmwMtgs2CHG8Mc g6cU9Q6vhGummrTqyTFzlyKFUQziR7eS15JPy5ItajHtO0wH/kQ1xJOoXm1om033gnU6 evEBOTVWnSGXVP50R3X9a/jYWejZbYoGxIPvbYPlLwd73n2AJi/OTY4814U/Z/7vo5fd u5eXIFYVkXLKiTQ40yfQ54hvbnNvw019eRAcIMj1oAuhBlaku46P4JX1uICDd1o9FHd+ RR7Q== X-Received: by 10.70.102.175 with SMTP id fp15mr2250335pdb.52.1407909863041; Tue, 12 Aug 2014 23:04:23 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <8738d1dlru.fsf@elektro.pacujo.net> References: <8738d1dlru.fsf@elektro.pacujo.net> From: Ian Kelly Date: Wed, 13 Aug 2014 00:03:42 -0600 Subject: Re: Using asyncio workers in a `concurrent.futures` interface To: Python Content-Type: multipart/alternative; boundary=001a11c2fce04330d605007c8f50 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 69 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1407909872 news.xs4all.nl 2862 [2001:888:2000:d::a6]:35787 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:76170 --001a11c2fce04330d605007c8f50 Content-Type: text/plain; charset=UTF-8 On Tue, Aug 12, 2014 at 11:03 PM, Marko Rauhamaa wrote: > > Ian Kelly : > > > On Tue, Aug 12, 2014 at 11:02 AM, cool-RR wrote: > >> And that's it, no coroutines, no `yield from`. Since, if I understand > >> correctly, asyncio requires a mainloop, it would make sense for the > >> AsyncIOExecutor to have a thread of its own in which it could run its > >> mainloop. > > > > I think that putting the event loop in a separate thread would be > > necessary if the intention is that the executor be invoked from > > outside. I'm not aware of this work having been done, but it sounds > > perfectly feasible. > > Multithreading will require normal locking to protect critical sections. > Care must be taken to never yield while holding a threading lock. You can use event_loop.call_soon_threadsafe() to schedule tasks and callbacks on the event loop, and Queue objects to pass futures back to the caller. Apart from whatever synchronization those use internally, I don't think any locking would be needed. However, it would of course require that the "functions" passed in to the executor be coroutines in actuality. You just can't pass a blocking function into an asynchronous framework and expect it to magically not block. --001a11c2fce04330d605007c8f50 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Tue, Aug 12, 2014 at 11:03 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>
&= gt; Ian Kelly <ian.g.kelly@gmai= l.com>:
>
> > On Tue, Aug 12, 2014 at 11:02 AM, cool-RR <ram.rachum@gmail.com> wrote:
> &g= t;> And that's it, no coroutines, no `yield from`. Since, if I under= stand
> >> correctly, asyncio requires a mainloop, it would make sense f= or the
> >> AsyncIOExecutor to have a thread of its own in whic= h it could run its
> >> mainloop.
> >
> > I t= hink that putting the event loop in a separate thread would be
> > necessary if the intention is that the executor be invoked from> > outside. I'm not aware of this work having been done, but i= t sounds
> > perfectly feasible.
>
> Multithreading wi= ll require normal locking to protect critical sections.
> Care must be taken to never yield while holding a threading lock.
<= br>
You can use event_loop.call_soon_threadsafe() to schedule tasks and= callbacks on the event loop, and Queue objects to pass futures back to the= caller. Apart from whatever synchronization those use internally, I don= 9;t think any locking would be needed.

However, it would of course require that the "func= tions" passed in to the executor be coroutines in actuality. You just = can't pass a blocking function into an asynchronous framework and expec= t it to magically not block.
--001a11c2fce04330d605007c8f50--