Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #78341

Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed1a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <ian.g.kelly@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.024
X-Spam-Evidence '*H*': 0.95; '*S*': 0.00; 'subject:same': 0.07; '"if': 0.09; 'callback': 0.09; 'caller': 0.09; 'def': 0.12; 'thread': 0.14; 'added.': 0.16; 'subject: \n ': 0.16; 'subject:run': 0.16; 'wanted.': 0.16; 'wrote:': 0.18; 'skip:f 30': 0.19; 'thu,': 0.19; 'example': 0.22; 'otherwise,': 0.22; 'handling': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'message-id:@mail.gmail.com': 0.30; 'code': 0.31; '25,': 0.31; 'bad.': 0.31; 'sep': 0.31; 'subject:the': 0.34; 'could': 0.34; "can't": 0.35; 'received:google.com': 0.35; 'there': 0.35; 'really': 0.36; 'returning': 0.36; 'subject:?': 0.36; 'should': 0.36; 'easily': 0.37; 'to:addr:python-list': 0.38; 'sure': 0.39; 'to:addr:python.org': 0.39; 'called': 0.40; 'future': 0.60; 'ian': 0.60; 'completed': 0.61; 'complete': 0.62; 'guarantee': 0.63; 'benefit': 0.68; 'subject:skip:T 10': 0.68; 'future,': 0.83
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=YATWwjGI2Hs4fAn6lDCHhsPMMbvBlKJDD7BiEFgreXs=; b=XenS9OJ/q7MoijojK0xjBlsXKwa1t2qVLuRRf0au6Kxef+CsxInjLK7P08SrMnCDKP gIqGpVJsUmzcUckR8le8K5i681e+jgvQVk8mRvQCel1/Aay2m53ZOGq2PPQnbQ6EZ/qp 5In08GWf8P9iEQXwn4RVRV21FnAiQKwGyfg0xJd6HUkykImxyvH+hr2Q/MtLmodzWDkO oNRiwE67VqqIlID8RV/F2UWHZ6LQbhDylsCS3fAassJyp7pPPTulk0nM+QLWvKDBtQj+ rlUy4rsFkSc6E/otgN91JffhnDW0Tta85gJd71iEws0Y2SrSzfet2/RyHzI/nonXNydg Vp+A==
X-Received by 10.70.128.199 with SMTP id nq7mr2703879pdb.156.1411654605418; Thu, 25 Sep 2014 07:16:45 -0700 (PDT)
MIME-Version 1.0
In-Reply-To <871tr06nhs.fsf@elektro.pacujo.net>
References <e2f400e6-359e-4070-a822-d734e85e6a75@googlegroups.com> <mailman.14345.1411627213.18130.python-list@python.org> <871tr06nhs.fsf@elektro.pacujo.net>
From Ian Kelly <ian.g.kelly@gmail.com>
Date Thu, 25 Sep 2014 08:16:05 -0600
Subject Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function?
To Python <python-list@python.org>
Content-Type text/plain; charset=UTF-8
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.14358.1411654608.18130.python-list@python.org> (permalink)
Lines 32
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1411654608 news.xs4all.nl 2836 [2001:888:2000:d::a6]:57292
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:78341

Show key headers only | View raw


On Thu, Sep 25, 2014 at 3:46 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Ian Kelly <ian.g.kelly@gmail.com>:
>
>> The documentation for Future.add_done_callback says: "If the future
>> has already completed or been cancelled, fn will be called
>> immediately."
>
> That sounds really bad.
>
> There should be a guarantee that the callback is not called from the
> same thread before returning from Future.add_done_callback. Otherwise,
> the caller may have a really difficult time handling preemption and
> avoiding deadlocks at the same time.
>
> Example (pseudocode):
>
>    def callback(self):
>       with self.lock:
>           ...
>
>    def xyz(self, f):
>       with self.lock:
>           ...
>           f.add_done_callback(self.callback)
>
> The code will deadlock if the callback is invoked immediately.

Easily solved using a re-entrant lock. Alternatively, one could use
synchronization to make sure the future can't complete until all
callbacks have been added. That would have the benefit of keeping the
callback processing on the same thread as the future, as the OP
wanted.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread


Thread

ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Praveen Gollakota <pgollakota@gmail.com> - 2014-09-24 07:44 -0700
  Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? dieter <dieter@handshake.de> - 2014-09-25 08:14 +0200
  Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 00:39 -0600
    Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 12:46 +0300
      Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 08:16 -0600
        Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 17:47 +0300
          Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 09:40 -0600
          Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Chris Angelico <rosuav@gmail.com> - 2014-09-26 02:04 +1000

csiph-web