Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #78343
| Path | csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.fsmpi.rwth-aachen.de!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail |
|---|---|
| From | Marko Rauhamaa <marko@pacujo.net> |
| Newsgroups | comp.lang.python |
| Subject | Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? |
| Date | Thu, 25 Sep 2014 17:47:54 +0300 |
| Organization | A noiseless patient Spider |
| Lines | 48 |
| Message-ID | <87sijf69k5.fsf@elektro.pacujo.net> (permalink) |
| References | <e2f400e6-359e-4070-a822-d734e85e6a75@googlegroups.com> <mailman.14345.1411627213.18130.python-list@python.org> <871tr06nhs.fsf@elektro.pacujo.net> <mailman.14358.1411654608.18130.python-list@python.org> |
| Mime-Version | 1.0 |
| Content-Type | text/plain; charset=us-ascii |
| Injection-Info | mx05.eternal-september.org; posting-host="ff5cf27ef3d5b31f034d3b72bdc27a41"; logging-data="511"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+URBf3mg1WZVFGlPzwj5xS" |
| User-Agent | Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) |
| Cancel-Lock | sha1:wdpQjt4uKIV23yE5Es22Q1MzcZo= sha1:VhGS6bet6eJGyKX9crShbG0/GJA= |
| Xref | csiph.com comp.lang.python:78343 |
Show key headers only | View raw
Ian Kelly <ian.g.kelly@gmail.com>:
> On Thu, Sep 25, 2014 at 3:46 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>> Example (pseudocode):
>>
>> def callback(self):
>> with self.lock:
>> ...
>>
>> def xyz(self, f):
>> with self.lock:
>> ...
>> f.add_done_callback(self.callback)
>>
>> The code will deadlock if the callback is invoked immediately.
>
> Easily solved using a re-entrant lock.
Not easily. Your state transition might not be complete at the time of
the preemption. Many such issues can be resolved by moving the
preemptive function calls to the end of the critical section, but that
is not always possible, easy or taken into account.
If preemption needs to be prepared for, the real solution is:
def xyz(self, f):
with self.lock:
...
def add_callback():
f.add_done_callback(self.callback)
self.executor.schedule(add_callback)
which is ugly, of course.
> Alternatively, one could use synchronization to make sure the future
> can't complete until all callbacks have been added. That would have
> the benefit of keeping the callback processing on the same thread as
> the future, as the OP wanted.
You can still keep the callback in the same thread if your executor is
tied to the the same thread as the calling function. That's the classic
event loop. The executor API should be independent of the implementation
(single vs multithreaded), but it should be able to guarantee that it
never makes callbacks preemptively.
Marko
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Praveen Gollakota <pgollakota@gmail.com> - 2014-09-24 07:44 -0700
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? dieter <dieter@handshake.de> - 2014-09-25 08:14 +0200
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 00:39 -0600
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 12:46 +0300
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 08:16 -0600
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 17:47 +0300
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 09:40 -0600
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Chris Angelico <rosuav@gmail.com> - 2014-09-26 02:04 +1000
csiph-web