Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #78345
| References | <e2f400e6-359e-4070-a822-d734e85e6a75@googlegroups.com> <mailman.14345.1411627213.18130.python-list@python.org> <871tr06nhs.fsf@elektro.pacujo.net> <mailman.14358.1411654608.18130.python-list@python.org> <87sijf69k5.fsf@elektro.pacujo.net> |
|---|---|
| From | Ian Kelly <ian.g.kelly@gmail.com> |
| Date | 2014-09-25 09:40 -0600 |
| Subject | Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? |
| Newsgroups | comp.lang.python |
| Message-ID | <mailman.14361.1411659684.18130.python-list@python.org> (permalink) |
On Thu, Sep 25, 2014 at 8:47 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Ian Kelly <ian.g.kelly@gmail.com>:
>
>> On Thu, Sep 25, 2014 at 3:46 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
>>> Example (pseudocode):
>>>
>>> def callback(self):
>>> with self.lock:
>>> ...
>>>
>>> def xyz(self, f):
>>> with self.lock:
>>> ...
>>> f.add_done_callback(self.callback)
>>>
>>> The code will deadlock if the callback is invoked immediately.
>>
>> Easily solved using a re-entrant lock.
>
> Not easily. Your state transition might not be complete at the time of
> the preemption. Many such issues can be resolved by moving the
> preemptive function calls to the end of the critical section, but that
> is not always possible, easy or taken into account.
Fair enough. In the simple example above, I see no harm in simply
moving the add_done_callback call outside (after) the "with self.lock"
block altogether. I can imagine more complex examples where the call
is further down the stack and harder to isolate from the lock,
although I would argue that's probably a code smell.
> If preemption needs to be prepared for, the real solution is:
>
>
> def xyz(self, f):
> with self.lock:
> ...
> def add_callback():
> f.add_done_callback(self.callback)
> self.executor.schedule(add_callback)
>
> which is ugly, of course.
YMMV, but I think I would prefer to make the executor.schedule call be
the callback:
def callback(self, f):
with self.lock:
...
def xyz(self, f):
with self.lock:
...
f.add_done_callback(
functools.partial(self.executor.schedule, self.callback))
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Praveen Gollakota <pgollakota@gmail.com> - 2014-09-24 07:44 -0700
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? dieter <dieter@handshake.de> - 2014-09-25 08:14 +0200
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 00:39 -0600
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 12:46 +0300
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 08:16 -0600
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Marko Rauhamaa <marko@pacujo.net> - 2014-09-25 17:47 +0300
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Ian Kelly <ian.g.kelly@gmail.com> - 2014-09-25 09:40 -0600
Re: ThreadPoolExecutor - callback guaranteed to run in same thread as the submitted function? Chris Angelico <rosuav@gmail.com> - 2014-09-26 02:04 +1000
csiph-web