Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #99367

Re: Futex hang when running event loop on a separated thread

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From Marc Aymerich <glicerinu@gmail.com>
Newsgroups comp.lang.python
Subject Re: Futex hang when running event loop on a separated thread
Date Tue, 24 Nov 2015 17:43:59 +0100
Lines 96
Message-ID <mailman.16.1448383462.20593.python-list@python.org> (permalink)
References <CA+DCN_s77howCh5ptdGwtsF9Z3-r0nRhu-q2t43jdXKTdqSMGw@mail.gmail.com> <CA+DCN_vp8nFS-gOrQMYYefhVt2W0oAP+rFcKQbQ7K+hOHwvuuQ@mail.gmail.com>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
X-Trace news.uni-berlin.de RHeE1ecH0x8aNHulgxhvFABVB9vdE4hw64xqgsdt9m7Q==
Return-Path <glicerinu@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'received:209.85.223': 0.03; 'elif': 0.04; 'handler': 0.04; 'finally:': 0.05; 'ignored': 0.05; 'python3': 0.05; '__name__': 0.07; 'exit': 0.07; 'skip:/ 10': 0.07; 'valueerror:': 0.07; 'work!': 0.07; '22,': 0.09; '__init__': 0.09; 'block.': 0.09; 'thread': 0.10; 'def': 0.13; "skip:' 30": 0.15; "'__main__':": 0.16; '24,': 0.16; 'deadlock': 0.16; 'fuse': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'subject:when': 0.16; 't.join()': 0.16; 'threading': 0.16; 'timeout):': 0.16; 'url:thread': 0.16; 'wrote:': 0.16; 'try:': 0.18; '2015': 0.20; 'skip:" 30': 0.20; 'skip:" 40': 0.20; 'appears': 0.23; 'second': 0.24; 'tried': 0.24; 'import': 0.24; '(most': 0.24; 'header:In-Reply-To:1': 0.24; "i've": 0.25; "doesn't": 0.26; 'skip:" 20': 0.26; 'skip:_ 20': 0.26; 'message- id:@mail.gmail.com': 0.27; 'idea': 0.28; 'marc': 0.29; 'separated': 0.29; 'thread,': 0.29; "i'm": 0.30; '[1]': 0.32; 'run': 0.33; 'url:python': 0.33; 'dies': 0.33; 'null': 0.33; 'traceback': 0.33; "i'll": 0.33; 'tue,': 0.34; 'file': 0.34; 'running': 0.34; 'server': 0.34; 'received:google.com': 0.35; 'nov': 0.35; 'supports': 0.35; 'but': 0.36; 'url:org': 0.36; 'received:209.85': 0.36; 'url:library': 0.36; 'to:addr:python- list': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'doing': 0.38; 'received:209': 0.38; 'skip:o 20': 0.38; 'hi,': 0.38; 'to:addr:python.org': 0.40; 'still': 0.40; 'url:3': 0.60; 'waiting': 0.60; 'situation': 0.67; 'finally': 0.70; '4:29': 0.84; 'definitive': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=Q+gJdZBClI4nY39h/tLOGZuROb8EJafvV1xHlCpHDf8=; b=WF4uyEJfMu5P9lmBZsoSPTeP7wsCmWDBtcWxOYPrvyKh9ZzW4rMMRaaX0QsqDWdw/5 gifwUN6PG7bZ0gPMkR20ttuEdiAqW1rpKzhtget743KK4c/WXlc6hTvoVEJ+oKvdp/xx E5kzygdDoHTw+5NgAXvvdBIUquUxAhWpteRgJojW4cU0CvpJM53SJHtCP6hX1xAQZVwX LU9KgKpWT+rfIZJITZaiR1zWbGap9SEyAjNKX1hgMZgAs4hj3G3dTl20YgpB3F6h4KYd 1RvqEcL7TLRu2n53QsqaSnksFW6KP1Z61o4o4fBu0cKq+BEqn2ZsW0NO4KEmrnAQyN4l bTQQ==
X-Received by 10.107.167.18 with SMTP id q18mr29921765ioe.141.1448383459273; Tue, 24 Nov 2015 08:44:19 -0800 (PST)
In-Reply-To <CA+DCN_vp8nFS-gOrQMYYefhVt2W0oAP+rFcKQbQ7K+hOHwvuuQ@mail.gmail.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.20+
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref csiph.com comp.lang.python:99367

Show key headers only | View raw


threading supports the 'daemon' option[1], when set to True and the
program is in *foreground* then the event-loop thread dies when
SIGTERM-ed, however, if the program is in the *background* it doesn't
work! still deadlocked :'''(

while I'm not finding a definitive solution I'll be doing a
os.kill(os.getpid(), signal.SIGKILL) inside the finally block.

[1] https://docs.python.org/3.5/library/threading.html#threading.Thread.daemon

On Tue, Nov 24, 2015 at 4:46 PM, Marc Aymerich <glicerinu@gmail.com> wrote:
> On Tue, Nov 24, 2015 at 4:29 PM, Marc Aymerich <glicerinu@gmail.com> wrote:
>> Hi,
>>
>> I have to run the asyncio.loop on a separated thread because the main
>> thread is running FUSE. Apparently fuse needs to run on the main
>> thread because it uses signal():
>>
>> ....
>> File "/usr/local/lib/python3.4/dist-packages/fuse.py", line 390, in __init__
>>     old_handler = signal(SIGINT, SIG_DFL)
>> ValueError: signal only works in main thread
>>
>>
>> Anyway, when I exit the program it appears that i run into a deadlock
>> with the eventloop thread, strace is stuck with:
>>
>> ....
>> futex(0x7f7ba0000c10, FUTEX_WAIT_PRIVATE, 0, NULL
>>
>>
>> I've tried to stop the event loop from the main thread but the
>> situation is exactly the same :(
>>
>> loop_container = {}
>> handler = threading.Thread(target=run_loop, args=(loop_container,))
>> try:
>>    handler.start()
>>    FUSE(fs)
>> finally:
>>     loop_container['loop'].stop()
>>     # handler.join()
>>
>> Any idea on how I can shutdown the hole thing? I have to manually kill
>> the program each time :(
>
>
> this can be reproduced with the following program:
>
> import asyncio
> import threading
> import time
>
> def run_loop(loop_container):
>     loop = asyncio.new_event_loop()
>     asyncio.set_event_loop(loop)
>     coro = asyncio.start_server(lambda: 1, '0.0.0.0', 8888, loop=loop)
>     server = loop.run_until_complete(coro)
>     loop_container['loop'] = loop
>     loop.run_forever()
>
> if __name__ == '__main__':
>     loop_container = {}
>     handler = threading.Thread(target=run_loop, args=(loop_container, ))
>     handler.start()
>     try:
>         time.sleep(10000)
>     finally:
>         loop_container['loop'].stop()
>
>
> glic3@XPS13:/tmp$ python3 /tmp/eventtest.py
> ^CTraceback (most recent call last):
>   File "/tmp/eventtest.py", line 22, in <module>
>     time.sleep(10000)
> KeyboardInterrupt
> ^CException ignored in: <module 'threading' from
> '/usr/lib/python3.4/threading.py'>
> Traceback (most recent call last):
>   File "/usr/lib/python3.4/threading.py", line 1294, in _shutdown
>     t.join()
>   File "/usr/lib/python3.4/threading.py", line 1060, in join
>     self._wait_for_tstate_lock()
>   File "/usr/lib/python3.4/threading.py", line 1076, in _wait_for_tstate_lock
>     elif lock.acquire(block, timeout):
> KeyboardInterrupt
>
> it is waiting for tstate_lock in the second ^C
>
> --
> Marc



-- 
Marc

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: Futex hang when running event loop on a separated thread Marc Aymerich <glicerinu@gmail.com> - 2015-11-24 17:43 +0100

csiph-web