Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Marc Aymerich <glicerinu@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Futex hang when running event loop on a separated thread
Date: Tue, 24 Nov 2015 21:22:49 +0100
Lines: 130
Message-ID: <mailman.32.1448396592.20593.python-list@python.org>
References: <CA+DCN_s77howCh5ptdGwtsF9Z3-r0nRhu-q2t43jdXKTdqSMGw@mail.gmail.com> <CA+DCN_vp8nFS-gOrQMYYefhVt2W0oAP+rFcKQbQ7K+hOHwvuuQ@mail.gmail.com> <CAKJDb-MY9W518kKhFf8vbZAfFad=uGXefDePf1KLpUJV9H3Rpg@mail.gmail.com> <CA+DCN_uC2k=rJxmtfEideAhGj1=2XVq8VHGU=OUkB9yHxV7bPw@mail.gmail.com> <CAKJDb-N5WD_cN2k0kd_aXXyj_2Bsjxye8Q34_3awtw7bGPRx5w@mail.gmail.com> <CA+DCN_uNHNHrBOSmd0JE58Ltt6HzrDWU_DhW_1N3dFMAVNpVOQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
In-Reply-To: <CA+DCN_uNHNHrBOSmd0JE58Ltt6HzrDWU_DhW_1N3dFMAVNpVOQ@mail.gmail.com>
Precedence: list
Xref: csiph.com comp.lang.python:99393

On Tue, Nov 24, 2015 at 9:17 PM, Marc Aymerich <glicerinu@gmail.com> wrote:
> On Tue, Nov 24, 2015 at 8:41 PM, Zachary Ware
> <zachary.ware+pylist@gmail.com> wrote:
>> On Tue, Nov 24, 2015 at 12:37 PM, Marc Aymerich <glicerinu@gmail.com> wrote:
>>> still it appears to work only if the main thread is in the foreground
>>> (as of calling Thread() with deamon=True), I don't get why it behaves
>>> differently :( maybe it is waiting for other stuff, but no idea how to
>>> confirm this with strace of other means.. i always see the same
>>> 'futex(0x7f9a70000c10, FUTEX_WAIT_PRIVATE, 0, NULL'
>>
>> What's your exact version of Python?  asyncio is still evolving
>> rapidly, and may behave differently between patch releases (say,
>> between 3.4.1 and 3.4.3).
>>
>> I have tried out your reproducer with my fix, and I can't provoke a
>> hang using Python 3.4.3 on either OSX or Ubuntu Trusty.  Can you give
>> some more specific directions to reproduce?
>>
>> --
>> Zach
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Yep, I'm a bit lost about how fuse implements going into background...
> so the only way I have to reproduce this so far is by using fuse :(
>
> # pip3 install fusepy
> import asyncio
> import threading
> import time
> import os
> import sys
> from fuse import FUSE, Operations
>
> def run_loop(loop_container):
>     loop = asyncio.new_event_loop()
>     asyncio.set_event_loop(loop)
>     coro = asyncio.start_server(lambda: 1, '0.0.0.0', 8888, loop=loop)
>     server = loop.run_until_complete(coro)
>     loop_container['loop'] = loop
>     loop.run_forever()
>
> if __name__ == '__main__':
>     mountpoint = sys.argv[1]
>     loop_container = {}
>     handler = threading.Thread(target=run_loop, args=(loop_container,))
>     handler.start()
>     try:
>         # with foreground=True works
>         FUSE(Operations(), mountpoint, foreground=False)
>     finally:
>         loop_container['loop'].call_soon_threadsafe(loop_container['loop'].stop)
>
>
> $ mkdir /tmp/7777
> $ python3.5 /tmp/eventtest.py /tmp/7777
>
> # Another terminal
> $ ps aux | grep python3.5
> glic3    19983  0.5  0.1 215540 10772 ?        Ssl  21:05   0:00
> python3.5 me.py /tmp/7777
> $ sudo strace -p 19983
> Process 19983 attached - interrupt to quit
> futex(0x7fff42affca0, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
>
> # Here i fire
> $ fusermount -u /tmp/7777
>
> tgkill(19983, 19985, SIGRTMIN)          = 0
> futex(0x7fff42affc30, FUTEX_WAKE_PRIVATE, 1) = 0
> futex(0x7f9840afc9d0, FUTEX_WAIT, 19984, NULL
>
>
> I've just tried with python3.5 and the same behavior. The thing is
> that with foreground=True it just works.
> thank you for your time!


In case this could be of some use:
strace using foreground=True

$ sudo strace -p 20326
Process 20326 attached - interrupt to quit
futex(0x7fffd9b8e5a0, FUTEX_WAIT_PRIVATE, 0, NULL
# Here umount command
) = 0
tgkill(20326, 20334, SIGRTMIN)          = 0
tgkill(20326, 20335, SIGRTMIN)          = 0
futex(0x7fffd9b8e530, FUTEX_WAKE_PRIVATE, 1) = 0
rt_sigaction(SIGHUP, NULL, {0x7fb60d850540, [], SA_RESTORER,
0x7fb61106ccb0}, 8) = 0
rt_sigaction(SIGINT, NULL, {0x7fb60d850540, [], SA_RESTORER,
0x7fb61106ccb0}, 8) = 0
rt_sigaction(SIGTERM, NULL, {0x7fb60d850540, [], SA_RESTORER,
0x7fb61106ccb0}, 8) = 0
rt_sigaction(SIGPIPE, NULL, {SIG_IGN, [], SA_RESTORER, 0x7fb61106ccb0}, 8) = 0
poll([{fd=5, events=0}], 1, 0)          = 1 ([{fd=5, revents=POLLERR}])
close(5)                                = 0
brk(0x126d000)                          = 0x126d000
rt_sigaction(SIGINT, {0x50b380, [], SA_RESTORER, 0x7fb61106ccb0},
{0x7fb60d850540, [], SA_RESTORER, 0x7fb61106ccb0}, 8) = 0
sendto(8, "\0", 1, 0, NULL, 0)          = 1
futex(0x7fb600001740, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x9d3204, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9d3200, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x9d31c0, FUTEX_WAKE_PRIVATE, 1)  = 1
futex(0x7fb600000c10, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fb61106ccb0},
{0x50b380, [], SA_RESTORER, 0x7fb61106ccb0}, 8) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 7,
{EPOLLWRBAND|EPOLLMSG|EPOLLERR|0xdea9800, {u32=32694,
u64=23586238107778998}}) = 0
close(7)                                = 0
close(8)                                = 0
close(4)                                = 0
getsockname(6, {sa_family=AF_INET, sin_port=htons(8888),
sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
getpeername(6, 0x7fffd9b8dc70, [16])    = -1 ENOTCONN (Transport
endpoint is not connected)
close(6)                                = 0
madvise(0x7fb600033000, 8192, MADV_DONTNEED) = 0
close(3)                                = 0
exit_group(0)                           = ?
Process 20326 detached



-- 
Marc