Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #96281 > unrolled thread

Signal SIGINT ignored during socket.accept

Back to article view | Back to comp.lang.python

Contents

  Signal SIGINT ignored during socket.accept "James Harris" <james.harris.1@gmail.com> - 2015-09-10 19:24 +0100
    Re: Signal SIGINT ignored during socket.accept Chris Angelico <rosuav@gmail.com> - 2015-09-11 04:36 +1000
      Re: Signal SIGINT ignored during socket.accept "James Harris" <james.harris.1@gmail.com> - 2015-09-10 20:11 +0100
        Re: Signal SIGINT ignored during socket.accept Chris Angelico <rosuav@gmail.com> - 2015-09-11 05:26 +1000
          Re: Signal SIGINT ignored during socket.accept "James Harris" <james.harris.1@gmail.com> - 2015-09-10 21:12 +0100
            Re: Signal SIGINT ignored during socket.accept Chris Angelico <rosuav@gmail.com> - 2015-09-11 12:01 +1000
              Re: Signal SIGINT ignored during socket.accept Grant Edwards <invalid@invalid.invalid> - 2015-09-11 13:50 +0000
                Re: Signal SIGINT ignored during socket.accept Marko Rauhamaa <marko@pacujo.net> - 2015-09-11 17:00 +0300
                Re: Signal SIGINT ignored during socket.accept Chris Angelico <rosuav@gmail.com> - 2015-09-12 00:27 +1000
                Re: Signal SIGINT ignored during socket.accept "James Harris" <james.harris.1@gmail.com> - 2015-09-11 18:14 +0100
                  Re: Signal SIGINT ignored during socket.accept "James Harris" <james.harris.1@gmail.com> - 2015-09-12 00:15 +0100

#96281 — Signal SIGINT ignored during socket.accept

I have a listening socket, self.lsock, which is used in an accept() call 
as follows

  endpoint = self.lsock.accept()

The problem is that if control-C is pressed it is not recognised until 
something connects to that socket. Only when the accept() call returns 
is the signal seen.

The question, then, is how to get the signal to break out of the 
accept() call. This is currently on Windows but I would like it to run 
on Unix too. I see from the web that this type of thing is a common 
problem with the underlying C libraries but I cannot quite relate the 
posts I have found to Python.

Any ideas?

James

[toc] | [next] | [standalone]

#96284

On Fri, Sep 11, 2015 at 4:24 AM, James Harris <james.harris.1@gmail.com> wrote:
> I have a listening socket, self.lsock, which is used in an accept() call as
> follows
>
>  endpoint = self.lsock.accept()
>
> The problem is that if control-C is pressed it is not recognised until
> something connects to that socket. Only when the accept() call returns is
> the signal seen.
>
> The question, then, is how to get the signal to break out of the accept()
> call. This is currently on Windows but I would like it to run on Unix too. I
> see from the web that this type of thing is a common problem with the
> underlying C libraries but I cannot quite relate the posts I have found to
> Python.

What version of Python are you using? Also (in case it matters), what
version of Windows?

Have you tested on any Unix system? I just tried on my Linux, and
Ctrl-C interrupted the accept() straight away, so this is quite
probably a Windows-only issue.

Can you produce an absolute minimal demo program? I'd try something like this:

import socket
s = socket.socket()
s.listen(1)
s.accept()

which is what worked for me (interactively, Python 2.7.9 and 3.6.0a0,
Debian Linux).

ChrisA

[toc] | [prev] | [next] | [standalone]

#96290

"Chris Angelico" <rosuav@gmail.com> wrote in message 
news:mailman.332.1441910212.8327.python-list@python.org...
> On Fri, Sep 11, 2015 at 4:24 AM, James Harris 
> <james.harris.1@gmail.com> wrote:

>> I have a listening socket, self.lsock, which is used in an accept() 
>> call as
>> follows
>>
>>  endpoint = self.lsock.accept()
>>
>> The problem is that if control-C is pressed it is not recognised 
>> until
>> something connects to that socket. Only when the accept() call 
>> returns is
>> the signal seen.
>>
>> The question, then, is how to get the signal to break out of the 
>> accept()
>> call. This is currently on Windows but I would like it to run on Unix 
>> too. I
>> see from the web that this type of thing is a common problem with the
>> underlying C libraries but I cannot quite relate the posts I have 
>> found to
>> Python.
>
> What version of Python are you using? Also (in case it matters), what
> version of Windows?

Good point. It turns out that it does matter. I have one implementation 
which fails (Windows) and one which works (Linux). The Linux one breaks 
out on Control-C. The Windows one does not recognise Control-C until the 
accept() call returns.

The implementations are:

$ uname -srm
Linux 3.18.7-v7+ armv7l
$ python -V
Python 2.7.3

And

c:\>ver
Microsoft Windows XP [Version 5.1.2600]
c:\>python -V
Python 2.7.9

> Have you tested on any Unix system? I just tried on my Linux, and
> Ctrl-C interrupted the accept() straight away,

Thanks.

> so this is quite probably a Windows-only issue.

That turns out to be exactly right.

> Can you produce an absolute minimal demo program? I'd try something 
> like this:
>
> import socket
> s = socket.socket()
> s.listen(1)
> s.accept()

Yes:

port = 8880
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("", port))
s.listen(1)
endpoint = s.accept()

> which is what worked for me (interactively, Python 2.7.9 and 3.6.0a0,
> Debian Linux).

On Linux I get

$ python socktest.py
^CTraceback (most recent call last):
  File "socktest.py", line 6, in <module>
    endpoint = s.accept()
  File "/usr/lib/python2.7/socket.py", line 202, in accept
    sock, addr = self._sock.accept()
KeyboardInterrupt
$

On Windows I get

S:\>python socktest.py
Traceback (most recent call last):
  File "socktest.py", line 6, in <module>
    endpoint = s.accept()
  File "C:\Python27\lib\socket.py", line 202, in accept
    sock, addr = self._sock.accept()
KeyboardInterrupt

S:\>

However, on Windows the recognition of Control-C does not happen until 
after something connects to the socket.

I will carry on researching it but maybe the above gives a clue to those 
in the know...!

James

[toc] | [prev] | [next] | [standalone]

#96295

On Fri, Sep 11, 2015 at 5:11 AM, James Harris <james.harris.1@gmail.com> wrote:
> S:\>python socktest.py
> Traceback (most recent call last):
>  File "socktest.py", line 6, in <module>
>    endpoint = s.accept()
>  File "C:\Python27\lib\socket.py", line 202, in accept
>    sock, addr = self._sock.accept()
> KeyboardInterrupt
>
> S:\>
>
> However, on Windows the recognition of Control-C does not happen until after
> something connects to the socket.
>
> I will carry on researching it but maybe the above gives a clue to those in
> the know...!

This is a known problem on Windows. I can't remember what the best
solution was, but there's a chance something got into 2.7.10, as it
was fairly recent. There's a significantly better chance that
something's different in Python 3.x. You may find it worth grabbing a
few different versions of Python and trying the same code on all of
them.

You may run into issues with XP, though. For instance, Python 3.5
doesn't support it, and (IIRC) won't install at all; 3.4 does work, as
will all releases of 2.7.x. Worst case, grab yourself a Windows 7 and
try a few tests.

But a quick test on one of my VMs, with 3.4 on Win 7, didn't show any
change. It's entirely possible that a blocking socket-accept call will
continue to block. There is one rather silly option, and that's to use
select() to effectively poll for Ctrl-C... or, possibly better, have a
separate program that shuts down your server (by connecting to it,
which thus breaks the stranglehold).

Of course, switching over to Unix is also a good option...

ChrisA

[toc] | [prev] | [next] | [standalone]

#96299

"Chris Angelico" <rosuav@gmail.com> wrote in message 
news:mailman.337.1441913195.8327.python-list@python.org...
> On Fri, Sep 11, 2015 at 5:11 AM, James Harris 
> <james.harris.1@gmail.com> wrote:

...

>> However, on Windows the recognition of Control-C does not happen 
>> until after
>> something connects to the socket.

...

> This is a known problem on Windows.

...

> It's entirely possible that a blocking socket-accept call will
> continue to block. There is one rather silly option, and that's to use
> select() to effectively poll for Ctrl-C... or, possibly better, have a
> separate program that shuts down your server (by connecting to it,
> which thus breaks the stranglehold).

Thanks for your help, Chris. Using select() is a very good option. I 
tried first without a timeout and even then this version of Windows does 
not recognise Control-C until after the select() call returns (which 
needs similar prompting as with the accept() call. However, select() 
with a timeout allows the code to work both on Windows and Linux. 
Hooray!

For anyone else who is looking for this the earlier test code was 
changed to

port = 8880
import select
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(0)
s.bind(("", port))
s.listen(1)
while 1:
  ready = select.select((s,), (), (), 0.5)
  #print '(ready %s)' % repr(ready)
  if (ready[0]):
    try:
      endpoint = s.accept()
    except socket.error, details:
      print 'Ignoring socket error:', repr(details)
      continue
    print '(endpoint %s)' % repr(endpoint)
    if endpoint:
      break

James

[toc] | [prev] | [next] | [standalone]

#96313

On Fri, Sep 11, 2015 at 6:12 AM, James Harris <james.harris.1@gmail.com> wrote:
> Thanks for your help, Chris. Using select() is a very good option. I tried
> first without a timeout and even then this version of Windows does not
> recognise Control-C until after the select() call returns (which needs
> similar prompting as with the accept() call. However, select() with a
> timeout allows the code to work both on Windows and Linux. Hooray!
>
> For anyone else who is looking for this the earlier test code was changed to
>
> port = 8880
> import select
> import socket
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.setblocking(0)
> s.bind(("", port))
> s.listen(1)
> while 1:
>  ready = select.select((s,), (), (), 0.5)
>  #print '(ready %s)' % repr(ready)
>  if (ready[0]):
>    try:
>      endpoint = s.accept()
>    except socket.error, details:
>      print 'Ignoring socket error:', repr(details)
>      continue
>    print '(endpoint %s)' % repr(endpoint)
>    if endpoint:
>      break

(Your indentation is looking like a single space, here. I would
suggest indenting a bit more, for readability; but it might just be an
artefact of posting.)

This is what I meant when I said you would be polling. Effectively,
you wake up your program every half-second, check if Ctrl-C has been
pressed, and if it hasn't, you go back to sleep again. This is pretty
inefficient.

Assuming you don't need stdin for any other purpose, one solution
would be to spin off a thread that simply watches for a keyboard
signal. I tested this on Windows 7 with 2.7.10 and 3.4.3, and it
appears to work:

import socket
import threading

# Python 2/3 compat
try: input = raw_input
except NameError: pass

PORT = 8880
mainsock = socket.socket()
mainsock.bind(("", PORT))
mainsock.listen(1)

def console():
    """Constantly read from stdin and discard"""
    try:
        while "moar console": input()
    except (KeyboardInterrupt, EOFError):
        socket.socket().connect(("127.0.0.1",PORT))

threading.Thread(target=console).start()

while "moar sockets":
    s = mainsock.accept()
    print("New connection: %r" % s)
    # Do whatever you want with the connection
    s.close()


As long as you have _some_ thread reading from the console, you can
get woken up by Ctrl-C. When that happens, it simply fires off a quick
socket connection to itself, thus waking up the main thread; and then
the main thread sees the KeyboardInterrupt. (I'm not sure why, but
instead of seeing KeyboardInterrupt in the console thread, I've been
seeing EOFError. Since I don't particularly care to explore the
problem, I just wrote the except clause to catch both.)

This eliminates the polling, but you have to sacrifice stdin to do it.
It may be worth bracketing all of that code with a platform check -
don't bother doing all this unless you're on Windows. Up to you.

Stupid Windows.

ChrisA

[toc] | [prev] | [next] | [standalone]

#96342

On 2015-09-11, Chris Angelico <rosuav@gmail.com> wrote:

> This is what I meant when I said you would be polling. Effectively,
> you wake up your program every half-second, check if Ctrl-C has been
> pressed, and if it hasn't, you go back to sleep again. This is pretty
> inefficient.

Though it offends one's engineering sensibilities[1], it's just not
that inefficient. I'd bet money you won't even be able to measure the
difference in CPU usage. Waking up twice per second and immediately
calling select() again on any hardware/OS built in the past 50 years
is going completely negligible (as long as you can ignore the smell).

Even waking up ten times per second won't be noticeable.

Waking up every millisecond or two might be noticeable.

> Stupid Windows.

Agreed.

[1] If offenses to engineering sensibility were of concern, then
    he wouldn't be using Windows in the first place.  ;)

-- 
Grant Edwards               grant.b.edwards        Yow! PIZZA!!
                                  at               
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#96343

Grant Edwards <invalid@invalid.invalid>:

> On 2015-09-11, Chris Angelico <rosuav@gmail.com> wrote:
>> This is what I meant when I said you would be polling. Effectively,
>> you wake up your program every half-second, check if Ctrl-C has been
>> pressed, and if it hasn't, you go back to sleep again. This is pretty
>> inefficient.
>
> Though it offends one's engineering sensibilities[1], it's just not
> that inefficient. I'd bet money you won't even be able to measure the
> difference in CPU usage. Waking up twice per second and immediately
> calling select() again on any hardware/OS built in the past 50 years
> is going completely negligible (as long as you can ignore the smell).
>
> Even waking up ten times per second won't be noticeable.
>
> Waking up every millisecond or two might be noticeable.

It can add up. In particular, it can prevent the CPU from staying in the
low-power mode, especially on battery-powered devices.

  <URL: https://lwn.net/Articles/549580/>

Another environment where such polling circuses can actually overwhelm a
CPU is virtual machines. When dozens or hundreds of VMs are each polling
left and right, the host may not get much actual work done.


Marko

[toc] | [prev] | [next] | [standalone]

#96344

On Fri, Sep 11, 2015 at 11:50 PM, Grant Edwards <invalid@invalid.invalid> wrote:
> On 2015-09-11, Chris Angelico <rosuav@gmail.com> wrote:
>
>> This is what I meant when I said you would be polling. Effectively,
>> you wake up your program every half-second, check if Ctrl-C has been
>> pressed, and if it hasn't, you go back to sleep again. This is pretty
>> inefficient.
>
> Though it offends one's engineering sensibilities[1], it's just not
> that inefficient. I'd bet money you won't even be able to measure the
> difference in CPU usage. Waking up twice per second and immediately
> calling select() again on any hardware/OS built in the past 50 years
> is going completely negligible (as long as you can ignore the smell).
>
> Even waking up ten times per second won't be noticeable.
>

True (although, as Marko says, it can add up). The other difference,
though, is that I like to keep "cope-with-stupidity" code to itself -
ideally, the clean path shouldn't be infected with it. Waking up
periodically with select(), when you otherwise just want a blocking
accept(), affects all your code. Spinning off a thread that monitors
stdin and signals the main thread when it's time to shut down can be
kept to a single block of code, guarded by some sort of platform
check:

import socket

PORT = 8880
mainsock = socket.socket()
mainsock.bind(("", PORT))
mainsock.listen(1)

if Windows: # however you want to go about checking this
    import threading

    # Python 2/3 compat
    try: input = raw_input
    except NameError: pass

    def console():
        """Constantly read from stdin and discard"""
        try:
            while "moar console": input()
        except (KeyboardInterrupt, EOFError):
            socket.socket().connect(("127.0.0.1",PORT))

    threading.Thread(target=console).start()

while "moar sockets":
    s = mainsock.accept()
    print("New connection: %r" % s)
    # Do whatever you want with the connection
    s.close()


As well as keeping the Unix variant stupidity-free, this allows you to
vary your platform check in the event that a future version of Windows
allows blocking calls to be interrupted. Or if a future version of
Python implements this kind of check globally. Or if a Python built
with Cygwin doesn't exhibit this behaviour. Or if PyPy... you get the
idea. The clean path is clearly delineated, and hopefully, the
complexity of your code will increase linearly with the number of
problems you're coping with, rather than having each patch affect each
other.

ChrisA

[toc] | [prev] | [next] | [standalone]

#96359

"Grant Edwards" <invalid@invalid.invalid> wrote in message 
news:msum6c$hv$1@reader1.panix.com...
> On 2015-09-11, Chris Angelico <rosuav@gmail.com> wrote:
>
>> This is what I meant when I said you would be polling. Effectively,
>> you wake up your program every half-second, check if Ctrl-C has been
>> pressed, and if it hasn't, you go back to sleep again. This is pretty
>> inefficient.
>
> Though it offends one's engineering sensibilities[1], it's just not
> that inefficient.

Indeed.

> I'd bet money you won't even be able to measure the
> difference in CPU usage.

You would win your bet.

> Waking up twice per second and immediately
> calling select() again on any hardware/OS built in the past 50 years
> is going completely negligible (as long as you can ignore the smell).

CPU usage is not the only issue but it is a consideration. I tested it 
before posting the code and couldn't see any significant increase in CPU 
use. To give it a better test I have just left running for a couple of 
hours or so. I am pleased to say it currently reports a cumulative total 
of 0:00:00, i.e. it has not clocked up even a second of CPU time yet!

...

> [1] If offenses to engineering sensibility were of concern, then
>    he wouldn't be using Windows in the first place.  ;)

LOL. I know that's tongue in cheek but I tend to favour portability over 
most other things. So running on Windows as well as Unix is, in my book, 
a Good Thing.

James

[toc] | [prev] | [next] | [standalone]

#96374

"James Harris" <james.harris.1@gmail.com> wrote in message 
news:msv21t$n1m$1@dont-email.me...
>
> "Grant Edwards" <invalid@invalid.invalid> wrote in message 
> news:msum6c$hv$1@reader1.panix.com...

...

>> Waking up twice per second and immediately
>> calling select() again on any hardware/OS built in the past 50 years
>> is going completely negligible (as long as you can ignore the smell).
>
> CPU usage is not the only issue but it is a consideration. I tested it 
> before posting the code and couldn't see any significant increase in 
> CPU use. To give it a better test I have just left running for a 
> couple of hours or so. I am pleased to say it currently reports a 
> cumulative total of 0:00:00, i.e. it has not clocked up even a second 
> of CPU time yet!

I am beginning to wonder if time is being accounted properly. It has now 
been running 8 hours and still shows CPU time as zero.

James

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web