Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #96827 > unrolled thread

Shutting down a cross-platform multithreaded app

Back to article view | Back to comp.lang.python

Contents

  Shutting down a cross-platform multithreaded app "James Harris" <james.harris.1@gmail.com> - 2015-09-18 18:17 +0100
    Re: Shutting down a cross-platform multithreaded app Paul Rubin <no.email@nospam.invalid> - 2015-09-18 11:23 -0700
      Re: Shutting down a cross-platform multithreaded app "James Harris" <james.harris.1@gmail.com> - 2015-09-18 20:09 +0100
        Re: Shutting down a cross-platform multithreaded app Laura Creighton <lac@openend.se> - 2015-09-18 22:50 +0200
          Re: Shutting down a cross-platform multithreaded app "James Harris" <james.harris.1@gmail.com> - 2015-09-19 10:56 +0100
    Re: Shutting down a cross-platform multithreaded app Marko Rauhamaa <marko@pacujo.net> - 2015-09-18 23:40 +0300
    Re: Shutting down a cross-platform multithreaded app Chris Angelico <rosuav@gmail.com> - 2015-09-19 07:40 +1000
      Re: Shutting down a cross-platform multithreaded app "James Harris" <james.harris.1@gmail.com> - 2015-09-19 10:49 +0100
        Re: Shutting down a cross-platform multithreaded app Chris Angelico <rosuav@gmail.com> - 2015-09-19 20:14 +1000
          Re: Shutting down a cross-platform multithreaded app "James Harris" <james.harris.1@gmail.com> - 2015-09-19 11:48 +0100
            Re: Shutting down a cross-platform multithreaded app Chris Angelico <rosuav@gmail.com> - 2015-09-19 20:59 +1000
    Re: Shutting down a cross-platform multithreaded app Random832 <random832@fastmail.com> - 2015-09-18 17:48 -0400
    Re: Shutting down a cross-platform multithreaded app Chris Angelico <rosuav@gmail.com> - 2015-09-19 08:09 +1000
    Re: Shutting down a cross-platform multithreaded app Akira Li <4kir4.1i@gmail.com> - 2015-09-19 02:56 +0300

#96827 — Shutting down a cross-platform multithreaded app

Well, this is fun ... for some definition of the word. ;-(

I have a multithreaded app that I want to be able to shut down easily 
such as by hitting control-c or sending it a signal. What follows is the 
way I have come up with given the common elements of different 
environments. Suggestions for improvement would be welcome or you may 
just find the convolutions and machinations interesting.

The first issue is that only the main thread can receive signals, 
according to

  https://docs.python.org/2/library/signal.html

It says: "the main thread will be the only one to receive signals (this 
is enforced by the Python signal module, even if the underlying thread 
implementation supports sending signals to individual threads)".

That's OK. I can get the main thread to accept suitable signals but then 
I need some way for it to tell the other threads to shut themselves down 
too. In most (probably all) cases they will be sitting waiting for 
network IO.

I could have the main thread set a value in a global variable and then 
have the sub-threads check the global in between accesses of the network 
in a polling loop (using select() with a timeout). But as has already 
been pointed out to me in the thread "Signal SIGINT ignored during 
socket.accept" such polling does not sit well with hosted OSes and it 
can keep the CPU from remaining at rest in a low-power mode.

I can, however, use select() to monitor two file descriptors. One would 
be the socket the thread is using to communicate with the client. The 
other would be a control connection from the master thread.

Now to make this cross platform.... According to the opening paragraph 
in the following link Windows select() won't work on arbitrary file 
descriptors but only works for sockets.

  https://docs.python.org/2/library/select.html

Well, that can be dealt with. I thought of using AF_UNIX or something 
else but it seems there is nothing else which could be considered 
universal and, according to the next link, if socket.AF_UNIX is not 
defined then even the Unix protocol is not supported.

  https://docs.python.org/2/library/socket.html

Needless to say, on a test Windows machine AF_UNIX is not present. The 
only cross-platform option, therefore, seems to be to use each 
subthread's select()s to monitor two AF_INET sockets: the one to the 
client and a control one from the master thread. I would seem to need IP 
socket pairs between the master thread and the subthreads. If the master 
thead receives a shutdown signal it will send a shutdown command to each 
subthread.

The above seems logical but would use quite a few IP sockets. I cannot 
think of a better way, though. Any comments on the ideas above?

James

[toc] | [next] | [standalone]

#96829

"James Harris" <james.harris.1@gmail.com> writes:
> I have a multithreaded app that I want to be able to shut down easily
> such as by hitting control-c or sending it a signal. 

Set the daemon flag on the worker threads, so when the main thread
exits, the workers also exit.

[toc] | [prev] | [next] | [standalone]

#96831

"Paul Rubin" <no.email@nospam.invalid> wrote in message 
news:87zj0jd1ta.fsf@jester.gateway.sonic.net...
> "James Harris" <james.harris.1@gmail.com> writes:

>> I have a multithreaded app that I want to be able to shut down easily
>> such as by hitting control-c or sending it a signal.
>
> Set the daemon flag on the worker threads, so when the main thread
> exits, the workers also exit.

Interesting idea, and I did not know that a *thread* could be a daemon. 
Unfortunately, I think what you suggest would kill the threads stone 
dead and not allow them to close connections.

That's a particular problem with TCP connections and would require the 
OS to keep TCP state around for a while. I would rather close the TCP 
connections or, rather, encourage the other end to close the connection 
so that the worker threads could then close the sockets in a way that 
would not hold on to resources.

For anyone who is interested see the TCP state diagram such as the one 
at

  http://no-shoveling.com/wp-content/uploads/2013/11/TCPfsm.png

The key transition is the way the server exits the ESTABLISHED state. If 
the server closes its end of the connection first the transition goes 
via the line labelled appl:close, send: FIN. In that case the socket 
will end up in the TIME_WAIT state wherein it can wait 2MSL or 2 maximum 
segment lifetimes before becoming free.

According to

  https://en.wikipedia.org/wiki/Maximum_segment_lifetime

MSL is arbitrarily defined to be two minutes. That means a TCP endpoint 
could sit in TIME_WAIT for a horribly long four minutes...!

So, I would rather get the other end to send the first FIN, if possible. 
On the TCP state diagram that is the exit from ESTABLISHED labelled 
recv:FIN, send ACK. My end can then shutdown the socket, which would 
send a FIN, and wait for a final ACK.

Bottom line: I need to do a controlled cleanup.

James

[toc] | [prev] | [next] | [standalone]

#96835

In a message of Fri, 18 Sep 2015 20:09:19 +0100, "James Harris" writes:
>> Set the daemon flag on the worker threads, so when the main thread
>> exits, the workers also exit.
>
>Interesting idea, and I did not know that a *thread* could be a daemon. 
>Unfortunately, I think what you suggest would kill the threads stone 
>dead and not allow them to close connections.

Can you stick your worker threads into a Queue.  When the main thread exits
have it tell the queue to clean itself up?

see:
http://code.activestate.com/recipes/82965-threads-tkinter-and-asynchronous-io/

The main thread doesn't have to be a gui ...

(but the author of that recipe and I are now drunkly celebrating a birthday
so maybe I ought not to be posting this idea ...)

Laura

[toc] | [prev] | [next] | [standalone]

#96849

"Laura Creighton" <lac@openend.se> wrote in message 
news:mailman.5.1442609448.21674.python-list@python.org...
> In a message of Fri, 18 Sep 2015 20:09:19 +0100, "James Harris" 
> writes:
>>> Set the daemon flag on the worker threads, so when the main thread
>>> exits, the workers also exit.
>>
>>Interesting idea, and I did not know that a *thread* could be a 
>>daemon.
>>Unfortunately, I think what you suggest would kill the threads stone
>>dead and not allow them to close connections.
>
> Can you stick your worker threads into a Queue.  When the main thread 
> exits
> have it tell the queue to clean itself up?
>
> see:
> http://code.activestate.com/recipes/82965-threads-tkinter-and-asynchronous-io/
>
> The main thread doesn't have to be a gui ...
>
> (but the author of that recipe and I are now drunkly celebrating a 
> birthday
> so maybe I ought not to be posting this idea ...)

:-)

I am not sure. The polling every 100ms or similar in periodicCall() is 
something I want to avoid. I think I have a way to do this without any 
polling.

James

[toc] | [prev] | [next] | [standalone]

#96834

"James Harris" <james.harris.1@gmail.com>:

> I have a multithreaded app that I want to be able to shut down easily
> such as by hitting control-c or sending it a signal.

The problem with threads is you cannot cause them to exit from the
outside. You can do that to asyncio coroutines, however. Maybe you
should consider porting your multithreaded app to asyncio.


Marko

[toc] | [prev] | [next] | [standalone]

#96838

On Sat, Sep 19, 2015 at 3:17 AM, James Harris <james.harris.1@gmail.com> wrote:
> Needless to say, on a test Windows machine AF_UNIX is not present. The only
> cross-platform option, therefore, seems to be to use each subthread's
> select()s to monitor two AF_INET sockets: the one to the client and a
> control one from the master thread. I would seem to need IP socket pairs
> between the master thread and the subthreads. If the master thead receives a
> shutdown signal it will send a shutdown command to each subthread.
>
> The above seems logical but would use quite a few IP sockets. I cannot think
> of a better way, though. Any comments on the ideas above?

If you're using select() to monitor the sockets, you don't actually
then have to _do_ anything with the shutdown socket. You could have a
single socket that sends the shutdown signal to all your workers.

Bear in mind, though, that Windows has no protection against other
processes shutting you down. You can restrict it to 127.0.0.1 (of
course) but any program running on the same computer as the server -
regardless of user permissions etc - will be able to connect to your
sockets. So it might be best to do something like this (all on the
main thread):

1) Open a listening socket
2) Connect to the listening socket
3) Accept a connection
4) Close the original listening socket
5) Spin off all your threads, passing them the socket from step 2
6) To terminate them all, write a byte to the socket from step 3.

This will make it difficult for ordinary userspace code to mess with
you. It'd still be possible, I think, for something with raw sockets
access to feign a termination signal; I have no idea what protections
Windows offers you there.

Give it a shot, see how it goes!

ChrisA

[toc] | [prev] | [next] | [standalone]

#96848

"Chris Angelico" <rosuav@gmail.com> wrote in message 
news:mailman.8.1442612439.21674.python-list@python.org...
> On Sat, Sep 19, 2015 at 3:17 AM, James Harris 
> <james.harris.1@gmail.com> wrote:
>> Needless to say, on a test Windows machine AF_UNIX is not present. 
>> The only
>> cross-platform option, therefore, seems to be to use each subthread's
>> select()s to monitor two AF_INET sockets: the one to the client and a
>> control one from the master thread. I would seem to need IP socket 
>> pairs
>> between the master thread and the subthreads. If the master thead 
>> receives a
>> shutdown signal it will send a shutdown command to each subthread.
>>
>> The above seems logical but would use quite a few IP sockets. I 
>> cannot think
>> of a better way, though. Any comments on the ideas above?
>
> If you're using select() to monitor the sockets, you don't actually
> then have to _do_ anything with the shutdown socket. You could have a
> single socket that sends the shutdown signal to all your workers.

I don't understand how a single socket could send the signal to all the 
workers. I did consider some form of multicast but thought it too 
complicated (and possibly infeasible).

Re. not understanding the single sending socket idea that you mention 
perhaps I had better explain a bit of what I had in mind:

1. A worker thread would have a TCP socket connection to its client, and 
another socket for communicating with the master. That second socket has 
to be AF_INET for portability. It could be TCP or UDP. A connected UDP 
socket may be most appropriate and seems worth trying.

2. If something happens to the master thread so that it determines that 
the application should shut down it would iterate over the sockets to 
the workers and tell each one to shut down. It would then shut itself 
down. (Am not sure at the moment whether to wait for the worker 
threads.)

3. A worker thread, being told to shutdown (basically a single byte 
received from the master thread) would tell its client to close the TCP 
connection and then it would wait a little while for it to do so - maybe 
a second or two. When the client closes the TCP connection (or the 
timeout wait period expires) the worker thread will close its end and 
exit.

> Bear in mind, though, that Windows has no protection against other
> processes shutting you down. You can restrict it to 127.0.0.1 (of
> course) but any program running on the same computer as the server -
> regardless of user permissions etc - will be able to connect to your
> sockets.

I was thinking of a connected UDP socket. That way, AIUI, at least in 
the absence of forged datagrams, only the master thread will be able to 
communicate with the worker, due to connected UDP sockets demulitplexing 
datagrams based on their source as well as their destination, i.e. on a 
5-tuple (UDP, source IP, source port, destination IP, destination port).

> So it might be best to do something like this (all on the
> main thread):
>
> 1) Open a listening socket
> 2) Connect to the listening socket
> 3) Accept a connection
> 4) Close the original listening socket
> 5) Spin off all your threads, passing them the socket from step 2
> 6) To terminate them all, write a byte to the socket from step 3.

That sounds similar to what I had in mind but I am not sure why you 
would close the listening socket. Connections could come in at any time 
and threads could therefore be needed at any time so I was thinking that 
the master thread (the one with the listening TCP socket) would just sit 
waiting for new connection requests (or an interrupting signal).

In reality, due to Windows not recognising signals while in the accept() 
call I think there would be a real master thread and a listening thread 
but I have omitted that in the descriptions above. As far as the normal 
worker threads are concerned they would be ready to be told to shut down 
by the listening thread, and it would be ready to be told to shut down 
by the master thread. Still with me? ;-)

> This will make it difficult for ordinary userspace code to mess with
> you. It'd still be possible, I think, for something with raw sockets
> access to feign a termination signal; I have no idea what protections
> Windows offers you there.

Yes, something which could forge a packet could tell a worker to close 
down. I don't think there's any significant problem here, thought, 
because:

* other programs are similarly vulnerable to forged packets
* the only forgery effect is to tell a worker thread to stop - not a big 
loss
* the shutdown protocol would/should cause the client to re-request
* a forger would have to know the specific port number used by the 
master thread to communicate with that particular worker, and the port 
number that worker was using.

Overall, I think it would be more than robust enough.

Notwithstanding your comment about a single socket, above, no one seems 
so far to have objected to the number of sockets this would need, which 
was my main concern, so that's good!

James

[toc] | [prev] | [next] | [standalone]

#96850

On Sat, Sep 19, 2015 at 7:49 PM, James Harris <james.harris.1@gmail.com> wrote:
> "Chris Angelico" <rosuav@gmail.com> wrote in message
> news:mailman.8.1442612439.21674.python-list@python.org...
>>
>> On Sat, Sep 19, 2015 at 3:17 AM, James Harris <james.harris.1@gmail.com>
>> wrote:
>>>
>>> Needless to say, on a test Windows machine AF_UNIX is not present. The
>>> only
>>> cross-platform option, therefore, seems to be to use each subthread's
>>> select()s to monitor two AF_INET sockets: the one to the client and a
>>> control one from the master thread. I would seem to need IP socket pairs
>>> between the master thread and the subthreads. If the master thead
>>> receives a
>>> shutdown signal it will send a shutdown command to each subthread.
>>>
>>> The above seems logical but would use quite a few IP sockets. I cannot
>>> think
>>> of a better way, though. Any comments on the ideas above?
>>
>>
>> If you're using select() to monitor the sockets, you don't actually
>> then have to _do_ anything with the shutdown socket. You could have a
>> single socket that sends the shutdown signal to all your workers.
>
>
> I don't understand how a single socket could send the signal to all the
> workers. I did consider some form of multicast but thought it too
> complicated (and possibly infeasible).

The way I'm describing it, the workers never actually read from the
socket. Once that socket becomes readable, they immediately shut down,
without making the socket no-longer-readable.

>> Bear in mind, though, that Windows has no protection against other
>> processes shutting you down. You can restrict it to 127.0.0.1 (of
>> course) but any program running on the same computer as the server -
>> regardless of user permissions etc - will be able to connect to your
>> sockets.
>
>
> I was thinking of a connected UDP socket. That way, AIUI, at least in the
> absence of forged datagrams, only the master thread will be able to
> communicate with the worker, due to connected UDP sockets demulitplexing
> datagrams based on their source as well as their destination, i.e. on a
> 5-tuple (UDP, source IP, source port, destination IP, destination port).

TCP sockets also work on that set of five. That's why I suggested a
pre-connected TCP socket, with the original listening socket closed.

(And as mentioned, Python 3.5 supports socketpair() on Windows. That
would definitely be the best option.)

> That sounds similar to what I had in mind but I am not sure why you would
> close the listening socket. Connections could come in at any time and
> threads could therefore be needed at any time so I was thinking that the
> master thread (the one with the listening TCP socket) would just sit waiting
> for new connection requests (or an interrupting signal).

TCP sockets work on the basis of a master socket and any number of
spawned sockets. The master is what gives you an open port; each
spawned socket represents one connection with one client. Once you
have an established connection, the master should be able to be closed
without disrupting that. No other process will be able to connect to
you, but you'll still be able to use one end of the socket to make the
other end readable.

>> This will make it difficult for ordinary userspace code to mess with
>> you. It'd still be possible, I think, for something with raw sockets
>> access to feign a termination signal; I have no idea what protections
>> Windows offers you there.
>
>
> Yes, something which could forge a packet could tell a worker to close down.
> I don't think there's any significant problem here, thought, because:
>
> * other programs are similarly vulnerable to forged packets
> * the only forgery effect is to tell a worker thread to stop - not a big
> loss
> * the shutdown protocol would/should cause the client to re-request
> * a forger would have to know the specific port number used by the master
> thread to communicate with that particular worker, and the port number that
> worker was using.
>
> Overall, I think it would be more than robust enough.

With UDP, any process that can send a UDP packet can flood the system
with them until your workers shut down. You wouldn't even notice until
it succeeds. With TCP, at least an attacker would need raw socket
access. It's still not as protected as a Unix domain socket, but it's
a bit harder for someone to do.

> Notwithstanding your comment about a single socket, above, no one seems so
> far to have objected to the number of sockets this would need, which was my
> main concern, so that's good!

Sure. Sockets are pretty cheap. Even if you had one for every worker,
there's room for you to have thousands (maybe tens of thousands) of
workers without a problem. I think you'll run into other scaling
problems with that many workers on one computer :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#96851

"Chris Angelico" <rosuav@gmail.com> wrote in message 
news:mailman.13.1442657702.21674.python-list@python.org...
> On Sat, Sep 19, 2015 at 7:49 PM, James Harris 
> <james.harris.1@gmail.com> wrote:
>> "Chris Angelico" <rosuav@gmail.com> wrote in message
>> news:mailman.8.1442612439.21674.python-list@python.org...

...

>>> If you're using select() to monitor the sockets, you don't actually
>>> then have to _do_ anything with the shutdown socket. You could have 
>>> a
>>> single socket that sends the shutdown signal to all your workers.
>>
>>
>> I don't understand how a single socket could send the signal to all 
>> the
>> workers. I did consider some form of multicast but thought it too
>> complicated (and possibly infeasible).
>
> The way I'm describing it, the workers never actually read from the
> socket. Once that socket becomes readable, they immediately shut down,
> without making the socket no-longer-readable.

Understood. Good idea. Initial thoughts on it: Would work for threads. 
Would save on the number of sockets required. Would not work for 
processes if the model was ever changed. Would make it easier for rogue 
packets to shut a worker down. Would not allow any way to distinguish 
between shutdown priorities (not something I have mentioned). Definitely 
feasible. I'll keep it in mind.

>>> Bear in mind, though, that Windows has no protection against other
>>> processes shutting you down. You can restrict it to 127.0.0.1 (of
>>> course) but any program running on the same computer as the server -
>>> regardless of user permissions etc - will be able to connect to your
>>> sockets.

...

>> That sounds similar to what I had in mind but I am not sure why you 
>> would
>> close the listening socket. Connections could come in at any time and
>> threads could therefore be needed at any time so I was thinking that 
>> the
>> master thread (the one with the listening TCP socket) would just sit 
>> waiting
>> for new connection requests (or an interrupting signal).
>
> TCP sockets work on the basis of a master socket and any number of
> spawned sockets. The master is what gives you an open port; each
> spawned socket represents one connection with one client. Once you
> have an established connection, the master should be able to be closed
> without disrupting that. No other process will be able to connect to
> you, but you'll still be able to use one end of the socket to make the
> other end readable.

Agreed but I need the listening socket to remain open and listening for 
new connections (at least until the whole program is told to shut down).

>>> This will make it difficult for ordinary userspace code to mess with
>>> you. It'd still be possible, I think, for something with raw sockets
>>> access to feign a termination signal; I have no idea what 
>>> protections
>>> Windows offers you there.
>>
>>
>> Yes, something which could forge a packet could tell a worker to 
>> close down.
>> I don't think there's any significant problem here, thought, because:
>>
>> * other programs are similarly vulnerable to forged packets
>> * the only forgery effect is to tell a worker thread to stop - not a 
>> big
>> loss
>> * the shutdown protocol would/should cause the client to re-request
>> * a forger would have to know the specific port number used by the 
>> master
>> thread to communicate with that particular worker, and the port 
>> number that
>> worker was using.
>>
>> Overall, I think it would be more than robust enough.
>
> With UDP, any process that can send a UDP packet can flood the system
> with them until your workers shut down. You wouldn't even notice until
> it succeeds.

Is that true? You seem to be describing a non-forged attack but to get 
the source UDP port right wouldn't the attacker have to be runing on the 
same machine *and* to bind to the same port that the machine had 
allocated to my program? I might be wrong but I don't think the UDP 
stack would allow the same port to be bound again before the original 
had been closed.

> With TCP, at least an attacker would need raw socket
> access. It's still not as protected as a Unix domain socket, but it's
> a bit harder for someone to do.
>
>> Notwithstanding your comment about a single socket, above, no one 
>> seems so
>> far to have objected to the number of sockets this would need, which 
>> was my
>> main concern, so that's good!
>
> Sure. Sockets are pretty cheap. Even if you had one for every worker,
> there's room for you to have thousands (maybe tens of thousands) of
> workers without a problem. I think you'll run into other scaling
> problems with that many workers on one computer :)

Let's see. If I stick with my original plan then each worker would have 
a TCP socket and a UDP socket. The "listener" thread would have its 
single listening TCP socket plus it would have a UDP socket for each 
worker thread. Total of three sockets per worker, two of which would be 
UDP sockets with port numbers assigned and thus consumed.

If I go with a single "shutdown socket" then I would have just one 
socket per worker. That would use fewer sockets, for sure.

James

[toc] | [prev] | [next] | [standalone]

#96854

On Sat, Sep 19, 2015 at 8:48 PM, James Harris <james.harris.1@gmail.com> wrote:
> "Chris Angelico" <rosuav@gmail.com> wrote in message
> news:mailman.13.1442657702.21674.python-list@python.org...
>>
>> On Sat, Sep 19, 2015 at 7:49 PM, James Harris <james.harris.1@gmail.com>
>> wrote:
>>>
>>> "Chris Angelico" <rosuav@gmail.com> wrote in message
>>> news:mailman.8.1442612439.21674.python-list@python.org...
>>>> If you're using select() to monitor the sockets, you don't actually
>>>> then have to _do_ anything with the shutdown socket. You could have a
>>>> single socket that sends the shutdown signal to all your workers.
>>>
>>>
>>>
>>> I don't understand how a single socket could send the signal to all the
>>> workers. I did consider some form of multicast but thought it too
>>> complicated (and possibly infeasible).
>>
>>
>> The way I'm describing it, the workers never actually read from the
>> socket. Once that socket becomes readable, they immediately shut down,
>> without making the socket no-longer-readable.
>
>
> Understood. Good idea. Initial thoughts on it: Would work for threads. Would
> save on the number of sockets required. Would not work for processes if the
> model was ever changed. Would make it easier for rogue packets to shut a
> worker down. Would not allow any way to distinguish between shutdown
> priorities (not something I have mentioned). Definitely feasible. I'll keep
> it in mind.

If you go to multiple processes, yeah, it probably wouldn't work *on
Windows*. On Unix, you can fork and have multiple processes with the
same socket. (It's very common to have, for instance, a subprocess's
stdin/stdout/stderr linked to pipes of some sort; the calling process
still retains control.)

It would be _harder_ for rogue packets to shut a worker down this way.

>> TCP sockets work on the basis of a master socket and any number of
>> spawned sockets. The master is what gives you an open port; each
>> spawned socket represents one connection with one client. Once you
>> have an established connection, the master should be able to be closed
>> without disrupting that. No other process will be able to connect to
>> you, but you'll still be able to use one end of the socket to make the
>> other end readable.
>
> Agreed but I need the listening socket to remain open and listening for new
> connections (at least until the whole program is told to shut down).

Not sure why. The sole purpose of this socket is to establish a
(single) socket pair used for the termination signals - nothing more.
You shouldn't need to listen for any new connections, even if you
spawn new workers.

>> With UDP, any process that can send a UDP packet can flood the system
>> with them until your workers shut down. You wouldn't even notice until
>> it succeeds.
>
> Is that true? You seem to be describing a non-forged attack but to get the
> source UDP port right wouldn't the attacker have to be runing on the same
> machine *and* to bind to the same port that the machine had allocated to my
> program? I might be wrong but I don't think the UDP stack would allow the
> same port to be bound again before the original had been closed.

UDP basically doesn't have protections like that. TCP does, and though
it _is_ possible to forge packets, it requires raw socket access. I
don't know what protections Windows has around that, but certainly a
software firewall should be able to notice that some program is
spewing raw packets.

> Let's see. If I stick with my original plan then each worker would have a
> TCP socket and a UDP socket. The "listener" thread would have its single
> listening TCP socket plus it would have a UDP socket for each worker thread.
> Total of three sockets per worker, two of which would be UDP sockets with
> port numbers assigned and thus consumed.
>
> If I go with a single "shutdown socket" then I would have just one socket
> per worker. That would use fewer sockets, for sure.

Yeah, it's pretty cheap either way. Your most scarce resource here
would be UDP port numbers, and there's 64K of those. That'd let you go
as far as 32K workers, and I don't think you can have that many
threads without saturating something somewhere else.

ChrisA

[toc] | [prev] | [next] | [standalone]

#96840

On Fri, Sep 18, 2015, at 17:40, Chris Angelico wrote:
> Bear in mind, though, that Windows has no protection against other
> processes shutting you down.

Neither does Unix. Any process that can send you a signal can send you
SIGKILL.

Of course, what Windows lacks is a generalized way for other processes
to send "less destructive" signals that do give you a chance to clean
up. (You can sometimes send a ctrl-break event, but that's it.) And most
frameworks for "emulating" them (including python's os module) simulate
sending other signals by calling TerminateProcess with an exit status
related to the signal.

[toc] | [prev] | [next] | [standalone]

#96841

On Sat, Sep 19, 2015 at 7:48 AM, Random832 <random832@fastmail.com> wrote:
> On Fri, Sep 18, 2015, at 17:40, Chris Angelico wrote:
>> Bear in mind, though, that Windows has no protection against other
>> processes shutting you down.
>
> Neither does Unix. Any process that can send you a signal can send you
> SIGKILL.

Incorrect. If your server is running as root, only root can kill it:

rosuav@sikorsky:~$ kill -9 17080
bash: kill: (17080) - Operation not permitted

If it's running as some other user, then that user can kill it (that
includes the simple case where a non-root user starts a process and
also tries to kill it), as can root, of course. So you have protection
against direct signals (and not just 9/KILL, naturally); and you also
have protection against an AF_UNIX socket, which is what I was talking
about here. The control over sockets is a bit more flexible, as I'm
fairly sure group permissions can't be set for process signals, but
they can for named sockets:

rosuav@sikorsky:~$ python3
Python 3.6.0a0 (default:30bc256f2346, Sep 17 2015, 02:01:45)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
>>> s.bind("/tmp/demo_socket")
>>> import os
>>> os.chmod("/tmp/demo_socket",0o750)
>>> s.listen(1)
>>> s.accept()
# program pauses here
(<socket.socket fd=4, family=AddressFamily.AF_UNIX,
type=SocketKind.SOCK_STREAM, proto=0, laddr=/tmp/demo_socket>, b'')

In another terminal, using Python 2 for variety:

rosuav@sikorsky:~$ sudo sudo -u tfr python
Python 2.7.9 (default, Mar  1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket(socket.AF_UNIX)
>>> s.connect("/tmp/demo_socket")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 13] Permission denied

Err, nope! What if I don't change users?

rosuav@sikorsky:~$ python
Python 2.7.9 (default, Mar  1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket(socket.AF_UNIX)
>>> s.connect("/tmp/demo_socket")
>>>

Looks good. (Feel free to concoct your own scenario that proves that
group permissions work here; I don't have any handy demo cases.)

Unix is designed for this exact sort of thing. Windows isn't, and
privilege escalation attacks are far more common there.

> Of course, what Windows lacks is a generalized way for other processes
> to send "less destructive" signals that do give you a chance to clean
> up. (You can sometimes send a ctrl-break event, but that's it.) And most
> frameworks for "emulating" them (including python's os module) simulate
> sending other signals by calling TerminateProcess with an exit status
> related to the signal.

Yeah, the whole notion of less-destructive (or even completely
non-destructive - look at how a lot of daemons use SIGHUP) signals is
absent on Windows. But that's not really the problem here; the problem
is that there's no way to say "this is a socket for my process ONLY",
which in Unix would be done with a socket.socketpair, but on Windows I
think has to be simulated.

That said, though.... socket.socketpair() IS supported on Windows...
as of Python 3.5. I haven't tested it to see what it's like. If you
can restrict your support to 3.5+, you might be able to do this
instead of what I was describing above.

ChrisA

[toc] | [prev] | [next] | [standalone]

#96842

"James Harris" <james.harris.1@gmail.com> writes:

...
> Needless to say, on a test Windows machine AF_UNIX is not present. The
> only cross-platform option, therefore, seems to be to use each
> subthread's select()s to monitor two AF_INET sockets: the one to the
> client and a control one from the master thread. I would seem to need
> IP socket pairs between the master thread and the subthreads. If the
> master thead receives a shutdown signal it will send a shutdown
> command to each subthread.

There is socket.socketpair() on Windows too (since Python 3.5)

  https://docs.python.org/3/library/socket.html#socket.socketpair
  http://bugs.python.org/issue18643

Note: you could use select() to handle signals in the main thread too
(even on Windows since Python 3.5) if you use signal.set_wakeup_fd()

  https://docs.python.org/3/library/signal.html#signal.set_wakeup_fd

It is known as a self-pipe trick

  http://www.sitepoint.com/the-self-pipe-trick-explained/

Look at *asyncio* source code, to see how to get a portable
implementation for various issues with signals. Some issues might still
be opened e.g., Ctrl+C behavior

  http://bugs.python.org/issue24080

Here's how to combine SIGCHLD signal handling with tkinter's event
loop

  http://stackoverflow.com/questions/30087506/event-driven-system-call-in-python

SIGCHLD, createfilehandler() are not portable but the code demonstrates
possible set_wakeup_fd() issues and their solutions (O_NONBLOCK, dummy
signal handler, SA_RESTART, signal coalescing).

On threads and signals in CPython

  http://bugs.python.org/issue5315#msg102829

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web