Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53013 > unrolled thread

How to check client shutdown?

Back to article view | Back to comp.lang.python

Contents

  How to check client shutdown? Paul Pittlerson <menkomigen6@gmail.com> - 2013-08-26 12:45 -0700
    Re: How to check client shutdown? Chris Angelico <rosuav@gmail.com> - 2013-08-27 06:09 +1000
      Re: How to check client shutdown? Paul Pittlerson <menkomigen6@gmail.com> - 2013-08-27 10:44 -0700
        Re: How to check client shutdown? Andrew Berg <robotsondrugs@gmail.com> - 2013-08-27 15:17 -0500
        Re: How to check client shutdown? Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2013-08-27 22:22 +0200
          Re: How to check client shutdown? Chris Angelico <rosuav@gmail.com> - 2013-08-28 06:40 +1000
            Re: How to check client shutdown? Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2013-08-27 22:49 +0200
    Re: How to check client shutdown? MRAB <python@mrabarnett.plus.com> - 2013-08-26 21:20 +0100
    Re: How to check client shutdown? Grant Edwards <invalid@invalid.invalid> - 2013-08-26 20:24 +0000

#53013 — How to check client shutdown?

I'm currently learning about the socket module. My question is how can I detect if a connection is closed from the other side, for example a KeyboardInterrupt as I frequently use. My code below:

##########################################################
#server script:

class client(threading.Thread):    
    def __init__(self, connection):
        super(client, self).__init__()
        
        self.recv = connection.recv
        
        print dir(connection)
        
    def start(self):
        self.run()
        
    def run(self):
        while True:
            data = self.recv(2000)
            
            if data:
                msg = self.decode(data)
                print msg
                #break;
                
    def decode(self, data):
        return pickle.loads(data)
        
                
class bullet_server:
    def __init__(self):
        self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        
    def start(self):
        server_address = ("localhost", 9006)
        print '/--server running--/'
        print server_address 
        self.server.bind(server_address)
        self.server.listen(5)

        while True:
            connection, client_address = self.server.accept()
            
            c = client(connection)
            c.start()
            
            
if __name__ == '__main__':            
    server = bullet_server()
    server.start()


##########################################################
#client script:

# set IP and port
host = '127.0.0.1'
port = 9006

# connect to server
connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
connection.connect((host, port))

def loop():
    print 'Enter message:'
    msg = raw_input()
    encoded = pickle.dumps(msg)
    connection.send(encoded)
    loop()
    
loop()

[toc] | [next] | [standalone]

#53014

On Tue, Aug 27, 2013 at 5:45 AM, Paul Pittlerson <menkomigen6@gmail.com> wrote:
> I'm currently learning about the socket module. My question is how can I detect if a connection is closed from the other side, for example a KeyboardInterrupt as I frequently use. My code below:
>

Once the remote end has terminated (as it will presumably do in this
case), any attempt to write to the socket will eventually result in an
error. You should be able to see that by simple experimentation. Be
aware that it might take a bit of time for the error to filter back to
you; it depends on whether the remote end actually sends back a RST
(reset) packet or not. (In a sane system, hitting Ctrl-C should result
in a clean shutdown and proper socket closure, but your server needs
to handle the case where it doesn't get one.) Play around with it, you
should be able to figure out what it's doing for you.

A couple of tangential comments about your script:

> ##########################################################
> #server script:
>
>     def decode(self, data):
>         return pickle.loads(data)

MAJOR security issue here. You are taking data from a networked source
and running it through a trusting system (pickle). This is NOT
recommended.

Also: You're using a stream socket, and depending - I think - on
getting an entire message in a single read() call. This is definitely
not guaranteed, though when you're working with localhost it's likely
to be true. To make this reliable, you'll need to do something like
prefix the message with its length (eg a Hollerith string), or use
some system like JSON that lets you detect the end of the message.

>From the client:

> def loop():
>     print 'Enter message:'
>     msg = raw_input()
>     encoded = pickle.dumps(msg)
>     connection.send(encoded)
>     loop()
>
> loop()

Why recurse? Why not simply loop? No Python has tail-call optimization
(as far as I know), so this will eventually blow up with a
RuntimeError: maximum recursion depth exceeded.

ChrisA

[toc] | [prev] | [next] | [standalone]

#53080

> MAJOR security issue here. You are taking data from a networked source
> 
> and running it through a trusting system (pickle). This is NOT
> 
> recommended.
> 

Security issue!? Do you mean someone could enter devious python h4xx into the chat or something? I had no idea using pickle was so dangerous, but I don't know any other method of transmitting data in python :(



> Also: You're using a stream socket, and depending - I think - on
> 
> getting an entire message in a single read() call. This is definitely
> 
> not guaranteed, though when you're working with localhost it's likely
> 
> to be true. To make this reliable, you'll need to do something like
> 
> prefix the message with its length (eg a Hollerith string), or use
> 
> some system like JSON that lets you detect the end of the message.
> 

I'm guessing the fix is to have a separate thread which only job is to send info about the size of the next data transmission. What is the actual downside of having the server set to anticipate a message length which is known to be more than will be sent (or be allowed to be sent?), for example connection.recv(10000). Does not the receiver know the size after the fact? Is it impacting performance somehow (I haven't noticed anything in my tests)

[toc] | [prev] | [next] | [standalone]

#53088

On 2013.08.27 12:44, Paul Pittlerson wrote:
> Security issue!? Do you mean someone could enter devious python h4xx into the chat or something? I had no idea using pickle was so dangerous, but I don't know any other method of transmitting data in python :(
JSON, XML, or any other format that doesn't have the capacity to serialize directly executable Python code. Pickles are okay for internal
data from trusted sources, but don't accept pickles from anyone you don't trust.
JSON is simple, easy, and can handle the common data types, and there is a module for it in the stdlib. It's also a standard format you can
use in pretty much any language, and it's human readable.
-- 
CPython 3.3.2 | Windows NT 6.2.9200 / FreeBSD 9.1

[toc] | [prev] | [next] | [standalone]

#53090

On 27-8-2013 19:44, Paul Pittlerson wrote:
>> MAJOR security issue here. You are taking data from a networked source
>> 
>> and running it through a trusting system (pickle). This is NOT
>> 
>> recommended.
>> 
> 
> Security issue!? Do you mean someone could enter devious python h4xx into the chat or
> something? I had no idea using pickle was so dangerous, but I don't know any other
> method of transmitting data in python :(
> 

Shameless plug: have a look at my remote object library Pyro:
http://pythonhosted.org/Pyro4/

It transmits arbitrary objects to remote machines by what seem to be normal method
calls. In the past, it was restricted to using pickle as serialization format, but since
a couple of releases, it now defaults to another -safe- serializer. Because of the same
reason Chris is warning you about: unpickling data from untrusted sources can result in
arbitrary code execution in your server.


Main thing is: don't use pickle in your networking code unless you can guarantee the
trustworthiness of your sources. Instead, use another serialization format that is safe
(such as marshal, json, serpent).



> I'm guessing the fix is to have a separate thread which only job is to send info
> about the size of the next data transmission.

I'm not sure what you're proposing here. What's a separate thread got to do with things?

? What is the actual downside of having
> the server set to anticipate a message length which is known to be more than will be
> sent (or be allowed to be sent?), for example connection.recv(10000). Does not the
> receiver know the size after the fact? Is it impacting performance somehow (I haven't
> noticed anything in my tests)

The issue is that recv() is not guaranteed to return you the full amount of data that is
requested. It may very well just return a single byte, and leave the rest for later. The
argument is an upper bound on the amount of data you receive. So to make your recv
reliable, you need to have a means of deciding when the 'full' amount of data has been
collected. As Chris already suggested, this is usually done by putting the recv() in a
loop and collecting data until it reaches a length that you precisely know beforehand,
or by detecting a special end-of-message marker in the data stream, such as a newline.


Irmen de Jong

[toc] | [prev] | [next] | [standalone]

#53093

On Wed, Aug 28, 2013 at 6:22 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote:
> ? What is the actual downside of having
>> the server set to anticipate a message length which is known to be more than will be
>> sent (or be allowed to be sent?), for example connection.recv(10000). Does not the
>> receiver know the size after the fact? Is it impacting performance somehow (I haven't
>> noticed anything in my tests)
>
> The issue is that recv() is not guaranteed to return you the full amount of data that is
> requested. It may very well just return a single byte, and leave the rest for later. The
> argument is an upper bound on the amount of data you receive. So to make your recv
> reliable, you need to have a means of deciding when the 'full' amount of data has been
> collected. As Chris already suggested, this is usually done by putting the recv() in a
> loop and collecting data until it reaches a length that you precisely know beforehand,
> or by detecting a special end-of-message marker in the data stream, such as a newline.

Right. When you use TCP sockets, there's no  boundaries, so you could
get two pickles in one recv, or you could get one and a half, or
anything. It depends partly on your buffer sizes and things; if you're
sending very short messages (less than a kilobyte), and have long
delays between them, chances are you'll get one write in one read; but
it's not guaranteed.

ChrisA

[toc] | [prev] | [next] | [standalone]

#53096

On 27-8-2013 22:40, Chris Angelico wrote:

> Right. When you use TCP sockets, there's no  boundaries, so you could
> get two pickles in one recv, or you could get one and a half, or
> anything. It depends partly on your buffer sizes and things; if you're
> sending very short messages (less than a kilobyte), and have long
> delays between them, chances are you'll get one write in one read; but
> it's not guaranteed.
> 
> ChrisA
> 

More horror stories: the same is also true for send().

Thankfully, Python provides a sendall(). But that doesn't work on some systems when the
socket is in non-blocking mode. Meh.

Irmen

[toc] | [prev] | [next] | [standalone]

#53017

On 26/08/2013 20:45, Paul Pittlerson wrote:
> I'm currently learning about the socket module. My question is how
> can I detect if a connection is closed from the other side, for
> example a KeyboardInterrupt as I frequently use. My code below:
>
[snip]
When reading from a socket, it'll return as much data as is available,
up to the given limit if specified.

If _no_ data is available, then it'll block, unless the socket has been
closed at the other end, in which case it'll return zero bytes (an
empty bytestring).

[toc] | [prev] | [next] | [standalone]

#53018

On 2013-08-26, Paul Pittlerson <menkomigen6@gmail.com> wrote:

> I'm currently learning about the socket module. My question is how
> can I detect if a connection is closed from the other side,

recv() will return an empty value of ''.

send() will eventually throw an exception.  [You may be able to call
send() once or twice after the connection has been closed without
getting the exception.] On Linux the exception is socket.error with
errno 32 (broken pipe).

-- 
Grant Edwards               grant.b.edwards        Yow! Is this an out-take
                                  at               from the "BRADY BUNCH"?
                              gmail.com            

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web