Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #44698 > unrolled thread

socket programming

Back to article view | Back to comp.lang.python

Contents

  socket programming Pedro <pedro@ncf.ca> - 2013-05-03 19:13 -0700
    Re: socket programming Chris Angelico <rosuav@gmail.com> - 2013-05-04 12:23 +1000
      Re: socket programming Pedro <pedro@ncf.ca> - 2013-05-03 20:37 -0700
        Re: socket programming Chris Angelico <rosuav@gmail.com> - 2013-05-04 13:56 +1000
          Re: socket programming Pedro <pedro@ncf.ca> - 2013-05-03 21:03 -0700
    Re: socket programming Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2013-05-04 11:37 +0200
      Re: socket programming Chris Angelico <rosuav@gmail.com> - 2013-05-04 20:00 +1000
      Re: socket programming Pedro <pedro@ncf.ca> - 2013-05-06 08:54 -0700
        Re: socket programming Chris Angelico <rosuav@gmail.com> - 2013-05-07 02:05 +1000
        Re: socket programming Arnaud Delobelle <arnodel@gmail.com> - 2013-05-06 19:54 +0100

#44698 — socket programming

I'm writing a simple app that uses socket programming, I'm using the following from tutorialspoint as a framework but I'm having a couple of issues implementing it. 

First - this code constantly loops around an open socket. Is there a way to use something like an interrupt so I don't have to loop constantly to monitor the socket?

Second, if any part of the program fails execution (crashes) the port often remains open on my windows machine and the only way to close it that i know of is through task manager or by rebooting the machine. Is there an easy way around this problem ? If I don't close the port the program can't open it again and crashes.

Thanks for looking



SERVER:

import socket               # Import socket module

s = socket.socket()         # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345                # Reserve a port for your service.
s.bind((host, port))        # Bind to the port

s.listen(5)                 # Now wait for client connection.
while True:
   c, addr = s.accept()     # Establish connection with client.
   print 'Got connection from', addr
   c.send('Thank you for connecting')
   c.close()                # Close the connection

CLIENT:
import socket               # Import socket module

s = socket.socket()         # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345                # Reserve a port for your service.

s.connect((host, port))
print s.recv(1024)
s.close                     # Close the socket when done

[toc] | [next] | [standalone]

#44699

On Sat, May 4, 2013 at 12:13 PM, Pedro <pedro@ncf.ca> wrote:
> First - this code constantly loops around an open socket. Is there a way to use something like an interrupt so I don't have to loop constantly to monitor the socket?

The accept() call should block. It's not going to spin or anything. If
you need to monitor multiple sockets, have a look at select().

> Second, if any part of the program fails execution (crashes) the port often remains open on my windows machine and the only way to close it that i know of is through task manager or by rebooting the machine. Is there an easy way around this problem ? If I don't close the port the program can't open it again and crashes.

It remains for a short time to ensure that there's no lurking
connections. You can bypass this check by setting the SO_REUSEADDR
option - lemme hunt that down in the Python docs, haven't done that in
Python for a while...

http://docs.python.org/3.3/library/socket.html#socket.socket.setsockopt

s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

That should do the job.

ChrisA

[toc] | [prev] | [next] | [standalone]

#44700

On Friday, May 3, 2013 10:23:38 PM UTC-4, Chris Angelico wrote:
> On Sat, May 4, 2013 at 12:13 PM, Pedro <pedro@ncf.ca> wrote:
> 
> > First - this code constantly loops around an open socket. Is there a way to use something like an interrupt so I don't have to loop constantly to monitor the socket?
> 
> 
> 
> The accept() call should block. It's not going to spin or anything. If
> 
> you need to monitor multiple sockets, have a look at select().
> 
> 
> 
> > Second, if any part of the program fails execution (crashes) the port often remains open on my windows machine and the only way to close it that i know of is through task manager or by rebooting the machine. Is there an easy way around this problem ? If I don't close the port the program can't open it again and crashes.
> 
> 
> 
> It remains for a short time to ensure that there's no lurking
> 
> connections. You can bypass this check by setting the SO_REUSEADDR
> 
> option - lemme hunt that down in the Python docs, haven't done that in
> 
> Python for a while...
> 
> 
> 
> http://docs.python.org/3.3/library/socket.html#socket.socket.setsockopt
> 
> 
> 
> s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
> 
> 
> 
> That should do the job.
> 
> 
> 
> ChrisA

Thanks Chris, can you elaborate on the accept() call should block?

[toc] | [prev] | [next] | [standalone]

#44701

On Sat, May 4, 2013 at 1:37 PM, Pedro <pedro@ncf.ca> wrote:
> On Friday, May 3, 2013 10:23:38 PM UTC-4, Chris Angelico wrote:
>> The accept() call should block. It's not going to spin or anything. If
>>
>> you need to monitor multiple sockets, have a look at select().
>
> Thanks Chris, can you elaborate on the accept() call should block?

When you call accept(), your program stops running until there's a
connection. It's like calling input() (or raw_input()) and your
program stopping until you type something. You can disable that by
setting the socket nonblocking, but I don't think you're doing that
here (and you probably don't want to).

Consider the accept() call to be, effectively, like reading from the
bound socket. In many ways it functions that way.

ChrisA

[toc] | [prev] | [next] | [standalone]

#44702

On Friday, May 3, 2013 11:56:01 PM UTC-4, Chris Angelico wrote:
> On Sat, May 4, 2013 at 1:37 PM, Pedro <pedro@ncf.ca> wrote:
> 
> > On Friday, May 3, 2013 10:23:38 PM UTC-4, Chris Angelico wrote:
> 
> >> The accept() call should block. It's not going to spin or anything. If
> 
> >>
> 
> >> you need to monitor multiple sockets, have a look at select().
> 
> >
> 
> > Thanks Chris, can you elaborate on the accept() call should block?
> 
> 
> 
> When you call accept(), your program stops running until there's a
> 
> connection. It's like calling input() (or raw_input()) and your
> 
> program stopping until you type something. You can disable that by
> 
> setting the socket nonblocking, but I don't think you're doing that
> 
> here (and you probably don't want to).
> 
> 
> 
> Consider the accept() call to be, effectively, like reading from the
> 
> bound socket. In many ways it functions that way.
> 
> 
> 
> ChrisA

Brilliant explanation Chris, I swear I read three different docs on that question and could not deduce what you just wrote. Thank you.

[toc] | [prev] | [next] | [standalone]

#44706

On 4-5-2013 4:13, Pedro wrote:
> SERVER:
> 
> import socket               # Import socket module
> 
> s = socket.socket()         # Create a socket object
> host = socket.gethostname() # Get local machine name
> port = 12345                # Reserve a port for your service.
> s.bind((host, port))        # Bind to the port

This won't always work as expected, particularly on machines with multiple network
interfaces. Depending on what your situation requires, a simpler

s.bind(('', port))

can be more suitable. (empty string means INADDR_ANY)

> 
> s.listen(5)                 # Now wait for client connection.
> while True:
>    c, addr = s.accept()     # Establish connection with client.
>    print 'Got connection from', addr
>    c.send('Thank you for connecting')

The send() call returns the number of bytes actually sent. This number can be LESS than
the length of the buffer you wanted to send! So you will need a loop here to make sure
all data is actually transmitted.

The easiest way out is to use this instead:

c.sendall('Thank you for connecting')

However, this might fail due to some I/O error and then it leaves your socket in an
undetermined state because you can't tell how much of the data was actually transmitted.
(Recovering from errors is problematic if not impossible with sendall, as far as I know
the only thing you can do is just close the bad socket and create a new one)


>    c.close()                # Close the connection
> 
> CLIENT:
> import socket               # Import socket module
> 
> s = socket.socket()         # Create a socket object
> host = socket.gethostname() # Get local machine name
> port = 12345                # Reserve a port for your service.
> 
> s.connect((host, port))
> print s.recv(1024)

While this usually seems to work (especially with small buffer sizes) it has the same
problem as pointed out above with send(): recv() might not retrieve all data at once. It
returns the amount of data actually received in that call.

[warning, hairy details below]

You HAVE to make a loop here to get all chunks of data until they add up to the total
data size that you expected. (There's a flag MSG_WAITALL you can pass to recv to get
around this. But it is not available on all systems, and on some systems where it is
provided, it doesn't work correctly.) Also you need to be careful with the buffer size
passed to recv, too large and it causes problems on some systems. Around 60kb seems a
good upper bound here.

This usually also means you need to somehow tell the receiving end of the socket how
much data is to be expected, and only then send that actual data. One way to do this is
to send the length first as a struct-packed integer (4 bytes), and the data after that.
Note that even reading the 4 bytes on the receiving side to determine the expected
length might be broken up in multiple chunks: you can't even expect recv(4) to always
return those 4 bytes. So even that needs to be in a loop...

Other ways to know on the receiving end when to stop reading from the socket is to
standardize on some sort of termination character (such as '\0' or perhaps '\n'), fixed
length buffers, or to always close the socket after every single message (but that is
hugely inefficient if you need to send multiple messages).


> s.close                     # Close the socket when done
> 

Oh, you forgot the parentheses here:

s.close()


Bottom line:
Socket programming on this level is hugely complicated. It doesn't seem too bad if you
start of with these simple example programs, but that's false hope. If at all possible,
avoid direct socket programming, and use a high-level protocol or library instead
(ftp/http/some IPC library/Twisted). Let them deal with the complexity of the socket layer.


Regards

Irmen de Jong

[toc] | [prev] | [next] | [standalone]

#44707

On Sat, May 4, 2013 at 7:37 PM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote:
> Bottom line:
> Socket programming on this level is hugely complicated. It doesn't seem too bad if you
> start of with these simple example programs, but that's false hope. If at all possible,
> avoid direct socket programming, and use a high-level protocol or library instead
> (ftp/http/some IPC library/Twisted). Let them deal with the complexity of the socket layer.
>

It's not quite as bad as this paragraph seems to imply. Sure, there
are a million things that can go wrong, but once you master that, it's
the very easiest way to do cross-language, cross-platform,
cross-computer communication. Practically EVERY language can do basic
TCP sockets, but not every language has higher level bindings.

ChrisA

[toc] | [prev] | [next] | [standalone]

#44833

On Saturday, May 4, 2013 5:37:42 AM UTC-4, Irmen de Jong wrote:
> On 4-5-2013 4:13, Pedro wrote: > SERVER: > > import socket # Import socket module > > s = socket.socket() # Create a socket object > host = socket.gethostname() # Get local machine name > port = 12345 # Reserve a port for your service. > s.bind((host, port)) # Bind to the port This won't always work as expected, particularly on machines with multiple network interfaces. Depending on what your situation requires, a simpler s.bind(('', port)) can be more suitable. (empty string means INADDR_ANY) > > s.listen(5) # Now wait for client connection. > while True: > c, addr = s.accept() # Establish connection with client. > print 'Got connection from', addr > c.send('Thank you for connecting') The send() call returns the number of bytes actually sent. This number can be LESS than the length of the buffer you wanted to send! So you will need a loop here to make sure all data is actually transmitted. The easiest way out is to use this instead: c.sendall('Thank you for connecting') However, this might fail due to some I/O error and then it leaves your socket in an undetermined state because you can't tell how much of the data was actually transmitted. (Recovering from errors is problematic if not impossible with sendall, as far as I know the only thing you can do is just close the bad socket and create a new one) > c.close() # Close the connection > > CLIENT: > import socket # Import socket module > > s = socket.socket() # Create a socket object > host = socket.gethostname() # Get local machine name > port = 12345 # Reserve a port for your service. > > s.connect((host, port)) > print s.recv(1024) While this usually seems to work (especially with small buffer sizes) it has the same problem as pointed out above with send(): recv() might not retrieve all data at once. It returns the amount of data actually received in that call. [warning, hairy details below] You HAVE to make a loop here to get all chunks of data until they add up to the total data size that you expected. (There's a flag MSG_WAITALL you can pass to recv to get around this. But it is not available on all systems, and on some systems where it is provided, it doesn't work correctly.) Also you need to be careful with the buffer size passed to recv, too large and it causes problems on some systems. Around 60kb seems a good upper bound here. This usually also means you need to somehow tell the receiving end of the socket how much data is to be expected, and only then send that actual data. One way to do this is to send the length first as a struct-packed integer (4 bytes), and the data after that. Note that even reading the 4 bytes on the receiving side to determine the expected length might be broken up in multiple chunks: you can't even expect recv(4) to always return those 4 bytes. So even that needs to be in a loop... Other ways to know on the receiving end when to stop reading from the socket is to standardize on some sort of termination character (such as '\0' or perhaps '\n'), fixed length buffers, or to always close the socket after every single message (but that is hugely inefficient if you need to send multiple messages). > s.close # Close the socket when done > Oh, you forgot the parentheses here: s.close() Bottom line: Socket programming on this level is hugely complicated. It doesn't seem too bad if you start of with these simple example programs, but that's false hope. If at all possible, avoid direct socket programming, and use a high-level protocol or library instead (ftp/http/some IPC library/Twisted). Let them deal with the complexity of the socket layer. Regards Irmen de Jong

Thanks for the reply. I'm sending short strings as commands to my server machine so the socket module seems to be doing the trick reliably. I'll try to add Twisted to my arsenal though. 
Cheers

[toc] | [prev] | [next] | [standalone]

#44835

On Tue, May 7, 2013 at 1:54 AM, Pedro <pedro@ncf.ca> wrote:
> Thanks for the reply. I'm sending short strings as commands to my server machine so the socket module seems to be doing the trick reliably. I'll try to add Twisted to my arsenal though.
> Cheers

I've never used Twisted, so I can't say how good it is. All I know is
that what I learned about socket programming in C on OS/2 is still
valid on Windows and on Linux, and in every language I've ever used
(bar JavaScript and ActionScript, which are deliberately sandboxed and
thus don't have direct socket calls available). So take your pick: Go
with Twisted, and bind yourself to a particular library, or go with
BSD sockets, and do the work yourself. Like everything else in
programming, it's a tradeoff.

ChrisA

[toc] | [prev] | [next] | [standalone]

#44841

On 6 May 2013 17:05, Chris Angelico <rosuav@gmail.com> wrote:

> I've never used Twisted, so I can't say how good it is. All I know is
> that what I learned about socket programming in C on OS/2 is still
> valid on Windows and on Linux, and in every language I've ever used
> (bar JavaScript and ActionScript, which are deliberately sandboxed and
> thus don't have direct socket calls available). So take your pick: Go
> with Twisted, and bind yourself to a particular library, or go with
> BSD sockets, and do the work yourself. Like everything else in
> programming, it's a tradeoff.

OTOH Twisted can handle much more than socket programming. On the
third hand Twisted has its own learning curve as well...

-- 
Arnaud

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web