Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #3585

Re: Pickling over a socket

Show key headers only | View raw

On Tue, Apr 19, 2011 at 11:53 AM, Roger Alexander <rtalexander@mac.com> wrote:
> Hi,
>
> I'm trying to understand how to pickle Python objects over a TCP
> socket.
>
> In the example below (based on code from Foundations of Python Network
> Programming), a client creates a dictionary (lines 34-38) and uses
> pickle.dump at line 42 to write the pickled object using file handle
> make from a socket. The server de-pickles with pickle.load  (line 24),
> again using a file handle made from a socket.
>
> When I run the program, the following output is produced:
>
>    Listening at ('127.0.0.1', 1060)
>    Accepted connection from ('127.0.0.1', 49938)
>    Traceback (most recent call last):
>    File "pickles.py", line 24, in <module>
>        d = pickle.load( s_fh )
>    File "/usr/local/lib/python2.7/pickle.py", line 1378, in load
>        return Unpickler(file).load()
>    File "/usr/local/lib/python2.7/pickle.py", line 857, in load
>        key = read(1)
>    File "/usr/local/lib/python2.7/socket.py", line 380, in read
>        data = self._sock.recv(left)
>    socket.error: [Errno 107] Transport endpoint is not connected
>
> I'm at a loss, can anyone provide any guidance?
>
> Thanks,
>
> Roger Alexander

I played around with it until something worked, and ended up with the
below.  The most significant change was probably using sc.makefile
instead of s.makefile in the server, but I seemed to need some data
framing too despite the pickling.  It's possible you won't need that
if you just flush your file in the client; I don't much pickling
experience - in particular, I don't know if you can concatenate
pickled objects and load them serially from a file-like object without
any (additional) framing.

I like to use bufsock for this sort of stuff, but I'm probably unique
in that.  ^_^   http://stromberg.dnsalias.org/~strombrg/bufsock.html

#!/usr/bin/python

import time
import pickle
import socket, sys
import pprint

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

HOST = sys.argv.pop() if len(sys.argv) == 3 else '127.0.0.1'
PORT = 1060

if sys.argv[1:] == ['server']:

    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind((HOST, PORT))
    s.listen(1)

    print 'Listening at', s.getsockname()

    sc, sockname = s.accept()

    print 'Accepted connection from', sockname

    sf = sc.makefile( "rb" )

    length_list = []
    while True:
        char = sf.read(1)
        if char == '\n':
            break
        else:
            length_list.append(int(char))
    length = 0
    for digit in length_list:
        length = length * 10 + digit
    data = sf.read(length)

    d = pickle.loads(data)

    pprint.pprint(d)

    sc.shutdown(socket.SHUT_RDWR)
    sc.close()
    s.close()

elif sys.argv[1:] == ['client']:

    s.connect((HOST, PORT))
    # s.shutdown(socket.SHUT_RD)

    d = dict()

    d[ 'Name' ]     = 'Jake Thompson.'
    d[ 'Age' ]      = 25
    d[ 'Location' ] = 'Washington, D.C.'

    sf = s.makefile( "wb" )

    string = pickle.dumps( d, pickle.HIGHEST_PROTOCOL )
    sf.write('%d\n' % len(string))
    sf.write(string)
    sf.flush()

    #time.sleep(10)
    sf.close()
    s.shutdown(socket.SHUT_RDWR)
    # s.close()

else:
    print >>sys.stderr, 'usage: streamer.py server|client [host]'

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Pickling over a socket Roger Alexander <rtalexander@mac.com> - 2011-04-19 11:53 -0700
  Re: Pickling over a socket Chris Rebert <clp2@rebertia.com> - 2011-04-19 12:21 -0700
  Re: Pickling over a socket Chris Angelico <rosuav@gmail.com> - 2011-04-20 05:29 +1000
  Re: Pickling over a socket Dan Stromberg <drsalists@gmail.com> - 2011-04-19 12:30 -0700
  Re: Pickling over a socket Chris Angelico <rosuav@gmail.com> - 2011-04-20 05:37 +1000
    Re: Pickling over a socket Roger Alexander <rtalexander@mac.com> - 2011-04-19 15:27 -0700
      Re: Pickling over a socket Jean-Paul Calderone <calderone.jeanpaul@gmail.com> - 2011-04-19 19:28 -0700
        Re: Pickling over a socket Bastian Ballmann <balle@chaostal.de> - 2011-04-20 08:44 +0200
        Re: Pickling over a socket Chris Angelico <rosuav@gmail.com> - 2011-04-20 16:59 +1000
        Re: Pickling over a socket Bastian Ballmann <balle@chaostal.de> - 2011-04-20 09:34 +0200
          Re: Pickling over a socket Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-04-20 10:25 +0200
            [OT] Re: Pickling over a socket Bastian Ballmann <balle@chaostal.de> - 2011-04-20 10:59 +0200
        Re: Pickling over a socket Chris Angelico <rosuav@gmail.com> - 2011-04-20 19:26 +1000
        Re: Pickling over a socket Bastian Ballmann <balle@chaostal.de> - 2011-04-20 11:41 +0200

csiph-web