Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!news2.arglkargh.de!news.wiretrip.org!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=JO/+DvNPZeSwGgqkJELh5pjUzUNhLOySEMsG6IIApOzG1bNOcN5E2xbYhRVq7mSq2d Bgpu3BGp4dSXYkvEVjde1ZiKjv90gWObr6XMMKLMI3934p1Xg4+LTbPPUO28KIiFumvm /ycTigYaimgGBPab2YgZ1idmN0wfN43Mk7YVY=
MIME-Version: 1.0
In-Reply-To: <61890800-f81a-4a1e-8905-a0237407f016@a21g2000prj.googlegroups.com>
References: <61890800-f81a-4a1e-8905-a0237407f016@a21g2000prj.googlegroups.com>
Date: Tue, 19 Apr 2011 12:30:53 -0700
Subject: Re: Pickling over a socket
From: Dan Stromberg <drsalists@gmail.com>
To: Roger Alexander <rtalexander@mac.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.581.1303241455.9059.python-list@python.org>
Lines: 117
NNTP-Posting-Host: 82.94.164.166
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:3585

On Tue, Apr 19, 2011 at 11:53 AM, Roger Alexander <rtalexander@mac.com> wro=
te:
> Hi,
>
> I'm trying to understand how to pickle Python objects over a TCP
> socket.
>
> In the example below (based on code from Foundations of Python Network
> Programming), a client creates a dictionary (lines 34-38) and uses
> pickle.dump at line 42 to write the pickled object using file handle
> make from a socket. The server de-pickles with pickle.load =A0(line 24),
> again using a file handle made from a socket.
>
> When I run the program, the following output is produced:
>
> =A0 =A0Listening at ('127.0.0.1', 1060)
> =A0 =A0Accepted connection from ('127.0.0.1', 49938)
> =A0 =A0Traceback (most recent call last):
> =A0 =A0File "pickles.py", line 24, in <module>
> =A0 =A0 =A0 =A0d =3D pickle.load( s_fh )
> =A0 =A0File "/usr/local/lib/python2.7/pickle.py", line 1378, in load
> =A0 =A0 =A0 =A0return Unpickler(file).load()
> =A0 =A0File "/usr/local/lib/python2.7/pickle.py", line 857, in load
> =A0 =A0 =A0 =A0key =3D read(1)
> =A0 =A0File "/usr/local/lib/python2.7/socket.py", line 380, in read
> =A0 =A0 =A0 =A0data =3D self._sock.recv(left)
> =A0 =A0socket.error: [Errno 107] Transport endpoint is not connected
>
> I'm at a loss, can anyone provide any guidance?
>
> Thanks,
>
> Roger Alexander

I played around with it until something worked, and ended up with the
below.  The most significant change was probably using sc.makefile
instead of s.makefile in the server, but I seemed to need some data
framing too despite the pickling.  It's possible you won't need that
if you just flush your file in the client; I don't much pickling
experience - in particular, I don't know if you can concatenate
pickled objects and load them serially from a file-like object without
any (additional) framing.

I like to use bufsock for this sort of stuff, but I'm probably unique
in that.  ^_^   http://stromberg.dnsalias.org/~strombrg/bufsock.html

#!/usr/bin/python

import time
import pickle
import socket, sys
import pprint

s =3D socket.socket(socket.AF_INET, socket.SOCK_STREAM)

HOST =3D sys.argv.pop() if len(sys.argv) =3D=3D 3 else '127.0.0.1'
PORT =3D 1060

if sys.argv[1:] =3D=3D ['server']:

    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind((HOST, PORT))
    s.listen(1)

    print 'Listening at', s.getsockname()

    sc, sockname =3D s.accept()

    print 'Accepted connection from', sockname

    sf =3D sc.makefile( "rb" )

    length_list =3D []
    while True:
        char =3D sf.read(1)
        if char =3D=3D '\n':
            break
        else:
            length_list.append(int(char))
    length =3D 0
    for digit in length_list:
        length =3D length * 10 + digit
    data =3D sf.read(length)

    d =3D pickle.loads(data)

    pprint.pprint(d)

    sc.shutdown(socket.SHUT_RDWR)
    sc.close()
    s.close()

elif sys.argv[1:] =3D=3D ['client']:

    s.connect((HOST, PORT))
    # s.shutdown(socket.SHUT_RD)

    d =3D dict()

    d[ 'Name' ]     =3D 'Jake Thompson.'
    d[ 'Age' ]      =3D 25
    d[ 'Location' ] =3D 'Washington, D.C.'

    sf =3D s.makefile( "wb" )

    string =3D pickle.dumps( d, pickle.HIGHEST_PROTOCOL )
    sf.write('%d\n' % len(string))
    sf.write(string)
    sf.flush()

    #time.sleep(10)
    sf.close()
    s.shutdown(socket.SHUT_RDWR)
    # s.close()

else:
    print >>sys.stderr, 'usage: streamer.py server|client [host]'