Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Sun, 10 Feb 2013 20:22:15 -0500
From: Dave Angel <davea@davea.name>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Python recv loop
References: <B52841CC-778D-41AF-AFCC-05A5F9C26A92@grep.my>
In-Reply-To: <B52841CC-778D-41AF-AFCC-05A5F9C26A92@grep.my>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1614.1360545756.2939.python-list@python.org>
Lines: 57
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:38627

On 02/10/2013 07:48 PM, Ihsan Junaidi Ibrahim wrote:
> Hi,
>
> I'm implementing a python client connecting to a C-backend server and am currently stuck to as to how to proceed with receiving variable-length byte stream coming in from the server.
>
> I have coded the first 4 bytes (in hexadecimal) of message coming in from the server to specify the length of the message payload i.e. 0xad{...}

Exactly how are you sending "hexadecimal" ?  If that 0xad (which is only 
one byte, what about the other 3 ?) is intended to be a C description, 
then it's certainly not hex, it's binary.  And probably little-endian, 
to boot.  That's a mistake, as network protocols almost always use 
big-endian.

So what is the range of values for the length, and how are they actually 
encoded?  Are they uint32 in native binary format?   And are you 
permitted to change the encoding, to fix it?  (If it were my choice, I'd 
make it a printable decimal value, first choice, or printable hex, 
second choice.)

>
> I've managed to receive and translate the message length until I reach my second recv which I readjusted the buffer size to include the new message length.
>
> However that failed and recv received 0 bytes. I implemented the same algorithm on the server side using C and it work so appreciate if you can help me on this.
>
> # receive message length
>      print 'receiving data'
>      mlen = sock.recv(4)
>      try:
>          nbuf = int(mlen, 16)

That supposes the count is being sent as a printable string of hex 
digits.  That's not what I concluded above.

>      except ValueError as e:

If the count starts as 0xad, I can't imagine why this exception wasn't 
thrown.

>          print 'invalid length type'
>          return -1
>
>      while True:
>          buf = sock.recv(nbuf)
>
>          if not buf:
>              break
>
>      slen = len(buf)
>      str = "{0} bytes received: {1}".format(slen, buf)
>      print str
>

You might need to play with hexlify, if you persist in sending the count 
in binary.

-- 
DaveA