Path: csiph.com!usenet.pasdenom.info!dedibox.gegeweb.org!gegeweb.eu!nntpfeed.proxad.net!proxad.net!feeder1-2.proxad.net!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Sun, 28 Oct 2012 06:31:10 -0700
From: Ethan Furman <ethan@stoneleaf.us>
User-Agent: Thunderbird 1.5.0.10 (Windows/20070221)
MIME-Version: 1.0
To: Travis Griggs <travisgriggs@gmail.com>
Subject: Re: Style help for a Smalltalk-hack
References: <168DF25C-0C43-45F2-BEDC-17AC4210F207@gmail.com>
In-Reply-To: <168DF25C-0C43-45F2-BEDC-17AC4210F207@gmail.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.2970.1351431671.27098.python-list@python.org>
Lines: 51
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:32312

Travis Griggs wrote:
> I'm writing some code that does a structured read from formatted binary file. The code I came up with looks like:
> 
> # get the first four bytes, the first gap field
> chunk = byteStream.read(4)
> while chunk:
>     # interpret the gap bytes
>     gap, = struct.unpack('>I', chunk)
>     # suck off the valveCount
>     valveCount, = struct.unpack('>I', byteStream.read(4))
>     # collect the next valveCount signatures
>     signatures = [struct.unpack('>I', byteStream.read(4))[0] for _ in range(valveCount)]
>     self.script.append(ScriptSpan(gap=gap, valveSet=signatures))
>     # now get the next 4 bytes for the gap of the next iteration, it'll be empty if we're at end
>     chunk = byteStream.read(4)
> 
> I can't help but thinking that there's some better way (i.e. more pythonic) to do this that doesn't involve having to use another module (Construct) or exploring generators or something like that. What bugs me about it is that there is two different styles for reading/decoding values from the byte stream. valveCount and signatures are both paired invocations of unpack() and read(). But to detect the end of the stream (file), I have to split the read() and unpack() of the gap value across 3 different lines of the code, and they don't even sit adjacent to each other.

What do you have against generators?  ;)

How about an iterator approach:

8<-------------------------------------------------------
class Gap_Valve_Reader(object):  # untested
     def __init__(self, filename):
         self.byteStream = open(filename)
     def __iter__(self):
         return self
     def __next__(self):                 # python 3 name
         chunk = self.byteStream.read(8)
         if not chunk:
             self.byteStream.close()
             raise StopIteration
         gap, valveCount = struct.unpack('>II', chunk)
         signatures = struct.unpack('>%sI' % valveCount,
                     self.byteStream.read(4 * valueCount))
         return gap, signatures
     next = __next__  # python 2 name
8<-------------------------------------------------------

then your main code can look like:

8<-------------------------------------------------------
gvr = Gap_Valve_Reader('binary_file.bin')
for gap, signatures in gvr:
     self.script.append(ScriptSpan(gap=gap, valveSet=signatures))
8<-------------------------------------------------------

Don't forget to add error checking where appropriate.

~Ethan~