Path: csiph.com!usenet.pasdenom.info!news.etla.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Date: Fri, 28 Nov 2014 11:15:32 -0500
From: Dave Angel <davea@davea.name>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Iterate over text file, discarding some lines via context manager
References: <CADjSo4Ro3xFj9T9gj0NO7YESE6G873vvrBD=Y=OAk-jNBRD5Bw@mail.gmail.com> <5478932A.1030509@davea.name> <m5a67t$qk4$1@ger.gmane.org>
In-Reply-To: <m5a67t$qk4$1@ger.gmane.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.16411.1417191349.18130.python-list@python.org>
Lines: 82
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:81691

On 11/28/2014 11:01 AM, Ned Batchelder wrote:
> On 11/28/14 10:22 AM, Dave Angel wrote:
>> On 11/28/2014 10:04 AM, fetchinson . wrote:
>>> Hi all,
>>>
>>> I have a feeling that I should solve this by a context manager but
>>> since I've never used them I'm not sure what the optimal (in the
>>> python sense) solution is. So basically what I do all the time is
>>> this:
>>>
>>> for line in open( 'myfile' ):
>>>      if not line:
>>>          # discard empty lines
>>>          continue
>>>      if line.startswith( '#' ):
>>>          # discard lines starting with #
>>>          continue
>>>      items = line.split( )
>>>      if not items:
>>>          # discard lines with only spaces, tabs, etc
>>>          continue
>>>
>>>      process( items )
>>>
>>> You see I'd like to ignore lines which are empty, start with a #, or
>>> are only white space. How would I write a context manager so that the
>>> above simply becomes
>>>
>>> with some_tricky_stuff( 'myfile' ) as items:
>>>      process( items )
>>>
>>
>> I see what you're getting at, but a context manager is the wrong
>> paradigm.  What you want is a generator.   (untested)
>>
>> def mygenerator(filename):
>>      with open(filename) as f:
>>          for line in f:
>>              if not line: continue
>>              if line.startswith('#'): continue
>>              items = line.split()
>>              if not items: continue
>>              yield items
>>
>> Now your caller simply does:
>>
>> for items in mygenerator(filename):
>>        process(items)
>>
>>
>
> I think it's slightly better to leave the open outside the generator:
>
> def interesting_lines(f):
>      for line in f:
>          line = line.strip()
>          if line.startswith('#'):
>              continue
>          if not line:
>              continue
>          yield line
>
> with open("my_config.ini") as f:
>      for line in interesting_lines(f):
>          do_something(line)
>
> This makes interesting_lines a pure filter, and doesn't care what sort
> of sequence of strings it's operating on.  This makes it easier to test,
> and more flexible.  The caller's code is also clearer in my opinion.
>
Thank you, I agree.  I was trying to preserve the factoring that the OP 
had implied.  I notice you also factored out the split.

> BTW: this example is taken verbatim from my PyCon presentation on
> iteration, it you are interested: http://nedbatchelder.com/text/iter.html
>

Thanks for the link.  I've started reading it, and I'll definitely read 
the whole thing.

-- 
DaveA