Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #81683 > unrolled thread

Iterate over text file, discarding some lines via context manager

Back to article view | Back to comp.lang.python

Contents

  Iterate over text file, discarding some lines via context manager "fetchinson ." <fetchinson@googlemail.com> - 2014-11-28 16:04 +0100
    Re: Iterate over text file, discarding some lines via context manager "ast" <nomail@invalid.com> - 2014-11-28 18:06 +0100
      Re: Iterate over text file, discarding some lines via context manager Dave Angel <davea@davea.name> - 2014-11-28 12:40 -0500
        Re: Iterate over text file, discarding some lines via context manager Marko Rauhamaa <marko@pacujo.net> - 2014-11-28 19:55 +0200
          Re: Iterate over text file, discarding some lines via context manager Chris Angelico <rosuav@gmail.com> - 2014-11-29 05:02 +1100
            Re: Iterate over text file, discarding some lines via context manager Marko Rauhamaa <marko@pacujo.net> - 2014-11-28 20:04 +0200

#81683 — Iterate over text file, discarding some lines via context manager

Hi all,

I have a feeling that I should solve this by a context manager but
since I've never used them I'm not sure what the optimal (in the
python sense) solution is. So basically what I do all the time is
this:

for line in open( 'myfile' ):
    if not line:
        # discard empty lines
        continue
    if line.startswith( '#' ):
        # discard lines starting with #
        continue
    items = line.split( )
    if not items:
        # discard lines with only spaces, tabs, etc
        continue

    process( items )

You see I'd like to ignore lines which are empty, start with a #, or
are only white space. How would I write a context manager so that the
above simply becomes

with some_tricky_stuff( 'myfile' ) as items:
    process( items )

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

[toc] | [next] | [standalone]

#81694

Hi

Here is a solution with a custom iterator which operate on files.
I tested it with a small file. Just a remark, there are no empty line
on a file, there is at least '\n' at the end of each lines but maybe the 
last one. If readline() got an emptyline, then the end of file has been 
reached.


class MyFileItr:
    
    def __init__(self, f):
        self.f = f
        self.line = ""

    def __next__(self):

        while True:
            
            self.line = self.f.readline()

            if (self.line == ''):
                raise StopIteration
       
            if not(self.line.startswith('#') or self.line.isspace() or self.line == '\n'):
                
                return self.line

    def __iter__(self):
        return self


        
f = open('test.txt', 'r')

for L in MyFileItr(f):
    print(L, end="")

[toc] | [prev] | [next] | [standalone]

#81697

Please don't start a new thread when responding to an existing topic. 
Just reply-List to the message you're responding to, and include some 
context from that message.

On 11/28/2014 12:06 PM, ast wrote:
> Hi
>
> Here is a solution with a custom iterator which operate on files.
> I tested it with a small file. Just a remark, there are no empty line
> on a file, there is at least '\n' at the end of each lines but maybe the
> last one. If readline() got an emptyline, then the end of file has been
> reached.
>
>
> class MyFileItr:
>     def __init__(self, f):
>         self.f = f
>         self.line = ""
>
>     def __next__(self):
>
>         while True:
>             self.line = self.f.readline()
>
>             if (self.line == ''):
>                 raise StopIteration
>             if not(self.line.startswith('#') or self.line.isspace() or
> self.line == '\n'):
>                 return self.line
>
>     def __iter__(self):
>         return self
>
>
> f = open('test.txt', 'r')
>
> for L in MyFileItr(f):
>     print(L, end="")
>

Why would you prefer that over a generator function, as given earlier in 
the thread?  See for example Ned's message.  By using 'yield', you get 
Python to generate all the class boilerplate for you.



-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#81698

Dave Angel <davea@davea.name>:

> Why would you prefer that over a generator function, as given earlier
> in the thread? See for example Ned's message. By using 'yield', you
> get Python to generate all the class boilerplate for you.

I think the OP can learn from the comparison. One question, many
lessons:

 * Here's how you write a generator.

 * Here's how you write a context manager class. You run into those
   quite often as well.

 * See how much more suitable a generator is in this case.

No need to shoot down those who only try to be helpful.


Marko

[toc] | [prev] | [next] | [standalone]

#81699

On Sat, Nov 29, 2014 at 4:55 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> I think the OP can learn from the comparison. One question, many
> lessons:
>
>  * Here's how you write a generator.
>
>  * Here's how you write a context manager class. You run into those
>    quite often as well.
>
>  * See how much more suitable a generator is in this case.
>
> No need to shoot down those who only try to be helpful.

Since we're so good at it on this list, I will nit-pick: that's not a
context manager class, that's an iterator class. A context manager has
__enter__ and __exit__, an iterator has __iter__ (returning self) and
__next__ (returning or raising StopIteration).

ChrisA

[toc] | [prev] | [next] | [standalone]

#81700

Chris Angelico <rosuav@gmail.com>:

> Since we're so good at it on this list, I will nit-pick: that's not a
> context manager class, that's an iterator class. A context manager has
> __enter__ and __exit__, an iterator has __iter__ (returning self) and
> __next__ (returning or raising StopIteration).

Excellent. Learnings will never end.


Marko

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web