Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #9318 > unrolled thread

How to write a file generator

Back to article view | Back to comp.lang.python

Contents

  How to write a file generator Billy Mays <noway@nohow.com> - 2011-07-12 10:46 -0400
    Re: How to write a file generator "bruno.desthuilliers@gmail.com" <bruno.desthuilliers@gmail.com> - 2011-07-12 08:34 -0700
    Re: How to write a file generator Thomas Jollans <t@jollybox.de> - 2011-07-12 17:52 +0200
      Re: How to write a file generator Billy Mays <noway@nohow.com> - 2011-07-12 12:42 -0400
        Re: How to write a file generator Thomas Jollans <t@jollybox.de> - 2011-07-12 21:04 +0200
        Re: How to write a file generator Thomas Jollans <t@jollybox.de> - 2011-07-12 21:05 +0200
    Re: How to write a file generator sturlamolden <sturlamolden@yahoo.no> - 2011-07-12 09:53 -0700
    Re: How to write a file generator Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-07-13 01:26 +0200

#9318 — How to write a file generator

I want to make a generator that will return lines from the tail of 
/var/log/syslog if there are any, but my function is reopening the file 
each call:

def getLines():
     with open('/var/log/syslog', 'rb') as f:
         while True:
             line = f.readline()
             if line:
                 yield line
             else:
                 raise StopIteration


I know the problem lies with the StopIteration, but I'm not sure how to 
tell the caller that there are no more lines for now.

--
Bill

[toc] | [next] | [standalone]

#9321

On Jul 12, 4:46 pm, Billy Mays <no...@nohow.com> wrote:
> I want to make a generator that will return lines from the tail of
> /var/log/syslog if there are any

Err... I must have missed something, but python files are their own
iterators.

Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
pythonrc start
pythonrc done
>>> f = open("/var/log/syslog")
>>> for line in f:
...     print line
...
(snip unintersting syslog stuff))

>, but my function is reopening the file
> each call:

How do you know, and how do you call your function ?

> def getLines():
>      with open('/var/log/syslog', 'rb') as f:
>          while True:
>              line = f.readline()
>              if line:
>                  yield line
>              else:
>                  raise StopIteration
>
> I know the problem lies with the StopIteration, but I'm not sure how to
> tell the caller that there are no more lines for now.

If you want the generator to wait until new content is available, just
remove the raise part - but you'll have a blocking call... Else, I
don't see what behaviour you are expecting exactly.

[toc] | [prev] | [next] | [standalone]

#9322

On 07/12/2011 04:46 PM, Billy Mays wrote:
> I want to make a generator that will return lines from the tail of
> /var/log/syslog if there are any, but my function is reopening the file
> each call:
> 
> def getLines():
>     with open('/var/log/syslog', 'rb') as f:
>         while True:
>             line = f.readline()
>             if line:
>                 yield line
>             else:
>                 raise StopIteration
> 
> 
> I know the problem lies with the StopIteration, but I'm not sure how to
> tell the caller that there are no more lines for now.
> 
> -- 
> Bill

http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep

[toc] | [prev] | [next] | [standalone]

#9325

On 07/12/2011 11:52 AM, Thomas Jollans wrote:
> On 07/12/2011 04:46 PM, Billy Mays wrote:
>> I want to make a generator that will return lines from the tail of
>> /var/log/syslog if there are any, but my function is reopening the file
>> each call:
>>
>> def getLines():
>>      with open('/var/log/syslog', 'rb') as f:
>>          while True:
>>              line = f.readline()
>>              if line:
>>                  yield line
>>              else:
>>                  raise StopIteration
>>
>>
>> I know the problem lies with the StopIteration, but I'm not sure how to
>> tell the caller that there are no more lines for now.
>>
>> --
>> Bill
>
> http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep


That was actually the behavior I was trying to avoid.  If there is no 
data to be read, the call will hang.  That function is actually called 
by a webserver (from wsgiref) so it cannot hang indefinitely.

--
Bill

[toc] | [prev] | [next] | [standalone]

#9341

On 07/12/2011 06:42 PM, Billy Mays wrote:
> On 07/12/2011 11:52 AM, Thomas Jollans wrote:
>> On 07/12/2011 04:46 PM, Billy Mays wrote:
>>> I want to make a generator that will return lines from the tail of
>>> /var/log/syslog if there are any, but my function is reopening the file
>>> each call:
>>>
>>> def getLines():
>>>      with open('/var/log/syslog', 'rb') as f:
>>>          while True:
>>>              line = f.readline()
>>>              if line:
>>>                  yield line
>>>              else:
>>>                  raise StopIteration
>>>
>>>
>>> I know the problem lies with the StopIteration, but I'm not sure how to
>>> tell the caller that there are no more lines for now.
>>>
>>> -- 
>>> Bill
>>
>> http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep
>>
> 
> 
> That was actually the behavior I was trying to avoid.  If there is no
> data to be read, the call will hang.  That function is actually called
> by a webserver (from wsgiref) so it cannot hang indefinitely.

In that case, what Bruno said.

[toc] | [prev] | [next] | [standalone]

#9342

On 07/12/2011 06:42 PM, Billy Mays wrote:
> On 07/12/2011 11:52 AM, Thomas Jollans wrote:
>> On 07/12/2011 04:46 PM, Billy Mays wrote:
>>> I want to make a generator that will return lines from the tail of
>>> /var/log/syslog if there are any, but my function is reopening the file
>>> each call:
>>>
>>> def getLines():
>>>      with open('/var/log/syslog', 'rb') as f:
>>>          while True:
>>>              line = f.readline()
>>>              if line:
>>>                  yield line
>>>              else:
>>>                  raise StopIteration
>>>
>>>
>>> I know the problem lies with the StopIteration, but I'm not sure how to
>>> tell the caller that there are no more lines for now.
>>>
>>> -- 
>>> Bill
>>
>> http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep
>>
> 
> 
> That was actually the behavior I was trying to avoid.  If there is no
> data to be read, the call will hang.  That function is actually called
> by a webserver (from wsgiref) so it cannot hang indefinitely.


What Terry said, then. (Not Bruno, I confused that. Sorry for sending a
mail both short and wrong.)

[toc] | [prev] | [next] | [standalone]

#9327

On 12 Jul, 16:46, Billy Mays <no...@nohow.com> wrote:

> I know the problem lies with the StopIteration, but I'm not sure how to
> tell the caller that there are no more lines for now.

Try 'yield None' instead of 'raise StopIteration'.

Sturla

[toc] | [prev] | [next] | [standalone]

#9355

Am 12.07.2011 16:46 schrieb Billy Mays:
> I want to make a generator that will return lines from the tail of
> /var/log/syslog if there are any, but my function is reopening the file
> each call

...

I have another solution: an object which is not an iterator, but an 
iterable.

class Follower(object):
     def __init__(self, file):
         self.file = file
     def __iter__(self):
         while True:
             l = self.file.readline()
             if not l: return
             yield l

if __name__ == '__main__':
     f = Follower(open("/var/log/messages"))
     while True:
         for i in f: print i,
         print "foo"
         import time
         time.sleep(4)

Here, you iterate over the object until it is exhausted, but you can 
iterate again to get the next entries.


Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web