Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #56171 > unrolled thread

How to streamingly read text file and display whenever updated text

Back to article view | Back to comp.lang.python

Contents

  How to streamingly read text file and display whenever updated text galeomaga@gmail.com - 2013-10-05 00:38 -0700
    Re: How to streamingly read text file and display whenever updated text galeomaga@gmail.com - 2013-10-05 00:54 -0700
      Re: How to streamingly read text file and display whenever updated text Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-05 09:12 +0100
    Re: How to streamingly read text file and display whenever updated text "James Harris" <james.harris.1@gmail.com> - 2013-10-05 09:06 +0100
      Re: How to streamingly read text file and display whenever updated text Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-05 09:14 +0100
    Re: How to streamingly read text file and display whenever updated text Nobody <nobody@nowhere.com> - 2013-10-05 11:25 +0100
    Re: How to streamingly read text file and display whenever updated text Joost Molenaar <j.j.molenaar@gmail.com> - 2013-10-05 13:02 +0200
      Re: How to streamingly read text file and display whenever updated text galeomaga@gmail.com - 2013-10-05 20:17 -0700
        Re: How to streamingly read text file and display whenever updated text Chris Angelico <rosuav@gmail.com> - 2013-10-06 14:28 +1100
        Re: How to streamingly read text file and display whenever updated text Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-10-06 04:06 +0000
          Re: How to streamingly read text file and display whenever updated text galeomaga@gmail.com - 2013-10-06 01:49 -0700
          Re: How to streamingly read text file and display whenever updated text Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-06 11:36 +0100
          Re: How to streamingly read text file and display whenever updated text Chris Angelico <rosuav@gmail.com> - 2013-10-06 22:03 +1100
          Re: How to streamingly read text file and display whenever updated text Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-06 12:13 +0100
          Re: How to streamingly read text file and display whenever updated text Chris Angelico <rosuav@gmail.com> - 2013-10-06 22:15 +1100
          Re: How to streamingly read text file and display whenever updated text Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-06 12:31 +0100
          Re: How to streamingly read text file and display whenever updated text Chris Angelico <rosuav@gmail.com> - 2013-10-06 22:44 +1100
            Re: How to streamingly read text file and display whenever updated text galeomaga@gmail.com - 2013-10-06 18:54 -0700
              Re: How to streamingly read text file and display whenever updated text Andreas Perstinger <andipersti@gmail.com> - 2013-10-07 11:52 +0200
    Re: How to streamingly read text file and display whenever updated text Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-05 12:08 +0100

#56171 — How to streamingly read text file and display whenever updated text

#!/usr/bin/python
import time
f = open('/home/martin/Downloads/a.txt')
while 1:
	for line in f:
		print line;
	time.sleep(1);

[toc] | [next] | [standalone]

#56173

gale...@gmail.com於 2013年10月5日星期六UTC+8下午3時38分51秒寫道：
> #!/usr/bin/python
> 
> import time
> 
> f = open('/home/martin/Downloads/a.txt')
> 
> while 1:
> 
> 	for line in f:
> 
> 		print line;
> 
> 	time.sleep(1);


if __name__ == '__main__':
    logfile = open("/home/martin/Downloads/a.txt","r");
    while True:
        line = logfile.readline();
        if not line:
		print line;    
        time.sleep(1);

this also failed

[toc] | [prev] | [next] | [standalone]

#56175

On 05/10/2013 08:54, galeomaga@gmail.com wrote:
>
> if __name__ == '__main__':
>      logfile = open("/home/martin/Downloads/a.txt","r");
>      while True:
>          line = logfile.readline();
>          if not line:
> 		print line;
>          time.sleep(1);
>
> this also failed
>

Usually please state your OS and Python versions, what you expected to 
happen, what actually happened and the full traceback if applicable.  In 
this case I'd hazard a guess that as you're trying to print something 
that evaluates to false you're not likely to see much.  You can also 
remove the semicolons as they're simply not needed.

-- 
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#56174

<galeomaga@gmail.com> wrote in message 
news:04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com...
>
>
> #!/usr/bin/python
> import time
> f = open('/home/martin/Downloads/a.txt')

Looks like you are on Unix so you can do this from the shell

  tail -F /home/martin/Downloads/a.txt

James

[toc] | [prev] | [next] | [standalone]

#56176

On 05/10/2013 09:06, James Harris wrote:
> <galeomaga@gmail.com> wrote in message
> news:04ee91f9-1cbf-4364-bca3-da25aa4db45f@googlegroups.com...
>>
>>
>> #!/usr/bin/python
>> import time
>> f = open('/home/martin/Downloads/a.txt')
>
> Looks like you are on Unix so you can do this from the shell
>
>    tail -F /home/martin/Downloads/a.txt
>
> James
>
>

Tail also works on Windows if you've unxutils installed :)

-- 
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#56177

On Sat, 05 Oct 2013 00:38:51 -0700, galeomaga wrote:

> #!/usr/bin/python
> import time
> f = open('/home/martin/Downloads/a.txt')
> while 1:
> 	for line in f:
> 		print line;
> 	time.sleep(1);

So you're trying to implement "tail -f"?

First, check that "tail -f" actually works for your particular use case.

If the process writing the file uses buffered output, data will only
actually be appended to the file when the buffer is full. You can't read
what isn't there.

And if the process creates a new file with the same name, rather than
appending to the existing file, you'll still be reading the old file. You
would need to open the file again to read the new file.

[toc] | [prev] | [next] | [standalone]

#56178

A bit of googling found me this:
http://www.linux-support.com/cms/implementation-of-tail-in-python/

import time
import sys

def tail_f(file):
  interval = 1.0
  while True:
    where = file.tell()
    line = file.readline()
    if not line:
      time.sleep(interval)
      file.seek(where)
    else:
      yield line

[toc] | [prev] | [next] | [standalone]

#56233

Joost Molenaar於 2013年10月5日星期六UTC+8下午7時02分05秒寫道：
> A bit of googling found me this:
> 
> http://www.linux-support.com/cms/implementation-of-tail-in-python/
> 
> 
> 
> import time
> 
> import sys
> 
> 
> 
> def tail_f(file):
> 
>   interval = 1.0
> 
>   while True:
> 
>     where = file.tell()
> 
>     line = file.readline()
> 
>     if not line:
> 
>       time.sleep(interval)
> 
>       file.seek(where)
> 
>     else:
> 
>       yield line

After tried many times, updated text file is not shown, it only print text at the first time. 

#!/usr/bin/python
import time 
import sys 
import thread

def tail_f(filehandler): 
  interval = 1.0 
  while True: 
	try:
		line = filehandler.readline()	
		where = filehandler.tell()
		if not line: 
			time.sleep(interval) 
			filehandler.seek(where) 
		else: 
			yield line 
	except:
		print "tail_f error"
	

def readfile(systemname):
	try:
		filehandler = open("/home/martin/Downloads/a.txt","r");
		while 1:
			#for line in tail_f(filehandler):
			#	print line 
			try:				
				interval = 1.0 
				line = filehandler.readline()	
				where = filehandler.tell()
				if not line: 
					time.sleep(interval) 
					filehandler.seek(where) 
					print where
				else: 
					print line 
			except:
				print "tail_f error"
	except:
		print "Error: readfile"

if __name__ == '__main__':
	try:
		thread.start_new_thread( readfile, ("Thread-1", ) )
	except:
   		print "Error: unable to start thread"

	while 1:
   		pass

[toc] | [prev] | [next] | [standalone]

#56234

On Sun, Oct 6, 2013 at 2:17 PM,  <galeomaga@gmail.com> wrote:
> After tried many times, updated text file is not shown, it only print text at the first time.

The implementation of tail has a lot of little oddities to deal with
edge cases. Why not simply use it?

A while ago, I wanted to make a system that would tail a bunch of logs
on a bunch of computers, and display it all to me in a single unified
view. Rather than write something that opened a whole lot of files and
monitored them, I simply forked a 'tail' process for each file and
reacted to its stdout. It was way WAY easier than dealing with
everything that could possibly happen (log rotation, etc, etc, etc) -
not that it'd be impossible to deal with, but it's a waste of time
reinventing this particular wheel. Build on top of what's already
there, save yourself the trouble.

ChrisA

[toc] | [prev] | [next] | [standalone]

#56236

On Sat, 05 Oct 2013 20:17:32 -0700, galeomaga wrote:


> if __name__ == '__main__':
> 	try:
> 		thread.start_new_thread( readfile, ("Thread-1", ) )
> 	except:
>    		print "Error: unable to start thread"


Why not? If you can't start a thread, you have a problem with your code. 
How do you expect to debug this problem?


"I find it amusing when novice programmers believe their main job is 
preventing programs from crashing. More experienced programmers realize 
that correct code is great, code that crashes could use improvement, but 
incorrect code that doesn’t crash is a horrible nightmare."
-- Chris Smith

http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#56241

I can start thread and no exception error print, and I do not know how to use tail in python script 
I need to cope with MySQL in python later as all file path stored in it, it is to monitor all text files

[toc] | [prev] | [next] | [standalone]

#56244

On 06/10/2013 05:06, Steven D'Aprano wrote:
> On Sat, 05 Oct 2013 20:17:32 -0700, galeomaga wrote:
>
>
>> if __name__ == '__main__':
>> 	try:
>> 		thread.start_new_thread( readfile, ("Thread-1", ) )
>> 	except:
>>     		print "Error: unable to start thread"
>
>
> Why not? If you can't start a thread, you have a problem with your code.
> How do you expect to debug this problem?
>
>
> "I find it amusing when novice programmers believe their main job is
> preventing programs from crashing. More experienced programmers realize
> that correct code is great, code that crashes could use improvement, but
> incorrect code that doesn’t crash is a horrible nightmare."
> -- Chris Smith
>
> http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/
>
>

Roughly translated for the benefit of newbies remove the try/except :)

Also note that a bare except is extremely bad practice, e.g. you can't 
stop rogue programs with a CTRL-C

-- 
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#56245

On Sun, Oct 6, 2013 at 9:36 PM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> Also note that a bare except is extremely bad practice, e.g. you can't stop
> rogue programs with a CTRL-C

Or to be more accurate, a Ctrl-C will cause a jump to your except
clause. Since, in this instance, that's going to emit and die, Ctrl-C
will still stop the program. But yes, catching KeyboardInterrupt
usually isn't your intention.

ChrisA

[toc] | [prev] | [next] | [standalone]

#56246

On 06/10/2013 12:03, Chris Angelico wrote:
> On Sun, Oct 6, 2013 at 9:36 PM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>> Also note that a bare except is extremely bad practice, e.g. you can't stop
>> rogue programs with a CTRL-C
>
> Or to be more accurate, a Ctrl-C will cause a jump to your except
> clause. Since, in this instance, that's going to emit and die, Ctrl-C
> will still stop the program. But yes, catching KeyboardInterrupt
> usually isn't your intention.
>
> ChrisA
>

Good point, but at least this time I typed "rogue" correctly, unlike on 
the tutor mailing list :)

-- 
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#56247

On Sun, Oct 6, 2013 at 10:13 PM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> Good point, but at least this time I typed "rogue" correctly, unlike on the
> tutor mailing list :)

Obligatory TVTropes link.

http://tvtropes.org/pmwiki/pmwiki.php/Main/RougeAnglesOfSatin

ChrisA

[toc] | [prev] | [next] | [standalone]

#56248

On 06/10/2013 12:15, Chris Angelico wrote:
> On Sun, Oct 6, 2013 at 10:13 PM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>> Good point, but at least this time I typed "rogue" correctly, unlike on the
>> tutor mailing list :)
>
> Obligatory TVTropes link.
>
> http://tvtropes.org/pmwiki/pmwiki.php/Main/RougeAnglesOfSatin
>
> ChrisA
>

<awful>
How do I know that you're not trying to send me to a rouge site?
</awful>

-- 
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#56250

On Sun, Oct 6, 2013 at 10:31 PM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> <awful>
> How do I know that you're not trying to send me to a rouge site?
> </awful>

I assure you, the background color is most distinctly white. They
probably contract with Google for their white pixel supply:

http://www.google.com.au/technology/pigeonrank.html

This is, however, quite distinctly off-topic for this list... though
Google does use Python extensively, and until not long ago had GvR on
their payroll.

ChrisA

[toc] | [prev] | [next] | [standalone]

#56298

https://docs.google.com/file/d/0B2D69u2pweEvelh1T25ra19oZEU/edit?usp=sharing

no matter call tail directly in python or using the script of tail
all failed
it seems it can not read next line

[toc] | [prev] | [next] | [standalone]

#56305

On 07.10.2013 03:54, galeomaga@gmail.com wrote:
> https://docs.google.com/file/d/0B2D69u2pweEvelh1T25ra19oZEU/edit?usp=sharing
>

For the readers who don't bother clicking on the link above: It's a 
short video where the OP demonstrates how her/his usage of tail doesn't 
work.

> no matter call tail directly in python or using the script of tail
> all failed
> it seems it can not read next line

In your video you use gedit to write some file and "tail -f <file>" to 
follow it. But "tail -f" will follow the file descriptor. Usually, 
editors like gedit won't save your changes to the original file but 
create a new temporary file and rename it later to the original file 
name after deleting the original one. Thus tail will follow an already 
deleted file.
See also this blog post:
http://tech.shantanugoel.com/2009/12/23/continuous-monitor-tail-fails.html

For your example you will have to use "tail -F <file>" which will follow 
the file name.

Alternatively you could write a simple script to simulate a continously 
growing file like

import time
for i in range(1000):
     with open("test.txt", "a") as f:
         f.write(str(i) + '\n')
     time.sleep(1)

which should work with "tail -f".

Bye, Andreas

[toc] | [prev] | [next] | [standalone]

#56179

On 05/10/2013 12:02, Joost Molenaar wrote:
> A bit of googling found me this:
> http://www.linux-support.com/cms/implementation-of-tail-in-python/
>
> import time
> import sys
>
> def tail_f(file):
>    interval = 1.0
>    while True:
>      where = file.tell()
>      line = file.readline()
>      if not line:
>        time.sleep(interval)
>        file.seek(where)
>      else:
>        yield line
>

In future could you please quote some context so that it's easier for us 
mere mortals to follow the thread, thanks in anticipation.

-- 
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web