Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.unix.programmer > #578 > unrolled thread

Tail -f with filter question

Back to article view | Back to comp.unix.programmer

Contents

  Tail -f  with filter question Matt Charles <mcharles48@gmail.com> - 2011-05-17 10:09 +0100
    Re: Tail -f  with filter question pk <pk@pk.invalid> - 2011-05-17 10:27 +0100
      Re: Tail -f  with filter question Mc <spam.mc@gmail.com> - 2011-05-17 10:55 +0100
        Re: Tail -f  with filter question pk <pk@pk.invalid> - 2011-05-17 11:04 +0100
    Re: Tail -f  with filter question Tobias Blass <tobiasblass@gmx.net> - 2011-05-17 09:59 +0000
      Re: Tail -f  with filter question scott@slp53.sl.home (Scott Lurndal) - 2011-05-17 14:08 +0000
        Re: Tail -f  with filter question Nobody <nobody@nowhere.com> - 2011-05-18 07:55 +0100
      Re: Tail -f  with filter question Geoff Clare <geoff@clare.See-My-Signature.invalid> - 2011-05-18 13:22 +0100
    Re: Tail -f  with filter question Damien Wyart <damien.wyart@free.fr> - 2011-05-17 16:36 +0200
      Re: Tail -f  with filter question James Waldby <not@valid.invalid> - 2011-05-17 18:22 +0000
        Re: Tail -f  with filter question Damien Wyart <damien.wyart@free.fr> - 2011-05-17 22:01 +0200

#578 — Tail -f with filter question

Hi folks,

When testing I have a log file that I tail with:

    tail -f log.txt

and watch the output.

However there's some log entries that start with INFO  that I don't want 
to see (the rest are error messages that I am interested in seeing).  Is 
there way of filtering those out?

I've tried this:

tail -f log.txt | grep -v INFO

but I think that's buffering and I don't get to see the error messages 
when they are produced (in real time).

Obviously I code write a C program to do this,  but wasn't sure if there 
was a better/easier way.

Cheers,

Matt

[toc] | [next] | [standalone]

#579

Matt Charles wrote:

> Hi folks,
> 
> When testing I have a log file that I tail with:
> 
>     tail -f log.txt
> 
> and watch the output.
> 
> However there's some log entries that start with INFO  that I don't want
> to see (the rest are error messages that I am interested in seeing).  Is
> there way of filtering those out?
> 
> I've tried this:
> 
> tail -f log.txt | grep -v INFO
> 
> but I think that's buffering and I don't get to see the error messages
> when they are produced (in real time).
> 
> Obviously I code write a C program to do this,  but wasn't sure if there
> was a better/easier way.

See if you have the "stdbuf" command installed.

[toc] | [prev] | [next] | [standalone]

#580

>
> See if you have the "stdbuf" command installed.
>
Doesn't look like I do unfortunately.

I should of mentioned that I'm running on a Solaris 10 Sparc host.

[toc] | [prev] | [next] | [standalone]

#583

Mc wrote:

>>
>> See if you have the "stdbuf" command installed.
>>
> Doesn't look like I do unfortunately.
> 
> I should of mentioned that I'm running on a Solaris 10 Sparc host.

Ah that makes it a bit more complicated. You can either install it (GNU 
coreutils), or maybe you can install expect, which comes with a sample 
script called "unbuffer" which should also do what you want.

[toc] | [prev] | [next] | [standalone]

#582

On 2011-05-17, Matt Charles <mcharles48@gmail.com> wrote:
> Hi folks,
>
> When testing I have a log file that I tail with:
>
>     tail -f log.txt
>
> and watch the output.
>
> However there's some log entries that start with INFO  that I don't
> want to see (the rest are error messages that I am interested in
> seeing).  Is there way of filtering those out?
>
> I've tried this:
>
> tail -f log.txt | grep -v INFO
>
> but I think that's buffering and I don't get to see the error messages
> when they are produced (in real time).
>
> Obviously I code write a C program to do this,  but wasn't sure if
> there was a better/easier way.
>
Do you use the syslog facility? If you do you could just change the rules
in the syslog configuration file. It would also allow the users of your
program to setup their own rules if they wish to do so.

If you don't want to use syslog I found the following option for grep
> --line-buffered Use line buffering on output.  This can cause a
> performance penalty.
I guess grep uses block buffering by default, so line buffering should solve
your problem

[toc] | [prev] | [next] | [standalone]

#584

Tobias Blass <tobiasblass@gmx.net> writes:

>I guess grep uses block buffering by default, so line buffering should solve
>your problem


stderr is line buffered by default.  stdout is line buffered if isatty() is
true for the file descriptor associated with stdout, otherwise it is full
buffered (512 to 4096+ byte buffer, depending on libc).  Note that isatty() is
false for a pipe.

scott

[toc] | [prev] | [next] | [standalone]

#607

On Tue, 17 May 2011 14:08:56 +0000, Scott Lurndal wrote:

>>I guess grep uses block buffering by default, so line buffering should
>>solve your problem
> 
> 
> stderr is line buffered by default.

On most Unices, stderr is initially *unbuffered*.

> stdout is line buffered if isatty() is true for the file descriptor
> associated with stdout, otherwise it is full buffered

This is the situation in practice for most Unices.

However, both the ISO C standard and POSIX have weaker requirements.

The ISO C standard (§7.19.3p7) says:

	As initially opened, the standard error stream is not fully
	buffered; the standard input and standard output streams are
	fully buffered if and only if the stream can be determined not
	to refer to an interactive device.

POSIX (§2.5) is practically identical:

	When opened, the standard error stream is not fully buffered;
	the standard input and standard output streams are fully
	buffered if and only if the stream can be determined not to
	refer to an interactive device.

<http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05>

[toc] | [prev] | [next] | [standalone]

#614

Tobias Blass wrote:

> On 2011-05-17, Matt Charles <mcharles48@gmail.com> wrote:
>>
>> I've tried this:
>>
>> tail -f log.txt | grep -v INFO
>>
>> but I think that's buffering and I don't get to see the error messages
>> when they are produced (in real time).
>
> I found the following option for grep
>> --line-buffered Use line buffering on output.  This can cause a
>> performance penalty.
> I guess grep uses block buffering by default, so line buffering should solve
> your problem

The buffering is being done in tail, not grep.

The solutions given by others involve running tail with its output
going to a pseudo-terminal, using a tool that will read that output
from the master side of the pseudo-terminal and write it (without
buffering) to the pipe.

-- 
Geoff Clare <netnews@gclare.org.uk>

[toc] | [prev] | [next] | [standalone]

#585

* Matt Charles <mcharles48@gmail.com> in comp.unix.programmer:
> tail -f log.txt | grep -v INFO

> but I think that's buffering and I don't get to see the error messages
> when they are produced (in real time).

A page detailing and comparing the solutions already posted can be found
here: http://mywiki.wooledge.org/BashFAQ/009

Best,
-- 
DW

[toc] | [prev] | [next] | [standalone]

#589

On Tue, 17 May 2011 16:36:05 +0200, Damien Wyart wrote:
> * Matt Charles <mcharles48@gmail.com> in comp.unix.programmer:
>> tail -f log.txt | grep -v INFO
> 
>> but I think that's buffering and I don't get to see the error messages
>> when they are produced (in real time).
> 
> A page detailing and comparing the solutions already posted can be found
> here: http://mywiki.wooledge.org/BashFAQ/009

In the first bullet of the 'less' section near the end of that 
page, perhaps you could list    less -p pattern 
as an alternative to starting less and then entering /, pattern.

-- 
jiw

[toc] | [prev] | [next] | [standalone]

#592

> > A page detailing and comparing the solutions already posted can be
> > found here: http://mywiki.wooledge.org/BashFAQ/009

* James Waldby <not@valid.invalid> in comp.unix.programmer:
> In the first bullet of the 'less' section near the end of that page,
> perhaps you could list less -p pattern as an alternative to starting
> less and then entering /, pattern.

Done, thanks for the suggestion.

Btw, I am not affiliated in any way to the site I quoted, so as edition
is not restricted to authenticated users (as I learnt while trying to
add your suggestion), you could have done this modification by yourself.

-- 
DW

[toc] | [prev] | [standalone]

Back to top | Article view | comp.unix.programmer

csiph-web