Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #24411

Re: Tailing a series of log files

Show all headers | View raw

On 6/15/2013 12:11 AM, Daniel Pitts wrote:
> On 6/14/13 6:06 PM, Eric Sosman wrote:
>> On 6/14/2013 7:00 PM, Arved Sandstrom wrote:
>>> On 06/12/2013 09:04 AM, SamXiao wrote:
>>>> Hi all,
>>>>
>>>> I am working on a solution to keep track of a series of log files and
>>>> read the tail lines. At the very beginning, there is only one log file
>>>> named “TEST-YYYYMMDD-00.log” (where YYYYMMDD is the current date).
>>>> When the size of this log reaches to 10 MB, another log file
>>>> “TEST-YYYYMMDD-01.log” will be generated and so forth. After midnight
>>>> (12:00), a new log file with the next date starting with 00 again will
>>>> be generated.
>>>>
>>>> I know Apache Commons IO containing tail.java. However, seems it is
>>>> for tailing a single log file only. How could I keep tailing a series
>>>> of log files?
>>>>
>>>> Any help would be highly appreciated. Thanks.
>>>>
>>> For what it's worth, a sane logging system with a rollover policy has a
>>> base log file which always has the same name, like TEST.log. Decent text
>>> editors can follow a filename, UNIX command line "tail" doesn't follow a
>>> filename but rather the precise file in question. But point being, if
>>> you're looking to follow an "active" log file it may help if it's always
>>> got the same name. It's the rolled-over logs that have the suffixes.
>>
>>      This is true, but I don't think it's going to help the
>> O.P. much.  Argument: The tailer (or any program) will use
>> the name to open the file, but subsequent accesses will use a
>> name-independent means -- "vnode," perhaps, or "handle."  If
>> the name-to-vnode/handle mapping changes after the program has
>> created its connection to the file, that's unlikely to affect
>> the existing connection: The program will, most likely, just
>> keep on tailing the original, oblivious to the name change.
>>
>>      A tailer might, I suppose, open TEST.log anew for each
>> access: open, tail, close, pause, open, tail, close, pause,
>> and so on.  That approach would catch the renames, but would
>> also risk missing the last few lines of the pre-rename file:
>>
>>      Tailer: open, tail, close, pause, open, tail, close, ...
>>      Writer: ... write ... write ... rename ... write ...
>>      Upshot:               ^^^^^
>>                     this batch isn't tailed
>>
>>      If the log writer switches from file to file occasionally,
>> I think the only reliable approach is to have the log reader be
>> explicitly aware of the fact.
>>
>
> from "man tail" on OS X:
>
>>      -F      The -F option implies the -f option, but tail will also
>> check to see if the file being followed has been renamed or rotated.
>> The file is closed and
>>              reopened when tail detects that the filename being read
>> from has a new inode number.  The -F option is ignored if reading from
>> standard input rather than a
>>              file.
>
>
> So, it will reopen the file if it sees that it has a new inode. If you
> are renaming the files frequently enough, that might miss a rename, but
> that seems to be an unlikely situation.

     We're straying into system specifics here, but I think the
quoted text makes my point.  On Unixoid systems (where inodes
live), renaming a file does not affect its inode number -- in
fact, a file can have N different names in M directories and
keep the same inode, unaltered, as those N names come and go
and change.

     Since checking the inode number of an already-open file
will not detect a rename (or even a deletion!), the program
must be doing something else instead.  I imagine it's looking
up the file name (probably with a variant of stat) and comparing
the inode number against that of the already-open file; if they
disagree, somebody has renamed the current file and it's time
to switch to the new one.  That's what I meant by "have the
log reader be explicitly aware of the fact:" It's got to poll
or arrange for change notification in some other way.

-- 
Eric Sosman
esosman@comcast-dot-net.invalid

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Tailing a series of log files SamXiao <foolsmart2005@gmail.com> - 2013-06-12 05:04 -0700
  Re: Tailing a series of log files Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-06-12 08:56 -0400
  Re: Tailing a series of log files Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-06-14 20:00 -0300
    Re: Tailing a series of log files Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-06-14 21:06 -0400
      Re: Tailing a series of log files Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2013-06-14 21:11 -0700
        Re: Tailing a series of log files Eric Sosman <esosman@comcast-dot-net.invalid> - 2013-06-15 09:20 -0400
          Re: Tailing a series of log files JLP <JLP@jlp.com> - 2013-06-15 18:59 +0200
            Re: Tailing a series of log files JLP <JLP@jlp.com> - 2013-06-15 19:15 +0200
      Re: Tailing a series of log files Arved Sandstrom <asandstrom2@eastlink.ca> - 2013-06-16 16:43 -0300

csiph-web