Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.albasani.net!eternal-september.org!feeder.eternal-september.org!mx05.eternal-september.org!.POSTED!not-for-mail From: Eric Sosman Newsgroups: comp.lang.java.programmer Subject: Re: Tailing a series of log files Date: Sat, 15 Jun 2013 09:20:24 -0400 Organization: A noiseless patient Spider Lines: 86 Message-ID: References: <3f65c1bd-6589-4b87-b6a9-646250731faa@googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Injection-Date: Sat, 15 Jun 2013 13:15:31 +0000 (UTC) Injection-Info: mx05.eternal-september.org; posting-host="0d73d8cc209bff1c6395088b400d0605"; logging-data="27847"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19BMN9EHk4vX1LuyPs8bvct" User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 In-Reply-To: Cancel-Lock: sha1:sH1sg9h8d9EBfQK0cIZcpSSpyzA= Xref: csiph.com comp.lang.java.programmer:24411 On 6/15/2013 12:11 AM, Daniel Pitts wrote: > On 6/14/13 6:06 PM, Eric Sosman wrote: >> On 6/14/2013 7:00 PM, Arved Sandstrom wrote: >>> On 06/12/2013 09:04 AM, SamXiao wrote: >>>> Hi all, >>>> >>>> I am working on a solution to keep track of a series of log files and >>>> read the tail lines. At the very beginning, there is only one log file >>>> named “TEST-YYYYMMDD-00.log” (where YYYYMMDD is the current date). >>>> When the size of this log reaches to 10 MB, another log file >>>> “TEST-YYYYMMDD-01.log” will be generated and so forth. After midnight >>>> (12:00), a new log file with the next date starting with 00 again will >>>> be generated. >>>> >>>> I know Apache Commons IO containing tail.java. However, seems it is >>>> for tailing a single log file only. How could I keep tailing a series >>>> of log files? >>>> >>>> Any help would be highly appreciated. Thanks. >>>> >>> For what it's worth, a sane logging system with a rollover policy has a >>> base log file which always has the same name, like TEST.log. Decent text >>> editors can follow a filename, UNIX command line "tail" doesn't follow a >>> filename but rather the precise file in question. But point being, if >>> you're looking to follow an "active" log file it may help if it's always >>> got the same name. It's the rolled-over logs that have the suffixes. >> >> This is true, but I don't think it's going to help the >> O.P. much. Argument: The tailer (or any program) will use >> the name to open the file, but subsequent accesses will use a >> name-independent means -- "vnode," perhaps, or "handle." If >> the name-to-vnode/handle mapping changes after the program has >> created its connection to the file, that's unlikely to affect >> the existing connection: The program will, most likely, just >> keep on tailing the original, oblivious to the name change. >> >> A tailer might, I suppose, open TEST.log anew for each >> access: open, tail, close, pause, open, tail, close, pause, >> and so on. That approach would catch the renames, but would >> also risk missing the last few lines of the pre-rename file: >> >> Tailer: open, tail, close, pause, open, tail, close, ... >> Writer: ... write ... write ... rename ... write ... >> Upshot: ^^^^^ >> this batch isn't tailed >> >> If the log writer switches from file to file occasionally, >> I think the only reliable approach is to have the log reader be >> explicitly aware of the fact. >> > > from "man tail" on OS X: > >> -F The -F option implies the -f option, but tail will also >> check to see if the file being followed has been renamed or rotated. >> The file is closed and >> reopened when tail detects that the filename being read >> from has a new inode number. The -F option is ignored if reading from >> standard input rather than a >> file. > > > So, it will reopen the file if it sees that it has a new inode. If you > are renaming the files frequently enough, that might miss a rename, but > that seems to be an unlikely situation. We're straying into system specifics here, but I think the quoted text makes my point. On Unixoid systems (where inodes live), renaming a file does not affect its inode number -- in fact, a file can have N different names in M directories and keep the same inode, unaltered, as those N names come and go and change. Since checking the inode number of an already-open file will not detect a rename (or even a deletion!), the program must be doing something else instead. I imagine it's looking up the file name (probably with a variant of stat) and comparing the inode number against that of the already-open file; if they disagree, somebody has renamed the current file and it's time to switch to the new one. That's what I meant by "have the log reader be explicitly aware of the fact:" It's got to poll or arrange for change notification in some other way. -- Eric Sosman esosman@comcast-dot-net.invalid