Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!.POSTED!not-for-mail From: Lew Newsgroups: comp.lang.java.programmer Subject: Re: analysis of java application logs Date: Mon, 23 May 2011 11:43:01 -0400 Organization: albasani.net Lines: 50 Message-ID: References: <8AuCp.7606$Mk4.5295@unlimited.newshosting.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Trace: news.albasani.net eBZjPuvMW/MhLPAKTvH7O1LuDr3v2NFvGOOvHmXeK2NMR/K2ntmo4632cnKv7DZPH5C2rMSgD4lNTI95t4xNTf/LSoa2Aa2xYO5ZCroqwa5F56UOguyfoR2m6nJq0YlP NNTP-Posting-Date: Mon, 23 May 2011 15:42:47 +0000 (UTC) Injection-Info: news.albasani.net; logging-data="cQVXesUxq1MCb8U5QtkYaZy3yegrH9lB4IdBfzbA380GWkio1MIGaNq7g5kUn+57huLLhZ4yn1z91U22HXV1ZvRWbtq+EssvgkAUXcOjwbCQa2jaAopdDQciH26oIptR"; mail-complaints-to="abuse@albasani.net" User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110424 Thunderbird/3.1.10 In-Reply-To: <8AuCp.7606$Mk4.5295@unlimited.newshosting.com> Cancel-Lock: sha1:g5xE6jzCzlO+Lj6xPSUXrGMBIxs= Xref: x330-a1.tempe.blueboxinc.net comp.lang.java.programmer:4462 CncShipper wrote: > I wrote one of these and thought about Open Sourcing it, but lost "open sourcing" > interest. I parsed the logs into a db, and assigned id's to the "DB, "IDs" (no greengrocer's apostrophe, and "id" is a different word from "ID", although the meaning you imputed by the substitution is poetic and interesting) > various fields. > > You could then search by Type, ( WARNING, SEVERE, etc... ) "type" > You could search a range of times > It could handle multiple log files into one run > could Sync on an event and stop analyzing on another trigger "synch" > Graphs to count trends, events, exceptions "graphs" > Used Reg-Ex a heck of a lot of work.. Sorted all the transactions in "used" "regex" > the logs, so you could also display by package name, really helped > me solve a lot of problems when I was working .. took me nearly two > years to complete everything to where it is today.. Double-dot, or two consecutive periods, is not legitimate punctuation in lieu of a comma or full stop. > I never found a package that even came close to it.. which is why I > wrote it You have made an important and useful point. Covering for a bad log format is a freeform-text parsing problem, inherently difficult and heuristic and probably never perfect. I wonder if your effort would have been better spent converting to a log format that is parser-friendly, as the OP should do. -- Lew Honi soit qui mal y pense. http://upload.wikimedia.org/wikipedia/commons/c/cf/Friz.jpg