Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #4485

Re: analysis of java application logs

Show all headers | View raw

On Mon, 23 May 2011 20:33:07 +0200, Robert Klemme wrote:

> On 23.05.2011 15:17, Patricia Shanahan wrote:
>> On 5/23/2011 12:50 AM, Ulrich Scholz wrote:
>>> I'm looking for an approach to the problem of analyzing application
>>> log files.
>>>
>>> I need to analyse Java log files from applications (i.e., not logs of
>>> web servers). These logs contain Java exceptions, thread dumps, and
>>> free-form log4j messages issued by log statements inserted by
>>> programmers during development. Right now, these man-made log entries
>>> do not have any specific format.
>>>
>>> What I'm looking for is a tool and/or strategy that supports in
>>> lexing/ parsing, tagging, and analysing the log entries. Because there
>>> is only little defined syntax and grammar - and because you might not
>>> know what you are looking for - the task requires the quick issuing of
>>> queries against the log data base. Some sort of visualization would be
>>> nice, too.
>>>
>>> Pointers to existing tools and approaches as well as appropriate
>>> tools/ algorithms to develop the required system would be welcome.
>>
>> I would use Perl, and begin by recognizing some of the more important
>> formats, such as thread dumps. I agree with the desirability of
>> introducing some organized formatting into the log messages, but an
>> ad-hoc Perl program can often get useful data out of a disorganized
>> log.
> 
> Only that Perl is so awful - YMMV of course.  But for these kinds of
> tasks (more correctly: for *any* task) I very much prefer to use Ruby
> because of its cleaner OO and cleaner syntax.
>
I do the same, but use gawk rather than Perl: I have the same objections 
to Perl as you, while gawk is pretty straight forward if you understand 
regexes and can write C.

So far, using gawk to extract the information I've needed from Linux 
system logs has been rather straight forward. Besides, I generally find 
gawk to be more concise and readable than Perl, for this type of job, 
anyway.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

analysis of java application logs Ulrich Scholz <d7@thispla.net> - 2011-05-23 00:50 -0700
  Re: analysis of java application logs Robert Klemme <shortcutter@googlemail.com> - 2011-05-23 02:20 -0700
  Re: analysis of java application logs jlp <jlp@jlp.com> - 2011-05-23 11:45 +0200
  Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-23 09:11 -0400
    Re: analysis of java application logs Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2011-05-23 19:16 +0200
      Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-23 15:02 -0400
        Re: analysis of java application logs Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2011-05-23 22:03 +0200
          Re: analysis of java application logs Michael Wojcik <mwojcik@newsguy.com> - 2011-05-26 14:43 -0400
        Re: analysis of java application logs Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2011-05-23 18:32 -0300
          Re: analysis of java application logs Ulrich Scholz <d7@thispla.net> - 2011-05-25 06:00 -0700
            Re: analysis of java application logs Arved Sandstrom <asandstrom3minus1@eastlink.ca> - 2011-05-25 19:04 -0300
        Re: analysis of java application logs Martin Gregorie <martin@address-in-sig.invalid> - 2011-05-23 22:25 +0000
          Re: analysis of java application logs Nigel Wade <nmw-news@ion.le.ac.uk> - 2011-05-24 12:26 +0100
            Re: analysis of java application logs Martin Gregorie <martin@address-in-sig.invalid> - 2011-05-24 12:29 +0000
              Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-24 08:49 -0400
                Re: analysis of java application logs Martin Gregorie <martin@address-in-sig.invalid> - 2011-05-24 14:37 +0000
                Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-24 12:26 -0400
                Re: analysis of java application logs Jim Gibson <jimsgibson@gmail.com> - 2011-05-24 11:00 -0700
                Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-24 14:35 -0400
                Re: analysis of java application logs Nigel Wade <nmw-news@ion.le.ac.uk> - 2011-05-25 09:53 +0100
            Re: analysis of java application logs Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2011-05-24 19:12 +0200
  Re: analysis of java application logs Patricia Shanahan <pats@acm.org> - 2011-05-23 06:17 -0700
    Re: analysis of java application logs Robert Klemme <shortcutter@googlemail.com> - 2011-05-23 20:33 +0200
      Re: analysis of java application logs Martin Gregorie <martin@address-in-sig.invalid> - 2011-05-23 19:07 +0000
  Re: analysis of java application logs CncShipper <anon@nowhere.com> - 2011-05-23 14:56 +0000
    Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-23 11:43 -0400
      Re: analysis of java application logs jlp <jlp@jlp.com> - 2011-05-23 18:00 +0200
        Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-23 12:20 -0400
          Re: analysis of java application logs Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2011-05-23 19:06 +0200
            Re: analysis of java application logs Robert Klemme <shortcutter@googlemail.com> - 2011-05-23 20:27 +0200
              Re: analysis of java application logs Daniele Futtorovic <da.futt.news@laposte-dot-net.invalid> - 2011-05-23 21:02 +0200
              Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-23 15:06 -0400
                Re: analysis of java application logs Robert Klemme <shortcutter@googlemail.com> - 2011-05-23 22:10 +0200
                Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-23 17:04 -0400
  Re: analysis of java application logs Tom Anderson <twic@urchin.earth.li> - 2011-05-24 15:04 +0100
    Re: analysis of java application logs Martin Gregorie <martin@address-in-sig.invalid> - 2011-05-24 14:50 +0000
      Re: analysis of java application logs Michael Wojcik <mwojcik@newsguy.com> - 2011-05-26 14:58 -0400
      Re: analysis of java application logs Lawrence D'Oliveiro <ldo@geek-central.gen.new_zealand> - 2011-05-30 16:23 +1200
        Re: analysis of java application logs Lew <noone@lewscanon.com> - 2011-05-30 01:08 -0400

csiph-web