Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #37499 > unrolled thread

Search log for string and display hourly/daily report

Back to article view | Back to comp.lang.python

Contents

  Search log for string and display hourly/daily report spek06@gmail.com - 2013-01-23 11:05 -0800
    Re: Search log for string and display hourly/daily report Tim Chase <python.list@tim.thechases.com> - 2013-01-23 14:08 -0600
    Re: Search log for string and display hourly/daily report John Gordon <gordon@panix.com> - 2013-01-23 21:25 +0000
      Re: Search log for string and display hourly/daily report spek06@gmail.com - 2013-01-23 16:57 -0800
    Re: Search log for string and display hourly/daily report Michael Torrie <torriem@gmail.com> - 2013-01-23 15:12 -0700

#37499 — Search log for string and display hourly/daily report

I need to search a log file for a specific string (Successfully Sent) and report the number of instances in the last hour (from when executed) and total for the day so far (midnight till the time executed). Can anyone provide any examples of such a program or get me started?

[toc] | [next] | [standalone]

#37500

On 01/23/13 13:05, spek06@gmail.com wrote:
> I need to search a log file for a specific string (Successfully
> Sent) and report the number of instances in the last hour (from
> when executed) and total for the day so far (midnight till the
> time executed). Can anyone provide any examples of such a program
> or get me started?

You'd have to specify additional details on how the log-file is 
formatted, presumably how it's delimited, and what the format of the 
corresponding time-stamp is.  Something like

   "I have a tab-delimited log file where the 4th column is the 
timestamp in the format 'YYYY-MM-DD HH:mm:ss' and the 18th column is 
the status.  I want to search for items where the status contains 
'Successfully Sent' and then further filter them by (1) events in 
the last hour, and (2) all events today"

-tkc

[toc] | [prev] | [next] | [standalone]

#37503

In <4f952e77-258d-47ae-9d76-a86daa8acaa2@googlegroups.com> spek06@gmail.com writes:

> I need to search a log file for a specific string (Successfully Sent) and
> report the number of instances in the last hour (from when executed) and
> total for the day so far (midnight till the time executed). Can anyone
> provide any examples of such a program or get me started?

from datetime import datetime, timedelta
from time import mktime, strptime

now = datetime.now()
midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
one_hour_ago = now - timedelta(hours=1)

daily_instances = 0
hourly_instances = 0

with open('myfile.log') as logfile:
    for line in logfile:
        if 'Successfully Sent' in line:
            time_string = line[0:19]
            struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
            log_time = datetime.fromtimestamp(mktime(struct))

            if log_time > midnight:
                daily_instances += 1

            if log_time > one_hour_ago:
                hourly_instances += 1

print "Instances in the last hour: ", hourly_instances
print "Instances since midnight: ", daily_instances


This code assumes that log lines begin with a timestamp similar to
"2013-01-23T09:27:01".  If the timestamp is in a different format, or
occurs elsewhere in the line, you'll have to adjust for that.

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon@panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"

[toc] | [prev] | [next] | [standalone]

#37528

On Wednesday, January 23, 2013 4:25:59 PM UTC-5, John Gordon wrote:
> In <4f952e77-258d-47ae-9d76-a86daa8acaa2@googlegroups.com> spek06 writes:
> 
> 
> 
> > I need to search a log file for a specific string (Successfully Sent) and
> 
> > report the number of instances in the last hour (from when executed) and
> 
> > total for the day so far (midnight till the time executed). Can anyone
> 
> > provide any examples of such a program or get me started?
> 
> 
> 
> from datetime import datetime, timedelta
> 
> from time import mktime, strptime
> 
> 
> 
> now = datetime.now()
> 
> midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
> 
> one_hour_ago = now - timedelta(hours=1)
> 
> 
> 
> daily_instances = 0
> 
> hourly_instances = 0
> 
> 
> 
> with open('myfile.log') as logfile:
> 
>     for line in logfile:
> 
>         if 'Successfully Sent' in line:
> 
>             time_string = line[0:19]
> 
>             struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
> 
>             log_time = datetime.fromtimestamp(mktime(struct))
> 
> 
> 
>             if log_time > midnight:
> 
>                 daily_instances += 1
> 
> 
> 
>             if log_time > one_hour_ago:
> 
>                 hourly_instances += 1
> 
> 
> 
> print "Instances in the last hour: ", hourly_instances
> 
> print "Instances since midnight: ", daily_instances
> 
> 
> 
> 
> 
> This code assumes that log lines begin with a timestamp similar to
> 
> "2013-01-23T09:27:01".  If the timestamp is in a different format, or
> 
> occurs elsewhere in the line, you'll have to adjust for that.
> 
> 
> 
> -- 
> 
> John Gordon                   A is for Amy, who fell down the stairs
> 
>              B is for Basil, assaulted by bears
> 
>                                 -- Edward Gorey, "The Gashlycrumb Tinies"

Thanks John, I think this will definitely help get me started!

[toc] | [prev] | [next] | [standalone]

#37508

On 01/23/2013 12:05 PM, spek06@gmail.com wrote:
> I need to search a log file for a specific string (Successfully Sent)
> and report the number of instances in the last hour (from when
> executed) and total for the day so far (midnight till the time
> executed). Can anyone provide any examples of such a program or get
> me started?

Take a look at this very interesting presentation/document:

http://www.dabeaz.com/generators/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web