Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #5507

Re: Convert AWK regex to Python

Show key headers only | View raw

J wrote:

> Good morning all,
> Wondering if you could please help me with the following query:-
> I have just started learning Python last weekend after a colleague of mine
> showed me how to dramatically cut the time a Bash script takes to execute
> by re-writing it in Python.  I was amazed at how fast it ran.  I would now
> like to do the same thing with another script I have.
> 
> This other script reads a log file and using AWK it filters certain fields
> from the log and writes them to a new file.  See below the regex the
> script is executing.  I would like to re-write this regex in Python as my
> script is currently taking about 1 hour to execute on a log file with
> about 100,000 lines.  I would like to cut this time down as much as
> possible.
> 
> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print
> $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service
> command status; do echo "Service: $service, Command: $command, Status:
> $status, Occurrences: `grep $service logs/pdu_log_fe.log | grep $command |
> grep $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log;
> done
> 
> This AWK command gets lines which look like this:-
> 
> 2011-05-16 09:46:22,361 [Thread-4847133] PDU D
> <G_CC_SMS_SERVICE_51408_656.O_
> CC_SMS_SERVICE_51408_656-ServerThread-
VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX
> - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 80000004
> Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
> 
> And outputs lines like this:-
> 
> CC_SMS_SERVICE_51408 submit_resp: 0
> 
> I have tried writing the Python script myself but I am getting stuck
> writing the regex.  So far I have the following:-

For the moment forget about the implementation. The first thing you should 
do is to describe the problem as clearly as possible, in plain English.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Convert AWK regex to Python J <jnr.gonzalez@googlemail.com> - 2011-05-16 01:19 -0700
  Re: Convert AWK regex to Python Chris Angelico <rosuav@gmail.com> - 2011-05-16 18:31 +1000
  Re: Convert AWK regex to Python Peter Otten <__peter__@web.de> - 2011-05-16 12:07 +0200
  Re: Convert AWK regex to Python Giacomo Boffi <giacomo.boffi@polimi.it> - 2011-05-16 16:19 +0200
    Re: Convert AWK regex to Python Matt Berends <matthewberends@gmail.com> - 2011-05-16 08:02 -0700
  Re: Convert AWK regex to Python MRAB <python@mrabarnett.plus.com> - 2011-05-16 17:39 +0100

csiph-web