Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!selfless.tophat.at!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Peter Otten <__peter__@web.de>
Subject: Re: Convert AWK regex to Python
Date: Mon, 16 May 2011 12:07:27 +0200
Organization: None
References: <d4b4303e-8002-4f5f-9cb8-af7ab7451f13@glegroupsg2000goo.googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7Bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1639.1305540395.9059.python-list@python.org>
Lines: 43
NNTP-Posting-Host: 82.94.164.166
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:5507

J wrote:

> Good morning all,
> Wondering if you could please help me with the following query:-
> I have just started learning Python last weekend after a colleague of mine
> showed me how to dramatically cut the time a Bash script takes to execute
> by re-writing it in Python.  I was amazed at how fast it ran.  I would now
> like to do the same thing with another script I have.
> 
> This other script reads a log file and using AWK it filters certain fields
> from the log and writes them to a new file.  See below the regex the
> script is executing.  I would like to re-write this regex in Python as my
> script is currently taking about 1 hour to execute on a log file with
> about 100,000 lines.  I would like to cut this time down as much as
> possible.
> 
> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print
> $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service
> command status; do echo "Service: $service, Command: $command, Status:
> $status, Occurrences: `grep $service logs/pdu_log_fe.log | grep $command |
> grep $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log;
> done
> 
> This AWK command gets lines which look like this:-
> 
> 2011-05-16 09:46:22,361 [Thread-4847133] PDU D
> <G_CC_SMS_SERVICE_51408_656.O_
> CC_SMS_SERVICE_51408_656-ServerThread-
VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX
> - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 80000004
> Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
> 
> And outputs lines like this:-
> 
> CC_SMS_SERVICE_51408 submit_resp: 0
> 
> I have tried writing the Python script myself but I am getting stuck
> writing the regex.  So far I have the following:-

For the moment forget about the implementation. The first thing you should 
do is to describe the problem as clearly as possible, in plain English.