Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!feeder.news-service.com!kanaga.switch.ch!switch.ch!newsserver.news.garr.it!newsserver.cilea.it!not-for-mail From: Giacomo Boffi Newsgroups: comp.lang.python Subject: Re: Convert AWK regex to Python Date: Mon, 16 May 2011 16:19:49 +0200 Organization: The Sun and The Rain Lines: 49 Message-ID: <86iptabud6.fsf@aiuole.stru.polimi.it> References: NNTP-Posting-Host: aiuole.stru.polimi.it Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: newsserver.cilea.it 1305555447 29938 131.175.189.141 (16 May 2011 14:17:27 GMT) X-Complaints-To: news@cilea.it NNTP-Posting-Date: 16 May 2011 14:17:27 GMT User-Agent: Gnus/5.1008 (Gnus v5.10.8) XEmacs/21.5-b31 (linux) Cancel-Lock: sha1:P9CTv+mhzAT7MR42TzDC+LCK5Pg= Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:5519 J writes: > cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service command status; do echo "Service: $service, Command: $command, Status: $status, Occurrences: `grep $service logs/pdu_log_fe.log | grep $command | grep $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log; done > > This AWK command gets lines which look like this:- > > 2011-05-16 09:46:22,361 [Thread-4847133] PDU D > > And outputs lines like this:- > > CC_SMS_SERVICE_51408 submit_resp: 0 > i see some discrepancies in the description of your problem 1. if i echo a properly quoted line "like this" above in the pipeline formed by the first three awk commands i get $ echo $likethis | awk -F\- '{print $1,$NF}' \ | awk -F\. '{print$1,$NF}' \ | awk '{print $1,$4,$5}' 2011 ) ) $ not a triple 'service command status' 2. with regard to the final product, you script outputs lines like in echo "Service: $service, [...]" and you say that it produces lines like CC_SMS_SERVICE_51408 submit_resp: WHATEVER, the abnormous run time is due to the fact that for every output line you rescan again and again the whole log file IF i had understood what you want, imho you should run your data through sort and uniq -c $ awk -F\- '{print $1,$NF}' < $file \ | awk -F\. '{print$1,$NF}' \ | awk '{print $1,$4,$5}' | sort | uniq -c | format_program uniq -c drops repeated lines from a sorted input AND prepends to each line the count of equal lines in the original stream hth g