Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #5519

Re: Convert AWK regex to Python

Show all headers | View raw

J <jnr.gonzalez@googlemail.com> writes:

> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service command status; do echo "Service: $service, Command: $command, Status: $status, Occurrences: `grep $service logs/pdu_log_fe.log | grep $command | grep $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log; done
>
> This AWK command gets lines which look like this:-
>
> 2011-05-16 09:46:22,361 [Thread-4847133] PDU D <G_CC_SMS_SERVICE_51408_656.O_ CC_SMS_SERVICE_51408_656-ServerThread-VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 80000004 Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
>
> And outputs lines like this:-
>
> CC_SMS_SERVICE_51408 submit_resp: 0
>

i see some discrepancies in the description of your problem

1. if i echo a properly quoted line "like this" above in the pipeline
   formed by the first three awk commands i get

$ echo $likethis | awk -F\- '{print $1,$NF}' \
                 | awk -F\. '{print$1,$NF}'  \
                 | awk '{print $1,$4,$5}'
2011 ) )
$ 
   not a triple 'service command status'

2. with regard to the final product, you script outputs lines like in

echo "Service: $service, [...]"

   and you say that it produces lines like

CC_SMS_SERVICE_51408 submit_resp: 


WHATEVER, the abnormous run time is due to the fact that for every
output line you rescan again and again the whole log file

IF i had understood what you want, imho you should run your data
through sort and uniq -c

$ awk -F\- '{print $1,$NF}' < $file \
        | awk -F\. '{print$1,$NF}'  \
        | awk '{print $1,$4,$5}'    | sort | uniq -c | format_program

uniq -c drops repeated lines from a sorted input AND prepends to each
line the count of equal lines in the original stream

hth
                                                                g

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Convert AWK regex to Python J <jnr.gonzalez@googlemail.com> - 2011-05-16 01:19 -0700
  Re: Convert AWK regex to Python Chris Angelico <rosuav@gmail.com> - 2011-05-16 18:31 +1000
  Re: Convert AWK regex to Python Peter Otten <__peter__@web.de> - 2011-05-16 12:07 +0200
  Re: Convert AWK regex to Python Giacomo Boffi <giacomo.boffi@polimi.it> - 2011-05-16 16:19 +0200
    Re: Convert AWK regex to Python Matt Berends <matthewberends@gmail.com> - 2011-05-16 08:02 -0700
  Re: Convert AWK regex to Python MRAB <python@mrabarnett.plus.com> - 2011-05-16 17:39 +0100

csiph-web