Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Sender: chris@rebertia.com
In-Reply-To: <f8598e49-67af-4a97-98db-9fd69d0182ae@googlegroups.com>
References: <f8598e49-67af-4a97-98db-9fd69d0182ae@googlegroups.com>
Date: Thu, 28 Mar 2013 21:29:17 -0700
Subject: Re: Doing both regex match and assignment within a If loop?
From: Chris Rebert <clp2@rebertia.com>
To: Victor Hooi <victorhooi@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.3934.1364531366.2939.python-list@python.org>
Lines: 87
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:42220

On Thu, Mar 28, 2013 at 9:00 PM, Victor Hooi <victorhooi@gmail.com> wrote:
> Hi,
>
> I have logline that I need to test against multiple regexes. E.g.:
>
>     import re
>
>     expression1 =3D re.compile(r'....')
>     expression2 =3D re.compile(r'....')
>
>     with open('log.txt') as f:
>         for line in f:
>             if expression1.match(line):
>                 # Do something - extract fields from line.
>             elif expression2.match(line):
>                 # Do something else - extract fields from line.
>             else:
>                 # Oh noes! Raise exception.
>
> However, in the "Do something" section - I need access to the match objec=
t itself, so that I can strip out certain fields from the line.
>
> Is it possible to somehow test for a match, as well as do assignment of t=
he re match object to a variable?
>
>     if expression1.match(line) =3D results:
>         results.groupsdict()...

AFAIK, not without hacks and/or being unidiomatic.

> Obviously the above won't work - however, is there a Pythonic way to tack=
le this?
>
> What I'm trying to avoid is this:
>
>     if expression1.match(line):
>         results =3D expression1.match(line)
>
> which I assume would call the regex match against the line twice - and wh=
en I'm dealing with a huge amount of log lines, slow things down.

def process(line):
    match =3D expr1.match(line)
    if match:
        # ...extract fields=E2=80=A6
        return something
    match =3D expr2.match(line)
    if match:
        # ...extract fields=E2=80=A6
        return something
    # etc=E2=80=A6
    raise SomeError()  # Oh noes!

with open('log.txt') as f:
    for line in f:
        results =3D process(line)


If you choose to further move the extractor snippets into their own
functions, then you can do:


# these could be lambdas if they're simple enough
def case1(match):
    # ...
def case2(match):
    # =E2=80=A6
# etc...

REGEX_EXTRACTOR_PAIRS =3D [
    (re.compile(r'....'), case1),
    (re.compile(r'....'), case2),
    # etc...
]

def process(line):
    for regex, extractor in REGEX_EXTRACTOR_PAIRS:
        match =3D regex.match(line)
        if match:
            return extractor(match)
    raise SomeError()

Although this second option is likely somewhat less performant, but it
definitely saves on repetition.

Cheers,
Chris