Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!rt.uk.eu.org!news-transit.tcx.org.uk!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <4f02c069$0$690$426a74cc@news.free.fr>
References: <4f02c069$0$690$426a74cc@news.free.fr>
From: Devin Jeanpierre <jeanpierreda@gmail.com>
Date: Tue, 3 Jan 2012 04:45:40 -0500
Subject: Re: Repeating assertions in regular expression
To: "comp.lang.python" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.4343.1325583985.27778.python-list@python.org>
Lines: 117
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:18390

> \\b\\b and \\b{2} aren't equivalent ?

This sounds suspiciously like a bug!

> Why the wording is "should never" ? Repeating a zero-width assertion is n=
ot
> forbidden, for instance :
>
>>>> import re
>>>> re.compile("\\b\\b\w+\\b\\b")
> <_sre.SRE_Pattern object at 0xb7831140>
>>>>

I believe this is meant to refer to arbitrary-length repetitions, such
as r'\b*', not simple concatenations like that. r'\b*' will abort the
whole match if is run on a boundary, because Python detects a
repetition of a zero-width match and decides this is an error.


A little OT:

I'm not really sure why Python does it this way. It's possible to
match r'\b*' fairly painlessly: instead of aborting the whole match on
a repeat of nothing, it can backtrack. So what should happen is that
the RE engine looks at \b* and does the following:

Is there one \b?
Yes.
Is there another \b?
Backtrack: You haven't read any more characters since the last match
in the * operation
# The result now is that we've matched '\b*' with exactly one '\b' and
can move on to the rest of the input


This is possible (I haven't tried implementing it, but it's e.g.
mentioned in Russ Cox's regular expression papers), you need to
augment the backtracking search with a notion of the regular
expression having "progress" so that you can require progress at
certain points. Otherwise you obviously get problems with nested stars
and things like that. (As it happens, Python's regex engine can't
handle nested stars either).

I'd be appreciative if anyone who works on re or regex can discuss
this a little.

-- Devin

On Tue, Jan 3, 2012 at 3:46 AM, candide <candide@free.invalid> wrote:
> The regular expression HOWTO
> (http://docs.python.org/howto/regex.html#more-metacharacters) explains th=
e
> following
>
> # ------------------------------
> zero-width assertions should never be repeated, because if they match onc=
e
> at a given location, they can obviously be matched an infinite number of
> times.
> # ------------------------------
>
>
> Why the wording is "should never" ? Repeating a zero-width assertion is n=
ot
> forbidden, for instance :
>
>>>> import re
>>>> re.compile("\\b\\b\w+\\b\\b")
> <_sre.SRE_Pattern object at 0xb7831140>
>>>>
>
> Nevertheless, the following doesn't execute :
>
>>>> re.compile("\\b{2}\w+\\b\\b")
> Traceback (most recent call last):
> =C2=A0File "<stdin>", line 1, in <module>
> =C2=A0File "/usr/lib/python2.7/re.py", line 190, in compile
> =C2=A0 =C2=A0return _compile(pattern, flags)
> =C2=A0File "/usr/lib/python2.7/re.py", line 245, in _compile
> =C2=A0 =C2=A0raise error, v # invalid expression
> sre_constants.error: nothing to repeat
>>>>
>
>
> \\b\\b and \\b{2} aren't equivalent ?
>
>
> Surprisingly, the engine doesn't optimize repeated boundary assertions, f=
or
> instance
>
> # ------------------------------
> import re
> import time
>
> a=3Dtime.clock()
> len("\\b\\b\\b"*100000+"\w+")
> b=3Dtime.clock()
> print "CPU time : %.2f s" %(b - a)
>
> a=3Dtime.clock()
> re.compile("\\b\\b\\b"*100000+"\w+")
> b=3Dtime.clock()
> print "CPU time : %.2f s" %(b - a)
> # ------------------------------
>
> outputs:
>
> # ------------------------------
> CPU time : 0.00 s
> CPU time : 1.33 s
> # ------------------------------
>
>
> Your comments are welcome!
> --
> http://mail.python.org/mailman/listinfo/python-list