Re: Why is regex so slow?

Path	csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<python-python-list@m.gmane.org>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.005
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'from:addr:yahoo.co.uk': 0.04; 'pypi': 0.07; 'lawrence': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'slow.': 0.09; 'subject:Why': 0.09; 'suggestions:': 0.09; 'bug': 0.12; 'assume': 0.14; '(use': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'roy': 0.16; 'subject:slow': 0.16; 'tempted': 0.16; 'version?': 0.16; 'wrote:': 0.18; 'module': 0.19; 'split': 0.19; 'seems': 0.21; '>>>': 0.22; 'creating': 0.23; 'header:User-Agent:1': 0.23; '2.x': 0.24; 'skip': 0.24; '---': 0.24; 'least': 0.26; 'header:X -Complaints-To:1': 0.27; 'header:In-Reply-To:1': 0.27; 'tried': 0.27; 'url:bugs': 0.29; 'matching': 0.30; "i'm": 0.30; '(which': 0.31; 'ball': 0.31; 'open': 0.33; 'url:python': 0.33; 'beginning': 0.33; 'could': 0.34; 'but': 0.35; 'next': 0.36; "didn't": 0.36; 'subject:?': 0.36; 'url:org': 0.36; 'easily': 0.37; 'performance': 0.37; 'to:addr:python-list': 0.38; 'pm,': 0.38; 'rather': 0.38; 'that,': 0.38; 'does': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'skip:u 10': 0.60; 'black': 0.61; 'gone': 0.61; 'new': 0.61; 'first': 0.61; 'making': 0.63; 'smith': 0.68; 'watching': 0.68; 'obvious': 0.74; 'yourself': 0.78; 'difference.': 0.84; 'email addr:panix.com': 0.84; 'white,': 0.84; 'received:89': 0.85; '2013,': 0.91
X-Injected-Via-Gmane	http://gmane.org/
To	python-list@python.org
From	Mark Lawrence <breamoreboy@yahoo.co.uk>
Subject	Re: Why is regex so slow?
Date	Tue, 18 Jun 2013 18:34:01 +0100
References	<kpq2r9$gg6$1@panix2.panix.com> <CANc-5UyouN5EQw_GDecaAR+inyAB26g=e2pZ=zBuspXdmxJh+Q@mail.gmail.com> <B2AEBA54-6098-47D6-B7E0-77F0B7D8D722@panix.com>
Mime-Version	1.0
Content-Type	text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding	7bit
X-Gmane-NNTP-Posting-Host	host-89-240-174-199.as13285.net
User-Agent	Mozilla/5.0 (Windows NT 6.0; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
In-Reply-To	<B2AEBA54-6098-47D6-B7E0-77F0B7D8D722@panix.com>
X-Antivirus	avast! (VPS 130618-0, 18/06/2013), Outbound message
X-Antivirus-Status	Clean
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.3549.1371576854.3114.python-list@python.org> (permalink)
Lines	40
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1371576854 news.xs4all.nl 16004 [2001:888:2000:d::a6]:43092
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:48648

Show key headers only | View raw

On 18/06/2013 18:08, Roy Smith wrote:
>
> On Jun 18, 2013, at 1:01 PM, Skip Montanaro wrote:
>
>>> I don't understand why the first way is so much slower.
>>
>> I have no obvious answers, but a couple suggestions:
>>
>> 1. Can you anchor the pattern at the beginning of the line?  (use
>> match() instead of search())
>
> That's one of the things we tried.  Didn't make any difference.
>
>> 2. Does it get faster it you eliminate the "(.*)" part of the pattern?
>
> Just tried that, it also didn't make any difference.
>
>> It seems that if you find a line matching the first part of the
>> pattern, you could just as easily split the line yourself instead of
>> creating a group.
>
>
> At this point, I'm not so much interested in making this faster as understanding why it's so slow.  I'm tempted to open this up as a performance bug against the regex module (which I assume will be rejected, at least for the 2.x series).
>
> ---
> Roy Smith
> roy@panix.com
>

Out of curiousity have the tried the new regex module from pypi rather 
than the stdlib version?  A heck of a lot of work has gone into it see 
http://bugs.python.org/issue2636

-- 
"Steve is going for the pink ball - and for those of you who are 
watching in black and white, the pink is next to the green." Snooker 
commentator 'Whispering' Ted Lowe.

Mark Lawrence

Thread

Why is regex so slow? roy@panix.com (Roy Smith) - 2013-06-18 12:45 -0400
  Re: Why is regex so slow? Skip Montanaro <skip@pobox.com> - 2013-06-18 12:01 -0500
  Re: Why is regex so slow? Roy Smith <roy@panix.com> - 2013-06-18 13:08 -0400
  Re: Why is regex so slow? Chris Angelico <rosuav@gmail.com> - 2013-06-19 03:20 +1000
    Re: Why is regex so slow? Johannes Bauer <dfnsonfsduifb@gmx.de> - 2013-06-18 20:10 +0200
      Re: Why is regex so slow? Roy Smith <roy@panix.com> - 2013-06-18 12:40 -0700
      Re: Why is regex so slow? André Malo <ndparker@gmail.com> - 2013-06-18 21:59 +0200
        Re: Why is regex so slow? André Malo <ndparker@gmail.com> - 2013-06-18 22:13 +0200
  Re: Why is regex so slow? MRAB <python@mrabarnett.plus.com> - 2013-06-18 18:31 +0100
  Re: Why is regex so slow? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-06-18 18:34 +0100
    Re: Why is regex so slow? roy@panix.com (Roy Smith) - 2013-06-18 15:21 -0400
      Re: Why is regex so slow? MRAB <python@mrabarnett.plus.com> - 2013-06-18 20:49 +0100
  Re: Why is regex so slow? Rick Johnson <rantingrickjohnson@gmail.com> - 2013-06-18 12:21 -0700
  Re: Why is regex so slow? Antoine Pitrou <solipsis@pitrou.net> - 2013-06-18 20:05 +0000
    Re: Why is regex so slow? Roy Smith <roy@panix.com> - 2013-06-18 13:23 -0700
      Re: Why is regex so slow? Duncan Booth <duncan.booth@invalid.invalid> - 2013-06-19 13:21 +0000
        Re: Why is regex so slow? Roy Smith <roy@panix.com> - 2013-06-19 12:55 -0700
    Re: Why is regex so slow? Grant Edwards <invalid@invalid.invalid> - 2013-06-18 20:30 +0000
      Re: Why is regex so slow? Terry Reedy <tjreedy@udel.edu> - 2013-06-18 17:29 -0400
      Re: Why is regex so slow? Johannes Bauer <dfnsonfsduifb@gmx.de> - 2013-06-19 10:29 +0200
  Re: Why is regex so slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-06-19 01:51 +0000
    Re: Why is regex so slow? Dave Angel <davea@davea.name> - 2013-06-18 22:11 -0400
      Re: Why is regex so slow? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-06-19 03:16 +0000

csiph-web