Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.009
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; '"""': 0.05; 'cache': 0.05; '*is*': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'cache:': 0.16; 'caching': 0.16; 'iteration': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'subject:search': 0.16; 'wrote:': 0.17; 'thu,': 0.17; '>>>': 0.18; 'feb': 0.19; 'import': 0.21; 'fraction': 0.22; 'setup.': 0.22; 'header:User-Agent:1': 0.26; 'compiled': 0.27; 'regular': 0.27; 'header:X-Complaints- To:1': 0.28; 'subject:/': 0.28; "we're": 0.30; 'function': 0.30; 'to:addr:python-list': 0.33; 'done': 0.34; 'clear': 0.35; 'needed': 0.35; 'whatever': 0.35; 'expected': 0.35; 'pm,': 0.35; 'add': 0.36; 'received:org': 0.36; 'expensive': 0.36; 'charset:us- ascii': 0.36; 'execute': 0.37; 'being': 0.37; 'why': 0.37; 'skip:3 10': 0.37; 'subject:: ': 0.38; 'mean': 0.38; 'to:addr:python.org': 0.39; 'apply': 0.39; 'header:Received:5': 0.40; 'think': 0.40; 'times': 0.63; 'here': 0.65; 'url:a': 0.72; 'savings': 0.75; '2013': 0.84; 'received:sd.cox.net': 0.84; 'subject:via': 0.84; 'hand,': 0.97
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: rh <richard_hubbe11@lavabit.com>
Subject: Re: Curious to see alternate approach on a search/replace via regex
Date: Thu, 7 Feb 2013 21:57:55 -0800
References: <mailman.1426.1360187770.2939.python-list@python.org> <511319c7$0$21812$c3e8da3$76491128@news.astraweb.com> <mailman.1437.1360207874.2939.python-list@python.org> <51142e96$0$6512$c3e8da3$5496439d@news.astraweb.com> <mailman.1460.1360278819.2939.python-list@python.org> <51143feb$0$29974$c3e8da3$5496439d@news.astraweb.com> <CALwzidmiR=ZaSF=woeHgOOmVtCrcngnFm_bMOizM+Lo6OOALBA@mail.gmail.com> <CALwzid=fkG9fFELE_xRmJwM2-KvhkBLkSw6_OBuUpZ+wshMbVA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Gmane-NNTP-Posting-Host: ip68-227-87-145.sb.sd.cox.net
User-Agent: dsodnetnin
X-Mailer: EZnn0.37p
X-Newsreader: EZnn0.37p
X-Gmane-NNTP-Posting-Host: EZnn0.37p
Original-Received: from slem by 1.1 with local
X-No-Archive: yes
Archive: no
X-Archive: expiry=11
X-Archive: encrypt
X-Operating-System: Barebones_6.1
X-Gmane-NNTP-Posting-Host: 192.168.1.1
X-NNTP-Posting-Host: 192.168.1.1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.1477.1360303085.2939.python-list@python.org>
Lines: 36
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1360303085 news.xs4all.nl 6892 [2001:888:2000:d::a6]:47878
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:38409

On Thu, 7 Feb 2013 18:08:00 -0700
Ian Kelly <ian.g.kelly@gmail.com> wrote:

> On Thu, Feb 7, 2013 at 5:55 PM, Ian Kelly <ian.g.kelly@gmail.com>
> wrote:
> > Whatever caching is being done by re.compile, that's still a 24%
> > savings by moving the compile calls into the setup.
> 
> On the other hand, if you add an re.purge() call to the start of t1 to
> clear the cache:
> 
> >>> t3 = Timer("""
> ... re.purge()
> ... nx = re.compile(r'https?://(.+)$')
> ... v = nx.search(u).group(1)
> ... ux = re.compile(r'([-:./?&=]+)')
> ... ux.sub('_', v)""", """
> ... import re
> ... u = 'http://alongnameofasite1234567.com/q?sports=run&a=1&b=1'""")
> >>> min(t3.repeat(number=10000))
> 3.5532990924824617
> 
> Which is approximately 30 times slower, so clearly the regular
> expression *is* being cached.  I think what we're seeing here is that
> the time needed to look up the compiled regular expression in the
> cache is a significant fraction of the time needed to actually execute
> it.

By "actually execute" you mean to apply the compiled expression
to the search or sub? Or do you mean the time needed to compile
the pattern into a regex obj?

I presumed that compiling the pattern at each iteration was expensive
and that's why I expected moving it out of the function to reduce the
time needed to search/sub.