Path: csiph.com!usenet.pasdenom.info!gegeweb.org!newsfeed0.kamp.net!newsfeed.kamp.net!feeder1.cambriumusenet.nl!82.197.223.103.MISMATCH!feeder3.cambriumusenet.nl!feed.tweaknews.nl!194.134.4.91.MISMATCH!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Sender: joshua.landau.ws@gmail.com
In-Reply-To: <ksf122$d5n$1@ger.gmane.org>
References: <mailman.4865.1374240179.3114.python-list@python.org> <51e967bb$0$29971$c3e8da3$5496439d@news.astraweb.com> <ksbt1a$5q1$1@ger.gmane.org> <CAN1F8qXttLWtMFDED-+gEdOR_5tZmDKtqDsF2kdojCRSMAn7eg@mail.gmail.com> <ksdtu7$ctm$1@ger.gmane.org> <CAN1F8qVv0N1D=JEkvWeQePhZg7fT7ULPPXp+ZTOmu=wQCzXiHQ@mail.gmail.com> <ksejfu$ist$1@ger.gmane.org> <CAN1F8qVj2nM6bzLETWm0mmvsrXxQog-qLQF=SRXepEyT4VNCuA@mail.gmail.com> <ksf122$d5n$1@ger.gmane.org>
From: Joshua Landau <joshua@landau.ws>
Date: Sat, 20 Jul 2013 23:33:23 +0100
Subject: Re: Find and Replace Simplification
To: Dave Angel <davea@davea.name>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: python-list <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.4933.1374359653.3114.python-list@python.org>
Lines: 49
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:50981

On 20 July 2013 22:56, Dave Angel <davea@davea.name> wrote:
> On 07/20/2013 02:37 PM, Joshua Landau wrote:
>>
>> The problem can be solved, I'd imagine, for builtin types. Just build
>> an internal representation upon calling .translate that's faster. It's
>> especially easy in the list case
>
> What "list case"?  list doesn't have a replace() method or translate()
> method.

I mean some_str.translate(some_list).

>> -- just build a C array=C2=B9 at the start
>> mapping int -> int and then have really fast C mapping speeds.
>
>
> As long as you can afford to have a list with a billion or so entries in =
it.
> We are talking about strings and version 3.3, aren't we?  Of course, one
> could always examine the mapping object (table) and see what the max valu=
e
> was, and only build a "C array" if it was smaller than say 50,000.

When talking about some_str.translate(some_list), this doesn't apply
very much -- they've already gotten a much bigger Python list.

In the dict case=C2=B2 I don't actually want to jump to the conclusion that
one should do array-based mappings because I can see the obvious
downsides and it's obviously not good to have 100 cases in there,
*but* I still think that there's a solution.

Here are some ideas:
=C2=B7 Latin and ASCII can obviously be done with a C array, and I imagine
that covers at least a fair portion of use-cases.
=C2=B7 If you only have a few characters in the mapping (so sys.getsizeof
is small) then it'll be a lot faster to just iterate through a C list
instead of checking the dict.
=C2=B7 Other cases are:
    =C2=B7 Full-character-set or equiv. mappings, which are already faster
than .replace(). Those should really be re-made into lists so that the
list optimisation can take place, and lists are much faster even in
versions without these hypothetical optimizations, too.
    =C2=B7 Custom objects. There's nothing we can do here.

I realise that this is a lot more code, so it's not something I'm
going to try to force. However, I think it's useful if it stops people
using .replace in a loop ;).

=C2=B2 some_str.translate(some_dict)