Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Terry Reedy <tjreedy@udel.edu>
Subject: Re: Coding challenge: Optimise a custom string encoding
Date: Mon, 18 Aug 2014 16:16:26 -0400
References: <6e869040-98e9-437b-b024-4ffe7abc3054@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0
In-Reply-To: <6e869040-98e9-437b-b024-4ffe7abc3054@googlegroups.com>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.13113.1408393206.18130.python-list@python.org>
Lines: 56
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:76508

On 8/18/2014 3:16 PM, Alex Willmer wrote:
> A challenge, just for fun. Can you speed up this function?

You should give a specification here, with examples. You should perhaps=20
be using .maketrans and .translate.

> import string
>
> charset =3D set(string.ascii_letters + string.digits + '@_-')
> byteseq =3D [chr(i) for i in xrange(256)]
> bytemap =3D {byte: byte if byte in charset else '+' + byte.encode('hex'=
)
>             for byte in byteseq}
>
> def plus_encode(s):
>      """Encode a unicode string with only ascii letters, digits, _, -, =
@, +
>      """
>      bytemap_ =3D bytemap
>      s_utf8 =3D s.encode('utf-8')
>      return ''.join([bytemap[byte] for byte in s_utf8])
>
> On my machine (Ubuntu 14.04, CPython 2.7.6, PyPy 2.2.1) this gets
>
> alex@martha:~$ python -m timeit -s 'import plus_encode' 'plus_encode.pl=
us_encode(u"""qwertyuiop1234567890!"=C2=A3$%^&*()EURO""")'
> 100000 loops, best of 3: 2.96 usec per loop
>
> alex@martha:~$ pypy -m timeit -s 'import plus_encode' 'plus_encode.plus=
_encode(u"""qwertyuiop1234567890!"=C2=A3$%^&*()EURO""")'
> 1000000 loops, best of 3: 1.24 usec per loop
>
> Back story:
> Last week we needed a custom encoding to store unicode usernames in a c=
onfig file that only allowed mixed case ascii, digits, underscore, dash, =
at-sign and plus sign. We also wanted to keeping the encoded usernames so=
mewhat human readable.
>
> My design was utf-8 and a variant of %-escaping, using the plus symbol.=
 So u'alic EURO 123' would be encoded as b'alic+e2+82+ac123'. This evenin=
g as a learning exercise I've tried to make it fast. This is the result.
>
> This challenge is just for fun. The chosen solution ended up being
>
> def name_encode(s):
>      return %s_%s' % (s.encode('utf-8').encode('hex'),
>                       re.replace('[A-Za-z0-9]', '', s))
>
> Regards, Alex
>


--=20
Terry Jan Reedy