Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!xlned.com!feeder3.xlned.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <lmmrb2$klg$1@ger.gmane.org>
References: <CAPTjJmr4iHdaCy61w2rz-oL6FcarRzzTeEU44Fxn2Z=gS0fh-Q@mail.gmail.com> <lmmkvk$73h$1@ger.gmane.org> <lmmrb2$klg$1@ger.gmane.org>
Date: Wed, 4 Jun 2014 20:44:09 +1000
Subject: Re: Unicode and Python - how often do you index strings?
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.10696.1401878658.18130.python-list@python.org>
Lines: 32
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:72623

On Wed, Jun 4, 2014 at 8:10 PM, Peter Otten <__peter__@web.de> wrote:
> The indices used for slicing typically don't come out of nowhere. A simple
> example would be
>
> def strip_prefix(text, prefix):
>     if text.startswith(prefix):
>         text = text[len(prefix):]
>     return text
>
> If both prefix and text use UTF-8 internally the byte offset is already
> known. The question is then how we can preserve that information.

Almost completely useless. First off, it solves only the problem of
operating on the string at exactly some point where you just got an
index; and secondly, you don't always get that index from a string
method. Suppose, for instance, that you iterate over a string thus:

for i, ch in enumerate(string):
    if ch=='{': start = i
    elif ch=='}': return string[start:end+1]

Okay, so this could be done by searching, but for something more
complicated, I can imagine it being better to enumerate. (But "I can
imagine" is much weaker than "Here's code that we use in production",
which is why I asked the question.)

Incidentally, the above code highlights the first problem too. With
direct indexing, you can ask for inclusive or exclusive slicing by
adding or subtracting one from the index. If you do that with a
byte-position-retaining special integer, you lose the byte position.

ChrisA