Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\))
Subject: =?utf-8?Q?Re=3A_=27Stra=C3=9Fe=27_=28=27Strasse=27=29_and_Python?= =?utf-8?Q?_2?=
From: Travis Griggs <travisgriggs@gmail.com>
In-Reply-To: <52D68402.6030407@chamonix.reportlab.co.uk>
Date: Wed, 15 Jan 2014 08:28:49 -0800
Content-Transfer-Encoding: quoted-printable
References: <30dfa6f1-61b2-49b8-bc65-5fd18d498c38@googlegroups.com> <52D67873.2010502@chamonix.reportlab.co.uk> <lb5u13$9hs$1@ger.gmane.org> <52D68402.6030407@chamonix.reportlab.co.uk>
To: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.5520.1389803336.18130.python-list@python.org>
Lines: 63
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:63995


On Jan 15, 2014, at 4:50 AM, Robin Becker <robin@reportlab.com> wrote:

> On 15/01/2014 12:13, Ned Batchelder wrote:
> ........
>>> On my utf8 based system
>>>=20
>>>=20
>>>> robin@everest ~:
>>>> $ cat ooo.py
>>>> if __name__=3D=3D'__main__':
>>>>    import sys
>>>>    s=3D'A=CC=85B'
>>>>    print('version_info=3D%s\nlen(%s)=3D%d' % =
(sys.version_info,s,len(s)))
>>>> robin@everest ~:
>>>> $ python ooo.py
>>>> version_info=3Dsys.version_info(major=3D3, minor=3D3, micro=3D3,
>>>> releaselevel=3D'final', serial=3D0)
>>>> len(A=CC=85B)=3D3
>>>> robin@everest ~:
>>>> $
>>>=20
>>>=20
> ........
>> You are right that more than one codepoint makes up a grapheme, and =
that you'll
>> need code to deal with the correspondence between them. But let's not =
muddy
>> these already confusing waters by referring to that mapping as an =
encoding.
>>=20
>> In Unicode terms, an encoding is a mapping between codepoints and =
bytes.  Python
>> 3's str is a sequence of codepoints.
>>=20
> Semantics is everything. For me graphemes are the endpoint (or should =
be); to get a proper rendering of a sequence of graphemes I can use =
either a sequence of bytes or a sequence of codepoints. They are both =
encodings of the graphemes; what unicode says is an encoding doesn't =
define what encodings are ie mappings from some source alphabet to a =
target alphabet.

But you=E2=80=99re talking about two levels of encoding. One runs on top =
of the other. So insisting that you be able to call them all encodings, =
makes the term pointless, because now it=E2=80=99s ambiguous as to what =
you=E2=80=99re referring to. Are you referring to encoding in the sense =
of representing code points with bytes? Or are you referring to what the =
unicode guys call =E2=80=9Cforms=E2=80=9D?

For example, the NFC form of =E2=80=98=C3=B1=E2=80=99 is =E2=80=99\u00F1=E2=
=80=99. =E2=80=98nThe NFD form represents the exact same grapheme, but =
is =E2=80=98\u006e\u0303=E2=80=99. You can call them encodings if you =
want, but I echo Ned=E2=80=99s sentiment that you keep that to yourself. =
Conventionally, they=E2=80=99re different forms, not different =
encodings. You can encode either form with an encoding, e.g.

'\u00F1'.encode('utf8=E2=80=99)
'\u00F1'.encode('utf16=E2=80=99)

'\u006e\u0303'.encode('utf8=E2=80=99)
'\u006e\u0303'.encode('utf16')