Path: csiph.com!usenet.pasdenom.info!dedibox.gegeweb.org!gegeweb.eu!nntpfeed.proxad.net!proxad.net!feeder1-2.proxad.net!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Newsgroups: comp.lang.python
Date: Sat, 25 Aug 2012 08:47:52 -0700 (PDT)
In-Reply-To: <mailman.3793.1345888006.4697.python-list@python.org>
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=83.79.66.203; posting-account=ung4FAoAAAC46zhHJ0Nsnuox7M5gDvs_
References: <a81cd504-d889-4aa1-9daa-6df3448b4da8@googlegroups.com> <1874857c-68ef-4c1b-b15a-46ef47df9445@googlegroups.com> <mailman.3784.1345854291.4697.python-list@python.org> <1cb3f062-eb45-4b0c-977b-76afb099923c@googlegroups.com> <k1a40u$r47$2@ger.gmane.org> <mailman.3793.1345888006.4697.python-list@python.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Subject: Re: Flexible string representation, unicode, typography, ...
From: wxjmfauth@gmail.com
To: comp.lang.python@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org
Precedence: list
Message-ID: <mailman.3805.1345909675.4697.python-list@python.org>
Lines: 114
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:27876

Le samedi 25 ao=FBt 2012 11:46:34 UTC+2, Frank Millman a =E9crit=A0:
> On 25/08/2012 10:58, Mark Lawrence wrote:
>=20
> > On 25/08/2012 08:27, wxjmfauth@gmail.com wrote:
>=20
> >>
>=20
> >> Unicode design: a flat table of code points, where all code
>=20
> >> points are "equals".
>=20
> >> As soon as one attempts to escape from this rule, one has to
>=20
> >> "pay" for it.
>=20
> >> The creator of this machinery (flexible string representation)
>=20
> >> can not even benefit from it in his native language (I think
>=20
> >> I'm correctly informed).
>=20
> >>
>=20
> >> Hint: Google -> "Das grosse Eszett"
>=20
> >>
>=20
> >> jmf
>=20
> >>
>=20
> >
>=20
> > It's Saturday morning, I'm stone cold sober, had a good sleep and I'm
>=20
> > still baffled as to the point if any.  Could someone please enlightem m=
e?
>=20
> >
>=20
>=20
>=20
> Here's what I think he is saying. I am posting this to test the water. I=
=20
>=20
> am also confused, and if I have got it wrong hopefully someone will=20
>=20
> correct me.
>=20
>=20
>=20
> In python 3.3, unicode strings are now stored as follows -
>=20
>    if all characters can be represented by 1 byte, the entire string is=
=20
>=20
> composed of 1-byte characters
>=20
>    else if all characters can be represented by 1 or 2 bytea, the entire=
=20
>=20
> string is composed of 2-byte characters
>=20
>    else the entire string is composed of 4-byte characters
>=20
>=20
>=20
> There is an overhead in making this choice, to detect the lowest number=
=20
>=20
> of bytes required.
>=20
>=20
>=20
> jmfauth believes that this only benefits 'english-speaking' users, as=20
>=20
> the rest of the world will tend to have strings where at least one=20
>=20
> character requires 2 or 4 bytes. So they incur the overhead, without=20
>=20
> getting any benefit.
>=20
>=20
>=20
> Therefore, I think he is saying that he would have preferred that python=
=20
>=20
> standardise on 4-byte characters, on the grounds that the saving in=20
>=20
> memory does not justify the performance overhead.
>=20
>=20
>=20
> Frank Millman

Very well explained. Thanks.

More precisely, affected are not only the 'english-speaking'
users, but all the users who are using not latin-1 characters.
(See the title of this topic, ... typography).

Being at the same time, latin-1 and unicode compliant is
a plain absurdity in the mathematical sense.

---

For those you do not know, the go language has introduced
the rune type. As far as I know, nobody is complaining, I
have not even seen a discussion related to this subject.


100% Unicode compliant from the day 0. Congratulations.

jmf