Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <1416533490.1620457.193598605.2BEB550F@webmail.messagingengine.com>
References: <m4kcu8$ta8$1@ger.gmane.org> <m4kn66$ma3$1@ger.gmane.org> <1416491999.209592.193345213.3A6FF9A4@webmail.messagingengine.com> <m4l20h$toc$1@ger.gmane.org> <CAPTjJmqjUGmTA-0JXrkeKYLeShmT=mOE=c38o8YD7XheCK_5Kw@mail.gmail.com> <m4l53q$ff4$1@ger.gmane.org> <mailman.16029.1416502283.18130.python-list@python.org> <87ppch8zvx.fsf@elektro.pacujo.net> <546e880d$0$12995$c3e8da3$5496439d@news.astraweb.com> <CAPTjJmrhXqsHaaaDUkj+F8goB_fatdZcCv7Lv4JdcwG8bNZWSQ@mail.gmail.com> <1416533490.1620457.193598605.2BEB550F@webmail.messagingengine.com>
Date: Fri, 21 Nov 2014 12:39:21 +1100
Subject: Re: python 2.7 and unicode (one more time)
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.16074.1416533965.18130.python-list@python.org>
Lines: 37
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:81158

On Fri, Nov 21, 2014 at 12:31 PM,  <random832@fastmail.us> wrote:
> On Thu, Nov 20, 2014, at 20:10, Chris Angelico wrote:
>> 2) Languages which use a different alphabet (eg Cyrillic - Russian,
>> Bulgarian). You could possibly cram them into an eight-bit encoding
>> without tipping ASCII out, but I'm not sure. In Unicode, these
>> languages are all easily supported by the BMP, as they don't use a
>> huge number of characters each.
>
> There are numerous eight-bit encodings that support latin and one other
> alphabet. Remember, ASCII is a seven-bit encoding, and an eight-bit
> encoding is basically two seven-bit encodings.

I'm aware of this; Greek, for instance, fits quite happily into
ISO-8859-7, which is eight-bit.

> The most difficult (of those still possible at all) language to encode
> in eight bits is actually Vietnamese, which uses the Latin alphabet, due
> to the sheer number of accented letters used. Windows' encoding of it
> (along with some other lesser used encodings, all for Vietnamese) is the
> only 8-bit encoding to use combining accents, in a way unfortunately
> incompatible with unicode normalization if naively translated, whereas
> VISCII sacrifices a handful of C0 control characters in addition to
> fully packing the high half with letters.

This is what I was suspicious of. The very notion of "combining
accents" already breaks the notion that "a byte is a character is a
glyph", which most eight-bit encodings try to pretend. In any case,
the BMP still easily copes with them all.

(Hmm. I wonder how you'd typeset the old "Self-Pronouncing Alphabet"
for English? It's basically English text with a few markings added to
letters - not standard diacriticals that already exist in Unicode, but
dots. Probably possible, one way or another... but I haven't seen SPA
text since the 90s, and that was in stuff published back in the 80s or
so.)

ChrisA