Re: python 2.7 and unicode (one more time)

References	(6 earlier) <mailman.16029.1416502283.18130.python-list@python.org> <87ppch8zvx.fsf@elektro.pacujo.net> <546e880d$0$12995$c3e8da3$5496439d@news.astraweb.com> <CAPTjJmrhXqsHaaaDUkj+F8goB_fatdZcCv7Lv4JdcwG8bNZWSQ@mail.gmail.com> <1416533490.1620457.193598605.2BEB550F@webmail.messagingengine.com>
Date	2014-11-21 12:39 +1100
Subject	Re: python 2.7 and unicode (one more time)
From	Chris Angelico <rosuav@gmail.com>
Newsgroups	comp.lang.python
Message-ID	<mailman.16074.1416533965.18130.python-list@python.org> (permalink)

Show all headers | View raw

On Fri, Nov 21, 2014 at 12:31 PM,  <random832@fastmail.us> wrote:
> On Thu, Nov 20, 2014, at 20:10, Chris Angelico wrote:
>> 2) Languages which use a different alphabet (eg Cyrillic - Russian,
>> Bulgarian). You could possibly cram them into an eight-bit encoding
>> without tipping ASCII out, but I'm not sure. In Unicode, these
>> languages are all easily supported by the BMP, as they don't use a
>> huge number of characters each.
>
> There are numerous eight-bit encodings that support latin and one other
> alphabet. Remember, ASCII is a seven-bit encoding, and an eight-bit
> encoding is basically two seven-bit encodings.

I'm aware of this; Greek, for instance, fits quite happily into
ISO-8859-7, which is eight-bit.

> The most difficult (of those still possible at all) language to encode
> in eight bits is actually Vietnamese, which uses the Latin alphabet, due
> to the sheer number of accented letters used. Windows' encoding of it
> (along with some other lesser used encodings, all for Vietnamese) is the
> only 8-bit encoding to use combining accents, in a way unfortunately
> incompatible with unicode normalization if naively translated, whereas
> VISCII sacrifices a handful of C0 control characters in addition to
> fully packing the high half with letters.

This is what I was suspicious of. The very notion of "combining
accents" already breaks the notion that "a byte is a character is a
glyph", which most eight-bit encodings try to pretend. In any case,
the BMP still easily copes with them all.

(Hmm. I wonder how you'd typeset the old "Self-Pronouncing Alphabet"
for English? It's basically English text with a few markings added to
letters - not standard diacriticals that already exist in Unicode, but
dots. Probably possible, one way or another... but I haven't seen SPA
text since the 90s, and that was in stuff published back in the 80s or
so.)

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Re: python 2.7 and unicode (one more time) Michael Torrie <torriem@gmail.com> - 2014-11-20 09:51 -0700
  Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-20 20:56 +0200
    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 10:49 +1100
      Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 08:36 +0200
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:02 +1100
          Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 09:14 +0200
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:29 +1100
              Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 10:16 +0200
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 19:24 +1100
        Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-21 08:03 -0800
          Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 18:36 +0200
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 03:52 +1100
            Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 15:38 +1100
              Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 08:40 +0200
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 00:50 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 01:17 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:18 +0200
                Re: python 2.7 and unicode (one more time) Roy Smith <roy@panix.com> - 2014-11-22 09:44 -0500
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:56 +0200
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 07:44 -0800
                Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-22 07:57 -0800
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 19:49 +0200
                Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 18:17 +0000
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:32 -0800
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:17 -0800
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 07:17 +1100
                Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 22:04 +0000
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 09:31 +1100
                Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 23:38 +0000
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 04:52 -0800
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:52 -0500
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 13:11 +1100
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 22:10 -0500
                Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 11:33 -0500
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 03:39 +1100
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-23 13:13 -0500
                Re: python 2.7 and unicode (one more time) Dave Angel <d@davea.name> - 2014-11-23 15:31 -0500
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:22 +1100
                Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 20:24 -0500
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-24 00:05 -0500
                Re: python 2.7 and unicode (one more time) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-11-24 11:51 +1300
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:59 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 08:57 +0200
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 18:18 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 09:40 +0200
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:56 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-25 10:08 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-25 08:23 +0200
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 22:56 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-26 00:07 +1100
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-25 06:26 -0800
              Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:35 -0500
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 17:17 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 17:37 +1100
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 07:40 -0800
    Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-21 11:32 +1100
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:10 +1100
        Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 02:23 +1100
          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 02:40 +1100
          Re: python 2.7 and unicode (one more time) Tim Chase <python.list@tim.thechases.com> - 2014-11-21 10:00 -0600
      Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-20 20:31 -0500
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:39 +1100

csiph-web