Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #81089 > unrolled thread

Re: python 2.7 and unicode (one more time)

Started byMichael Torrie <torriem@gmail.com>
First post2014-11-20 09:51 -0700
Last post2014-11-21 12:39 +1100
Articles 2 on this page of 62 — 13 participants

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: python 2.7 and unicode (one more time) Michael Torrie <torriem@gmail.com> - 2014-11-20 09:51 -0700
    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-20 20:56 +0200
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 10:49 +1100
        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 08:36 +0200
          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:02 +1100
            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 09:14 +0200
              Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:29 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 10:16 +0200
                  Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 19:24 +1100
          Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-21 08:03 -0800
            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 18:36 +0200
              Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 03:52 +1100
              Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 15:38 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 08:40 +0200
                  Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 00:50 +1100
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 01:17 +1100
                    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:18 +0200
                      Re: python 2.7 and unicode (one more time) Roy Smith <roy@panix.com> - 2014-11-22 09:44 -0500
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:56 +0200
                      Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 07:44 -0800
                        Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-22 07:57 -0800
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 19:49 +0200
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 18:17 +0000
                            Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:32 -0800
                          Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:17 -0800
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 07:17 +1100
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 22:04 +0000
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 09:31 +1100
                          Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 23:38 +0000
                            Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 04:52 -0800
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:52 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 13:11 +1100
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 22:10 -0500
                          Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 11:33 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 03:39 +1100
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-23 13:13 -0500
                          Re: python 2.7 and unicode (one more time) Dave Angel <d@davea.name> - 2014-11-23 15:31 -0500
                          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:22 +1100
                          Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 20:24 -0500
                          Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-24 00:05 -0500
                  Re: python 2.7 and unicode (one more time) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-11-24 11:51 +1300
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:59 +1100
                    Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 08:57 +0200
                      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 18:18 +1100
                        Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 09:40 +0200
                          Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:56 +1100
                            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-25 10:08 +1100
                            Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-25 08:23 +0200
                              Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 22:56 +1100
                                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-26 00:07 +1100
                                  Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-25 06:26 -0800
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:35 -0500
                  Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 17:17 +1100
                    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 17:37 +1100
                      Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 07:40 -0800
      Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-21 11:32 +1100
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:10 +1100
          Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 02:23 +1100
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 02:40 +1100
            Re: python 2.7 and unicode (one more time) Tim Chase <python.list@tim.thechases.com> - 2014-11-21 10:00 -0600
        Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-20 20:31 -0500
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:39 +1100

Page 4 of 4 — ← Prev page 1 2 3 [4]


#81157

Fromrandom832@fastmail.us
Date2014-11-20 20:31 -0500
Message-ID<mailman.16073.1416533493.18130.python-list@python.org>
In reply to#81154
On Thu, Nov 20, 2014, at 20:10, Chris Angelico wrote:
> 2) Languages which use a different alphabet (eg Cyrillic - Russian,
> Bulgarian). You could possibly cram them into an eight-bit encoding
> without tipping ASCII out, but I'm not sure. In Unicode, these
> languages are all easily supported by the BMP, as they don't use a
> huge number of characters each.

There are numerous eight-bit encodings that support latin and one other
alphabet. Remember, ASCII is a seven-bit encoding, and an eight-bit
encoding is basically two seven-bit encodings.

The most difficult (of those still possible at all) language to encode
in eight bits is actually Vietnamese, which uses the Latin alphabet, due
to the sheer number of accented letters used. Windows' encoding of it
(along with some other lesser used encodings, all for Vietnamese) is the
only 8-bit encoding to use combining accents, in a way unfortunately
incompatible with unicode normalization if naively translated, whereas
VISCII sacrifices a handful of C0 control characters in addition to
fully packing the high half with letters.


-- 
Random832

[toc] | [prev] | [next] | [standalone]


#81158

FromChris Angelico <rosuav@gmail.com>
Date2014-11-21 12:39 +1100
Message-ID<mailman.16074.1416533965.18130.python-list@python.org>
In reply to#81154
On Fri, Nov 21, 2014 at 12:31 PM,  <random832@fastmail.us> wrote:
> On Thu, Nov 20, 2014, at 20:10, Chris Angelico wrote:
>> 2) Languages which use a different alphabet (eg Cyrillic - Russian,
>> Bulgarian). You could possibly cram them into an eight-bit encoding
>> without tipping ASCII out, but I'm not sure. In Unicode, these
>> languages are all easily supported by the BMP, as they don't use a
>> huge number of characters each.
>
> There are numerous eight-bit encodings that support latin and one other
> alphabet. Remember, ASCII is a seven-bit encoding, and an eight-bit
> encoding is basically two seven-bit encodings.

I'm aware of this; Greek, for instance, fits quite happily into
ISO-8859-7, which is eight-bit.

> The most difficult (of those still possible at all) language to encode
> in eight bits is actually Vietnamese, which uses the Latin alphabet, due
> to the sheer number of accented letters used. Windows' encoding of it
> (along with some other lesser used encodings, all for Vietnamese) is the
> only 8-bit encoding to use combining accents, in a way unfortunately
> incompatible with unicode normalization if naively translated, whereas
> VISCII sacrifices a handful of C0 control characters in addition to
> fully packing the high half with letters.

This is what I was suspicious of. The very notion of "combining
accents" already breaks the notion that "a byte is a character is a
glyph", which most eight-bit encodings try to pretend. In any case,
the BMP still easily copes with them all.

(Hmm. I wonder how you'd typeset the old "Self-Pronouncing Alphabet"
for English? It's basically English text with a few markings added to
letters - not standard diacriticals that already exist in Unicode, but
dots. Probably possible, one way or another... but I haven't seen SPA
text since the 90s, and that was in stuff published back in the 80s or
so.)

ChrisA

[toc] | [prev] | [standalone]


Page 4 of 4 — ← Prev page 1 2 3 [4]

Back to top | Article view | comp.lang.python


csiph-web