Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #81336

Re: python 2.7 and unicode (one more time)

Path csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!bcyclone05.am1.xlned.com!bcyclone05.am1.xlned.com!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <rosuav@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.002
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'python.': 0.02; 'programmer': 0.03; 'sufficient': 0.05; 'ascii': 0.09; 'character,': 0.09; 'english,': 0.09; 'forms,': 0.09; 'subject:2.7': 0.09; 'worse': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'windows': 0.15; '(code': 0.16; '23,': 0.16; 'correctness': 0.16; 'distinct': 0.16; 'encodings': 0.16; 'former,': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'latter,': 0.16; 'mess,': 0.16; 'subject:unicode': 0.16; 'unicode.': 0.16; 'subject:python': 0.16; 'wrote:': 0.18; 'code.': 0.18; 'not,': 0.20; 'cc:addr:python.org': 0.22; "aren't": 0.24; 'fraction': 0.24; 'text.': 0.24; 'unicode': 0.24; "world's": 0.24; 'fine': 0.24; 'cc:2**0': 0.24; 'supported': 0.26; 'header:In-Reply-To:1': 0.27; 'idea': 0.28; 'character': 0.29; "doesn't": 0.30; 'characters': 0.30; 'especially': 0.30; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; 'gives': 0.31; 'code': 0.31; '(maybe': 0.31; "d'aprano": 0.31; 'steven': 0.31; 'probably': 0.32; 'languages': 0.32; 'says': 0.33; 'running': 0.33; 'guess': 0.33; 'subject:time': 0.33; 'trouble': 0.34; 'subject: (': 0.35; 'etc': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'really': 0.36; 'subject:one': 0.36; 'represent': 0.38; 'system,': 0.38; 'nov': 0.38; 'pm,': 0.38; 'supporting': 0.39; 'even': 0.60; 'han': 0.60; 'today.': 0.61; 'world.': 0.61; 'design,': 0.64; 'pick': 0.64; 'subject:more': 0.64; 'interest': 0.64; 'more': 0.64; 'american': 0.66; 'between': 0.67; 'combining': 0.68; 'legal': 0.71; 'etc,': 0.84; 'horrible': 0.84; 'lazy': 0.91; 'trustworthy': 0.91; 'to:none': 0.92; 'yourself,': 0.95
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type:content-transfer-encoding; bh=lUr9ymm3uOUiFwJTWVZWF5dta/CWA8x5lF9j2I9JAro=; b=Qh5cLXdCace85FrTSXgOhV5UMKwnFuMXdB+LXcbGyxKM5qN4hzYP0HEaav0WEgwG/N XDagy9TvQ8PkLecCWdXdmh50Wm4yH9SjDOgTgrh4lcCiTqLqLSKXLXNlU3E2lefVdCZS SvZxbhRIsbLG1UEX58wmnCtVuVCjweOe7GLCLlpQcD/iXGB79e3pCIE9bSwp6QnGuPYO NO08SarehRclwBQXqIESt9Tbitk47FqWNdzmqVnnUISZQAw8phQNewXEmzdtVGD4eonH RSkxP3Rh6dUSeE3/KXLpEXAXcy0Quvk8Douz7QEvzf/DvqQ1fYB6oHNtjXHw1Yke3s6p U9Hw==
MIME-Version 1.0
X-Received by 10.50.152.104 with SMTP id ux8mr6009755igb.14.1416724650321; Sat, 22 Nov 2014 22:37:30 -0800 (PST)
In-Reply-To <54717c05$0$12985$c3e8da3$5496439d@news.astraweb.com>
References <m4kcu8$ta8$1@ger.gmane.org> <m4kn66$ma3$1@ger.gmane.org> <1416491999.209592.193345213.3A6FF9A4@webmail.messagingengine.com> <m4l20h$toc$1@ger.gmane.org> <CAPTjJmqjUGmTA-0JXrkeKYLeShmT=mOE=c38o8YD7XheCK_5Kw@mail.gmail.com> <m4l53q$ff4$1@ger.gmane.org> <mailman.16029.1416502283.18130.python-list@python.org> <87ppch8zvx.fsf@elektro.pacujo.net> <mailman.16066.1416527371.18130.python-list@python.org> <87k32p6ow8.fsf@elektro.pacujo.net> <f8bbae79-6196-43fc-bd9f-f95ddbcf95f3@googlegroups.com> <871towik8j.fsf@elektro.pacujo.net> <5470134c$0$12986$c3e8da3$5496439d@news.astraweb.com> <mailman.16149.1416706556.18130.python-list@python.org> <54717c05$0$12985$c3e8da3$5496439d@news.astraweb.com>
Date Sun, 23 Nov 2014 17:37:30 +1100
Subject Re: python 2.7 and unicode (one more time)
From Chris Angelico <rosuav@gmail.com>
Cc "python-list@python.org" <python-list@python.org>
Content-Type text/plain; charset=UTF-8
Content-Transfer-Encoding quoted-printable
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.16162.1416724653.18130.python-list@python.org> (permalink)
Lines 43
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1416724653 news.xs4all.nl 2884 [2001:888:2000:d::a6]:58772
X-Complaints-To abuse@xs4all.nl
X-Received-Bytes 6667
X-Received-Body-CRC 3477346858
Xref csiph.com comp.lang.python:81336

Show key headers only | View raw


On Sun, Nov 23, 2014 at 5:17 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> If Python treated the character set as an implementation detail, the
> programmer would have no way of knowing whether
>
> s = u"ö"
>
> is legal or not, since you cannot know whether or not ö is a supported
> character in the running Python. It might work on your system, and fail for
> other people. That is worse than the old distinction between "narrow"
> and "wide" builds. It would be a lazy and stupid design, and especially
> stupid since there really in no good alternative to Unicode today. ASCII is
> not even sufficient for American English, the whole Windows code page idea
> is a horrible mess, none of the legacy encodings are suitable for more than
> a tiny fraction of the world.

(Code pages aren't a Windows concept, of course, though I guess that's
the main place where they're found on PCs today.)

The only trouble with enforcing Unicode is Japanese encodings and the
whole Han unification debate. Ultimately, you have to pick a side: are
you siding with those who say there are fewer characters with multiple
forms, or with those who say there are more distinct characters? If
the former, go with Unicode. If the latter, be prepared to do heaps of
work yourself, and probably be stuck with supporting only Japanese,
because encodings like Shift-JIS aren't going to be able to represent
Scandinavian text.

Me, I'm siding with Unicode. The politicking of Han unification
doesn't interest me, so I'm happy to accept a position that says that
they're all the same character, just as the Roman letter A can be used
in English, Italian, German, Swedish, etc, etc, etc (maybe with some
combining characters for diacriticals). That gives me access to all
the world's languages with a single character set and some trustworthy
encodings. I think it's a fine trade-off: philosophy I don't care
about versus correctness in my code.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread


Thread

Re: python 2.7 and unicode (one more time) Michael Torrie <torriem@gmail.com> - 2014-11-20 09:51 -0700
  Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-20 20:56 +0200
    Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 10:49 +1100
      Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 08:36 +0200
        Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:02 +1100
          Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 09:14 +0200
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:29 +1100
              Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 10:16 +0200
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 19:24 +1100
        Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-21 08:03 -0800
          Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 18:36 +0200
            Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 03:52 +1100
            Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 15:38 +1100
              Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 08:40 +0200
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 00:50 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 01:17 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:18 +0200
                Re: python 2.7 and unicode (one more time) Roy Smith <roy@panix.com> - 2014-11-22 09:44 -0500
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:56 +0200
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 07:44 -0800
                Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-22 07:57 -0800
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 19:49 +0200
                Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 18:17 +0000
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:32 -0800
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:17 -0800
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 07:17 +1100
                Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 22:04 +0000
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 09:31 +1100
                Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 23:38 +0000
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 04:52 -0800
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:52 -0500
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 13:11 +1100
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 22:10 -0500
                Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 11:33 -0500
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 03:39 +1100
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-23 13:13 -0500
                Re: python 2.7 and unicode (one more time) Dave Angel <d@davea.name> - 2014-11-23 15:31 -0500
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:22 +1100
                Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 20:24 -0500
                Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-24 00:05 -0500
                Re: python 2.7 and unicode (one more time) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-11-24 11:51 +1300
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:59 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 08:57 +0200
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 18:18 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 09:40 +0200
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:56 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-25 10:08 +1100
                Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-25 08:23 +0200
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 22:56 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-26 00:07 +1100
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-25 06:26 -0800
              Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:35 -0500
                Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 17:17 +1100
                Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 17:37 +1100
                Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 07:40 -0800
    Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-21 11:32 +1100
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:10 +1100
        Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 02:23 +1100
          Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 02:40 +1100
          Re: python 2.7 and unicode (one more time) Tim Chase <python.list@tim.thechases.com> - 2014-11-21 10:00 -0600
      Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-20 20:31 -0500
      Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:39 +1100

csiph-web