Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!border1.nntp.ams2.giganews.com!border2.nntp.ams2.giganews.com!border4.nntp.ams.giganews.com!border2.nntp.ams.giganews.com!nntp.giganews.com!feeder1.cambriumusenet.nl!feed.tweaknews.nl!85.12.40.138.MISMATCH!xlned.com!feeder5.xlned.com!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.007
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'reject': 0.07; 'utf-8': 0.07; 'deemed': 0.09; 'encode': 0.09; 'subject:few': 0.09; 'translate': 0.10; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'planes': 0.16; 'unlikely': 0.16; 'wrote:': 0.18; 'thu,': 0.19; 'seems': 0.21; 'bytes': 0.24; 'unicode': 0.24; '(or': 0.24; 'header:In-Reply-To:1': 0.27; 'point': 0.28; 'wondering': 0.29; 'am,': 0.29; 'message-id:@mail.gmail.com': 0.30; "i'm": 0.30; '13,': 0.31; "d'aprano": 0.31; 'steven': 0.31; 'used,': 0.33; "can't": 0.35; 'skip:u 20': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'even': 0.60; 'most': 0.60; 'more': 0.64; 'needing': 0.65; '2013': 0.98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=PoUroB5NGewo5wut6igKG0R/OQsFdiNYxylLJkCkrQQ=; b=YEeLWizE1Pko3yuruP6Ppxr/p2ezSdfLhrTEA4arrTSXvY0RhATK52A4C8G+RAosoz eJAHCrxwHyaMat5rMUyqW8E3vDYTPdtMEGtF47s+HeR15jTtC2UzrVuswfoeaozd/33Q oO6vMYxXXRqqObb5WTCdJrgRSPGuhF1QELaJVP6bdRMSYzl3OTxEfja4jpTMfOF/z2DI 5D/81LhinBLRvSn6xALKrPZ6kZtJozskR8G5CQHQQUGxAjcuceg3IWz6fAuGml6VmUnS kNsI4vMDiQF50C47+/22mknDQ5ZOY57Y2e46bAth3ZnY0++KhzGW83of9EmOHlJpLUbG reOA==
MIME-Version: 1.0
X-Received: by 10.52.121.99 with SMTP id lj3mr8993773vdb.52.1371088916073; Wed, 12 Jun 2013 19:01:56 -0700 (PDT)
In-Reply-To: <51b9231b$0$29997$c3e8da3$5496439d@news.astraweb.com>
References: <6dfa3707-80f4-407a-a109-66dbb0130513@googlegroups.com> <mailman.2923.1370797972.3114.python-list@python.org> <kp9drh$1o0t$1@news.ntua.gr> <51b83e5a$0$29998$c3e8da3$5496439d@news.astraweb.com> <kp9lo6$9l5$2@news.ntua.gr> <pan.2013.06.12.20.30.22.31000@nowhere.com> <51b9231b$0$29997$c3e8da3$5496439d@news.astraweb.com>
Date: Thu, 13 Jun 2013 12:01:55 +1000
Subject: Re: A few questiosn about encoding
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.3153.1371088918.3114.python-list@python.org>
Lines: 15
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1371088918 news.xs4all.nl 15895 [2001:888:2000:d::a6]:54940
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:47886

On Thu, Jun 13, 2013 at 11:40 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> The *mechanism* of UTF-8 can go up to 6 bytes (or even 7 perhaps?), but
> that's not UTF-8, that's UTF-8-plus-extra-codepoints.

And a proper UTF-8 decoder will reject "\xC0\x80" and "\xed\xa0\x80",
even though mathematically they would translate into U+0000 and U+D800
respectively. The UTF-16 *mechanism* is limited to no more than
Unicode has currently used, but I'm left wondering if that's actually
the other way around - that Unicode planes were deemed to stop at the
point where UTF-16 can't encode any more. Not that it matters; with
most of the current planes completely unallocated, it seems unlikely
we'll be needing more.

ChrisA