Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail From: eryk sun Newsgroups: comp.lang.python Subject: Re: Unicode failure Date: Sat, 5 Dec 2015 07:21:58 -0600 Lines: 41 Message-ID: References: <20151204130738.76313c43@imp> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: news.uni-berlin.de AmcTac0yQnVdF/TIzuTALQa43d8lDlNPuX24qxLJPEhg== Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.006 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'received:209.85.223': 0.03; 'pages.': 0.07; 'url:msdn': 0.07; '1200,': 0.09; '[1]:': 0.09; "(it's": 0.16; 'fail,': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'reedy': 0.16; 'subject:Unicode': 0.16; 'wrote:': 0.16; '>>>': 0.20; 'windows': 0.20; '2015': 0.20; 'to:name:python-list@python.org': 0.20; "aren't": 0.22; 'am,': 0.23; 'dec': 0.23; 'sat,': 0.23; 'header:In-Reply-To:1': 0.24; 'chris': 0.26; 'message-id:@mail.gmail.com': 0.27; 'subset': 0.29; 'windows,': 0.29; 'character': 0.29; "i'm": 0.30; 'print': 0.30; 'code': 0.30; '[1]': 0.32; 'idle': 0.33; 'received:google.com': 0.35; 'but': 0.36; 'received:209.85': 0.36; 'url:library': 0.36; 'to:addr:python-list': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'missing': 0.37; 'received:209': 0.38; 'test': 0.39; 'sure': 0.39; 'to:addr:python.org': 0.40; 'entire': 0.61; 'is.': 0.63; '>>>>>': 0.66; "they're": 0.66; 'url:en-us': 0.72; 'euro': 0.75; 'china.': 0.79 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=ejlnUMAzxQh95j8O63bDGa2JBsVVbcPEMw3vt+UdDsY=; b=nhFyGPr8vPhvb9jrDVPwynCNI8jdYKSLVnkNYpJkhj5/TpiLGfsv/zCXdE1EwepZey wbjj5h99zyq9pykbg0mYMP2McKYwEB4PXO86U2iEW6h8IQ4CB9IevCivY2WkLJVCoyP4 obI2/jLIK/PPkf025j/BUBg1R3SztlFMdNdy8xVQAfiR/wgI7GZXbk5fH9IOz3RjYmIB SC5ofUKAtjsBR88wHAxc4mRc3hvfVWaXidbXeu20r1iMBig83OlXu5PvPkU9SjtImX+3 0A+6KrCyWv5OwuyhLaAyNfCs4ToCo9qdBTg8VUzOIJO9Y+IqAt7uXa0mjY8L7nlNAUCv aYHg== X-Received: by 10.107.164.227 with SMTP id d96mr17876464ioj.73.1449321758413; Sat, 05 Dec 2015 05:22:38 -0800 (PST) In-Reply-To: X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.20+ Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Xref: csiph.com comp.lang.python:100037 On Sat, Dec 5, 2015 at 12:10 AM, Chris Angelico wrote: > On Sat, Dec 5, 2015 at 5:06 PM, Terry Reedy wrote: >> On 12/4/2015 10:22 PM, Random832 wrote: >>> >>> On 2015-12-04, Terry Reedy wrote: >>>> >>>> Tk widgets, and hence IDLE windows, will print any character from \u00= 00 >>>> to \uffff without raising, even if the result is blank or =EF=BF=BD. = Higher >>>> codepoints fail, but allowing the entire BMP is better than any Window= s >>>> codepage. >>> >>> >>> Well, any bar 1200, 1201, 12000, 12001, 65000, 65001, and 54936. >> >> >> Test before you post. >> >>>>> for cp in 1200, 1201, 12000, 12001, 65000, 65001, 54936: >> print(chr(cp)) >> >> >> =D2=B0 >> =D2=B1 >> =E2=BB=A0 >> =E2=BB=A1 >> =EF=BF=BD >> =EF=BF=BD >> =ED=9A=98 > > Those numbers aren't codepoints, they're code pages. Specifically, > they're UTF-16, UTF-32, UTF-8, and I'm not sure what 54936 is. Codepage 65000 is UTF-7. Codepage 54936 [1] is GB18030, the official character set of China. It's a UTF superset of GBK. For comparison, codepage 936 is a subset of GBK (it's missing 95 characters) plus the Euro symbol. [1]: https://msdn.microsoft.com/en-us/library/dd317756