Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'string.': 0.04; 'subject:Python': 0.05; 'encode': 0.09; 'literal': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'tcl/tk': 0.09; 'terry': 0.09; 'unicode,': 0.09; '(the': 0.15; 'weird': 0.15; '-s,': 0.16; '3.3,': 0.16; 'codec': 0.16; 'fine.': 0.16; 'printing.': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'reedy': 0.16; 'subject:Problem': 0.16; 'subject:Unicode': 0.16; 'surrogate': 0.16; 'surrogates': 0.16; 'string': 0.17; 'wrote:': 0.17; 'char': 0.17; 'widget': 0.17; '(in': 0.18; 'jan': 0.18; '>>>': 0.18; 'windows': 0.19; 'skip:" 40': 0.20; '"",': 0.22; '(on': 0.22; 'header:In-Reply-To:1': 0.25; 'header:User-Agent:1': 0.26; '---': 0.26; 'creating': 0.26; 'supported': 0.26; '(most': 0.27; 'am,': 0.27; 'translated': 0.27; 'header:X-Complaints-To:1': 0.28; 'behavior.': 0.29; 'character': 0.29; 'file': 0.32; 'message.': 0.33; 'idle': 0.33; 'traceback': 0.33; 'problem': 0.33; 'to:addr :python-list': 0.33; 'version': 0.34; "can't": 0.34; 'text': 0.34; 'especially': 0.35; 'received:org': 0.36; 'but': 0.36; 'characters': 0.36; 'subject:with': 0.36; 'problems': 0.36; 'does': 0.37; 'subject:: ': 0.38; 'shows': 0.38; 'to:addr:python.org': 0.39; 'where': 0.40; 'header:Received:5': 0.40; 'skip:u 10': 0.60; 'first': 0.61; 'is.': 0.62; 'different': 0.63; 'skip:c 50': 0.66; 'cut': 0.71; 'printing,': 0.75; 'square': 0.75; 'brand': 0.78; 'music': 0.79; '<<<': 0.84; 'ion': 0.84; 'received:fios.verizon.net': 0.84; 'windows)': 0.84
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: Terry Reedy <tjreedy@udel.edu>
Subject: Re: Problem with Unicode char in Python 3.3.0
Date: Tue, 08 Jan 2013 03:40:47 -0500
References: <nobody-672426.17430906012013@news.free.fr> <mailman.175.1357492817.2939.python-list@python.org> <nobody-940F87.13572507012013@news.free.fr> <kcehls$9gc$1@ger.gmane.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable
X-Gmane-NNTP-Posting-Host: pool-173-75-251-66.phlapa.fios.verizon.net
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0
In-Reply-To: <kcehls$9gc$1@ger.gmane.org>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.266.1357634500.2939.python-list@python.org>
Lines: 51
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1357634500 news.xs4all.nl 6931 [2001:888:2000:d::a6]:50855
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:36419

On 1/7/2013 8:12 AM, Terry Reedy wrote:
> On 1/7/2013 7:57 AM, Franck Ditter wrote:
>
>> <<< print('\U0001d11e')
>> Traceback (most recent call last):
>>    File "<pyshell#1>", line 1, in <module>
>>      print('\U0001d11e')
>> UnicodeEncodeError: 'UCS-2' codec can't encode character '\U0001d11e'
>> in position 0: Non-BMP character not supported in Tk
>
> The message comes from printing to a tk text widget (the IDLE shell),
> not from creating the 1 char string. c =3D '\U0001d11e' works fine. Whe=
n
> you have problems with creating and printing unicode, *separate*
> creating from printing to see where the problem is. (I do not know if
> the brand new tcl/tk 8.6 is any better.)
>
> The windows console also chokes, but with a different message.
>
>  >>> c=3D'\U0001d11e'
>  >>> print(c)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "C:\Programs\Python33\lib\encodings\cp437.py", line 19, in enco=
de
>      return codecs.charmap_encode(input,self.errors,encoding_map)[0]
> UnicodeEncodeError: 'charmap' codec can't encode character '\U0001d11e'=

> in posit
> ion 0: character maps to <undefined>
>
> Yes, this is very annoying, especially in Win 7.

The above is in 3.3, in which '\U0001d11e' is actually translated to a=20
length 1 string. In 3.2-, that literal is translated (on 3.2- narrow=20
builds, as on Windows) to a length 2 string surrogate pair (in the BMP). =

On printing, the pair of surrogates got translated to a square box used=20
for all characters for which the font does not have a glyph.  =F0=9D=84=9E=
When cut=20
and pasted, it shows in this mail composer as a weird music sign with=20
peculiar behavior.
3 -s, 3 spaces, paste, 3 spaces, 3 -s, but it may disappear.
---   =F0=9D=84=9E   ---
So 3.3 is the first Windows version to get the UnicodeEncodeError on=20
printing.

--=20
Terry Jan Reedy