Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Terry Reedy <tjreedy@udel.edu>
Newsgroups: comp.lang.python
Subject: Re: Pyhon 2.x or 3.x, which is faster?
Date: Wed, 9 Mar 2016 10:42:53 -0500
Lines: 71
Message-ID: <mailman.84.1457538207.15725.python-list@python.org>
References: <mailman.238.1457265255.20602.python-list@python.org> <nbjmv7$ad5$1@dont-email.me> <87d1r6iltx.fsf@elektro.pacujo.net> <nbjp1e$jhv$1@dont-email.me> <nbjrjm$m16$1@gioia.aioe.org> <nbjvas$h22$1@dont-email.me> <mailman.17.1457364684.10335.python-list@python.org> <nbkhei$dg6$1@dont-email.me> <mailman.43.1457377845.10335.python-list@python.org> <nbknir$avu$1@dont-email.me> <56de28a1$0$1604$c3e8da3$5496439d@news.astraweb.com> <nblae9$nl0$1@dont-email.me> <56de57b5$0$1590$c3e8da3$5496439d@news.astraweb.com> <nbml2q$l4n$1@dont-email.me> <56df6873$0$1588$c3e8da3$5496439d@news.astraweb.com> <nbnu1m$u1m$1@dont-email.me> <56df87f7$0$1620$c3e8da3$5496439d@news.astraweb.com> <nbpaaa$uic$1@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0
In-Reply-To: <nbpaaa$uic$1@dont-email.me>
Precedence: list
Xref: csiph.com comp.lang.python:104426

On 3/9/2016 9:03 AM, BartC wrote:

> I've just tried a UTF-8 file and getting some odd results. With a file
> containing [three euro symbols]:
>
> =E2=82=AC=E2=82=AC=E2=82=AC
>
> (including a 3-byte utf-8 marker at the start), and opened in text mode=
,
> Python 3 gives me this series of bytes (ie. the ord() of each character=
):
>
> 239
> 187
> 191
> 226
> 8218
> 172
> 226
> 8218
> 172
> 226
> 8218
> 172
>
> And prints the resulting string as: =C3=AF=C2=BB=C2=BF=C3=A2=E2=80=9A=C2=
=AC=C3=A2=E2=80=9A=C2=AC=C3=A2=E2=80=9A=C2=AC. Although this latter
> might depend on my console's code page setting.

It definitely does.

> Changing it to UTF-8 however (CHCP 65001 in Windows)

CP65001 is MS's ugly pretense of unicode compatibility.  It has been=20
known to be buggy for over a decade, though some people claim to have=20
gotten some use of it.

 > gives me this error when I run the  program again:
>
> ----------
> Fatal Python error: Py_Initialize: can't initialize sys standard stream=
s
> LookupError: unknown encoding: cp65001
>
> This application has requested the Runtime to terminate it in an unusua=
l
> way.
> Please contact the application's support team for more information.
> ----------

> So I think I'll skip Unicode handling to start off with! (I've already
> had plenty of fun and games with it in the past.)

At least on Windows, use IDLE for the BMP subset of unicode.  tk and=20
hence tkinter and IDLE can handle any char in the BMP subset.  I believe =

that which are actually displayed and which are shown as boxes depends=20
on the font.  On my US Win10 system:

IDLE with Lucida Console:
 >>> s =3D '=E2=82=AC=E2=82=AC=E2=82=AC'
 >>> s
'=E2=82=AC=E2=82=AC=E2=82=AC'

In the console interpreter: '???' is printed.


--=20
Terry Jan Reedy