Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32980

Re: Right solution to unicode error?

References (4 earlier) <mailman.3436.1352383603.27098.python-list@python.org> <65910cea-f145-409c-a579-9f0cda499546@googlegroups.com> <CAHVvXxT9msy1HMwTtJBwGCXM_gKwtovOV5dNUQUYcugdoypWtw@mail.gmail.com> <mailman.3459.1352400535.27098.python-list@python.org> <a0073458-3b60-4c19-909d-c3d6dda7dccc@googlegroups.com>
From Ian Kelly <ian.g.kelly@gmail.com>
Date 2012-11-08 13:41 -0700
Subject Re: Right solution to unicode error?
Newsgroups comp.lang.python
Message-ID <mailman.3465.1352407330.27098.python-list@python.org> (permalink)

Show all headers | View raw

On Thu, Nov 8, 2012 at 12:54 PM,  <wxjmfauth@gmail.com> wrote:
> Font has nothing to do here.
> You are "simply" wrongly encoding your "unicode".
>
>>>> '\u2013'
> '–'
>>>> '\u2013'.encode('utf-8')
> b'\xe2\x80\x93'
>>>> '\u2013'.encode('utf-8').decode('cp1252')
> '–'

No, it seriously is the font.  This is what I get using the default
("Raster") font:

C:\>chcp 65001
Active code page: 65001

C:\>c:\python33\python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> '\u2013'
'–'
>>> import sys
>>> sys.stdout.buffer.write('\u2013\n'.encode('utf-8'))
–
4

I should note here that the characters copied and pasted do not
correspond to the glyphs actually displayed in my terminal window.  In
the terminal window I actually see:

ΓÇô

If I change the font to Lucida Console and run the *exact same code*,
I get this:

C:\>chcp 65001
Active code page: 65001

C:\>c:\python33\python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> '\u2013'
'–'

>>> import sys
>>> sys.stdout.buffer.write('\u2013\n'.encode('utf-8'))
–
4

Why is the font important?  I have no idea.  Blame Microsoft.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Right solution to unicode error? Anders <aschneiderman@asha.org> - 2012-11-07 14:17 -0800
  RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-07 23:07 +0000
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-07 23:27 +0000
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-07 17:51 -0600
  Re: Right solution to unicode error? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-11-07 23:53 +0000
    Re: Right solution to unicode error? Hans Mulder <hansmu@xs4all.nl> - 2012-11-08 12:40 +0100
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 00:44 +0000
  Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 03:01 -0800
  RE: Right solution to unicode error? Anders Schneiderman <ASchneiderman@asha.org> - 2012-11-08 09:00 -0500
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 14:06 +0000
    Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 07:05 -0800
      Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 18:32 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
      Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 11:48 -0700
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 13:41 -0700
            Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-09 02:06 -0800
          RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-08 20:54 +0000
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 14:07 -0700
          Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 21:37 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-08 21:30 -0600

csiph-web