Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32980

Re: Right solution to unicode error?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin1!goblin.stu.neva.ru!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <ian.g.kelly@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.003
X-Spam-Evidence '*H*': 0.99; '*S*': 0.00; 'win32': 0.03; 'sys': 0.05; 'correspond': 0.07; 'see:': 0.07; 'python': 0.09; 'sep': 0.09; 'subject:error': 0.11; 'encoding': 0.15; 'subject:unicode': 0.16; 'wrongly': 0.16; 'wrote:': 0.17; 'copied': 0.17; 'thu,': 0.17; '>>>': 0.18; 'bit': 0.21; 'import': 0.21; 'displayed': 0.22; "skip:' 40": 0.22; 'this:': 0.23; 'header:In-Reply-To:1': 0.25; 'message-id:@mail.gmail.com': 0.27; 'skip:( 20': 0.28; 'run': 0.28; '>>>>': 0.29; 'no,': 0.29; 'window': 0.30; 'code': 0.31; "skip:' 20": 0.32; 'to:addr:python-list': 0.33; 'received:google.com': 0.34; 'nov': 0.35; 'pm,': 0.35; 'subject:?': 0.35; 'received:209.85': 0.35; 'characters': 0.36; 'should': 0.36; 'why': 0.37; 'received:209': 0.37; 'subject:: ': 0.38; 'nothing': 0.38; 'to:addr:python.org': 0.39; 'header:Received:5': 0.40; 'your': 0.60; 'more': 0.63; 'here': 0.65; '\xe2\x80\x93': 0.75; 'to:name:python': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=DDLmiPEsc6JLFcECamZS2J/3FJMbBnySiQDvI+QkoOI=; b=jGPBh/8DlXnbKl09Pakkw4voiSzFQfyRiBadSd/Kp8LknEMbHddCJvh6FcmWoBzGcW oW3FKV24O0pyBXi4XZLRMd1hvOugejOsY8RTELmN89auQ+bdYFQqzsa5ZRJrKo6PfJL6 FmtJjMHr/QAKvlohyNYxMU1nRYVyeEEB7QimbHrBWhXR0XRbAL4FBMcpWHPJMFnFhpmp 152Ua5wufpJE7wnkv4gKjZd5AwVmqAj+rUxHhRPP8IW4IJc3FURS/+ronnvf8283tijl 6WCSvZmBkWnVyM698F9zukWIUZW3nXWFcnl24apCMnGywSJFHjRHvoXJhPQBD9e6Jttb Kf0g==
MIME-Version 1.0
In-Reply-To <a0073458-3b60-4c19-909d-c3d6dda7dccc@googlegroups.com>
References <09a3d20b-5871-47f4-9218-df119698e405@m4g2000yqf.googlegroups.com> <CAHVvXxT8SpK24HtsS8XTfs0CHTBofh3=deYw5xY1nq5ctATjrg@mail.gmail.com> <509AF3EF.8050108@gmail.com> <CAHVvXxSg0XWjZmA1pnEvBB_b2T9pS91YdjhF2duZHmBuLbXNAw@mail.gmail.com> <mailman.3436.1352383603.27098.python-list@python.org> <65910cea-f145-409c-a579-9f0cda499546@googlegroups.com> <CAHVvXxT9msy1HMwTtJBwGCXM_gKwtovOV5dNUQUYcugdoypWtw@mail.gmail.com> <mailman.3459.1352400535.27098.python-list@python.org> <a0073458-3b60-4c19-909d-c3d6dda7dccc@googlegroups.com>
From Ian Kelly <ian.g.kelly@gmail.com>
Date Thu, 8 Nov 2012 13:41:37 -0700
Subject Re: Right solution to unicode error?
To Python <python-list@python.org>
Content-Type text/plain; charset=UTF-8
Content-Transfer-Encoding quoted-printable
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.3465.1352407330.27098.python-list@python.org> (permalink)
Lines 53
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1352407330 news.xs4all.nl 6926 [2001:888:2000:d::a6]:44678
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:32980

Show key headers only | View raw

On Thu, Nov 8, 2012 at 12:54 PM,  <wxjmfauth@gmail.com> wrote:
> Font has nothing to do here.
> You are "simply" wrongly encoding your "unicode".
>
>>>> '\u2013'
> '–'
>>>> '\u2013'.encode('utf-8')
> b'\xe2\x80\x93'
>>>> '\u2013'.encode('utf-8').decode('cp1252')
> '–'

No, it seriously is the font.  This is what I get using the default
("Raster") font:

C:\>chcp 65001
Active code page: 65001

C:\>c:\python33\python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> '\u2013'
'–'
>>> import sys
>>> sys.stdout.buffer.write('\u2013\n'.encode('utf-8'))
–
4

I should note here that the characters copied and pasted do not
correspond to the glyphs actually displayed in my terminal window.  In
the terminal window I actually see:

ΓÇô

If I change the font to Lucida Console and run the *exact same code*,
I get this:

C:\>chcp 65001
Active code page: 65001

C:\>c:\python33\python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> '\u2013'
'–'

>>> import sys
>>> sys.stdout.buffer.write('\u2013\n'.encode('utf-8'))
–
4

Why is the font important?  I have no idea.  Blame Microsoft.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Right solution to unicode error? Anders <aschneiderman@asha.org> - 2012-11-07 14:17 -0800
  RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-07 23:07 +0000
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-07 23:27 +0000
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-07 17:51 -0600
  Re: Right solution to unicode error? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-11-07 23:53 +0000
    Re: Right solution to unicode error? Hans Mulder <hansmu@xs4all.nl> - 2012-11-08 12:40 +0100
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 00:44 +0000
  Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 03:01 -0800
  RE: Right solution to unicode error? Anders Schneiderman <ASchneiderman@asha.org> - 2012-11-08 09:00 -0500
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 14:06 +0000
    Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 07:05 -0800
      Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 18:32 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
      Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 11:48 -0700
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 13:41 -0700
            Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-09 02:06 -0800
          RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-08 20:54 +0000
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 14:07 -0700
          Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 21:37 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-08 21:30 -0600

csiph-web