Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32983

Re: Right solution to unicode error?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <oscar.j.benjamin@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.001
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'remaining': 0.07; 'responding': 0.07; 'utf-8': 0.07; 'python': 0.09; 'encoding.': 0.09; 'inserted': 0.09; 'cc:addr:python-list': 0.10; 'subject:error': 0.11; 'encoding': 0.15; '"import': 0.16; '11:32': 0.16; 'benjamin': 0.16; 'correctly,': 0.16; 'cp1252': 0.16; 'interpreting': 0.16; 'subject:unicode': 0.16; 'wrongly': 0.16; '\xe9crit': 0.16; 'wrote:': 0.17; 'bytes': 0.17; 'thu,': 0.17; 'windows': 0.19; 'displayed': 0.22; "skip:' 40": 0.22; 'cc:2**0': 0.23; 'mention': 0.23; 'seems': 0.23; 'cc:no real name:2**0': 0.24; 'least': 0.25; 'cc:addr:python.org': 0.25; 'header:In-Reply- To:1': 0.25; 'ago': 0.27; 'am,': 0.27; 'forgot': 0.27; 'message- id:@mail.gmail.com': 0.27; 'lines': 0.28; '>>>>': 0.29; 'accidentally': 0.29; 'code': 0.31; "skip:' 20": 0.32; 'received:google.com': 0.34; 'text': 0.34; 'identified': 0.35; 'nov': 0.35; 'subject:?': 0.35; 'received:209.85': 0.35; 'characters': 0.36; 'correctly': 0.37; 'option': 0.37; 'received:209': 0.37; 'subject:: ': 0.38; 'skip:o 20': 0.38; 'some': 0.38; 'nothing': 0.38; 'gives': 0.39; 'header:Received:5': 0.40; 'your': 0.60; 'charset:windows-1252': 0.65; '\xe2\x80\x93': 0.75; 'confusing': 0.84; 'console,': 0.84; 'novembre': 0.84; 'oscar': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ns8akyQhnnrMfHT15laz0IQ3Rkc3oc9u8TV1TNwremI=; b=0TR2hdLLAJC8DiFasNeC5oMwMe96VgWwo1jCWXy4xPlgAmJk3goJHjyebQr4j6+FaS aMftj7WzDhRlM/+5vVd1oRv6OFQqdGh376PaZJFnX4x8ZrEY3Olccs7IhW6pA6X0Z+zw pLHhsRrrjMbub2jUNiy19bGjN07PbRJJPFsxSeem+xf+BqE5mRGPSXClNdOZzwYSQJLF bEx4HwBKPmvr6dpFz08r48ryOPc8H/f1G5nNa/DidKCCkOMM2HursviRg8ZtSAYpD52F rfEOlhAR0p3aTsibvW3pCe3cfdFrl+BelExW2V4sBEA+/xbL9lrWlSCReAO1/EY7imQ3 RQmQ==
MIME-Version 1.0
In-Reply-To <a0073458-3b60-4c19-909d-c3d6dda7dccc@googlegroups.com>
References <09a3d20b-5871-47f4-9218-df119698e405@m4g2000yqf.googlegroups.com> <CAHVvXxT8SpK24HtsS8XTfs0CHTBofh3=deYw5xY1nq5ctATjrg@mail.gmail.com> <509AF3EF.8050108@gmail.com> <CAHVvXxSg0XWjZmA1pnEvBB_b2T9pS91YdjhF2duZHmBuLbXNAw@mail.gmail.com> <mailman.3436.1352383603.27098.python-list@python.org> <65910cea-f145-409c-a579-9f0cda499546@googlegroups.com> <CAHVvXxT9msy1HMwTtJBwGCXM_gKwtovOV5dNUQUYcugdoypWtw@mail.gmail.com> <mailman.3459.1352400535.27098.python-list@python.org> <a0073458-3b60-4c19-909d-c3d6dda7dccc@googlegroups.com>
Date Thu, 8 Nov 2012 21:37:48 +0000
Subject Re: Right solution to unicode error?
From Oscar Benjamin <oscar.j.benjamin@gmail.com>
To wxjmfauth@gmail.com
Content-Type text/plain; charset=windows-1252
Content-Transfer-Encoding quoted-printable
Cc python-list@python.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.3468.1352410669.27098.python-list@python.org> (permalink)
Lines 55
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1352410670 news.xs4all.nl 6842 [2001:888:2000:d::a6]:60705
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:32983

Show key headers only | View raw

On 8 November 2012 19:54,  <wxjmfauth@gmail.com> wrote:
> Le jeudi 8 novembre 2012 19:49:24 UTC+1, Ian a écrit :
>> On Thu, Nov 8, 2012 at 11:32 AM, Oscar Benjamin
>>
>> <oscar.j.benjamin@gmail.com> wrote:
>>
>> > If I want the other characters to work I need to change the code page:
>>
>> >
>>
>> > O:\>chcp 65001
>>
>> > Active code page: 65001
>>
>> >
>>
>> > O:\>Q:\tools\Python33\python -c "import sys;
>>
>> I find that I also need to change the font.  With the default font,
>>
>> printing '\u2013' gives me:
>>
>> –
>>
>>
>>
>> The only alternative font option I have in Windows XP is Lucida
>>
>> Console, which at least works correctly, although it seems to be
>>
>> lacking a lot of glyphs.
>
> Font has nothing to do here.
> You are "simply" wrongly encoding your "unicode".
>
>>>> '\u2013'
> '–'
>>>> '\u2013'.encode('utf-8')
> b'\xe2\x80\x93'
>>>> '\u2013'.encode('utf-8').decode('cp1252')
> '–'

You have correctly identified that the displayed characters are the
result of accidentally interpreting utf-8 bytes as if they were cp1252
or similar. However, it is not Ian or Python that is confusing the
encoding. It is cmd.exe that is confusing the encoding in a
font-dependent way. I also had to change the font as Ian describes
though I did it some time ago and forgot to mention it here.

jmf, can you please trim the text you quote removing the parts you are
not responding to and then any remaining blank lines that were
inserted by your reader/editor?


Oscar

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Right solution to unicode error? Anders <aschneiderman@asha.org> - 2012-11-07 14:17 -0800
  RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-07 23:07 +0000
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-07 23:27 +0000
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-07 17:51 -0600
  Re: Right solution to unicode error? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-11-07 23:53 +0000
    Re: Right solution to unicode error? Hans Mulder <hansmu@xs4all.nl> - 2012-11-08 12:40 +0100
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 00:44 +0000
  Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 03:01 -0800
  RE: Right solution to unicode error? Anders Schneiderman <ASchneiderman@asha.org> - 2012-11-08 09:00 -0500
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 14:06 +0000
    Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 07:05 -0800
      Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 18:32 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
      Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 11:48 -0700
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 13:41 -0700
            Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-09 02:06 -0800
          RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-08 20:54 +0000
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 14:07 -0700
          Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 21:37 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-08 21:30 -0600

csiph-web