Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32981

RE: Right solution to unicode error?

Show key headers only | View raw

wxjmfauth@gmail.com wrote:
> 
> Le jeudi 8 novembre 2012 19:49:24 UTC+1, Ian a écrit :
> > On Thu, Nov 8, 2012 at 11:32 AM, Oscar Benjamin
> >
> > <oscar.j.benjamin@gmail.com> wrote:
> >
> > > If I want the other characters to work I need to change the code page:
> > >
> > > O:\>chcp 65001
> > > Active code page: 65001
> > >
> > > O:\>Q:\tools\Python33\python -c "import sys;
> > > sys.stdout.buffer.write('\u03b1\n'.encode('utf-8'))"
> > > α
> > >
> > > O:\>Q:\tools\Python33\python -c "import sys;
> > > sys.stdout.buffer.write('\u03b1\n'.encode(sys.stdout.en
> > > coding))"
> > > α
> >
> > I find that I also need to change the font.  With the default font,
> >
> > printing '\u2013' gives me:
> > –
> >
> > The only alternative font option I have in Windows XP is Lucida
> > Console, which at least works correctly, although it seems to be
> > lacking a lot of glyphs.
> 
> --------
> 
> Font has nothing to do here.
> You are "simply" wrongly encoding your "unicode".
> 


Why would font not matter? Unicode is the abstract definition 
of all characters right? From that we map the abstract 
character to a code page/set, which gives real values for an
abstract character. From that code page we then visually display 
the "real value" based on the font. If that font does
not have a glyph for a specific character page (or a different
glyph) then that is a problem and not related encoding. 

Unicode->code page->font


> >>> '\u2013'
> '–'
> >>> '\u2013'.encode('utf-8')
> b'\xe2\x80\x93'
> >>> '\u2013'.encode('utf-8').decode('cp1252')
> '–'
> 

This is a mismatched translation between code pages; not
font related but is instead one abstraction "level" up. 


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Right solution to unicode error? Anders <aschneiderman@asha.org> - 2012-11-07 14:17 -0800
  RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-07 23:07 +0000
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-07 23:27 +0000
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-07 17:51 -0600
  Re: Right solution to unicode error? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-11-07 23:53 +0000
    Re: Right solution to unicode error? Hans Mulder <hansmu@xs4all.nl> - 2012-11-08 12:40 +0100
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 00:44 +0000
  Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 03:01 -0800
  RE: Right solution to unicode error? Anders Schneiderman <ASchneiderman@asha.org> - 2012-11-08 09:00 -0500
  Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 14:06 +0000
    Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 07:05 -0800
      Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 18:32 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:30 -0800
      Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 11:48 -0700
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 13:41 -0700
            Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-09 02:06 -0800
          RE: Right solution to unicode error? "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-11-08 20:54 +0000
          Re: Right solution to unicode error? Ian Kelly <ian.g.kelly@gmail.com> - 2012-11-08 14:07 -0700
          Re: Right solution to unicode error? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-11-08 21:37 +0000
        Re: Right solution to unicode error? wxjmfauth@gmail.com - 2012-11-08 11:54 -0800
  Re: Right solution to unicode error? Andrew Berg <bahamutzero8825@gmail.com> - 2012-11-08 21:30 -0600

csiph-web