Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #49336

Re: Devnagari Unicode Conversion Issues

Show key headers only | View raw

On 06/27/2013 11:39 AM, darpan6aya wrote:
> That worked out. I was trying to encode it the entire time.
> Now I realise how silly I am.
>
> Thanks MRAB. Once Again. :D
>

you're not silly, it's a complex question.  MRAB is good at guessing 
which part is messing you up.

However, when you're writing a real Python program with a real text 
editor, and when you're not using a newsgroup in between to mangle or 
unmangle things, you have a few things to match up to get it right.


The file is just a bunch of bytes.  Those bytes are being inserted in 
there by your editor, and interpreted by the compiler.  So if you have a 
non-ASCII character on your keyboard and you hit it, the editor will 
decode it (from Unicode to byte(s)) and put it in the file.  If you tell 
the editor to use utf-8, then you also want to tell the compiler to 
decode it using utf-8.

The most polite way to do that looks something like:
# -*- coding: <encoding-name> -*-
# -*- coding: <utf-8> -*-

http://docs.python.org/release/2.7.5/reference/lexical_analysis.html#encoding-declarations

Once you've got that straight, you don't need to explicitly decode byte 
strings.  You can just use
   u"This is my string"

with whatever characters you need.  As long as the declarations match, 
this should "just work."  If the data comes from a byte string other 
than a literal string, you might need the more verbose form.

Your original message was sent in Western (ISO 8859-1), and MRAB's 
response was in utf-8, and my mail program decoded the string the same 
way.  However, I don't know anything about Devnagari, so I can't say if 
it looked reasonable here.


-- 
DaveA

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Devnagari Unicode Conversion Issues darpan6aya <akshay.ksth@gmail.com> - 2013-06-27 08:05 -0700
  Re: Devnagari Unicode Conversion Issues MRAB <python@mrabarnett.plus.com> - 2013-06-27 16:28 +0100
  Re: Devnagari Unicode Conversion Issues darpan6aya <akshay.ksth@gmail.com> - 2013-06-27 08:39 -0700
    Re: Devnagari Unicode Conversion Issues Dave Angel <davea@davea.name> - 2013-06-27 12:28 -0400

csiph-web