Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #6746

Re: sqlalchemy and Unicode strings: errormessage

Show all headers | View raw

On Wed, Jun 1, 2011 at 2:31 AM, Prasad, Ramit <ramit.prasad@jpmchase.com> wrote:
>>line = unicode(line.strip(),'utf8')
>>and now i get really utf8-strings. It does work but i dont know why it works. For me it looks like i change an utf8-string to an utf8-string.
>
>
> I would like to point out that UTF-8 is not exactly "Unicode". From what I understand, Unicode is a standard while UTF-8 is like an implementation of that standard (called an encoding). Being able to convert to Unicode (the standard) should mean you are then able to convert to any encoding that supports the Unicode characters used.

Unicode defines characters; UTF-8 is one way (of many) to represent
those characters in bytes. UTF-16 and UTF-32 are other ways of
representing those characters in bytes, and internally, Python
probably uses one of them - but there is no guarantee, and you should
never need to know. Unicode strings can be stored in memory and
manipulated in various ways, but they're a high level construct on par
with lists and dictionaries - they can't be stored on disk or
transmitted to another computer without using an encoding system.

UTF-8 is an efficient way to translate Unicode text consisting
primarily of low codepoint characters into bytes. It's not so much an
implementation of Unicode as a means of converting a mythical concept
of "Unicode characters" into a concrete stream of bytes.

Hope that clarifies things a little!

Chris Angelico

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

sqlalchemy and Unicode strings: errormessage Wolfgang Meiners <WolfgangMeiners01@web.de> - 2011-05-30 23:40 +0200
  Re: sqlalchemy and Unicode strings: errormessage Chris Withers <chris@simplistix.co.uk> - 2011-05-31 10:55 +0100
    Re: sqlalchemy and Unicode strings: errormessage Wolfgang Meiners <WolfgangMeiners01@web.de> - 2011-05-31 17:47 +0200
  Re: sqlalchemy and Unicode strings: errormessage Daniel Kluev <dan.kluev@gmail.com> - 2011-05-31 22:32 +1100
    Re: sqlalchemy and Unicode strings: errormessage Wolfgang Meiners <WolfgangMeiners01@web.de> - 2011-05-31 17:45 +0200
      Re: sqlalchemy and Unicode strings: errormessage Wolfgang Meiners <WolfgangMeiners01@web.de> - 2011-05-31 18:10 +0200
      Re: sqlalchemy and Unicode strings: errormessage Benjamin Kaplan <benjamin.kaplan@case.edu> - 2011-05-31 09:42 -0700
      RE: sqlalchemy and Unicode strings: errormessage "Prasad, Ramit" <ramit.prasad@jpmchase.com> - 2011-05-31 12:31 -0400
      Re: sqlalchemy and Unicode strings: errormessage Chris Angelico <rosuav@gmail.com> - 2011-06-01 03:19 +1000
      Thanks for all responses Wolfgang Meiners <WolfgangMeiners01@web.de> - 2011-05-31 21:52 +0200
        Re: Thanks for all responses Chris Angelico <rosuav@gmail.com> - 2011-06-01 07:56 +1000
          Re: Thanks for all responses Wolfgang Meiners <WolfgangMeiners01@web.de> - 2011-06-01 19:29 +0200
            Re: Thanks for all responses Chris Angelico <rosuav@gmail.com> - 2011-06-02 04:38 +1000

csiph-web