Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #46292

Re: Encodign issue in Python 3.3.1 (once again)

Date 2013-05-28 08:34 -0600
From Michael Torrie <torriem@gmail.com>
Subject Re: Encodign issue in Python 3.3.1 (once again)
References <7823093c-2c07-4fa0-ae97-960c62f8ff9d@googlegroups.com> <69f5q8hkra42rvpfmbng4f0airgikqf5js@4ax.com> <9560d43f-8100-4914-9a71-c30ab479ec35@googlegroups.com> <mailman.2251.1369665785.3114.python-list@python.org> <f4c1b04a-7a50-418a-8be9-8a24609e261d@googlegroups.com>
Newsgroups comp.lang.python
Message-ID <mailman.2299.1369751680.3114.python-list@python.org> (permalink)

Show all headers | View raw


On 05/27/2013 02:17 PM, Νίκος Γκρ33κ wrote:
> I have checked the database through phpMyAdmin and it is indeed UTF-8.
> 
> I have no idea why python 3.3.1 chooses to work with latin-iso only....

It's not python that is doing this here...

If you look at the source code to pymysql, I'm sure you will identify
the problem.  In fact I'll even walk you through things.

The traceback you posted tells you what's going on and why, with
superfluous details removed for clarity:

File "/opt/python3/lib/python3.3/site-packages/pymysql/cursors.py", line
108, in execute:
     query = query.encode(charset)
UnicodeEncodeError: 'latin-1' codec can't encode characters in position
46-52: ordinal not in range(256)

So there we have it.  pymysql is actually explicitly calling .encode()
on a string and is using whatever character set is specified by the
local variable charset.  If you look in cursors.py at that line, and
then a few lines above, you will find that charset is assigned to
conn.charset.  This means that the charset is actually defined in the
connection object.  So go look at connections.py.  Sure enough, it shows
that charset is defined by default as "latin-1."  That's no good for
you.  So take a look at the __init__ method in connections.py.  In there
you should find the necessary keyword argument you need to use when
creating the mysql connection to make sure the charset is utf-8.

Back to comp.lang.python | Previous | NextPrevious in thread | Next in thread | Find similar | Unroll thread


Thread

Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-26 09:36 -0700
  Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-26 11:20 -0700
  Re: Encodign  issue in Python 3.3.1 (once again) Terry Jan Reedy <tjreedy@udel.edu> - 2013-05-26 16:53 -0400
    Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-26 14:26 -0700
      Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-27 07:44 +1000
      Re: Encodign  issue in Python 3.3.1 (once again) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-05-26 23:21 +0100
      RE: Encodign  issue in Python 3.3.1 (once again) Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-05-27 01:31 +0300
      Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-27 08:32 +1000
      Re: Encodign issue in Python 3.3.1 (once again) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-05-26 23:51 +0100
  Re: Encodign  issue in Python 3.3.1 (once again) Tim Roberts <timr@probo.com> - 2013-05-26 19:01 -0700
    Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-26 22:06 -0700
      Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-27 07:16 -0700
        Re: Encodign  issue in Python 3.3.1 (once again) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-05-27 15:45 +0100
          Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-27 08:37 -0700
            Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-28 01:52 +1000
              Re: Encodign issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-27 09:56 -0700
                Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-28 08:18 +1000
                Re: Encodign issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-27 22:55 -0700
                Re: Encodign issue in Python 3.3.1 (once again) rusi <rustompmody@gmail.com> - 2013-05-28 04:56 -0700
      Re: Encodign  issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-27 08:42 -0600
        Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-27 13:17 -0700
          Re: Encodign  issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-28 08:34 -0600
            Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-05-28 09:00 -0700
              Re: Encodign  issue in Python 3.3.1 (once again) MRAB <python@mrabarnett.plus.com> - 2013-05-28 17:26 +0100
              Re: Encodign  issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-28 10:40 -0600
                Re: Encodign  issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <support@superhost.gr> - 2013-05-28 09:45 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-28 11:17 -0600
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-28 11:08 -0700
                Re: Encodign issue in Python 3.3.1 (once again) Joel Goldstick <joel.goldstick@gmail.com> - 2013-05-28 14:14 -0400
                Re: Encodign  issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-28 12:19 -0600
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-28 12:03 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-28 12:48 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-29 03:30 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-30 00:18 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-30 01:16 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-30 03:29 -0600
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-30 03:53 -0700
                Re: Encodign  issue in Python 3.3.1 (once again) nagia.retsina@gmail.com - 2013-05-30 03:56 -0700
                Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-30 21:33 +1000
                Re: Encodign issue in Python 3.3.1 (once again) Νίκος Γκρ33κ <support@superhost.gr> - 2013-05-30 04:47 -0700
                Re: Encodign issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-30 08:35 -0600
                Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-31 00:40 +1000
                Re: Encodign issue in Python 3.3.1 (once again) Michael Torrie <torriem@gmail.com> - 2013-05-30 11:01 -0600
                Re: Encodign issue in Python 3.3.1 (once again) Chris Angelico <rosuav@gmail.com> - 2013-05-31 03:28 +1000
                Re: Encodign issue in Python 3.3.1 (once again) Νικόλαος Κούρας <nikos.gr33k@gmail.com> - 2013-05-31 08:13 -0700

csiph-web