Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #57138

Re: Looking for UNICODE to ASCII Conversioni Example Code

Show all headers | View raw

In article <mailman.1267.1382220612.18130.python-list@python.org>,
 Chris Angelico <rosuav@gmail.com> wrote:

> On Sun, Oct 20, 2013 at 3:49 AM, Roy Smith <roy@panix.com> wrote:
> > So, yesterday, I tracked down an uncaught exception stack in our logs to a 
> > user whose username included the unicode character 'SMILING FACE WITH 
> > SUNGLASSES' (U+1F60E).  It turns out, that's perfectly fine as a user name, 
> > except that in one obscure error code path, we try to str() it during some 
> > error processing.
> 
> How is that a problem? Surely you have to deal with non-ASCII
> characters all the time - how is that particular one a problem? I'm
> looking at its UTF-8 and UTF-16 representations and not seeing
> anything strange, unless it's the \x0e in UTF-16 - but, again, you
> must surely have had to deal with
> non-ASCII-encoded-whichever-way-you-do-it.
> 
> Or are you saying that that particular error code path did NOT handle
> non-ASCII characters?

Exactly.  The fundamental error was caught, and then we raised another 
UnicodeEncodeError generating the text of the error message to log!

> If so, that's a strong argument for moving to
> Python 3, to get full Unicode support in _all_ branches.

Well, yeah.  The problem is, my pip requirements file lists 76 modules 
(and installing all those results in 144 modules, including the cascaded 
dependencies).  Until most of those are P3 ready, we can't move.

Heck, I can't even really move off 2.6 because we use Amazon's EMR 
service, which is stuck on 2.6.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Looking for UNICODE to ASCII Conversioni Example Code caldwellinva@gmail.com - 2013-10-18 13:45 -0700
  Re: Looking for UNICODE to ASCII Conversioni Example Code Zero Piraeus <z@etiol.net> - 2013-10-18 19:02 -0300
  Re: Looking for UNICODE to ASCII Conversioni Example Code Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-10-19 09:19 +0000
    Re: Looking for UNICODE to ASCII Conversioni Example Code Zero Piraeus <z@etiol.net> - 2013-10-19 11:14 -0300
      Re: Looking for UNICODE to ASCII Conversioni Example Code Roy Smith <roy@panix.com> - 2013-10-19 11:10 -0400
        Re: Looking for UNICODE to ASCII Conversioni Example Code rusi <rustompmody@gmail.com> - 2013-10-19 08:26 -0700
      Re: Looking for UNICODE to ASCII Conversioni Example Code Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-10-19 16:16 +0000
        Re: Looking for UNICODE to ASCII Conversioni Example Code Roy Smith <roy@panix.com> - 2013-10-19 09:49 -0700
          Re: Looking for UNICODE to ASCII Conversioni Example Code Chris Angelico <rosuav@gmail.com> - 2013-10-20 09:10 +1100
            Re: Looking for UNICODE to ASCII Conversioni Example Code Roy Smith <roy@panix.com> - 2013-10-19 21:52 -0400
              Re: Looking for UNICODE to ASCII Conversioni Example Code Chris Angelico <rosuav@gmail.com> - 2013-10-20 13:09 +1100
                Re: Looking for UNICODE to ASCII Conversioni Example Code Roy Smith <roy@panix.com> - 2013-10-19 22:13 -0400
                Re: Looking for UNICODE to ASCII Conversioni Example Code Ben Finney <ben+python@benfinney.id.au> - 2013-10-20 13:26 +1100
                Re: Looking for UNICODE to ASCII Conversioni Example Code Chris Angelico <rosuav@gmail.com> - 2013-10-20 13:29 +1100
                Re: Looking for UNICODE to ASCII Conversioni Example Code Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-20 10:11 +0100
  Re: Looking for UNICODE to ASCII Conversioni Example Code Roy Smith <roy@panix.com> - 2013-10-19 08:28 -0400
  Re: Looking for UNICODE to ASCII Conversioni Example Code caldwellinva@gmail.com - 2013-10-19 05:50 -0700

csiph-web