Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'encoding': 0.05; '"""': 0.07; 'sys': 0.07; 'utf-8': 0.07; 'ascii': 0.09; 'encode': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:into': 0.09; 'subject:string': 0.09; 'subject:How': 0.10; 'python': 0.11; 'charset': 0.16; 'codec': 0.16; 'lie,': 0.16; 'ordinal': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'used:': 0.16; 'using,': 0.16; 'wrote:': 0.18; 'module': 0.19; '>>>': 0.22; 'import': 0.22; 'print': 0.22; 'header:User- Agent:1': 0.23; 'skip:l 30': 0.24; 'header:X-Complaints-To:1': 0.27; 'function': 0.29; 'character': 0.29; "doesn't": 0.30; 'characters': 0.30; 'skip:g 30': 0.30; 'subject:list': 0.30; "skip:' 10": 0.31; '"",': 0.31; "d'aprano": 0.31; 'enforce': 0.31; 'steven': 0.31; 'file': 0.32; '(most': 0.33; "can't": 0.35; 'skip:s 30': 0.35; 'but': 0.35; 'set.': 0.36; 'subject:?': 0.36; 'to:addr:python-list': 0.38; 'recent': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'according': 0.40; 'skip:u 10': 0.60; 'conversion': 0.61; 'more': 0.64; 'mar': 0.68; '8bit%:43': 0.74; 'manner.': 0.74; '*really*': 0.84; '2014,': 0.84; 'technically': 0.84; 'treats': 0.84; 'refuse': 0.93
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: Peter Otten <__peter__@web.de>
Subject: Re: How to turn a string into a list of integers?
Date: Sat, 06 Sep 2014 10:22:30 +0200
Organization: None
References: <h2ejdb-mdk.ln1@chris.zbmc.eu> <mailman.13738.1409748804.18130.python-list@python.org> <1amjdb-p3n.ln1@chris.zbmc.eu> <mailman.13776.1409864831.18130.python-list@python.org> <1k9odb-1qs.ln1@chris.zbmc.eu> <mailman.13801.1409939785.18130.python-list@python.org> <540aa002$0$29968$c3e8da3$5496439d@news.astraweb.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8Bit
X-Gmane-NNTP-Posting-Host: p57bd8c29.dip0.t-ipconnect.de
User-Agent: KNode/4.13.3
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.13826.1409991776.18130.python-list@python.org>
Lines: 69
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1409991776 news.xs4all.nl 2861 [2001:888:2000:d::a6]:39333
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:77644

Steven D'Aprano wrote:

>>>>> import sys
>>>>> sys.getdefaultencoding()
>> 'ascii'
> 
> That's technically known as a "lie", since if it were *really* ASCII it
> would refuse to deal with characters with the high-bit set. But it
> doesn't, it treats them in an unpredictable and implementation-dependent
> manner.

It's not a lie, it just doesn't control the unicode-to-bytes conversion when 
printing:

$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getdefaultencoding()
'ascii'
>>> print u"äöü"
äöü
>>> str(u"äöü")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: 
ordinal not in range(128)
>>> reload(sys)
<module 'sys' (built-in)>
>>> sys.setdefaultencoding("latin1")
>>> print u"äöü"
äöü
>>> str(u"äöü")
'\xe4\xf6\xfc'
>>> sys.setdefaultencoding("utf-8")
>>> print u"äöü"
äöü
>>> str(u"äöü")
'\xc3\xa4\xc3\xb6\xc3\xbc'

You can enforce ascii-only printing:

$ LANG=C python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print unichr(228)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 
0: ordinal not in range(128)

To find out the encoding that is used:

$ python -c 'import locale; print locale.getpreferredencoding()'
UTF-8
$ LANG=C python -c 'import locale; print locale.getpreferredencoding()'
ANSI_X3.4-1968

"""
Help on function getpreferredencoding in module locale:

getpreferredencoding(do_setlocale=True)
    Return the charset that the user is likely using,
    according to the system configuration.
"""