Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <mailing@franzoni.eu>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'handler': 0.04; '-*-': 0.07; 'practice,': 0.07; 'skip:/ 10': 0.07; 'utf-8': 0.07; 'python': 0.09; 'alan': 0.09; 'codecs': 0.09; 'coding:': 0.09; 'encode': 0.09; 'encoding.': 0.09; 'received:151': 0.09; 'subject:()': 0.09; 'def': 0.10; 'encoding': 0.15; 'codec': 0.16; 'missing?': 0.16; 'ordinal': 0.16; 'url:py': 0.16; 'string': 0.17; 'instance,': 0.17; 'unicode': 0.17; 'trying': 0.21; 'import': 0.21; 'doc': 0.22; 'exceptions': 0.22; 'errors': 0.23; 'seems': 0.23; 'header:User-Agent:1': 0.26; 'charset:iso-8859-15': 0.26; 'skip:" 20': 0.26; '(most': 0.27; 'handling': 0.27; 'behaviour': 0.29; 'cat': 0.29; 'piece': 0.29; 'subject:skip:u 10': 0.29; 'character': 0.29; '8bit%:5': 0.29; 'probably': 0.29; "i'm": 0.29; 'error': 0.30; 'expect': 0.31; 'url:python': 0.32; 'file': 0.32; 'print': 0.32; 'says': 0.33; 'traceback': 0.33; 'to:addr:python- list': 0.33; "can't": 0.34; 'thanks': 0.34; 'there': 0.35; 'skip:u 20': 0.36; 'but': 0.36; 'url:org': 0.36; 'url:library': 0.36; 'why': 0.37; 'some': 0.38; 'url:docs': 0.38; 'to:addr:python.org': 0.39; 'hello,': 0.39; 'called': 0.39; 'skip:" 10': 0.40; 'think': 0.40; 'skip:u 10': 0.60; 'strange': 0.62; 'information': 0.63; 'more': 0.63; 'making': 0.64; 'url:u': 0.66; 'contact': 0.68; 'discovered': 0.83; '2.7.1': 0.84; 'links:': 0.84; 'preventing': 0.91
Date: Fri, 27 Jul 2012 01:03:01 +0200
From: Alan Franzoni <mailing@franzoni.eu>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0
MIME-Version: 1.0
To: python-list@python.org
Subject: codecs.register_error for "strict", unicode.encode() and str.decode()
Content-Type: text/plain; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2641.1343344059.4697.python-list@python.org>
Lines: 70
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1343344059 news.xs4all.nl 6974 [2001:888:2000:d::a6]:59223
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:26125

Hello,
I think I'm missing some piece here.

I'm trying to register a default error handler for handling exceptions
for preventing encoding/decoding errors (I know how this works and that
making this global is probably not a good practice, but I found this
strange behaviour while writing a proof of concept of how to let Python
work in a more forgiving way).

What I discovered is that register_error() for "strict" seems to work in
the way I expect for string decoding, not for unicode encoding.

That's what happens on Mac, Python 2.7.1 from Apple:

melquiades:tmp alan$ cat minimal_test_encode.py
# -*- coding: utf-8 -*-

import codecs

def handle_encode(e):
    return ("ASD", e.end)

codecs.register_error("strict", handle_encode)

print u"à".encode("ascii")

melquiades:tmp alan$ python minimal_test_encode.py
Traceback (most recent call last):
  File "minimal_test_encode.py", line 10, in <module>
    u"à".encode("ascii")
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in
position 0: ordinal not in range(128)


OTOH this works properly:

melquiades:tmp alan$ cat minimal_test_decode.py
# -*- coding: utf-8 -*-

import codecs

def handle_decode(e):
    return (u"ASD", e.end)

codecs.register_error("strict", handle_decode)

print "à".decode("ascii")

melquiades:tmp alan$ python minimal_test_decode.py
ASDASD


What piece am I missing? The doc at 
http://docs.python.org/library/codecs.html says " For
encoding /error_handler/ will be called with a UnicodeEncodeError
<http://docs.python.org/library/exceptions.html#exceptions.UnicodeEncodeError> instance,
which contains information about the location of the error.", is there
any reason why the standard "strict" handler cannot be replaced?


Thanks for any clue.

File links:
https://dl.dropbox.com/u/249926/minimal_test_decode.py
https://dl.dropbox.com/u/249926/minimal_test_encode.py

-- 
Alan Franzoni
contact me at public@[mysurname].eu