Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <python-python-list@m.gmane.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'exception': 0.03; 'exit': 0.07; 'rewrite': 0.07; 'try:': 0.07; '(without': 0.09; 'exception,': 0.09; 'exits': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'vast': 0.09; 'files.': 0.13; 'bad:': 0.16; 'did.': 0.16; 'encoded.': 0.16; 'from:addr:behnel.de': 0.16; 'from:addr:stefan_ml': 0.16; 'from:name:stefan behnel': 0.16; 'hint': 0.16; 'message-id:@dough.gmane.org': 0.16; 'occurred.': 0.16; 'originate': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'semantically': 0.16; 'suddenly,': 0.16; 'traceback.': 0.16; 'stefan': 0.17; '>>>': 0.18; 'code.': 0.20; 'file.': 0.20; 'parse': 0.22; 'tells': 0.22; '(this': 0.24; 'header:In-Reply-To:1': 0.25; 'header:User- Agent:1': 0.26; "doesn't": 0.28; 'header:X-Complaints-To:1': 0.28; 'implicitly': 0.29; 'character': 0.29; 'error': 0.30; 'code': 0.31; 'problem': 0.33; 'to:addr:python-list': 0.33; 'skip:d 20': 0.34; 'text': 0.34; 'open': 0.35; 'there': 0.35; 'received:org': 0.36; 'except': 0.36; 'skip:u 20': 0.36; 'but': 0.36; 'should': 0.36; 'bad': 0.37; 'correctly': 0.37; 'uses': 0.37; 'why': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'sure': 0.38; 'to:addr:python.org': 0.39; 'where': 0.40; 'header:Received:5': 0.40; 'think': 0.40; 'your': 0.60; 'skip:u 10': 0.60; 'most': 0.61; 'places': 0.61; 'is.': 0.62; 'repeat': 0.62; 'different': 0.63; 'here': 0.65; 'today': 0.67; 'case?': 0.84
X-Injected-Via-Gmane: http://gmane.org/
To: python-list@python.org
From: Stefan Behnel <stefan_ml@behnel.de>
Subject: Re: catch UnicodeDecodeError
Date: Thu, 26 Jul 2012 13:15:37 +0200
References: <04f7ff8d-9881-4a04-ab2e-b5573b5f3cd1@googlegroups.com> <mailman.2570.1343216119.4697.python-list@python.org> <b8723e64-12fa-4e53-8914-8f2b8e9c0f1d@googlegroups.com> <mailman.2581.1343242258.4697.python-list@python.org> <38f5cdaf-c021-4ccd-8fcb-c68b21d3aeb2@w24g2000vby.googlegroups.com> <mailman.2593.1343291337.4697.python-list@python.org> <17bf754d-b1e9-4bb7-bf42-190325ee969a@q29g2000vby.googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Gmane-NNTP-Posting-Host: fw-snc-frn5-de01.fw.telefonica.de
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120714 Thunderbird/14.0
In-Reply-To: <17bf754d-b1e9-4bb7-bf42-190325ee969a@q29g2000vby.googlegroups.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2599.1343301351.4697.python-list@python.org>
Lines: 42
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1343301351 news.xs4all.nl 6888 [2001:888:2000:d::a6]:40699
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:26081

Jaroslav Dobrek, 26.07.2012 12:51:
>>> try:
>>>     for line in f: # here text is decoded implicitly
>>>        do_something()
>>> except UnicodeDecodeError():
>>>     do_something_different()
> 
> the code above (without the brackets) is semantically bad: The
> exception is not caught.

Sure it is. Just to repeat myself: if the above doesn't catch the
exception, then the exception did not originate from the place where you
think it did. Again: look at the traceback.


>>> The problem is that vast majority of the thousands of files that I
>>> process are correctly encoded. But then, suddenly, there is a bad
>>> character in a new file. (This is so because most files today are
>>> generated by people who don't know that there is such a thing as
>>> encodings.) And then I need to rewrite my very complex program just
>>> because of one single character in one single file.
>>
>> Why would that be the case? The places to change should be very local in
>> your code.
> 
> This is the case in a program that has many different functions which
> open and parse different
> types of files. When I read and parse a directory with such different
> types of files, a program that
> uses
> 
> for line in f:
> 
> will not exit with any hint as to where the error occurred. I just
> exits with a UnicodeDecodeError.

... that tells you the exact code line where the error occurred. No need to
look around.

Stefan