Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #26040

Re: catch UnicodeDecodeError

Path csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!npeer01.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
From jaroslav.dobrek@gmail.com
Newsgroups comp.lang.python
Subject Re: catch UnicodeDecodeError
Date Wed, 25 Jul 2012 05:09:40 -0700 (PDT)
Organization http://groups.google.com
Lines 35
Message-ID <b8723e64-12fa-4e53-8914-8f2b8e9c0f1d@googlegroups.com> (permalink)
References <04f7ff8d-9881-4a04-ab2e-b5573b5f3cd1@googlegroups.com> <mailman.2570.1343216119.4697.python-list@python.org>
NNTP-Posting-Host 2001:4ca0:4f01:1:230:5ff:fed2:39da
Mime-Version 1.0
Content-Type text/plain; charset=ISO-8859-1
Content-Transfer-Encoding quoted-printable
X-Trace posting.google.com 1343218302 23709 127.0.0.1 (25 Jul 2012 12:11:42 GMT)
X-Complaints-To groups-abuse@google.com
NNTP-Posting-Date Wed, 25 Jul 2012 12:11:42 +0000 (UTC)
Cc jaroslav.dobrek@gmail.com, python-list@python.org
In-Reply-To <mailman.2570.1343216119.4697.python-list@python.org>
Complaints-To groups-abuse@google.com
Injection-Info glegroupsg2000goo.googlegroups.com; posting-host=2001:4ca0:4f01:1:230:5ff:fed2:39da; posting-account=74hy7woAAACQhc1ciracbLCiA9j-fl9e
User-Agent G2/1.0
X-Received-Bytes 2492
Xref csiph.com comp.lang.python:26040

Show key headers only | View raw

On Wednesday, July 25, 2012 1:35:09 PM UTC+2, Philipp Hagemeister wrote:
> Hi Jaroslav,
> 
> you can catch a UnicodeDecodeError just like any other exception. Can
> you provide a full example program that shows your problem?
> 
> This works fine on my system:
> 
> 
> import sys
> open(&#39;tmp&#39;, &#39;wb&#39;).write(b&#39;\xff\xff&#39;)
> try:
>     buf = open(&#39;tmp&#39;, &#39;rb&#39;).read()
>     buf.decode(&#39;utf-8&#39;)
> except UnicodeDecodeError as ude:
>     sys.exit(&quot;Found a bad char in file &quot; + &quot;tmp&quot;)
> 

Thank you. I got it. What I need to do is explicitly decode text.

But I think trial and error with moving files around will in most cases be faster. Usually, such a problem occurs with some (usually complex) program that I wrote quite a long time ago. I don't like editing old and complex programs that work under all normal circumstances.

What I am missing (especially for Python3) is something like:

try:
    for line in sys.stdin:
except UnicodeDecodeError:
    sys.exit("Encoding problem in line " + str(line_number))

I got the point that there is no such thing as encoding-independent lines. But if no line ending can be found, then the file simply has one single line.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

catch UnicodeDecodeError jaroslav.dobrek@gmail.com - 2012-07-25 04:05 -0700
  Re: catch UnicodeDecodeError Andrew Berg <bahamutzero8825@gmail.com> - 2012-07-25 06:34 -0500
  Re: catch UnicodeDecodeError Philipp Hagemeister <phihag@phihag.de> - 2012-07-25 13:35 +0200
    Re: catch UnicodeDecodeError jaroslav.dobrek@gmail.com - 2012-07-25 05:09 -0700
    Re: catch UnicodeDecodeError jaroslav.dobrek@gmail.com - 2012-07-25 05:09 -0700
      Re: catch UnicodeDecodeError Dave Angel <d@davea.name> - 2012-07-25 14:50 -0400
        Re: catch UnicodeDecodeError Jaroslav Dobrek <jaroslav.dobrek@gmail.com> - 2012-07-26 00:46 -0700
          Re: catch UnicodeDecodeError Stefan Behnel <stefan_ml@behnel.de> - 2012-07-26 10:28 +0200
            Re: catch UnicodeDecodeError Jaroslav Dobrek <jaroslav.dobrek@gmail.com> - 2012-07-26 03:51 -0700
              Re: catch UnicodeDecodeError Stefan Behnel <stefan_ml@behnel.de> - 2012-07-26 13:15 +0200
                Re: catch UnicodeDecodeError jaroslav.dobrek@gmail.com - 2012-07-26 04:58 -0700
                Re: catch UnicodeDecodeError jaroslav.dobrek@gmail.com - 2012-07-26 04:58 -0700
              Re: catch UnicodeDecodeError Philipp Hagemeister <phihag@phihag.de> - 2012-07-26 14:17 +0200
              Re: catch UnicodeDecodeError Stefan Behnel <stefan_ml@behnel.de> - 2012-07-26 14:24 +0200
          Re: catch UnicodeDecodeError Chris Angelico <rosuav@gmail.com> - 2012-07-26 19:46 +1000
          Re: catch UnicodeDecodeError wxjmfauth@gmail.com - 2012-07-26 03:19 -0700
      Re: catch UnicodeDecodeError Philipp Hagemeister <phihag@phihag.de> - 2012-07-26 14:43 +0200

csiph-web