Path: csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail From: Adam Funk Newsgroups: comp.lang.python Subject: Re: trying to force stdout to utf-8 with errors='ignore' or 'replace' Date: Fri, 11 Dec 2015 16:54:27 +0000 Organization: $CABAL Lines: 24 Message-ID: <3nbrjcxcaf.ln2@news.ducksburg.com> References: X-Trace: individual.net eSJAMaWWbPq8uZKdbSvf7gwr8jmzWRpJQFh6YFWm0y7nrLFn6r X-Orig-Path: news.ducksburg.com!not-for-mail Cancel-Lock: sha1:PkQlt0cBPSPuC78jk2B4l4PLOnY= sha1:SCo5QGXOGlH6DLB8PRRJQCb/W+0= User-Agent: slrn/pre1.0.3-5 (Linux) Xref: csiph.com comp.lang.python:100273 On 2015-12-11, Peter Otten wrote: > Adam Funk wrote: >> but with either or both of those, I get the dreaded >> "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position >> 562: ordinal not in range(128)". How can I force the output to be in >> UTF-8 & silently suppress invalid characters? > > (I'm assuming you are using Python 2 and that main_body is a unicode > instance) The short answer turned out to be 'switch to Python 3', which I think is what I'll do from now on unless I absolutely need a library that isn't available there. (AFAICT, the email parser in 2.7 returns the body as a bytestring & doesn't actually look at the Content-Type header, & trying to decode the body with that just made it barf in different places.) -- Science is what we understand well enough to explain to a computer. Art is everything else we do. --- Donald Knuth