Path: csiph.com!usenet.pasdenom.info!gegeweb.org!usenet-fr.net!nerim.net!novso.com!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Ben Finney <ben+python@benfinney.id.au>
Subject: Re: python-list@python.org
Date: Thu, 16 Jan 2014 11:52:41 +1100
References: <11036720.dCfMmLrdqv@horus> <52D56D40.1060401@mrabarnett.plus.com> <mailman.5488.1389749137.18130.python-list@python.org> <52d729fd$0$29970$c3e8da3$5496439d@news.astraweb.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux)
Cancel-Lock: sha1:3LZHyZ+qBkfim8l4tOMKHoyLwpY=
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.5553.1389833576.18130.python-list@python.org>
Lines: 60
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:64031

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> On Wed, 15 Jan 2014 02:25:34 +0100, Florian Lindner wrote:
> >> On 2014-01-14 16:37, Florian Lindner wrote:
> >> > I'm using python 3.2.3 on debian wheezy. My script is called from
> >> > my mail delivery agent (MDA) maildrop (like procmail) through
> >> > it's xfilter directive.
> >> >
> >> > Script works fine when used interactively, e.g. ./script.py <
> >> > testmail but when called from maildrop it's producing an infamous
> >> > UnicodeDecodeError:
>
> What's maildrop? When using third party libraries, it's often helpful to 
> point to give some detail on what they are and where they are from.

It's not a library; as he says, it's an MDA program. It is from the
Courier mail application <URL:http://www.courier-mta.org/maildrop/>.

>From that, I understand Florian to be saying his Python program is
invoked via command-line from some configuration directive for Maildrop.

> What does the maildrop documentation say about encodings and/or
> malformed email?

I think this is the more likely line of enquiry to diagnose the problem.

> For a modern Linux system to be using the C encoding is not a good
> sign.

That's true, but it's likely a configuration problem: the encoding needs
to be set *and* obeyed at an administrative and user-profile level.

> It's not 1970 anymore. I would expect it should be using UTF-8. But I 
> don't think that's relevant to your problem (although a mis-configured 
> system may make it worse).

Since the MDA runs usually not as a system service, but rather at a
user-specific level, I would expect some interaction of the host locale
and the user-specific locale is the problem.

> Who knows? What's maildrop? What does its documentation say about 
> encodings?

I hope the original poster enjoys manpages, since that's how the program
is documented <URL:http://www.courier-mta.org/maildrop/documentation.html>.

> The fact that it is using ASCII apparently by default does not give me
> confidence that it knows how to deal with 8-bit emails, but I might be
> completely wrong.

I've found that the problem is often that Python is the party assuming
that stdin and stdout are ASCII, largely because it hasn't been told
otherwise.

-- 
 \        “The greatest tragedy in mankind's entire history may be the |
  `\       hijacking of morality by religion.” —Arthur C. Clarke, 1991 |
_o__)                                                                  |
Ben Finney