Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Date: Wed, 15 Jan 2014 03:49:26 +0000
From: MRAB <python@mrabarnett.plus.com>
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: python-list@python.org
References: <11036720.dCfMmLrdqv@horus> <52D56D40.1060401@mrabarnett.plus.com> <5496632.M1CKVtg2ju@horus>
In-Reply-To: <5496632.M1CKVtg2ju@horus>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Reply-To: python-list@python.org
Newsgroups: comp.lang.python
Message-ID: <mailman.5493.1389757767.18130.python-list@python.org>
Lines: 52
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:63962

On 2014-01-15 01:25, Florian Lindner wrote:
> Am Dienstag, 14. Januar 2014, 17:00:48 schrieb MRAB:
>> On 2014-01-14 16:37, Florian Lindner wrote:
>> > Hello!
>> >
>> > I'm using python 3.2.3 on debian wheezy. My script is called from my mail delivery agent (MDA) maildrop (like procmail) through it's xfilter directive.
>> >
>> > Script works fine when used interactively, e.g. ./script.py < testmail but when called from maildrop it's producing an infamous UnicodeDecodeError:
>> >
>> > File "/home/flindner/flofify.py", line 171, in main
>> >       mail = sys.stdin.read()
>> > File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
>> >       return codecs.ascii_decode(input, self.errors)[0]
>> >
>> > Exception for example is always like
>> >
>> > UnicodeDecodeError: 'ascii' codec can't decode byte 0x82 in position 869: ordinal not in range(128)
>> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1176: ordinal not in range(128)
>> > UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 846: ordinal not in range(128)
>> >
>> > I read mail from stdin "mail = sys.stdin.read()"
>> >
>> > Environment when called is:
>> >
>> > locale.getpreferredencoding(): ANSI_X3.4-1968
>> > environ["LANG"]: C
>> >
>> > System environment when using shell is:
>> >
>> > ~ % echo $LANG
>> > en_US.UTF-8
>> >
>> > As far as I know when reading from stdin I don't need an decode(...) call, since stdin has a decoding. I also tried some decoding/encoding stuff but changed nothing.
>> >
>> > Any ideas to help me?
>> >
>> When run from maildrop it thinks that the encoding of stdin is ASCII.
>
> Well, true. But what encoding does maildrop actually gives me? It obviously does not inherit LANG or is called from the MTA that way. I also tried:
>
locale.getpreferredencoding() said "ANSI_X3.4-1968", which is ASCII
(ask Wikipedia if you want to know why it's called that!).

>          inData = codecs.getreader('utf-8')(sys.stdin)
>          mail = inData.read()
>
> Failed also. But I'm not exactly an encoding expert.
>
Try:

sys.stdin = codecs.getreader('utf-8')(sys.stdin.detach())