Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!easy.in-chemnitz.de!news.swapon.de!news.stack.nl!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <mailinglists@xgm.de>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.013
X-Spam-Evidence: '*H*': 0.97; '*S*': 0.00; 'subject:application': 0.07; 'directive.': 0.09; 'received:localnet': 0.09; 'subject:script': 0.09; 'python': 0.11; 'codec': 0.16; 'ordinal': 0.16; 'subject:when': 0.16; 'exception': 0.16; 'producing': 0.19; 'example': 0.22; 'shell': 0.22; 'header:User-Agent:1': 0.23; 'byte': 0.24; 'skip:l 30': 0.24; 'fine': 0.24; 'environment': 0.24; 'script': 0.25; 'skip:" 40': 0.26; 'skip:" 20': 0.27; 'tried': 0.27; '(like': 0.30; "i'm": 0.30; 'file': 0.32; 'me?': 0.32; 'stuff': 0.32; 'thanks!': 0.32; 'subject:from': 0.34; "can't": 0.35; 'received:84': 0.35; 'but': 0.35; 'e.g.': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'changed': 0.39; 'called': 0.40; 'skip:u 10': 0.60; 'read': 0.60; '26,': 0.68; 'florian': 0.84; 'infamous': 0.84
From: Florian Lindner <mailinglists@xgm.de>
To: python-list@python.org
Subject: Encoding trouble when script called from application
Date: Tue, 14 Jan 2014 17:37:36 +0100
User-Agent: KMail/4.12 (Linux/3.12.7-1-ARCH; KDE/4.12.0; x86_64; ; )
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="utf-8"
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.5464.1389717465.18130.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1389717467 news.xs4all.nl 2876 [2001:888:2000:d::a6]:52496
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:63915

Hello!

I'm using python 3.2.3 on debian wheezy. My script is called from my mail delivery agent (MDA) maildrop (like procmail) through it's xfilter directive.

Script works fine when used interactively, e.g. ./script.py < testmail but when called from maildrop it's producing an infamous UnicodeDecodeError:

File "/home/flindner/flofify.py", line 171, in main
     mail = sys.stdin.read()
File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
     return codecs.ascii_decode(input, self.errors)[0]

Exception for example is always like

UnicodeDecodeError: 'ascii' codec can't decode byte 0x82 in position 869: ordinal not in range(128) 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1176: ordinal not in range(128)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 846: ordinal not in range(128)

I read mail from stdin "mail = sys.stdin.read()"

Environment when called is:

locale.getpreferredencoding(): ANSI_X3.4-1968
environ["LANG"]: C

System environment when using shell is:

~ % echo $LANG
en_US.UTF-8

As far as I know when reading from stdin I don't need an decode(...) call, since stdin has a decoding. I also tried some decoding/encoding stuff but changed nothing.

Any ideas to help me?

Thanks!
Florian