Path: csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: Laura Creighton <lac@openend.se>
Newsgroups: comp.lang.python
Subject: Re: getting fileinput to do errors='ignore' or 'replace'?
Date: Thu, 03 Dec 2015 21:40:16 +0100
Lines: 35
Message-ID: <mailman.185.1449175233.14615.python-list@python.org>
References: <fn26jcxltl.ln2@news.ducksburg.com> <mailman.179.1449161213.14615.python-list@python.org> <v3h6jcxtas.ln2@news.ducksburg.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de FCHk5Rvw4STWCBu9j1IrlgsSamCg9BNafx274DAsskTQ==
Return-Path: <lac@openend.se>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; "subject:' ": 0.07; 'subject:getting': 0.07; 'cc:addr:python-list': 0.09; 'agrees': 0.09; 'creighton': 0.09; 'effect.': 0.09; 'received:openend.se': 0.09; 'received:theraft.openend.se': 0.09; 'subject:ignore': 0.09; 'useful,': 0.13; 'ignore': 0.14; 'encoding': 0.15; 'skip:f 30': 0.15; 'thu,': 0.15; '>on': 0.16; 'adam': 0.16; 'add,': 0.16; 'approval,': 0.16; 'cc:addr:lac': 0.16; 'cc:addr:openend.se': 0.16; 'from:addr:lac': 0.16; 'from:addr:openend.se': 0.16; 'from:name:laura creighton': 0.16; 'hacked': 0.16; 'message- id:@fido.openend.se': 0.16; 'patch,': 0.16; 'received:fido': 0.16; 'received:fido.openend.se': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'wrote:': 0.16; 'laura': 0.18; 'input': 0.18; 'thanks.': 0.18; '>>>': 0.20; '2015': 0.20; 'cc:addr:python.org': 0.20; 'cc:2**1': 0.22; 'mixed': 0.22; 'dec': 0.23; 'url:bugs': 0.24; "i've": 0.25; 'moved': 0.27; 'received:se': 0.29; 'cc:no real name:2**1': 0.29; "i'd": 0.31; 'url:python': 0.33; 'file': 0.34; 'this?': 0.34; 'so,': 0.35; 'trouble': 0.35; 'skip:> 10': 0.35; 'problem.': 0.35; 'but': 0.36; 'should': 0.36; 'instead': 0.36; 'url:org': 0.36; 'possible': 0.36; 'subject:: ': 0.37; 'being': 0.37; 'charset:us-ascii': 0.37; "won't": 0.38; 'files': 0.38; 'along': 0.39; 'some': 0.40; 'easy': 0.60; 'future': 0.60; 'header:Message-Id:1': 0.61; "'you": 0.84; 'header:In-reply-to:1': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openend.se; s=default; t=1449175219; bh=rSTIdvXrvA+X1eyKAbIkQZWllS6ginuhdrPtIINIP4g=; h=To:cc:From:Subject:In-reply-to:References:Date:From; b=sJBKsyyjOyNL4YOARy897QsvNGjOCY2DtyM4Hkr+2N22+F6MnY2wAr8VTexyHaN88 cehMgpIuZR8J0yy+8Il6oF4/wOwmUcKdRRU8FrnZ1/V2kyoME0k2f/zENQTNBVyCh0 d/2PubDk07p69O4ViG94cUIb4QuBk7G7QPlfFyfY=
In-reply-to: <v3h6jcxtas.ln2@news.ducksburg.com>
Comments: In-reply-to Adam Funk <a24061@ducksburg.com> message dated "Thu, 03 Dec 2015 19:17:51 +0000."
Content-ID: <21801.1449175216.1@fido>
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.9 (theraft.openend.se [82.96.5.2]); Thu, 03 Dec 2015 21:40:19 +0100 (CET)
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.20+
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref: csiph.com comp.lang.python:99985

In a message of Thu, 03 Dec 2015 19:17:51 +0000, Adam Funk writes:
>On 2015-12-03, Laura Creighton wrote:
>
>> In a message of Thu, 03 Dec 2015 15:12:15 +0000, Adam Funk writes:
>>>I'm having trouble with some input files that are almost all proper
>>>UTF-8 but with a couple of troublesome characters mixed in, which I'd
>>>like to ignore instead of throwing ValueError.  I've found the
>>>openhook for the encoding
>>>
>>>for line in fileinput.input(options.files, openhook=3Dfileinput.hook_en=
coded("utf-8")):
>>>    do_stuff(line)
>>>
>>>which the documentation describes as "a hook which opens each file
>>>with codecs.open(), using the given encoding to read the file", but
>>>I'd like codecs.open() to also have the errors=3D'ignore' or
>>>errors=3D'replace' effect.  Is it possible to do this?
>>>
>>>Thanks.
>>
>> This should be both easy to add, and useful, and I happen to know that
>> fileinput is being hacked on by Serhiy Storchaka right now, who agrees
>> that this would be easy.  So, with his approval, I stuck this into the
>> tracker.  http://bugs.python.org/issue25788  =

>>
>> Future Pythons may not have the problem.
>
>Good to know, thanks.

Well, we have moved right along to 'You write the patch, Laura' so I
can pretty much guarantee that future Pythons won't have the problem. :)

Laura