Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #99985

Re: getting fileinput to do errors='ignore' or 'replace'?

Path csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From Laura Creighton <lac@openend.se>
Newsgroups comp.lang.python
Subject Re: getting fileinput to do errors='ignore' or 'replace'?
Date Thu, 03 Dec 2015 21:40:16 +0100
Lines 35
Message-ID <mailman.185.1449175233.14615.python-list@python.org> (permalink)
References <fn26jcxltl.ln2@news.ducksburg.com> <mailman.179.1449161213.14615.python-list@python.org> <v3h6jcxtas.ln2@news.ducksburg.com>
Mime-Version 1.0
Content-Type text/plain; charset="us-ascii"
Content-Transfer-Encoding quoted-printable
X-Trace news.uni-berlin.de FCHk5Rvw4STWCBu9j1IrlgsSamCg9BNafx274DAsskTQ==
Return-Path <lac@openend.se>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; "subject:' ": 0.07; 'subject:getting': 0.07; 'cc:addr:python-list': 0.09; 'agrees': 0.09; 'creighton': 0.09; 'effect.': 0.09; 'received:openend.se': 0.09; 'received:theraft.openend.se': 0.09; 'subject:ignore': 0.09; 'useful,': 0.13; 'ignore': 0.14; 'encoding': 0.15; 'skip:f 30': 0.15; 'thu,': 0.15; '>on': 0.16; 'adam': 0.16; 'add,': 0.16; 'approval,': 0.16; 'cc:addr:lac': 0.16; 'cc:addr:openend.se': 0.16; 'from:addr:lac': 0.16; 'from:addr:openend.se': 0.16; 'from:name:laura creighton': 0.16; 'hacked': 0.16; 'message- id:@fido.openend.se': 0.16; 'patch,': 0.16; 'received:fido': 0.16; 'received:fido.openend.se': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'wrote:': 0.16; 'laura': 0.18; 'input': 0.18; 'thanks.': 0.18; '>>>': 0.20; '2015': 0.20; 'cc:addr:python.org': 0.20; 'cc:2**1': 0.22; 'mixed': 0.22; 'dec': 0.23; 'url:bugs': 0.24; "i've": 0.25; 'moved': 0.27; 'received:se': 0.29; 'cc:no real name:2**1': 0.29; "i'd": 0.31; 'url:python': 0.33; 'file': 0.34; 'this?': 0.34; 'so,': 0.35; 'trouble': 0.35; 'skip:> 10': 0.35; 'problem.': 0.35; 'but': 0.36; 'should': 0.36; 'instead': 0.36; 'url:org': 0.36; 'possible': 0.36; 'subject:: ': 0.37; 'being': 0.37; 'charset:us-ascii': 0.37; "won't": 0.38; 'files': 0.38; 'along': 0.39; 'some': 0.40; 'easy': 0.60; 'future': 0.60; 'header:Message-Id:1': 0.61; "'you": 0.84; 'header:In-reply-to:1': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=openend.se; s=default; t=1449175219; bh=rSTIdvXrvA+X1eyKAbIkQZWllS6ginuhdrPtIINIP4g=; h=To:cc:From:Subject:In-reply-to:References:Date:From; b=sJBKsyyjOyNL4YOARy897QsvNGjOCY2DtyM4Hkr+2N22+F6MnY2wAr8VTexyHaN88 cehMgpIuZR8J0yy+8Il6oF4/wOwmUcKdRRU8FrnZ1/V2kyoME0k2f/zENQTNBVyCh0 d/2PubDk07p69O4ViG94cUIb4QuBk7G7QPlfFyfY=
In-reply-to <v3h6jcxtas.ln2@news.ducksburg.com>
Comments In-reply-to Adam Funk <a24061@ducksburg.com> message dated "Thu, 03 Dec 2015 19:17:51 +0000."
Content-ID <21801.1449175216.1@fido>
X-Greylist Sender IP whitelisted, not delayed by milter-greylist-4.3.9 (theraft.openend.se [82.96.5.2]); Thu, 03 Dec 2015 21:40:19 +0100 (CET)
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.20+
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref csiph.com comp.lang.python:99985

Show key headers only | View raw

In a message of Thu, 03 Dec 2015 19:17:51 +0000, Adam Funk writes:
>On 2015-12-03, Laura Creighton wrote:
>
>> In a message of Thu, 03 Dec 2015 15:12:15 +0000, Adam Funk writes:
>>>I'm having trouble with some input files that are almost all proper
>>>UTF-8 but with a couple of troublesome characters mixed in, which I'd
>>>like to ignore instead of throwing ValueError.  I've found the
>>>openhook for the encoding
>>>
>>>for line in fileinput.input(options.files, openhook=fileinput.hook_encoded("utf-8")):
>>>    do_stuff(line)
>>>
>>>which the documentation describes as "a hook which opens each file
>>>with codecs.open(), using the given encoding to read the file", but
>>>I'd like codecs.open() to also have the errors='ignore' or
>>>errors='replace' effect.  Is it possible to do this?
>>>
>>>Thanks.
>>
>> This should be both easy to add, and useful, and I happen to know that
>> fileinput is being hacked on by Serhiy Storchaka right now, who agrees
>> that this would be easy.  So, with his approval, I stuck this into the
>> tracker.  http://bugs.python.org/issue25788  
>>
>> Future Pythons may not have the problem.
>
>Good to know, thanks.

Well, we have moved right along to 'You write the patch, Laura' so I
can pretty much guarantee that future Pythons won't have the problem. :)

Laura

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 15:12 +0000
  Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 15:18 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Peter Otten <__peter__@web.de> - 2015-12-03 17:11 +0100
      Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 19:17 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Terry Reedy <tjreedy@udel.edu> - 2015-12-03 11:48 -0500
      Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 19:21 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-12-03 22:26 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Serhiy Storchaka <storchaka@gmail.com> - 2015-12-04 10:34 +0200
    Re: getting fileinput to do errors='ignore' or 'replace'? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-12-04 09:00 +0000
      Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-07 14:46 +0000
  Re: getting fileinput to do errors='ignore' or 'replace'? MRAB <python@mrabarnett.plus.com> - 2015-12-03 16:12 +0000
  Re: getting fileinput to do errors='ignore' or 'replace'? Laura Creighton <lac@openend.se> - 2015-12-03 17:46 +0100
    Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 19:17 +0000
      Re: getting fileinput to do errors='ignore' or 'replace'? Laura Creighton <lac@openend.se> - 2015-12-03 21:40 +0100

csiph-web