Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #99999

Re: getting fileinput to do errors='ignore' or 'replace'?

Path csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From Oscar Benjamin <oscar.j.benjamin@gmail.com>
Newsgroups comp.lang.python
Subject Re: getting fileinput to do errors='ignore' or 'replace'?
Date Fri, 4 Dec 2015 09:00:32 +0000
Lines 46
Message-ID <mailman.193.1449226525.14615.python-list@python.org> (permalink)
References <fn26jcxltl.ln2@news.ducksburg.com> <8336jcxi2m.ln2@news.ducksburg.com> <n3prpd$pt4$1@ger.gmane.org> <CAHVvXxQCuHQdt3W5yUc02n_8UL=PqtQhp8ro63T_u4_d0bnpDw@mail.gmail.com> <n3rj72$fic$1@ger.gmane.org>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
X-Trace news.uni-berlin.de c201zAMI7jn162a9eAJ9YwBzf83OSkLO8oFSVdPEEalA==
Return-Path <oscar.j.benjamin@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'context': 0.05; "subject:' ": 0.07; 'subject:getting': 0.07; 'cc:addr:python-list': 0.09; 'benjamin': 0.09; 'closed.': 0.09; 'files:': 0.09; 'subject:ignore': 0.09; 'whichever': 0.09; '2.7': 0.13; '&gt;&gt;&gt;': 0.15; '"terry': 0.16; 'cc:name:python list': 0.16; 'consume': 0.16; 'defaulting': 0.16; 'email addr:udel.edu&gt;': 0.16; 'email name:&lt;tjreedy': 0.16; 'iteration': 0.16; 'iterators': 0.16; 'iterators,': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'reedy"': 0.16; 'statement.': 0.16; 'wrote:': 0.16; '&gt;': 0.18; 'email addr:gmail.com&gt;': 0.18; '>>>': 0.20; '2015': 0.20; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'dec': 0.23; 'slightly': 0.23; 'header:In-Reply-To:1': 0.24; 'module': 0.25; "doesn't": 0.26; 'message-id:@mail.gmail.com': 0.27; 'function': 0.28; "skip:' 10": 0.28; 'fine': 0.28; 'convert': 0.29; 'statement': 0.32; 'wrap': 0.33; 'open': 0.33; 'correctly': 0.34; 'file': 0.34; 'received:google.com': 0.35; 'skip:c 30': 0.35; 'replace': 0.35; 'received:209.85': 0.36; '(and': 0.36; 'subject:: ': 0.37; 'received:209': 0.38; 'files': 0.38; 'end': 0.39; 'does': 0.39; 'easily': 0.39; 'your': 0.60; 'managers': 0.63; 'skip:\xc2 10': 0.67; 'guaranteed': 0.67; 'oscar': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2u3FNH0hRy0aImTcfkylLLt8VI6ZDnr6Imgw04U8gOY=; b=ZVue2+7iXjsJj1P8yDQv+BBevzhnxrXRzYOGqvxSrAJ77RNSoB+tkmsEh3lShmQy0c u2GE19CSPqg19CWzbKwwSjo0rc02GGLA0linhf462HJdgY4ZNJjasW4C2SKp45qI7DzZ l+WryhmI6f32/czgZK1gvCsY2wqtD0KRbz+x3bXXLqQ57TWhkRi+50oawGyaWdWfNmFB RE5j9IdE7uzdwrHkSHQX5qpc1FrGGHeWIJKwuWz89WIreHDSN66K7aqIKuZXH8vhmDER o8gDEyV8xcVT/+j2/3dHIOfhXGz+BcuJceoDoCSLNId/0+L8bdOGDXBpHYUuZoa0Q4u5 TknA==
X-Received by 10.112.148.198 with SMTP id tu6mr7146514lbb.39.1449219633089; Fri, 04 Dec 2015 01:00:33 -0800 (PST)
In-Reply-To <n3rj72$fic$1@ger.gmane.org>
X-Content-Filtered-By Mailman/MimeDel 2.1.20+
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.20+
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref csiph.com comp.lang.python:99999

Show key headers only | View raw

On 4 Dec 2015 08:36, "Serhiy Storchaka" <storchaka@gmail.com> wrote:
>
> On 04.12.15 00:26, Oscar Benjamin wrote:
>>
>> On 3 Dec 2015 16:50, "Terry Reedy" <tjreedy@udel.edu> wrote:
>>>
>>> fileinput is an ancient module that predates iterators (and generators)
>>
>> and context managers. Since by 2.7 open files are both context managers
and
>> line iterators, you can easily write your own multi-file line iteration
>> that does exactly what you want.  At minimum:
>>>
>>>
>>> for file in files:
>>>      with codecs.open(file, errors='ignore') as f
>>>      # did not look up signature,
>>>          for line in f:
>>>              do_stuff(line)
>>
>>
>> The above is fine but...
>>
>>> To make this reusable, wrap in 'def filelines(files):' and replace
>>
>> 'do_stuff(line)' with 'yield line'.
>>
>> That doesn't work entirely correctly as you end up yielding from inside a
>> with statement. If the user of your generator function doesn't fully
>> consume the generator then whichever file is currently open is not
>> guaranteed to be closed.
>
>
> You can convert the generator to context manager and use it in the with
statement to guarantee closing.
>
> with contextlib.closing(filelines(files)) as f:
>     for line in f:
>         ...

Or you can use fileinput which is designed to be exactly this kind of
context manager and to be used in this way. Although fileinput is slightly
awkward in defaulting to reading stdin.

--
Oscar

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 15:12 +0000
  Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 15:18 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Peter Otten <__peter__@web.de> - 2015-12-03 17:11 +0100
      Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 19:17 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Terry Reedy <tjreedy@udel.edu> - 2015-12-03 11:48 -0500
      Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 19:21 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-12-03 22:26 +0000
    Re: getting fileinput to do errors='ignore' or 'replace'? Serhiy Storchaka <storchaka@gmail.com> - 2015-12-04 10:34 +0200
    Re: getting fileinput to do errors='ignore' or 'replace'? Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-12-04 09:00 +0000
      Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-07 14:46 +0000
  Re: getting fileinput to do errors='ignore' or 'replace'? MRAB <python@mrabarnett.plus.com> - 2015-12-03 16:12 +0000
  Re: getting fileinput to do errors='ignore' or 'replace'? Laura Creighton <lac@openend.se> - 2015-12-03 17:46 +0100
    Re: getting fileinput to do errors='ignore' or 'replace'? Adam Funk <a24061@ducksburg.com> - 2015-12-03 19:17 +0000
      Re: getting fileinput to do errors='ignore' or 'replace'? Laura Creighton <lac@openend.se> - 2015-12-03 21:40 +0100

csiph-web