Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #36934

Re: iterating over the lines of a file - difference between Python 2.7 and 3?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'example:': 0.03; 'subsequent': 0.04; 'subject:Python': 0.05; 'binary': 0.05; 'returned.': 0.07; 'subject:file': 0.07; 'python': 0.09; 'buffer,': 0.09; 'methods,': 0.09; 'mode,': 0.09; 'read-ahead': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:2.7': 0.09; 'unexpected': 0.09; 'def': 0.10; 'assume': 0.11; '2.7': 0.13; '3.3,': 0.16; '314': 0.16; 'exists?': 0.16; 'iterator': 0.16; 'iterators': 0.16; 'oserror:': 0.16; 'received:80.91.229.3': 0.16; 'received:dip.t-dialin.net': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-dialin.net': 0.16; 'subject:between': 0.16; 'wrote:': 0.17; 'documented': 0.17; 'skip:i 40': 0.17; 'file.': 0.20; 'assuming': 0.22; 'disabled': 0.22; 'either.': 0.22; 'skip:_ 20': 0.22; 'raise': 0.24; 'idea': 0.24; 'header:User-Agent:1': 0.26; 'mix': 0.27; 'header:X -Complaints-To:1': 0.28; 'lines': 0.28; 'behavior.': 0.29; 'behaviour': 0.29; 'mode.': 0.29; 'prints': 0.29; 'class': 0.29; 'call.': 0.30; 'mode': 0.30; 'code': 0.31; 'file': 0.32; 'could': 0.32; 'print': 0.32; 'anywhere': 0.33; 'shorter': 0.33; 'to:addr :python-list': 0.33; 'text': 0.34; 'ahead': 0.35; 'so,': 0.35; 'subject:?': 0.35; 'there': 0.35; 'next': 0.35; 'received:org': 0.36; 'but': 0.36; 'anything': 0.36; 'bad': 0.37; 'beyond': 0.37; 'quite': 0.37; 'subject:: ': 0.38; 'fact': 0.38; 'some': 0.38; 'to:addr:python.org': 0.39; 'header:Received:5': 0.40; 'telling': 0.61; '8bit%:20': 0.62; 'between': 0.63; 'behavior': 0.64; 'completing': 0.71; '2.7.': 0.84; 'basically,': 0.84; 'official,': 0.84; 'subject:over': 0.84
X-Injected-Via-Gmane http://gmane.org/
To python-list@python.org
From Peter Otten <__peter__@web.de>
Subject Re: iterating over the lines of a file - difference between Python 2.7 and 3?
Date Thu, 17 Jan 2013 13:04:25 +0100
Organization None
References <000301cdf49e$602fd510$208f7f30$@biologie.uni-freiburg.de>
Mime-Version 1.0
Content-Type text/plain; charset="UTF-8"
Content-Transfer-Encoding 8Bit
X-Gmane-NNTP-Posting-Host p5084a0c3.dip.t-dialin.net
User-Agent KNode/4.7.3
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.588.1358424269.2939.python-list@python.org> (permalink)
Lines 53
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1358424269 news.xs4all.nl 6843 [2001:888:2000:d::a6]:48284
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:36934

Show key headers only | View raw


Wolfgang Maier wrote:

> I just came across an unexpected behavior in Python 3.3, which has to do
> with file iterators and their interplay with other methods of file/IO
> class methods, like readline() and tell(): Basically, I got used to the
> fact that it is a bad idea to mix them because the iterator would use that
> hidden read-ahead buffer, so what you got with subsequent calls to
> readline() or tell() was what was beyond that buffer, but not the next
> thing after what the iterator just returned.
> 
> Example:
> 
> in_file_object=open(‘some_file’,’rb’)
> 
> for line in in_file_object:
> 
>                 print (line)
> 
>                 if in_file_object.tell() > 300:
> 
>                                # assuming that individual lines are
>                                # shorter
> 
>                                break
> 
>  
> 
> This wouldn´t print anything in Python 2.7 since next(in_file_object)
> would read ahead beyond the 300 position immediately, as evidenced by a
> subsequent call to in_file_object.tell() (returning 8192 on my system).
> 
> However, I find that under Python 3.3 this same code works: it prints some
> lines from my file and after completing in_file_object.tell() returns a
> quite reasonable 314 as the current position in the file.
> 
> I couldn´t find this difference anywhere in the documentation. Is the 3.3
> behavior official, and if so, when was it introduced and how is it
> implemented? I assume the read-ahead buffer still exists?

You can get the Python 3 behaviour with io.open() in Python 2.7. There is an 
implementation in Python in _pyio.py:

    def tell(self):
        return _BufferedIOMixin.tell(self) - len(self._read_buf) + 
self._read_pos


> By the way, the 3.3 behavior only works in binary mode. In text mode, the
> code will raise an OSError:  telling position disabled by next() call. In
> Python 2.7 there was no difference between the binary and text mode
> behavior. Could not find this documented either.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: iterating over the lines of a file - difference between Python 2.7 and 3? Peter Otten <__peter__@web.de> - 2013-01-17 13:04 +0100

csiph-web