Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #36933

iterating over the lines of a file - difference between Python 2.7 and 3?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <wolfgang.maier@biologie.uni-freiburg.de>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'example:': 0.03; 'subsequent': 0.04; 'subject:Python': 0.05; 'binary': 0.05; 'returned.': 0.07; 'subject:file': 0.07; 'python': 0.09; 'buffer,': 0.09; 'methods,': 0.09; 'mode,': 0.09; 'read-ahead': 0.09; 'skip:r 50': 0.09; 'subject:2.7': 0.09; 'unexpected': 0.09; 'assume': 0.11; '2.7': 0.13; '3.3,': 0.16; '314': 0.16; 'exists?': 0.16; 'iterator': 0.16; 'iterators': 0.16; 'oserror:': 0.16; 'skip:3 30': 0.16; 'subject:between': 0.16; 'documented': 0.17; 'file.': 0.20; 'assuming': 0.22; 'disabled': 0.22; 'either.': 0.22; '&gt;': 0.23; 'raise': 0.24; 'idea': 0.24; 'mix': 0.27; 'lines': 0.28; 'skip:( 20': 0.28; 'behavior.': 0.29; 'mode.': 0.29; 'prints': 0.29; 'received:132': 0.29; 'class': 0.29; 'call.': 0.30; 'mode': 0.30; 'code': 0.31; 'file': 0.32; 'could': 0.32; 'print': 0.32; 'anywhere': 0.33; 'skip:s 30': 0.33; 'shorter': 0.33; 'to:addr:python-list': 0.33; 'text': 0.34; 'ahead': 0.35; 'so,': 0.35; 'subject:?': 0.35; 'there': 0.35; 'next': 0.35; 'but': 0.36; 'url:org': 0.36; 'anything': 0.36; 'bad': 0.37; 'beyond': 0.37; 'url:rec-html40': 0.37; 'quite': 0.37; 'fact': 0.38; 'some': 0.38; 'received:10': 0.38; 'to:addr:python.org': 0.39; 'url:schemas': 0.39; 'url:office': 0.39; 'url:omml': 0.39; 'url:2004': 0.39; 'url:microsoft': 0.39; 'url:12': 0.40; 'telling': 0.61; 'between': 0.63; 'behavior': 0.64; '8bit%:50': 0.65; 'completing': 0.71; 'basically,': 0.84; 'official,': 0.84; 'subject:over': 0.84
Delivery-date Thu, 17 Jan 2013 11:59:35 +0100
From "Wolfgang Maier" <wolfgang.maier@biologie.uni-freiburg.de>
To <python-list@python.org>
Subject iterating over the lines of a file - difference between Python 2.7 and 3?
Date Thu, 17 Jan 2013 11:35:31 +0100
MIME-Version 1.0
Content-Type multipart/alternative; boundary="----=_NextPart_000_0004_01CDF4A6.C1F52770"
X-Mailer Microsoft Outlook 14.0
Thread-Index Ac30m57l1mBijQe6SfSJ4fiw9ZbnhA==
Content-Language en-us
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.587.1358420377.2939.python-list@python.org> (permalink)
Lines 161
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1358420377 news.xs4all.nl 6913 [2001:888:2000:d::a6]:33287
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:36933

Show key headers only | View raw


[Multipart message — attachments visible in raw view] - view raw

I just came across an unexpected behavior in Python 3.3, which has to do
with file iterators and their interplay with other methods of file/IO class
methods, like readline() and tell(): Basically, I got used to the fact that
it is a bad idea to mix them because the iterator would use that hidden
read-ahead buffer, so what you got with subsequent calls to readline() or
tell() was what was beyond that buffer, but not the next thing after what
the iterator just returned.

Example:

in_file_object=open(‘some_file’,’rb’)

for line in in_file_object:

                print (line)

                if in_file_object.tell() > 300:

                               # assuming that individual lines are shorter

                               break

 

This wouldn´t print anything in Python 2.7 since next(in_file_object) would
read ahead beyond the 300 position immediately, as evidenced by a subsequent
call to in_file_object.tell() (returning 8192 on my system).

However, I find that under Python 3.3 this same code works: it prints some
lines from my file and after completing in_file_object.tell() returns a
quite reasonable 314 as the current position in the file.

I couldn´t find this difference anywhere in the documentation. Is the 3.3
behavior official, and if so, when was it introduced and how is it
implemented? I assume the read-ahead buffer still exists?

 

By the way, the 3.3 behavior only works in binary mode. In text mode, the
code will raise an OSError:  telling position disabled by next() call. In
Python 2.7 there was no difference between the binary and text mode
behavior. Could not find this documented either.

 

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

iterating over the lines of a file - difference between Python 2.7 and 3? "Wolfgang Maier" <wolfgang.maier@biologie.uni-freiburg.de> - 2013-01-17 11:35 +0100

csiph-web