Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #31763 > unrolled thread

RE: Python does not take up available physical memory

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  RE: Python does not take up available physical memory "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-10-19 19:38 +0000

#31763 — RE: Python does not take up available physical memory

Chris Angelico wrote:
> On Sat, Oct 20, 2012 at 4:08 AM, Pradipto Banerjee
> <pradipto.banerjee@adainvestments.com> wrote:
> > I am trying to read a file into memory. The size of the file is around 1 GB.
> > I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB available
> > physical memory when I was trying to read the file. I tried to read the file
> > as follows:
> >
> >
> >
> >>>> fdata = open(filename, 'r').read()
> 
> Is this Python 2 or Python 3? Just throwing a random possibility out
> there, could it be that reading it in and converting it to Unicode
> text requires more memory than you have?
> 
> My recommendation: Unless you actually need to search the whole file
> as a single string, iterate over the file instead:
> 
> for line in open(filename):
>     # do something with line

If you (OP) are in Python 2.5+ I would do the following instead.

with open(filename) as f:
    for line in f:
        # do something with line

This will automatically close the file when it is done. I doubt
it will help with memory issues, but closing files after you
are done with them is a Good practice.


Ramit Prasad


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web