Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!eweka.nl!lightspeed.eweka.nl!194.109.133.83.MISMATCH!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Wed, 02 Jan 2013 18:54:18 -0500
From: Dave Angel <d@davea.name>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/20121011 Thunderbird/16.0.1
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: avoding the accumulation of array when using loop.
References: <d00e1c2c-56a8-4d19-ba4b-da4d3281e275@googlegroups.com>
In-Reply-To: <d00e1c2c-56a8-4d19-ba4b-da4d3281e275@googlegroups.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Precedence: list
Reply-To: d@davea.name
Newsgroups: comp.lang.python
Message-ID: <mailman.5.1357170888.2939.python-list@python.org>
Lines: 74
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:36016

On 01/02/2013 05:21 PM, Isaac Won wrote:
> Hi all,
>
> Thanks to Hans, I have had a good progress on my problem. 
>
> Followings are Hans's Idea:
>
> import numpy as np 
>
> b = [] 
> c = 4 
> f = open("text.file", "r") 
>
> while c < 10: 
>         c = c + 1 
>
>
>         f.seek(0,0) 
>
>         for  columns in ( raw.strip().split() for raw in f ): 
>                 b.append(columns[c]) 
>
>         y = np.array(b, float) 
>         print c, y 
>
>
> It's a bit inefficient to read the same file several times. 

Don't bet on it.  The OS and the libraries and Python each do some
buffering, so it might be nearly as fast to just reread if it's a small
file.  And if it's a huge one, the list would be even bigger.  So the
only sizes where the second approach is likely better is the mid-size file.

> You might consider reading it just once.  For example: 
>
>
> import numpy as np 
>
> b = [] 
>
>
>
> f = open("text.file", "r") 
>
> data = [ line.strip().split() for line in f ] 
> f.close() 
>
> for c in xrange(5, 11): 
>         for row in data: 
>                 b.append(row[c]) 
>
>
>         y = np.array(b, float) 
>         print c, y 
> -------------------------------------------------------------------------------
>
> It is a great idea, but I found some problems. I want each individual array of y. However, these two codes prodce accumulated array such as [1,2,3], [1,2,3,4,5,6], [1,2,3,4,5,6,7,8,9] and so on. I have tried to initialize for loop for each time to produce array. This effort has not been very successful.
> Do you guys have any idea? I will really appreciate ant help and idea.

Your description is very confusing.  But i don't see why you just don't
just set b=[] inside the outer loop, rather than doing it at the begin
of the program.

for c in xrange(5, 11): 
        b = []
        for row in data: 
                b.append(row[c]) 



-- 

DaveA