Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #99203 > unrolled thread

Problem to read from array

Back to article view | Back to comp.lang.python

Contents

  Problem to read from array vostrushka@gmail.com - 2015-11-21 02:41 -0800
    Re: Problem to read from array Peter Otten <__peter__@web.de> - 2015-11-21 12:23 +0100
    Re: Problem to read from array Laura Creighton <lac@openend.se> - 2015-11-21 15:22 +0100
    Re: Problem to read from array Laura Creighton <lac@openend.se> - 2015-11-21 16:05 +0100
    Re: Problem to read from array BartC <bc@freeuk.com> - 2015-11-21 15:26 +0000
      Re: Problem to read from array Nathan Hilterbrand <nhilterbrand@gmail.com> - 2015-11-21 10:52 -0500
        Re: Problem to read from array Crane Ugly <vostrushka@gmail.com> - 2015-11-23 12:58 -0800
          Re: Problem to read from array sohcahtoa82@gmail.com - 2015-11-23 16:44 -0800
            Re: Problem to read from array Crane Ugly <vostrushka@gmail.com> - 2015-11-24 00:56 -0800
              Re: Problem to read from array Peter Otten <__peter__@web.de> - 2015-11-24 10:31 +0100

#99203 — Problem to read from array

Hi,
I have a file with one parameter per line:
a1
b1
c1
a2
b2
c2
a3
b3
c3
...
The parameters are lines of characters (not numbers)

I need to load it to 2D array for further manipulations.
So far I managed to upload this file into 1D array:

ParametersRaw = []
with open(file1) as fh:
    ParametersRaw = fh.readlines()
fh.close()

NumberOfColumns = 7
NumberOfRows = len(ParametersRaw)/NumberOfColumns

Parameters = [[],[]]
i=0
j=0
k=0
while (i < NumberOfRows):
    while (j < NumberOfColumns):
        k = (i*NumberOfColumns)+j
        Parameters[i][j] = ParametersRaw[k]
        j = j + 1
    i = i + 1
    j = 0

it fails at the line "Parameters[i][j] = ParametersRaw[k]" with error:

IndexError: index 0 is out of bounds for axis 0 with size 0

In case of populating 1D array I would use append() method.
But in case of 2D I am lost of how append() can be applied.
Could some one help newbie.

CU

[toc] | [next] | [standalone]

#99204

vostrushka@gmail.com wrote:

> Hi,
> I have a file with one parameter per line:
> a1
> b1
> c1
> a2
> b2
> c2
> a3
> b3
> c3
> ...
> The parameters are lines of characters (not numbers)
> 
> I need to load it to 2D array for further manipulations.
> So far I managed to upload this file into 1D array:
> 
> ParametersRaw = []

No need for this assignment as you throw the value away.
> with open(file1) as fh:
>     ParametersRaw = fh.readlines()
> fh.close()

The with open() ... statement already ensures that the file is closed.

> NumberOfColumns = 7
> NumberOfRows = len(ParametersRaw)/NumberOfColumns
> 
> Parameters = [[],[]]
> i=0
> j=0
> k=0
> while (i < NumberOfRows):
>     while (j < NumberOfColumns):
>         k = (i*NumberOfColumns)+j
>         Parameters[i][j] = ParametersRaw[k]

The inner lists Parameters[i] are empty so every assignment

Parameters[i][k] = ...

will fail. While the minimal fix is to use append()

          Parameters[i].append(ParametersRaw[k])

you will run into the same problem with the rows if there are more than the 
two you prepared for in the line

> Parameters = [[],[]]

>         j = j + 1
>     i = i + 1
>     j = 0
> 
> it fails at the line "Parameters[i][j] = ParametersRaw[k]" with error:
> 
> IndexError: index 0 is out of bounds for axis 0 with size 0
> 
> In case of populating 1D array I would use append() method.
> But in case of 2D I am lost of how append() can be applied.
> Could some one help newbie.

As a general remark in idiomatic Python the loop

i = 0
while i < THRESHOLD:
   ...
   i = i + 1

is written as a for loop with range():

for i in range(THRESHOLD):
   ...

For your specific problem list slicing is a good approach:

COLUMNS = 7
with open(file1) as f:
    parameters_raw = f.readlines()

parameters = []
for row_start in range(0, len(parameters_raw), COLUMNS):
    parameters.append(parameters_raw[row_start:row_start+COLUMNS])


 

[toc] | [prev] | [next] | [standalone]

#99211

In Python we don't write while loops that often.
instead we do:

for i in range(NumberOfColumns):
    for j in range(NumberOfRows):
    	do_something()

But for the particular case that you are after, constructing lists,
we have something even neater -- list comprehensions.

So you might write:

 >>> newarray=[]
 >>> for i in range(3):
 ...     newarray[i] = i

And this would not work.

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
IndexError: list assignment index out of range

At this point you might get a bit frustrated.  Python is happily telling
you that you don't have a newarray[0][0] which is hardly news to you who
was trying to initialise the thing.

The trick is to initialise it with a list comprehension.

You write one like this:
[ calculate_something() for i in range(LIMIT)]

This works:
 >>> newarray = [i for i in range(3)]
 >>> newarray 
[0, 1, 2]
 >>> 

As does
 >>> newarray = ["hello" for i in range(3)]
 >>> newarray
['hello', 'hello', 'hello']

You can even build your lists with a condition:
 >>> even_numbers=[x for x in range(11) if x % 2 == 0]
 >>> even_numbers
[0, 2, 4, 6, 8, 10]

You are not limited to one dimentional arrays (lists) here.  You
can have much more complicated expressions as the calculate_something.

 >>> newarray =[[ [x+2000, y*y] for x in range(3)] for y in range(5)]
 >>> newarray
[[[2000, 0], [2001, 0], [2002, 0]], [[2000, 1], [2001, 1], [2002, 1]], 
[[2000, 4], [2001, 4], [2002, 4]], [[2000, 9], [2001, 9], [2002, 9]], 
[[2000, 16], [2001, 16], [2002, 16]]]

Which leads us to a particularly useful construct:
 >>> emptyarray =[[ [] for x in range(3)] for y in range(5)]
 >>> emptyarray
[[[], [], []], [[], [], []], [[], [], []], [[], [], []], [[], [], []]]

Bingo.  Now you won't have any index errors when you try to append
to your x,y values.

Hope this is useful.
Also, there is the tutor mailing list
https://mail.python.org/mailman/listinfo/tutor
which you might be interested in, where we discuss things like this.
Peter Otten is there, too.

Laura

[toc] | [prev] | [next] | [standalone]

#99213

In a message of Sat, 21 Nov 2015 15:22:55 +0100, Laura Creighton writes:
>At this point you might get a bit frustrated.  Python is happily telling
>you that you don't have a newarray[0][0] which is hardly news to you who
>was trying to initialise the thing.

My bad.  I was thinking about 2 dimentional arrays, but of course that
error is just that newarray[0] doesn't exit.

(So why is it that I can read an article 3 times, find no more errors,
post it, and _then_ discover one?!)

Laura

[toc] | [prev] | [next] | [standalone]

#99214

On 21/11/2015 10:41, vostrushka@gmail.com wrote:
> Hi,
> I have a file with one parameter per line:
> a1
> b1
> c1
> a2
> b2
> c2
> a3
> b3
> c3
> ...
> The parameters are lines of characters (not numbers)
>
> I need to load it to 2D array for further manipulations.
> So far I managed to upload this file into 1D array:
>
> ParametersRaw = []
> with open(file1) as fh:
>      ParametersRaw = fh.readlines()
> fh.close()

I tried this code based on yours:

with open("input") as fh:
     lines=fh.readlines()

rows = len(lines)//3

params=[]
index=0

for row in range(rows):
     params.append([lines[index],lines[index+1],lines[index+2]])
     index += 3

for row in range(rows):
     print (row,":",params[row])

For the exact input you gave, it produced this output:

0 : ['a1\n', 'b1\n', 'c1\n']
1 : ['a2\n', 'b2\n', 'c2\n']
2 : ['a3\n', 'b3\n', 'c3\n']

Probably you'd want to get rid of those \n characters. (I don't know how 
off-hand as I'm not often write in Python.)

The last bit could also be written:

for param in params:
     print (params)

but I needed the row index.

-- 
BartC

[toc] | [prev] | [next] | [standalone]

#99215

On 11/21/2015 10:26 AM, BartC wrote:
> On 21/11/2015 10:41, vostrushka@gmail.com wrote:
>> Hi,
>> I have a file with one parameter per line:
>> a1
>> b1
>> c1
>> a2
>> b2
>> c2
>> a3
>> b3
>> c3
>> ...
>> The parameters are lines of characters (not numbers)
>>
>> I need to load it to 2D array for further manipulations.
>> So far I managed to upload this file into 1D array:
>>
>> ParametersRaw = []
>> with open(file1) as fh:
>>      ParametersRaw = fh.readlines()
>> fh.close()
>
> I tried this code based on yours:
>
> with open("input") as fh:
>     lines=fh.readlines()
>
> rows = len(lines)//3
>
> params=[]
> index=0
>
> for row in range(rows):
>     params.append([lines[index],lines[index+1],lines[index+2]])
>     index += 3
>
> for row in range(rows):
>     print (row,":",params[row])
>
> For the exact input you gave, it produced this output:
>
> 0 : ['a1\n', 'b1\n', 'c1\n']
> 1 : ['a2\n', 'b2\n', 'c2\n']
> 2 : ['a3\n', 'b3\n', 'c3\n']
>
> Probably you'd want to get rid of those \n characters. (I don't know 
> how off-hand as I'm not often write in Python.)
>
> The last bit could also be written:
>
> for param in params:
>     print (params)
>
> but I needed the row index.
>
To get rid of the '\n' (lineend) characters:

with open(file1) as fh:
     ParametersRaw = [line.strip() for line in fh.readlines()]

or, more succinctly..

  with open(file1) as fh:
      ParametersRaw = [line.strip() for line in fh]

Comprehensions are your friend.

Nathan

[toc] | [prev] | [next] | [standalone]

#99282

Thank you all.
Here is the last piece of code that caused me so much troubles but now working the way I wanted it: 

	fRawData = []
	with open(fStagingFile2) as fStagingFile2FH:
		fRawData = [line.strip() for line in fStagingFile2FH.readlines()]         # #### This is to read each element from the file and chop off the end of line character
	
	fNumberOfColumns = 7
	fNumberOfRows = len(fRawData)/fNumberOfColumns

	fRowID = 0
	fParameters = []
	for fRowID in range(0, len(fRawData), fNumberOfColumns):
		fParameters.append(fRawData[fRowID:fRowID+fNumberOfColumns])     # #### This is to convert 1D array to 2D

    # #### ... and down below section is an example of how to read each element of the list
    # #### and how to update it if I need so. That was also a problem before.
	fRowID = 0
	fColumnID = 0
	for fRowID in range(fNumberOfRows):
		for fColumnID in range(fNumberOfColumns):
			if fColumnID == 0:
				fParameters[fRowID][fColumnID] = "XXXX"
			Message2Log("fParameters[" + str(fRowID) + "][" + str(fColumnID) + "] = " + str(fParameters[fRowID][fColumnID]))

CU

On Saturday, November 21, 2015 at 4:52:35 PM UTC+1, Nathan Hilterbrand wrote:
> On 11/21/2015 10:26 AM, BartC wrote:
> > On 21/11/2015 10:41, vostrushka@gmail.com wrote:
> >> Hi,
> >> I have a file with one parameter per line:
> >> a1
> >> b1
> >> c1
> >> a2
> >> b2
> >> c2
> >> a3
> >> b3
> >> c3
> >> ...
> >> The parameters are lines of characters (not numbers)
> >>
> >> I need to load it to 2D array for further manipulations.
> >> So far I managed to upload this file into 1D array:
> >>
> >> ParametersRaw = []
> >> with open(file1) as fh:
> >>      ParametersRaw = fh.readlines()
> >> fh.close()
> >
> > I tried this code based on yours:
> >
> > with open("input") as fh:
> >     lines=fh.readlines()
> >
> > rows = len(lines)//3
> >
> > params=[]
> > index=0
> >
> > for row in range(rows):
> >     params.append([lines[index],lines[index+1],lines[index+2]])
> >     index += 3
> >
> > for row in range(rows):
> >     print (row,":",params[row])
> >
> > For the exact input you gave, it produced this output:
> >
> > 0 : ['a1\n', 'b1\n', 'c1\n']
> > 1 : ['a2\n', 'b2\n', 'c2\n']
> > 2 : ['a3\n', 'b3\n', 'c3\n']
> >
> > Probably you'd want to get rid of those \n characters. (I don't know 
> > how off-hand as I'm not often write in Python.)
> >
> > The last bit could also be written:
> >
> > for param in params:
> >     print (params)
> >
> > but I needed the row index.
> >
> To get rid of the '\n' (lineend) characters:
> 
> with open(file1) as fh:
>      ParametersRaw = [line.strip() for line in fh.readlines()]
> 
> or, more succinctly..
> 
>   with open(file1) as fh:
>       ParametersRaw = [line.strip() for line in fh]
> 
> Comprehensions are your friend.
> 
> Nathan

[toc] | [prev] | [next] | [standalone]

#99290

On Monday, November 23, 2015 at 12:58:49 PM UTC-8, Crane Ugly wrote:
> Thank you all.
> Here is the last piece of code that caused me so much troubles but now working the way I wanted it: 
> 
> 	fRawData = []
> 	with open(fStagingFile2) as fStagingFile2FH:
> 		fRawData = [line.strip() for line in fStagingFile2FH.readlines()]         # #### This is to read each element from the file and chop off the end of line character
> 	
> 	fNumberOfColumns = 7
> 	fNumberOfRows = len(fRawData)/fNumberOfColumns
> 
> 	fRowID = 0
> 	fParameters = []
> 	for fRowID in range(0, len(fRawData), fNumberOfColumns):
> 		fParameters.append(fRawData[fRowID:fRowID+fNumberOfColumns])     # #### This is to convert 1D array to 2D
> 
>     # #### ... and down below section is an example of how to read each element of the list
>     # #### and how to update it if I need so. That was also a problem before.
> 	fRowID = 0
> 	fColumnID = 0
> 	for fRowID in range(fNumberOfRows):
> 		for fColumnID in range(fNumberOfColumns):
> 			if fColumnID == 0:
> 				fParameters[fRowID][fColumnID] = "XXXX"
> 			Message2Log("fParameters[" + str(fRowID) + "][" + str(fColumnID) + "] = " + str(fParameters[fRowID][fColumnID]))
> 
> CU
> 
> On Saturday, November 21, 2015 at 4:52:35 PM UTC+1, Nathan Hilterbrand wrote:
> > On 11/21/2015 10:26 AM, BartC wrote:
> > > On 21/11/2015 10:41, vostrushka@gmail.com wrote:
> > >> Hi,
> > >> I have a file with one parameter per line:
> > >> a1
> > >> b1
> > >> c1
> > >> a2
> > >> b2
> > >> c2
> > >> a3
> > >> b3
> > >> c3
> > >> ...
> > >> The parameters are lines of characters (not numbers)
> > >>
> > >> I need to load it to 2D array for further manipulations.
> > >> So far I managed to upload this file into 1D array:
> > >>
> > >> ParametersRaw = []
> > >> with open(file1) as fh:
> > >>      ParametersRaw = fh.readlines()
> > >> fh.close()
> > >
> > > I tried this code based on yours:
> > >
> > > with open("input") as fh:
> > >     lines=fh.readlines()
> > >
> > > rows = len(lines)//3
> > >
> > > params=[]
> > > index=0
> > >
> > > for row in range(rows):
> > >     params.append([lines[index],lines[index+1],lines[index+2]])
> > >     index += 3
> > >
> > > for row in range(rows):
> > >     print (row,":",params[row])
> > >
> > > For the exact input you gave, it produced this output:
> > >
> > > 0 : ['a1\n', 'b1\n', 'c1\n']
> > > 1 : ['a2\n', 'b2\n', 'c2\n']
> > > 2 : ['a3\n', 'b3\n', 'c3\n']
> > >
> > > Probably you'd want to get rid of those \n characters. (I don't know 
> > > how off-hand as I'm not often write in Python.)
> > >
> > > The last bit could also be written:
> > >
> > > for param in params:
> > >     print (params)
> > >
> > > but I needed the row index.
> > >
> > To get rid of the '\n' (lineend) characters:
> > 
> > with open(file1) as fh:
> >      ParametersRaw = [line.strip() for line in fh.readlines()]
> > 
> > or, more succinctly..
> > 
> >   with open(file1) as fh:
> >       ParametersRaw = [line.strip() for line in fh]
> > 
> > Comprehensions are your friend.
> > 
> > Nathan

What is the significance of prefixing all your variables with "f"?  I've frequently seen people use it to signify the variable is a float, but you're using it for things that aren't floats.

[toc] | [prev] | [next] | [standalone]

#99301

It is nothing more but naming convention. This code was part of the function.
Actually, in the final version I removed that "f" and changed some names of variables to shorter versions.
It is all about your own preferences towards readability of the code and ability of visual capture of the code. I always try to have code that easy to read. At least for myself.

CU

> What is the significance of prefixing all your variables with "f"?  I've frequently seen people use it to > signify the variable is a float, but you're using it for things that aren't floats.

[toc] | [prev] | [next] | [standalone]

#99303

Crane Ugly wrote:

> I always try to have code that easy to read. 

That's laudable, code is a means of communication.

> At least for myself.

and only for yourself, unfortunately.

Recommended read:

https://www.python.org/dev/peps/pep-0008/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web