Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #105531 > unrolled thread

numpy arrays

Back to article view | Back to comp.lang.python

Contents

  numpy arrays Heli <hemla21@gmail.com> - 2016-03-23 03:06 -0700
    Re: numpy arrays Steven D'Aprano <steve@pearwood.info> - 2016-03-23 21:32 +1100
    Re: numpy arrays Chris Angelico <rosuav@gmail.com> - 2016-03-23 21:35 +1100
      Re: numpy arrays Nobody <nobody@nowhere.invalid> - 2016-03-23 13:45 +0000
        Re: numpy arrays Heli <hemla21@gmail.com> - 2016-04-06 09:26 -0700
          Re: numpy arrays Heli <hemla21@gmail.com> - 2016-04-07 07:31 -0700
            Re: numpy arrays Heli <hemla21@gmail.com> - 2016-04-11 02:17 -0700
            Re: numpy arrays Heli <hemla21@gmail.com> - 2016-04-11 02:32 -0700
    Re: numpy arrays Manolo Martínez <manolo@austrohungaro.com> - 2016-03-23 11:26 +0100
    Re: numpy arrays Heli <hemla21@gmail.com> - 2016-03-23 03:47 -0700
    Re: numpy arrays Simon Ward <simon+python@bleah.co.uk> - 2016-03-23 10:47 +0000

#105531 — numpy arrays

Hi, 

I have a 2D numpy array like this:

[[1,2,3,4],
 [1,2,3,4], 
 [1,2,3,4]
 [1,2,3,4]]

Is there any fast way to convert this array to 

[[1,1,1,1],
 [2,2,2,2]
 [3,3,3,3]
 [4,4,4,4]]

In general I would need to retrieve every nth element of the interior arrays in to single arrays. I know I can loop over and do this, but I have really big arrays and I need the fastest way to do this. 

Thanks for your help,  

[toc] | [next] | [standalone]

#105532

On Wed, 23 Mar 2016 09:06 pm, Heli wrote:

> Hi,
> 
> I have a 2D numpy array like this:
> 
> [[1,2,3,4],
>  [1,2,3,4],
>  [1,2,3,4]
>  [1,2,3,4]]
> 
> Is there any fast way to convert this array to
> 
> [[1,1,1,1],
>  [2,2,2,2]
>  [3,3,3,3]
>  [4,4,4,4]]



Mathematically, this is called the "transpose" of the array, and you can do
that in numpy:

py> import numpy as np
py> arr = np.array(
... [[1,2,3,4],
...  [1,2,3,4],
...  [1,2,3,4],
...  [1,2,3,4]])
py> arr
array([[1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4]])
py> arr.T
array([[1, 1, 1, 1],
       [2, 2, 2, 2],
       [3, 3, 3, 3],
       [4, 4, 4, 4]])



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#105534

On Wed, Mar 23, 2016 at 9:06 PM, Heli <hemla21@gmail.com> wrote:
> I have a 2D numpy array like this:
>
> [[1,2,3,4],
>  [1,2,3,4],
>  [1,2,3,4]
>  [1,2,3,4]]
>
> Is there any fast way to convert this array to
>
> [[1,1,1,1],
>  [2,2,2,2]
>  [3,3,3,3]
>  [4,4,4,4]]

What you want is called *transposing* the array:

http://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html

That should be a sufficiently fast operation.

ChrisA

[toc] | [prev] | [next] | [standalone]

#105583

> What you want is called *transposing* the array:
> 
> http://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html
> 
> That should be a sufficiently fast operation.

Transposing itself is fast, as it just swaps the strides and dimensions
without touching the data (i.e. it returns a new view of the original
array), but subsequent operations may be slower as the data is no longer
contiguous (i.e. iterating over the flattened array in order won't result
in sequential memory access).

If that's an issue, you can use numpy.ascontiguousarray() to make a
contiguous copy of the data.

[toc] | [prev] | [next] | [standalone]

#106587

Thanks for your replies. I have a question in regard with my previous question. I have a file that contains x,y,z and a value for that coordinate on each line. Here I am giving an example of the file using a numpy array called f. 

f=np.array([[1,1,1,1],
            [1,1,2,2], 
            [1,1,3,3],
            [1,2,1,4],
            [1,2,2,5],
            [1,2,3,6],
            [1,3,1,7],
            [1,3,2,8],
            [1,3,3,9],
            [2,1,1,10],
            [2,1,2,11],
            [2,1,3,12],
            [2,2,1,13],
            [2,2,2,14],
            [2,2,3,15],
            [2,3,1,16],
            [2,3,2,17],
            [2,3,3,18],
            [3,1,1,19],
            [3,1,2,20],
            [3,1,3,21],
            [3,2,1,22],
            [3,2,2,23],
            [3,2,3,24],
            [3,3,1,25],
            [3,3,2,26],
            [3,3,3,27],
            ])

then after tranposing f, I get the x,y and z coordinates:
f_tranpose=f.T
x=np.sort(np.unique(f_tranpose[0]))
y=np.sort(np.unique(f_tranpose[1]))
z=np.sort(np.unique(f_tranpose[2]))

Then I will create a 3D array to put the values inside. The only way I see to do this is the following:
arr_size=x.size
val2=np.empty([3, 3,3])

for sub_arr in f:
    idx = (np.abs(x-sub_arr[0])).argmin()
    idy = (np.abs(y-sub_arr[1])).argmin()
    idz = (np.abs(z-sub_arr[2])).argmin()
    val2[idx,idy,idz]=sub_arr[3]

I know that in the example above I could simple reshape f_tranpose[3] to a three by three by three array, but in my real example the coordinates are not in order and the only way I see to do this is by looping over the whole file which takes a lot of time. 

I would appreciate any workarounds to make this quicker. 

Thanks,

[toc] | [prev] | [next] | [standalone]

#106630

Thanks a lot Oscar, 

The lexsort you suggested was the way to go. 

import h5py
import numpy as np
f=np.loadtxt(inputFile,delimiter=None)
xcoord=np.sort(np.unique(f[:,0]))
ycoord=np.sort(np.unique(f[:,1]))
zcoord=np.sort(np.unique(f[:,2]))

x=f[:,0]
y=f[:,1]
z=f[:,2]
val=f[:,3]

ind = np.lexsort((val,z,y,x)) # Sort by x, then by y, then by z, then by val
sortedVal=np.array([(val[i]) for i in ind]).reshape((xcoord.size,ycoord.size,zcoord.size))

[toc] | [prev] | [next] | [standalone]

#106868

Thanks Oscar, 

In my case this did the trick. 
sortedVal=np.array(val[ind]).reshape((xcoord.size,ycoord.size,zcoord.size))

[toc] | [prev] | [next] | [standalone]

#106870

As you said, this did the trick. 
sortedVal=np.array(val[ind]).reshape((xcoord.size,ycoord.size,zcoord.size)) 

Only val[ind] instead of val[ind,:] as val is 1D. 

Thanks Oscar, 

[toc] | [prev] | [next] | [standalone]

#105535

On 03/23/16 at 03:06am, Heli wrote:

> I have a 2D numpy array like this:
> 
> [[1,2,3,4],
>  [1,2,3,4], 
>  [1,2,3,4]
>  [1,2,3,4]]
> 
> Is there any fast way to convert this array to 
> 
> [[1,1,1,1],
>  [2,2,2,2]
>  [3,3,3,3]
>  [4,4,4,4]]

You don't mean just transposing your original array, as in
original_array.T, right?

> In general I would need to retrieve every nth element of the interior
> arrays in to single arrays. I know I can loop over and do this, but I
> have really big arrays and I need the fastest way to do this. 

I didn't really get this, so transposing is probably not what you need.
Anyway, just in case!

Manolo

[toc] | [prev] | [next] | [standalone]

#105536

On Wednesday, March 23, 2016 at 11:07:27 AM UTC+1, Heli wrote:
> Hi, 
> 
> I have a 2D numpy array like this:
> 
> [[1,2,3,4],
>  [1,2,3,4], 
>  [1,2,3,4]
>  [1,2,3,4]]
> 
> Is there any fast way to convert this array to 
> 
> [[1,1,1,1],
>  [2,2,2,2]
>  [3,3,3,3]
>  [4,4,4,4]]
> 
> In general I would need to retrieve every nth element of the interior arrays in to single arrays. I know I can loop over and do this, but I have really big arrays and I need the fastest way to do this. 
> 
> Thanks for your help,

Thanks a lot everybody, 

Transposing is exactly what I want. I just was not sure if that would work on internal arrays of a 2D array. Problem solved thanks everyone.

[toc] | [prev] | [next] | [standalone]

#105541

On 23 March 2016 10:06:56 GMT+00:00, Heli <hemla21@gmail.com> wrote:
>Hi, 
>
>I have a 2D numpy array like this:
>
>[[1,2,3,4],
> [1,2,3,4], 
> [1,2,3,4]
> [1,2,3,4]]
>
>Is there any fast way to convert this array to 
>
>[[1,1,1,1],
> [2,2,2,2]
> [3,3,3,3]
> [4,4,4,4]]

Use the transpose() method:
http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.ndarray.transpose.html

Simon
-- 
Sent from Kaiten Mail. Please excuse my brevity.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web