Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92137 > unrolled thread

So what's happening here?

Back to article view | Back to comp.lang.python

Contents

  So what's happening here? Paul Appleby <pap@nowhere.invalid> - 2015-06-05 12:46 +0000
    Re: So what's happening here? Fabien <fabien.maussion@gmail.com> - 2015-06-05 14:52 +0200
    Re: So what's happening here? Larry Martell <larry.martell@gmail.com> - 2015-06-05 08:55 -0400
    Re: So what's happening here? Peter Otten <__peter__@web.de> - 2015-06-05 15:09 +0200
    Re: So what's happening here? Paul Appleby <pap@nowhere.invalid> - 2015-06-05 13:11 +0000
      Re: So what's happening here? Gary Herron <gary.herron@islandtraining.com> - 2015-06-05 06:23 -0700
      Re: So what's happening here? Steven D'Aprano <steve@pearwood.info> - 2015-06-05 23:51 +1000
      Re: So what's happening here? Nobody <nobody@nowhere.invalid> - 2015-06-05 15:13 +0100

#92137 — So what's happening here?

I saw somewhere on the net that you can copy a list with slicing. So 
what's happening when I try it with a numpy array?

>>> a = numpy.array([1,2,3])
>>> b = a[:]
>>> a is b
False
>>> b[1] = 9
>>> a
array([1, 9, 3])

[toc] | [next] | [standalone]

#92138

On 06/05/2015 02:46 PM, Paul Appleby wrote:
> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?

Python lists and numpy arrays are NOT the same thing. This is one of the 
reasons why numpy was developed in the first place. Numpy uses views, 
not copies. There is however a .copy() method on numpy arrays

Fabien

[toc] | [prev] | [next] | [standalone]

#92139

On Fri, Jun 5, 2015 at 8:46 AM, Paul Appleby <pap@nowhere.invalid> wrote:
> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?
>
>>>> a = numpy.array([1,2,3])
>>>> b = a[:]
>>>> a is b
> False
>>>> b[1] = 9
>>>> a
> array([1, 9, 3])

is is identity testing, == is equality testing

>>> a = numpy.array([1,2,3])
>>> b = a[:]
>>> a == b
array([ True,  True,  True], dtype=bool)
>>> id(a)
4510409872
>>> id(b)
4510410192

[toc] | [prev] | [next] | [standalone]

#92141

Paul Appleby wrote:

> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?
> 
>>>> a = numpy.array([1,2,3])
>>>> b = a[:]
>>>> a is b
> False
>>>> b[1] = 9
>>>> a
> array([1, 9, 3])

Copy or view -- have a look under the hood:

>>> a = numpy.array([1,2,3])
>>> v = a.view()
>>> c = a.copy()
>>> s = a[:]
>>> a.flags["OWNDATA"]
True
>>> v.flags["OWNDATA"]
False
>>> c.flags["OWNDATA"]
True
>>> s.flags["OWNDATA"]
False

You only get a copy if you ask for one; slicing produces a view.

[toc] | [prev] | [next] | [standalone]

#92142

On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:

> Numpy arrays are not lists, they are numpy arrays. They are two
> different data types with different behaviors.  In lists, slicing is a
> copy.  In numpy arrays, it is a view (a data structure representing some
> part of another data structure).  You need to explicitly copy the numpy
> array using the "copy" method to get a copy rather than a view:

OK, thanks.  I see. 

(I'd have thought that id(a[1]) and id(b[1]) would be the same if they 
were the same element via different "views", but the id's seem to change 
according to rules that I can't fathom.)

[toc] | [prev] | [next] | [standalone]

#92146

On 06/05/2015 06:11 AM, Paul Appleby wrote:
> On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
>
>> Numpy arrays are not lists, they are numpy arrays. They are two
>> different data types with different behaviors.  In lists, slicing is a
>> copy.  In numpy arrays, it is a view (a data structure representing some
>> part of another data structure).  You need to explicitly copy the numpy
>> array using the "copy" method to get a copy rather than a view:
> OK, thanks.  I see.
>
> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)
Nope.  It's odder than that.  a[1] is still a view into the inderlying 
numpy array, and your id is the id of that view. Each such index 
produces a new such view object.  Check this out:

 >>> import numpy
 >>> a = numpy.array([1,2,3])
 >>> id(a[1])
28392768
 >>> id(a[1])
28409872

This produces two different view of the same underlying object.

Gary Herron

[toc] | [prev] | [next] | [standalone]

#92149

On Fri, 5 Jun 2015 11:11 pm, Paul Appleby wrote:

> On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
> 
>> Numpy arrays are not lists, they are numpy arrays. They are two
>> different data types with different behaviors.  In lists, slicing is a
>> copy.  In numpy arrays, it is a view (a data structure representing some
>> part of another data structure).  You need to explicitly copy the numpy
>> array using the "copy" method to get a copy rather than a view:
> 
> OK, thanks.  I see.
> 
> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)

They're the same element, but not the same object.

The id() function operates on an object, and returns some arbitrary ID
number for that object. The only thing promised about that ID number is
that for any two distinct objects existing at the same time, they will have
distinct IDs.

Now, let's see what happens when we extract elements from a regular Python
list:

py> a = [1, 2, 3]
py> x = a[0]
py> y = a[0]
py> x is y
True
py> id(x) == id(y)
True

This tells us that extracting the first (or zeroth, if you prefer) element
from the list gives us the same object each time.

Now let's try it with a numpy array:

py> import numpy as np
py> b = np.array([1, 2, 3])
py> x = b[0]
py> y = b[0]
py> x is y
False
py> id(x), id(y)
(149859472, 151810312)


The IDs are clearly different, therefore they are different objects. What's
going on?

The secret is that lists contain objects, so when you extract the zeroth
item using a[0], you get the same object each time. But numpy arrays do not
contain objects, they are a wrapper around a C array of machine ints.

(The numpy array itself is an object, but the elements of that array are
not.)

This is one of the reasons why numpy is so fast: it can bypass all the
high-level Python object-oriented machinery, and perform calculations using
high-speed, low-level C code taking advantage of unboxed machine primitive
values.

But when you extract an element using b[0], numpy has to give you an object.
(Python itself has no concept of low-level machine values.) So it grabs the
32-bit integer at offset 0, converts it into an object, and returns that.
When you do it again, numpy goes through the same process, and returns a
second object with the same numeric value. Hence, the IDs are different.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#92153

On Fri, 05 Jun 2015 13:11:13 +0000, Paul Appleby wrote:

> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)

First, a[1] and b[1] aren't views, they're scalars.

Second, different views on the same data are different objects, they just
share the same underlying data. Consider the case where the slice doesn't
cover the entire range:

> a = np.array([1,2,3])
> b = a[:2]
> a
array([1, 2, 3])
> b
array([1, 2])
> id(a)
139682716078288
> id(b)
139682716078368
> b[0] = 99
> a
array([99,  2,  3])
> b
array([99,  2])

The case where a slice *does* cover the entire range isn't special; the
resulting view is still a different object.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web