Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #29858 > unrolled thread

List Problem

Back to article view | Back to comp.lang.python

Contents

  List Problem jimbo1qaz <jimmyli1528@gmail.com> - 2012-09-23 14:31 -0700
    Re: List Problem jimbo1qaz <jimmyli1528@gmail.com> - 2012-09-23 14:44 -0700
      Re: List Problem Chris Angelico <rosuav@gmail.com> - 2012-09-24 07:57 +1000
      Re: List Problem Dave Angel <d@davea.name> - 2012-09-23 18:03 -0400
      Re: List Problem Chris Angelico <rosuav@gmail.com> - 2012-09-24 08:27 +1000
        Re: List Problem Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-09-23 22:37 +0000
          Re: List Problem Chris Angelico <rosuav@gmail.com> - 2012-09-24 08:45 +1000
      Re: List Problem Chris Angelico <rosuav@gmail.com> - 2012-09-24 08:30 +1000
      Re: List Problem "Littlefield, Tyler" <tyler@tysdomain.com> - 2012-09-23 18:56 -0600
      Re: List Problem Chris Angelico <rosuav@gmail.com> - 2012-09-24 11:52 +1000
    Re: List Problem Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-09-23 22:07 +0000
    Re: List Problem jimbo1qaz <jimmyli1528@gmail.com> - 2012-09-23 15:44 -0700

#29858 — List Problem

I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
Link to broken code: http://jimbopy.pastebay.net/1090401

[toc] | [next] | [standalone]

#29860

On Sunday, September 23, 2012 2:31:48 PM UTC-7, jimbo1qaz wrote:
> I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
> 
> Link to broken code: http://jimbopy.pastebay.net/1090401

No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

[toc] | [prev] | [next] | [standalone]

#29861

On Mon, Sep 24, 2012 at 7:44 AM, jimbo1qaz <jimmyli1528@gmail.com> wrote:
> On Sunday, September 23, 2012 2:31:48 PM UTC-7, jimbo1qaz wrote:
>> I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
>>
>> Link to broken code: http://jimbopy.pastebay.net/1090401
>
> No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

The first thing I'd change about that code is the whole thing of using
try/exec/except to suppress IndexError. Definitely not good code. I'm
not wholly certain, but I think you might run into weird issues with
negative OOBounds indices (since Python treats a negative list index
as counting from the far end).

This is nothing to do with your originally requested issue, which I
can't see the cause of in your script there. But when you assign a
list, you just get another reference to the same list.

ChrisA

[toc] | [prev] | [next] | [standalone]

#29862

On 09/23/2012 05:44 PM, jimbo1qaz wrote:
> On Sunday, September 23, 2012 2:31:48 PM UTC-7, jimbo1qaz wrote:
>> I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
>>
>> Link to broken code: http://jimbopy.pastebay.net/1090401
> No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

I also would prefer an inline posting of the code, but if it's too big
to post here, it's probably too big for me to debug here.

The usual reason for such a symptom is a nested list, where you have
multiple references to the same inner list inside the outer.  When you
change one of those, you change all of them.

alist = [1, 2, 3]
blist = [alist, alist, alist]   #  or blist = alist * 3
print blist
alist.append(49)
print blist

davea@think:~/temppython$ python jimbo.py
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3, 49], [1, 2, 3, 49], [1, 2, 3, 49]]

Solution to this is to make sure that only copies of alist get into
blist.  One way is

blist = [alist[:], alist[:], alist[:]]

More generally, you can get into this type of trouble whenever you have
non-immutable objects inside the list.

Understand, this is NOT a flaw in the language.  It's perfectly
reasonable to be able to do so, in fact essential in many cases, when
you want it to be the SAME item.


-- 

DaveA

[toc] | [prev] | [next] | [standalone]

#29865

On Mon, Sep 24, 2012 at 8:03 AM, Dave Angel <d@davea.name> wrote:
> blist = [alist, alist, alist]   #  or blist = alist * 3

(Minor point: I think you mean this.)
#  or blist = [alist] * 3

> Understand, this is NOT a flaw in the language.  It's perfectly
> reasonable to be able to do so, in fact essential in many cases, when
> you want it to be the SAME item.

And this is the real part. There's no other way to handle complex
objects that makes as much sense. PHP's system of references and
copy-on-write assignment doesn't truly cover all cases, and it can
make operations unexpectedly run vastly faster or slower depending on
external circumstances, which gets annoying (the first write to an
assigned array has to copy the array). C simply doesn't let you move
arrays around, only pointers to them, so semantics are actually pretty
similar to high level languages, only in a completely different way.

These sorts of issues only ever seem to crop up with nested arrays,
which strengthens this next point: Deep copying is a really REALLY
hairy concept. It seems so simple at first:

a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
b = deepcopy(a) # b is a new list with three new sublists

But then it gets messy.

a = [[1, 2, 3]]*2 + [[7, 8, 9]]

It's much better to make copying a very explicit thing; it's so
expensive that you really should make it very clear in your code when
this happens.

ChrisA

[toc] | [prev] | [next] | [standalone]

#29867

On Mon, 24 Sep 2012 08:27:28 +1000, Chris Angelico wrote:

> C simply doesn't let you move
> arrays around, only pointers to them, so semantics are actually pretty
> similar to high level languages, only in a completely different way.

I once dated a girl who looked exactly like Scarlett Johannsen only 
completely different.

Pascal let's you pass arrays around either by value (which copies them, 
and may be expensive) or by reference (which doesn't), neither of which 
is what Python (or Java, Ruby, etc.) do. Passing pointers by value is not 
the same as Python object semantics.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#29869

On Mon, Sep 24, 2012 at 8:37 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Mon, 24 Sep 2012 08:27:28 +1000, Chris Angelico wrote:
>
>> C simply doesn't let you move
>> arrays around, only pointers to them, so semantics are actually pretty
>> similar to high level languages, only in a completely different way.
>
> I once dated a girl who looked exactly like Scarlett Johannsen only
> completely different.
>
> Pascal let's you pass arrays around either by value (which copies them,
> and may be expensive) or by reference (which doesn't), neither of which
> is what Python (or Java, Ruby, etc.) do. Passing pointers by value is not
> the same as Python object semantics.

If your arrays are allocated on the heap, you can pretty much treat
the pointer like a HLL list object. The only real difference is that
C, by default, won't garbage-collect your heap allocations, so you'll
leak memory. The behaviours you observe will be pretty similar,
though.

ChrisA

[toc] | [prev] | [next] | [standalone]

#29866

On Mon, Sep 24, 2012 at 8:27 AM, Chris Angelico <rosuav@gmail.com> wrote:
> a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
> b = deepcopy(a) # b is a new list with three new sublists
>

Oops, this should be:

b = copy.deepcopy(a)

as in Steven's post. And in case I wasn't clear about it, the
deepcopy() function does deal with the issue I mention, but that's
part of why it's an expensive operation.

ChrisA

[toc] | [prev] | [next] | [standalone]

#29877

On 9/23/2012 3:44 PM, jimbo1qaz wrote:
> On Sunday, September 23, 2012 2:31:48 PM UTC-7, jimbo1qaz wrote:
>> I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
>>
>> Link to broken code: http://jimbopy.pastebay.net/1090401
> No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

I've not been following this thread fully, but why not just use 
x=list(y) to copy the list?
The issue is that when you assign i=[1,2,3] and then j = i, j is just a 
reference to i, which is why you change either and the other changes.

-- 
Take care,
Ty
http://tds-solutions.net
The aspen project: a barebones light-weight mud engine:
http://code.google.com/p/aspenmud
He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

[toc] | [prev] | [next] | [standalone]

#29882

On Mon, Sep 24, 2012 at 10:56 AM, Littlefield, Tyler
<tyler@tysdomain.com> wrote:
> I've not been following this thread fully, but why not just use x=list(y) to
> copy the list?
> The issue is that when you assign i=[1,2,3] and then j = i, j is just a
> reference to i, which is why you change either and the other changes.

The problem is with lists as elements of that list, so the key is deepcopy.

ChrisA

[toc] | [prev] | [next] | [standalone]

#29863

On Sun, 23 Sep 2012 14:31:48 -0700, jimbo1qaz wrote:

> I have a nested list. Whenever I make a copy of the list, changes in one
> affect the other, 

Then you aren't making a copy.

py> first_list = [1, 2, 3]
py> second_list = first_list  # THIS IS NOT A COPY
py> second_list.append(9999)
py> print first_list
[1, 2, 3, 9999]


> even when I use list(orig) 

Nonsense. Either you are confused, or there is something you aren't 
telling us. Calling list *does* make a copy:

py> first_list = [1, 2, 3]
py> second_list = list(first_list)
py> second_list.append(9999)
py> print first_list
[1, 2, 3]

What aren't you telling us? My guess is that there are TWO lists 
involved, and you're only copying ONE:


py> a = [1, 2, 3]
py> a.append(["ham", "spam", "cheese"])
py> print a
[1, 2, 3, ['ham', 'spam', 'cheese']]
py> b = list(a)  # make a copy of a, but not the contents of a
py> b.append(99)  # change b
py> b[-1].append("tomato")
py> print a
[1, 2, 3, ['ham', 'spam', 'cheese', 'tomato']]

Notice that b is a copy of a: changing b does not change a. But the 
embedded list within a is *not* copied, so whether you append to that 
list via a or b is irrelevant, both see the same change because there is 
only one inner list in two places.

It might be more obvious if you give the shared sublist a name:

c = ['ham', 'spam', 'tomato']
a = [1, 2, 3, c]
b = [1, 2, 3, c]  # a copy of a, but not a copy of c

Now it should be obvious that any changes to c will show up in both a and 
b, regardless of how you change it. All three of these will have the 
exact same effect:

a[-1].append('eggs')
b[-1].append('eggs')
c.append('eggs')


The way to copy lists, and all their sublists, and their sublists, and so 
on all the way down, is with the copy module:

import copy
b = copy.deepcopy(a)

but if you are doing this a lot, (1) your code will be slow, and (2) you 
can probably redesign your code to avoid so many copies.

By the way, instead of dumping lots of irrelevant code on us, please take 
the time to narrow your problem down to the smallest possible piece of 
code. We're volunteers here, and you are not paying us to wade through 
your code trying to determine where your problem lies. That is up to you: 
you narrow down to the actual problem, then ask for help.

Please read http://sscce.org/ for more details of how, and why, you 
should do this.


Thank you.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#29868

On Sunday, September 23, 2012 2:31:48 PM UTC-7, jimbo1qaz wrote:
> I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
> 
> Link to broken code: http://jimbopy.pastebay.net/1090401

OK, deepcopy fixed it!
And I fixed the catch indexerror thing too.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web