Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #3700 > unrolled thread

Re: dictionary size changed during iteration

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: dictionary size changed during iteration Peter Otten <__peter__@web.de> - 2011-04-20 15:33 +0200
    Re: dictionary size changed during iteration Roy Smith <roy@panix.com> - 2011-04-22 09:15 -0400
      Re: dictionary size changed during iteration Laszlo Nagy <gandalf@shopzeus.com> - 2011-05-06 18:21 +0200
      Re: dictionary size changed during iteration Paul Rubin <no.email@nospam.invalid> - 2011-05-07 14:07 -0700
        Re: dictionary size changed during iteration Roy Smith <roy@panix.com> - 2011-05-07 18:12 -0400
          Re: dictionary size changed during iteration Hans Mulder <hansmu@xs4all.nl> - 2011-05-08 21:32 +0200
            Re: dictionary size changed during iteration Paul Rubin <no.email@nospam.invalid> - 2011-05-08 12:42 -0700

#3700 — Re: dictionary size changed during iteration

Laszlo Nagy wrote:

> Given this iterator:
> 
> class SomeIterableObject(object):
>      ....
>      ....
> 
>      def __iter__(self):
>          ukeys = self.updates.keys()
>          for key in ukeys:
>              if self.updates.has_key(key):
>                  yield self.updates[key]
>          for rec in self.inserts:
>              yield rec
>      ....
>      ....
> 
> How can I get this exception:
> 
> RuntimeError: dictionary changed size during iteration
> 
> 
> It is true that self.updates is being changed during the iteration. But
> I have created the "ukeys" variable solely to prevent this kind of
> error. Here is a proof of correctness:
> 
>>>>  d = {1:1,2:2}
>>>>  k = d.keys()
>>>>  del d[1]
>>>>  k
> [1, 2]
>>>>  k is d.keys()
> False
> 
> So what is wrong with this iterator? Why am I getting this error message?

The keys() method which used to return a list in 2.x was changed in 3.x to 
return a view object and to become more or less the equivalent of the old 
dict.iterkeys():

>>> d = dict(a=1)
>>> keys = d.keys()
>>> keys
dict_keys(['a'])
>>> for k in keys:
...     d["b"] = 42
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
>>> keys
dict_keys(['a', 'b'])

You now have to create the list explicitly to avoid the error:

>>> d = dict(a=1)
>>> keys = list(d.keys())
>>> for k in keys:
...     d["b"] = 42
...
>>> d
{'a': 1, 'b': 42}
>>> keys
['a']

[toc] | [next] | [standalone]

#3862

In article <mailman.644.1303306435.9059.python-list@python.org>,
 Peter Otten <__peter__@web.de> wrote:

> You now have to create the list explicitly to avoid the error:
> 
> >>> d = dict(a=1)
> >>> keys = list(d.keys())
> >>> for k in keys:
> ...     d["b"] = 42
> ...

That works, but if d is large, it won't be very efficient because it has 
to generate a large list.

If d is large, and the number of keys to be mutated is relatively small, 
a better solution may be to do it in two passes.  The first loop 
traverses the iterator and builds a list of things to be changed.  The 
second loop changes them.

changes = [ ]
for key in d.iterkeys():
  if is_bad(key):
    changes.append(key)
for key in changes:
  d[key] = "I'm not dead yet"

Both solutions are O(n), but the second may run significantly faster and 
use less memory.

[toc] | [prev] | [next] | [standalone]

#4917

>> ...
> That works, but if d is large, it won't be very efficient because it has
> to generate a large list.
It is not large. But I'm using Python 2.6 , not Python 3.

I did not get this error again in the last two days. I'll post a new 
reply if I encounter it again. (It happened just a few times out of many 
thousand program invocations)

[toc] | [prev] | [next] | [standalone]

#4920

Roy Smith <roy@panix.com> writes:
> changes = [ ]
> for key in d.iterkeys():
>   if is_bad(key):
>     changes.append(key)

    changes = list(k for k in d if is_bad(k))

is a little bit more direct.

[toc] | [prev] | [next] | [standalone]

#4922

In article <7xd3jukyn9.fsf@ruckus.brouhaha.com>,
 Paul Rubin <no.email@nospam.invalid> wrote:

> Roy Smith <roy@panix.com> writes:
> > changes = [ ]
> > for key in d.iterkeys():
> >   if is_bad(key):
> >     changes.append(key)
> 
>     changes = list(k for k in d if is_bad(k))
> 
> is a little bit more direct.

This is true.  I still file list comprehensions under "new fangled 
toys".  While I use them, and appreciate their value, I admit they're 
not always the first thing that comes to my mind.

OBTW,

>     changes = [k for k in d if is_bad(k)]

is even more direct :-)

[toc] | [prev] | [next] | [standalone]

#4963

On 08/05/2011 00:12, Roy Smith wrote:
> In article<7xd3jukyn9.fsf@ruckus.brouhaha.com>,
>   Paul Rubin<no.email@nospam.invalid>  wrote:
>
>> Roy Smith<roy@panix.com>  writes:
>>> changes = [ ]
>>> for key in d.iterkeys():
>>>    if is_bad(key):
>>>      changes.append(key)
>>
>>      changes = list(k for k in d if is_bad(k))
>>
>> is a little bit more direct.
>
> This is true.  I still file list comprehensions under "new fangled
> toys".  While I use them, and appreciate their value, I admit they're
> not always the first thing that comes to my mind.
>
> OBTW,
>
>>      changes = [k for k in d if is_bad(k)]
>
> is even more direct :-)

How about:

	changes = filter(is_bad, d)

Or would that be too compact?

-- HansM

[toc] | [prev] | [next] | [standalone]

#4964

Hans Mulder <hansmu@xs4all.nl> writes:
> How about:
> 	changes = filter(is_bad, d)
> Or would that be too compact?

I thought of writing something like that but filter in python 3 creates
an iterator that would have the same issue of walking the dictionary
while the dictionary is mutating.

    changes = list(filter(is_bad, d))

should work.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web