Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #31034 > unrolled thread

surprising behaviour of global dictionaries

Back to article view | Back to comp.lang.python

Contents

  surprising behaviour of global dictionaries Michele Simionato <michele.simionato@gmail.com> - 2012-10-09 08:08 -0700
    Re: surprising behaviour of global dictionaries Peter Otten <__peter__@web.de> - 2012-10-09 17:24 +0200
      Re: surprising behaviour of global dictionaries Michele Simionato <michele.simionato@gmail.com> - 2012-10-09 08:36 -0700
        Re: surprising behaviour of global dictionaries Dave Angel <d@davea.name> - 2012-10-09 12:10 -0400
      Re: surprising behaviour of global dictionaries Michele Simionato <michele.simionato@gmail.com> - 2012-10-09 08:36 -0700
      Re: surprising behaviour of global dictionaries Grant Edwards <invalid@invalid.invalid> - 2012-10-09 15:53 +0000
        Re: surprising behaviour of global dictionaries Peter Otten <__peter__@web.de> - 2012-10-09 19:45 +0200

#31034 — surprising behaviour of global dictionaries

I have the following module implementing a registry of functions with a decorator:

$ cat x.py
registry = {} # global dictionary

def dec(func):
    registry[func.__name__] = func
    print registry, id(registry)
    return func

if __name__ == '__main__':
    import xlib
    print registry, id(registry)

The library xlib just defines two dummy functions:

$ cat xlib.py
from x import dec

@dec
def f1():
    pass

@dec
def f2():
    pass

Then I get the following output:

$ python x.py
{'f1': <function f1 at 0x7f7bce0cd668>} 27920352
{'f1': <function f1 at 0x7f7bce0cd668>, 'f2': <function f2 at 0x7f7bce0cd6e0>} 27920352
{} 27395472

This is surprising since I would expect to have a single global dictionary, not two: how comes the registry inside the ``if __name__ == '__main__'`` block is different from the one seen in the library?

This is python 2.7.3 on Ubuntu.

[toc] | [next] | [standalone]

#31038

Michele Simionato wrote:

> I have the following module implementing a registry of functions with a
> decorator:
> 
> $ cat x.py
> registry = {} # global dictionary
> 
> def dec(func):
>     registry[func.__name__] = func
>     print registry, id(registry)
>     return func
> 
> if __name__ == '__main__':
>     import xlib
>     print registry, id(registry)
> 
> The library xlib just defines two dummy functions:
> 
> $ cat xlib.py
> from x import dec
> 
> @dec
> def f1():
>     pass
> 
> @dec
> def f2():
>     pass
> 
> Then I get the following output:
> 
> $ python x.py
> {'f1': <function f1 at 0x7f7bce0cd668>} 27920352
> {'f1': <function f1 at 0x7f7bce0cd668>, 'f2': <function f2 at
> {0x7f7bce0cd6e0>} 27920352 } 27395472
> 
> This is surprising since I would expect to have a single global
> dictionary, not two: how comes the registry inside the ``if __name__ ==
> '__main__'`` block is different from the one seen in the library?
> 
> This is python 2.7.3 on Ubuntu.

Welcome to python -- this is a trap every newbie falls into ;)

Seriously, you shouldn't use the main script as a library; it is put into 
the sys.modules cache under the "__main__" key. Subsequent imports under its 
real name will not find that name in the cache and import another instance 
of the module, with puzzling effects, like

$ cat x.py 
import x
class A: pass
a = A()
assert isinstance(a, x.A)

$ python x.py 
Traceback (most recent call last):
  File "x.py", line 4, in <module>
    assert isinstance(a, x.A)
AssertionError
$ 

[toc] | [prev] | [next] | [standalone]

#31040

On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
> Seriously, you shouldn't use the main script as a library; it is put into 
> 
> the sys.modules cache under the "__main__" key. Subsequent imports under its 
> 
> real name will not find that name in the cache and import another instance 
> 
> of the module, with puzzling effects

Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.

[toc] | [prev] | [next] | [standalone]

#31046

On 10/09/2012 11:36 AM, Michele Simionato wrote:
> On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
>> Seriously, you shouldn't use the main script as a library; it is put into 
>>
>> the sys.modules cache under the "__main__" key. Subsequent imports under its 
>>
>> real name will not find that name in the cache and import another instance 
>>
>> of the module, with puzzling effects
> Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.
More generally, you should arrange your imports so that there are no
cycles.  If module/script "a" imports "b", directly or indirectly, "b"
should not try to import "a."  Move the common code to a third place,
and import it from both "a" and from "b".

The other symptoms you can get are more subtle than this one, but just
as surprising.


-- 

DaveA

[toc] | [prev] | [next] | [standalone]

#31041

On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
> Seriously, you shouldn't use the main script as a library; it is put into 
> 
> the sys.modules cache under the "__main__" key. Subsequent imports under its 
> 
> real name will not find that name in the cache and import another instance 
> 
> of the module, with puzzling effects

Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.

[toc] | [prev] | [next] | [standalone]

#31043

On 2012-10-09, Peter Otten <__peter__@web.de> wrote:

> Welcome to python -- this is a trap every newbie falls into ;)
>
> Seriously, you shouldn't use the main script as a library;

There must be something wrong with me.  It never even occurred to me
to try to import a file from within that same file.  I don't think
I've ever even heard of that before...

-- 
Grant Edwards               grant.b.edwards        Yow! The Korean War must
                                  at               have been fun.
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#31051

Grant Edwards wrote:

> On 2012-10-09, Peter Otten <__peter__@web.de> wrote:
> 
>> Welcome to python -- this is a trap every newbie falls into ;)
>>
>> Seriously, you shouldn't use the main script as a library;
> 
> There must be something wrong with me.  It never even occurred to me
> to try to import a file from within that same file.  

It is typically done in two steps: 

(1) Write module x, use it in module y.
(2) For convenience add "if __name__ == '__main__'" and import module y.

Hilarity ensues.

> I don't think I've ever even heard of that before...

As I was poking fun at Michele who really is an expert and likely knows more 
about Python than I do I may have exaggerated a bit ;)

But it does come up, see

"Singleton implementation problems"
http://mail.python.org/pipermail/python-list/2008-July/470770.html

"Dynamically declared shared constant/variable imported twice problem"
http://mail.python.org/pipermail/python-list/2009-May/535619.html

There are more, but I don't know a convenient way to find the posts.
Related problems with reload() or paths into a package occur even more 
frequently.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web