Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #81942 > unrolled thread

About Modifying Globals

Back to article view | Back to comp.lang.python

Contents

  About Modifying Globals LJ <luisjosenovoa@gmail.com> - 2014-12-04 12:09 -0800
    Re: About Modifying Globals Chris Angelico <rosuav@gmail.com> - 2014-12-05 07:22 +1100
    Re: About Modifying Globals Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-04 13:21 -0700
    Re: About Modifying Globals Dave Angel <davea@davea.name> - 2014-12-04 18:54 -0500
    Re: About Modifying Globals Chris Angelico <rosuav@gmail.com> - 2014-12-05 10:58 +1100
    Re: About Modifying Globals Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 13:09 +1100
      The binding operator, and what gets bound to what (was: About Modifying Globals) Ben Finney <ben+python@benfinney.id.au> - 2014-12-05 16:36 +1100
        Re: The binding operator, and what gets bound to what (was: About Modifying Globals) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 20:53 +1100
          Re: The binding operator, and what gets bound to what (was: About Modifying Globals) Chris Angelico <rosuav@gmail.com> - 2014-12-05 21:03 +1100
            Re: The binding operator, and what gets bound to what (was: About Modifying Globals) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 23:26 +1100
              Re: The binding operator, and what gets bound to what (was: About Modifying Globals) Chris Angelico <rosuav@gmail.com> - 2014-12-05 23:42 +1100
          Re: The binding operator, and what gets bound to what Ned Batchelder <ned@nedbatchelder.com> - 2014-12-05 07:06 -0500
            Re: The binding operator, and what gets bound to what Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-05 23:34 +1100
              Re: The binding operator, and what gets bound to what Chris Angelico <rosuav@gmail.com> - 2014-12-05 23:48 +1100
              Re: The binding operator, and what gets bound to what Ned Batchelder <ned@nedbatchelder.com> - 2014-12-05 07:56 -0500
    Re: About Modifying Globals "shiyao.ma" <i@introo.me> - 2014-12-05 10:53 +0800

#81942 — About Modifying Globals

Hi All,

I have a quick question regarding the modification of global variables within functions. To illustrate, consider the following toy example:

a={"1": set()}
b=9

def gt(l):
   a["1"] = a["1"] | set([l])

When calling this last function and checking the a dictionary, I get:

>>> gt(5)
>>> a
{"1": set([5])}


The set in the dictionary was modified. The question is, why isn't it necessary to declare a as global within the gt function, as apposed to a case like  

def gt2(l):
   b=b+l

where I need to declare b as global within the function to avoid:

UnboundLocalError: local variable 'b' referenced before assignment.


I apologize if this question has been answered before.

Thank you.

[toc] | [next] | [standalone]

#81943

On Fri, Dec 5, 2014 at 7:09 AM, LJ <luisjosenovoa@gmail.com> wrote:
> def gt(l):
>    a["1"] = a["1"] | set([l])
>
>
> def gt2(l):
>    b=b+l

These two may both look like they're assigning something, but one of
them is assigning directly to the name "b", while the other assigns to
a subscripted element of "a". In the first function, "a" is still the
same object after the change; in the second, "b" has been replaced by
a new object. That's why your second one needs a global declaration.
It's all about names and assignment; if you're not assigning to the
actual name, there's no need to declare where the name comes from,
because you're only referencing it, not replacing it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81944

[Multipart message — attachments visible in raw view] — view raw

On Thu, Dec 4, 2014 at 1:09 PM, LJ <luisjosenovoa@gmail.com> wrote:
>
> Hi All,
>
> I have a quick question regarding the modification of global variables
within functions. To illustrate, consider the following toy example:
>
> a={"1": set()}
> b=9
>
> def gt(l):
>    a["1"] = a["1"] | set([l])
>
> When calling this last function and checking the a dictionary, I get:
>
> >>> gt(5)
> >>> a
> {"1": set([5])}
>
>
> The set in the dictionary was modified. The question is, why isn't it
necessary to declare a as global within the gt function, as apposed to a
case like
>
> def gt2(l):
>    b=b+l
>
> where I need to declare b as global within the function to avoid:
>
> UnboundLocalError: local variable 'b' referenced before assignment.

https://docs.python.org/3/faq/programming.html#what-are-the-rules-for-local-and-global-variables-in-python

In the first case, you never assign to a; you only modify it. Since it's
never assigned locally the compiler knows it can't be a local variable, so
it automatically makes it a global.

In the second case, the assignment to b causes the compiler to treat the
variable as local by default, so it needs to be explicitly marked as global.

[toc] | [prev] | [next] | [standalone]

#81962

On 12/04/2014 03:09 PM, LJ wrote:
> Hi All,
>
> I have a quick question regarding the modification of global variables within functions. To illustrate, consider the following toy example:
>
> a={"1": set()}
> b=9
>
> def gt(l):
>     a["1"] = a["1"] | set([l])
>
> When calling this last function and checking the a dictionary, I get:
>
>>>> gt(5)
>>>> a
> {"1": set([5])}
>
>
> The set in the dictionary was modified. The question is, why isn't it necessary to declare a as global within the gt function, as apposed to a case like
>
> def gt2(l):
>     b=b+l
>
> where I need to declare b as global within the function to avoid:
>
> UnboundLocalError: local variable 'b' referenced before assignment.
>

The reason this sort of thing seems to confuse lots of people is that we 
insist on calling these things assignments.  The thing that's global is 
the name 'a'.  It's global because it's in the global() namespace of 
some module.

The data that's "assigned" to it is an object.  You bind the object to 
the name 'a' with an assignment statement.  If that object is immutable, 
then the only way to change 'a' is to do another assignment.

But if the object is mutable, as a set is, then it can change as much as 
you like without rebinding the name.

A given object may have any number of names bound to it, including no 
names.  An example of that could be objects that are referenced only in 
some list.

Python doesn't have declarations, so when a function is compiled, the 
compiler has to infer what names are to be local and what are not.  The 
rule it normally uses is roughly based on whether an assignment occurs 
somewhere inside the function.


-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#81964

On Fri, Dec 5, 2014 at 10:54 AM, Dave Angel <davea@davea.name> wrote:
> Python doesn't have declarations, so when a function is compiled, the
> compiler has to infer what names are to be local and what are not.  The rule
> it normally uses is roughly based on whether an assignment occurs somewhere
> inside the function.

Not strictly true; Python just inverts the C model. In C, you declare
your locals; in Python, you declare your globals. The "global x"
statement is a declaration. But otherwise, yes. When a function is
compiled, the compiler has to figure out what's local and what's
global.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81968

LJ wrote:

> Hi All,
> 
> I have a quick question regarding the modification of global variables
> within functions. To illustrate, consider the following toy example:
> 
> a={"1": set()}
> b=9
> 
> def gt(l):
>    a["1"] = a["1"] | set([l])

The difference between this example and your second one:

> def gt2(l):
>    b=b+l


is that the second is a "binding operation" and the first is not.

Binding operations include direct assignment to a name, deletions, imports
and a few other things:

    b = 23
    del b
    import sys as b


are all considered binding operations, since they bind a value to a name
(or, in the case of del, unbind it). Inside functions, the compiler follows
the rule that any binding operation makes that variable a local for the
entire function, unless declared global.

So in the gt2() example, "b = b + 1" is interpreted as this:

- look up local variable b;
- add one to the result;
- assign it to local variable b.

which fails because b hasn't got a value yet.

This is different from, say, Lua, where that line would be interpreted as: 

- look up local variable b;
- if that doesn't exist, look up global variable b instead;
- add one to the result;
- assign it to local variable b.

But that's not what Python does.

On the other hand, your first example doesn't include a direct assignment to
a name:

    a["1"] = a["1"] | set([l])


looks like an assignment, but it's actually a method call! That is
approximately equivalent to:

    a.__setitem__("1", X)

with X equal to the right hand side. Since that's a method call, it doesn't
count as a binding operation and the compiler doesn't treat a as a local,
it treats it as global, so the lookup on the right hand side succeeds, and
the method call modifies the set in place.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#81976 — The binding operator, and what gets bound to what (was: About Modifying Globals)

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> LJ wrote:
>
> > def gt(l):
> >    a["1"] = a["1"] | set([l])
>
> The difference between this example and your second one:
>
> > def gt2(l):
> >    b=b+l
>
>
> is that the second is a "binding operation" and the first is not.

I disagree; they're both binding operations (and they both use the
binding operator ‘=’ to express this).

The difference between the two is: what reference is being bound?

* In the first example, the reference is ‘a["1"]’.

  That reference is re-bound to a different object; whatever object it
  was previously bound to is forgotten.

* In the second example, the reference is ‘b’.

  That reference is re-bound to a different object; whatever object it
  was previously bound to is forgotten.

So, both of them bind (or re-bind) a reference to an object. The only
difference between the operations is the parameters.

-- 
 \      “People always ask me, ‘Where were you when Kennedy was shot?’ |
  `\                        Well, I don't have an alibi.” —Emo Philips |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#81992 — Re: The binding operator, and what gets bound to what (was: About Modifying Globals)

Ben Finney wrote:

> Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:
> 
>> LJ wrote:
>>
>> > def gt(l):
>> >    a["1"] = a["1"] | set([l])
>>
>> The difference between this example and your second one:
>>
>> > def gt2(l):
>> >    b=b+l
>>
>>
>> is that the second is a "binding operation" and the first is not.
> 
> I disagree; they're both binding operations (and they both use the
> binding operator ‘=’ to express this).

Define "operator" :-)

In Python terms, operators include + - * etc. They return a result, and are
implemented by a special __dunder__ method. (Sometimes by two dunder
methods, e.g. __add__ and method __radd__.)

By this definition, = is not an operator. And indeed, the Python docs don't
list = as an operator:

https://docs.python.org/2/reference/lexical_analysis.html#operators

but it is listed as a delimiter:

https://docs.python.org/2/reference/lexical_analysis.html#delimiters

That's why I was careful to use the term "binding operation" and
avoid "operator", although I admit that was pretty subtle :-)

As for the rest of your comments:

> The difference between the two is: what reference is being bound?

I don't think we disagree, we merely have a different emphasis. The rules
for assignments in Python are a bit complicated, and the official language
used is often different from what we say informally:

https://docs.python.org/2/reference/simple_stmts.html#assignment-statements

Of course you are right that other assignment statements bind a value to a
reference of some sort:

spam.method().attribute['key'].thingy[0] = eggs

but *not for the purpose of deciding what is a local variable*. That was the
point I was emphasising.

The docs referenced above say:

    Assignment of an object to a single target is recursively 
    defined as follows.

    If the target is an identifier (name):

        If the name does not occur in a global statement in the 
        current code block: the name is bound to the object in 
        the current local namespace.
        Otherwise: the name is bound to the object in the current
        global namespace.

which is a little less than clear, but basically it means that assignments
of the form:

    name = something

force name to be a local variable unless it is explicitly declared global.

More information here:

https://docs.python.org/2/reference/simple_stmts.html#the-global-statement

Oh, I learned something new: strictly speaking, this is implementation-
dependent and not guaranteed to work in the future!

def func():
    global math
    import math




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#81993 — Re: The binding operator, and what gets bound to what (was: About Modifying Globals)

On Fri, Dec 5, 2014 at 8:53 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Oh, I learned something new: strictly speaking, this is implementation-
> dependent and not guaranteed to work in the future!
>
> def func():
>     global math
>     import math

When would you actually *want* this, though? Given that 'import'
already caches, there's not much point caching globally, and the idea
that a function could cause a module-level import to happen is pretty
weird!

The only scheme I could concoct would be something like this:

def use_foo():
    global useme
    import foo as useme
    # code to initialize foo for our use

def use_bar():
    global useme
    import bar as useme
    # likewise

def do_stuff():
    useme.blah(1,2,3,4)

So, for instance, if you need a JavaScript executor, you might support
several different runtimes, and allow one to be selected like this.
But this would be serious code smell.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81999 — Re: The binding operator, and what gets bound to what (was: About Modifying Globals)

Chris Angelico wrote:

> On Fri, Dec 5, 2014 at 8:53 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> Oh, I learned something new: strictly speaking, this is implementation-
>> dependent and not guaranteed to work in the future!
>>
>> def func():
>>     global math
>>     import math
> 
> When would you actually *want* this, though? Given that 'import'
> already caches, there's not much point caching globally, and the idea
> that a function could cause a module-level import to happen is pretty
> weird!

That's one way of dealing with circular imports. That way the problematic
import is deferred until the function is called, which hopefully is after
both modules have finished initialising their global names. And by making
the imported name a global, you don't have to bother writing "import math"
in every single function. You just need to ensure your initialisation
function is called before doing any real work.

I'm not saying that this is an ideal solution, but there are no ideal
solutions to circular imports. 


> The only scheme I could concoct would be something like this:
> 
> def use_foo():
>     global useme
>     import foo as useme
>     # code to initialize foo for our use
> 
> def use_bar():
>     global useme
>     import bar as useme
>     # likewise
> 
> def do_stuff():
>     useme.blah(1,2,3,4)

Oooh! That's nice too!


> So, for instance, if you need a JavaScript executor, you might support
> several different runtimes, and allow one to be selected like this.
> But this would be serious code smell.

No more than any other global variable. Just because something is unusual
doesn't make it harmful. If you have globals (constants, or variables), you
might put their initialisation code into a function:

def init():
    global x, y, z
    x = some_calculation(arg1)
    y = another_calculation(arg2)
    z = yet_a_third_calculation(arg3)


import is just a special case of a particular calculation.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82001 — Re: The binding operator, and what gets bound to what (was: About Modifying Globals)

On Fri, Dec 5, 2014 at 11:26 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Chris Angelico wrote:
>
>>> def func():
>>>     global math
>>>     import math
>>
>> When would you actually *want* this, though? Given that 'import'
>> already caches, there's not much point caching globally, and the idea
>> that a function could cause a module-level import to happen is pretty
>> weird!
>
> That's one way of dealing with circular imports. That way the problematic
> import is deferred until the function is called, which hopefully is after
> both modules have finished initialising their global names. And by making
> the imported name a global, you don't have to bother writing "import math"
> in every single function. You just need to ensure your initialisation
> function is called before doing any real work.

Ah. I've never liked circular imports at all, so I've never actually
looked into ways of dealing with them. So basically, func() is a sort
of "second phase initialization" routine, to be called before any of
the module's real work gets called.

> Oooh! That's nice too!
>
>
>> So, for instance, if you need a JavaScript executor, you might support
>> several different runtimes, and allow one to be selected like this.
>> But this would be serious code smell.
>
> No more than any other global variable. Just because something is unusual
> doesn't make it harmful. If you have globals (constants, or variables), you
> might put their initialisation code into a function:

That's why I said "code smell". Imagine the usage of a module like that:

import module
module.use_foo() # What happens if you omit this line?
module.do_stuff()

This is true of your circular imports example, too. Anything that
requires a two-step initialization runs the risk of someone either
forgetting, or having two imports in different modules and having them
out of sync. (With the code example I gave, whichever one is imported
second will "stick", because the use_* functions overwrite each
other.) So it's going to be a bit confusing to people using the
module. It might be better to have a deliberately-instantiated object:

import module
stuff_doer = module.use_foo()
stuff_doer.do_stuff()

It also might not, of course, which is the point of code smell - it
*is* sometimes correct - but I'd want to see some justifying comments
(or docstring) if anyone does this in production code.

ChrisA

[toc] | [prev] | [next] | [standalone]

#81998 — Re: The binding operator, and what gets bound to what

On 12/5/14 4:53 AM, Steven D'Aprano wrote:
> Oh, I learned something new: strictly speaking, this is implementation-
> dependent and not guaranteed to work in the future!
>
> def func():
>      global math
>      import math

I don't think this is implementation-dependent.  The import statement is 
an assignment to a name (as are class, def, for, with-as and except-as). 
  Where that name is scoped is the same for all of them, and does not 
depend on the implementation.  The usual behavior of locals and globals 
happens for all of them:

 >>> import math
 >>> math
<module 'math' from '/blah/.../math.so'>
 >>> def f():
...   print math
...   import math
...
 >>> f()
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "<stdin>", line 2, in f
UnboundLocalError: local variable 'math' referenced before assignment


 >>> a
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
 >>> def f():
...   global a
...   for a in range(10):
...     pass
...
 >>> f()
 >>> a
9

This also means that you can do odd things like:

    for self.foo in range(10):

or:

    for my_list[17] in range(10):

(but please don't!)


-- 
Ned Batchelder, http://nedbatchelder.com

[toc] | [prev] | [next] | [standalone]

#82000 — Re: The binding operator, and what gets bound to what

Ned Batchelder wrote:

> On 12/5/14 4:53 AM, Steven D'Aprano wrote:
>> Oh, I learned something new: strictly speaking, this is implementation-
>> dependent and not guaranteed to work in the future!
>>
>> def func():
>>      global math
>>      import math
> 
> I don't think this is implementation-dependent.

The docs say that it is:

https://docs.python.org/3/reference/simple_stmts.html#the-global-statement

    Names listed in a global statement MUST NOT [emphasis added] be 
    defined as formal parameters or in a for loop control target, 
    class definition, function definition, or import statement.

    CPython implementation detail: The current implementation does not
    enforce the two restrictions, but programs should not abuse this
    freedom, as future implementations may enforce them or silently 
    change the meaning of the program.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82002 — Re: The binding operator, and what gets bound to what

On Fri, Dec 5, 2014 at 11:34 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
>> I don't think this is implementation-dependent.
>
> The docs say that it is:
>
> https://docs.python.org/3/reference/simple_stmts.html#the-global-statement
>
>     Names listed in a global statement MUST NOT [emphasis added] be
>     defined as formal parameters or in a for loop control target,
>     class definition, function definition, or import statement.
>
>     CPython implementation detail: The current implementation does not
>     enforce the two restrictions, but programs should not abuse this
>     freedom, as future implementations may enforce them or silently
>     change the meaning of the program.

Interesting, I didn't know that either.

But the first restriction _is_ enforced:

rosuav@sikorsky:~$ python3
Python 3.5.0a0 (default:23ab1197df0b, Nov 20 2014, 12:57:44)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def x(y):
...     global y
...
  File "<stdin>", line 2
SyntaxError: name 'y' is parameter and global

Likewise in CPython 2.7:

rosuav@sikorsky:~$ python
Python 2.7.3 (default, Mar 13 2014, 11:03:55)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def x(y):
...     global y
...
  File "<stdin>", line 1
SyntaxError: name 'y' is local and global

ChrisA

[toc] | [prev] | [next] | [standalone]

#82003 — Re: The binding operator, and what gets bound to what

On 12/5/14 7:34 AM, Steven D'Aprano wrote:
> Ned Batchelder wrote:
>
>> On 12/5/14 4:53 AM, Steven D'Aprano wrote:
>>> Oh, I learned something new: strictly speaking, this is implementation-
>>> dependent and not guaranteed to work in the future!
>>>
>>> def func():
>>>       global math
>>>       import math
>>
>> I don't think this is implementation-dependent.
>
> The docs say that it is:
>
> https://docs.python.org/3/reference/simple_stmts.html#the-global-statement
>
>      Names listed in a global statement MUST NOT [emphasis added] be
>      defined as formal parameters or in a for loop control target,
>      class definition, function definition, or import statement.
>
>      CPython implementation detail: The current implementation does not
>      enforce the two restrictions, but programs should not abuse this
>      freedom, as future implementations may enforce them or silently
>      change the meaning of the program.

Ah, good.  TIL! :)

-- 
Ned Batchelder, http://nedbatchelder.com

[toc] | [prev] | [next] | [standalone]

#81970

On 12/04, LJ wrote:
> Hi All,
>
> I have a quick question regarding the modification of global variables within functions. To illustrate, consider the following toy example:
>
> a={"1": set()}
> b=9
>
> def gt(l):
>    a["1"] = a["1"] | set([l])
>
> When calling this last function and checking the a dictionary, I get:
>
> >>> gt(5)
> >>> a
> {"1": set([5])}
>
>
> The set in the dictionary was modified. The question is, why isn't it necessary to declare a as global within the gt function, as apposed to a case like
>
> def gt2(l):
>    b=b+l
>
> where I need to declare b as global within the function to avoid:
>
> UnboundLocalError: local variable 'b' referenced before assignment.

Well. To understand the difference, one first has to know the mechanism of CPython.
When you have a script, CPython will first parse it, and then generate bytecode representation, and then finally execute it.
If you take a look at the bytecode of both your source code, you will notice, that.

In the first snippet, there is something like LOAD_NAME, which loads the global name a.
However, in the second snippet, there would be a LOAD_FAST, which loads the local name b (which is, technically speaking, stored in the PyFrameObj).

The reason Python treats it differently, is, IIUC, for better semantic meaning, and also, for huge performance improvement.


Hope that explains.


Regards.

-- 
Shiyao Ma
http://introo.me

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web