Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78554 > unrolled thread

Clearing globals in CPython

Back to article view | Back to comp.lang.python

Contents

  Clearing globals in CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-02 02:00 +1000
    Re: Clearing globals in CPython Chris Angelico <rosuav@gmail.com> - 2014-10-02 02:14 +1000
    Re: Clearing globals in CPython Chris Kaynor <ckaynor@zindagigames.com> - 2014-10-01 10:08 -0700
    Re: Clearing globals in CPython Peter Otten <__peter__@web.de> - 2014-10-01 19:24 +0200
      Re: Clearing globals in CPython Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-02 18:11 +1000
        Re: Clearing globals in CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-10-02 09:30 +0100
      Re: Clearing globals in CPython jakubo <jakubo@example.org> - 2014-10-06 09:05 +0000
        Re: Clearing globals in CPython Steven D'Aprano <steve@pearwood.info> - 2014-10-06 09:08 +0000
          Re: Clearing globals in CPython Chris Angelico <rosuav@gmail.com> - 2014-10-06 20:30 +1100
    Re: Clearing globals in CPython Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-10-01 19:17 +0100
    Re: Clearing globals in CPython Terry Reedy <tjreedy@udel.edu> - 2014-10-01 14:44 -0400
    Re: Clearing globals in CPython Marco Buttu <marco.buttu@gmail.com> - 2014-10-01 18:39 +0200
    Re: Clearing globals in CPython Grant Edwards <invalid@invalid.invalid> - 2014-10-02 14:07 +0000
      Re: Clearing globals in CPython Chris Angelico <rosuav@gmail.com> - 2014-10-03 00:12 +1000
      Re: Clearing globals in CPython Stefan Behnel <stefan_ml@behnel.de> - 2014-10-03 07:26 +0200

#78554 — Clearing globals in CPython

Out of curiosity, I ran:

globals().clear()

in the interactive interpreter. It broke much more than I expected!
Built-ins were no longer available, and import stopped working.

I expected that global variables would be all lost, but built-ins would
remain, since they don't live in the global namespace. I was wrong:

>>> globals().clear()
>>> x = len([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'len' is not defined

The reason, I think, is that CPython has a special __builtins__ global
variable that the interpreter uses to access built-in functions. What
*appears* to be happening is that if that __builtins__ global is missing,
CPython can not access the built-ins.

(Supporting this interpretation: IronPython, like CPython, has a
__builtins__ global, and behaves the same when the globals() are cleared.
Jython, which does not have a __builtins__ global, does not.)

Well that's okay, I thought to myself, I'll just import the built-in
functions:

>>> from builtins import len  # use '__builtin__' in Python 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: __import__ not found

Oops.

So, with no built-ins available, import no longer works. That makes things
rather tricky.

Obviously the easiest way to recover is to exit the current session and
restart it, but as a challenge, can we recover from this state?


-- 
Steven

[toc] | [next] | [standalone]

#78555

On Thu, Oct 2, 2014 at 2:00 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Obviously the easiest way to recover is to exit the current session and
> restart it, but as a challenge, can we recover from this state?

Oooh interesting. This is kinda like breaking out of a sandbox, and I
know there are people here who are experts at that. However... I'm not
entirely sure how to get a backtrace, when you don't have any built-in
exceptions! In Python 2, I can define an old-style class and raise
that. However, with the removal of __builtins__, Python throws a bunch
of errors about restricted mode, so I'm really not sure where to go
from there.

ChrisA

[toc] | [prev] | [next] | [standalone]

#78559

[Multipart message — attachments visible in raw view] — view raw

On Wed, Oct 1, 2014 at 9:14 AM, Chris Angelico <rosuav@gmail.com> wrote:

> On Thu, Oct 2, 2014 at 2:00 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
> > Obviously the easiest way to recover is to exit the current session and
> > restart it, but as a challenge, can we recover from this state?
>
> Oooh interesting. This is kinda like breaking out of a sandbox, and I
> know there are people here who are experts at that. However... I'm not
> entirely sure how to get a backtrace, when you don't have any built-in
> exceptions! In Python 2, I can define an old-style class and raise
> that. However, with the removal of __builtins__, Python throws a bunch
> of errors about restricted mode, so I'm really not sure where to go
> from there.
>

I've played with it a bit, and I haven't  found any way to break it yet.

I have discovered that you cannot directly declare classes, although you
can access the type built-in (indirectly) so it might be possible to
indirectly declare a class.

It is also possible to access the types of many of the built-ins (str,
bytes, dict, set, type, object, frame, function, code, generator, int,
float at a minimum).

Note that I've been playing with Python 3.4.1.

[toc] | [prev] | [next] | [standalone]

#78560

Steven D'Aprano wrote:

> Out of curiosity, I ran:
> 
> globals().clear()
> 
> in the interactive interpreter. It broke much more than I expected!
> Built-ins were no longer available, and import stopped working.
> 
> I expected that global variables would be all lost, but built-ins would
> remain, since they don't live in the global namespace. I was wrong:
> 
>>>> globals().clear()
>>>> x = len([])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> NameError: name 'len' is not defined
> 
> The reason, I think, is that CPython has a special __builtins__ global
> variable that the interpreter uses to access built-in functions. What
> *appears* to be happening is that if that __builtins__ global is missing,
> CPython can not access the built-ins.
> 
> (Supporting this interpretation: IronPython, like CPython, has a
> __builtins__ global, and behaves the same when the globals() are cleared.
> Jython, which does not have a __builtins__ global, does not.)
> 
> Well that's okay, I thought to myself, I'll just import the built-in
> functions:
> 
>>>> from builtins import len  # use '__builtin__' in Python 2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ImportError: __import__ not found
> 
> Oops.
> 
> So, with no built-ins available, import no longer works. That makes things
> rather tricky.
> 
> Obviously the easiest way to recover is to exit the current session and
> restart it, but as a challenge, can we recover from this state?

$ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:11) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> globals().clear()
>>> import that
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: __import__ not found
>>> object = 1 .__class__.__bases__[0]
>>> Quitter = [c for c in object.__subclasses__() if c.__name__ == 
"Quitter"][0]
>>> __builtins__ = Quitter.__call__.__globals__["__builtins__"]
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

[toc] | [prev] | [next] | [standalone]

#78586

Peter Otten wrote:

> Steven D'Aprano wrote:

>> Obviously the easiest way to recover is to exit the current session and
>> restart it, but as a challenge, can we recover from this state?
> 
> $ python3
> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> globals().clear()
>>>> import that
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ImportError: __import__ not found
>>>> object = 1 .__class__.__bases__[0]
>>>> Quitter = [c for c in object.__subclasses__() if c.__name__ ==
> "Quitter"][0]
>>>> __builtins__ = Quitter.__call__.__globals__["__builtins__"]
>>>> import this
> The Zen of Python, by Tim Peters

Nicely done! 



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#78587

On 02/10/2014 09:11, Steven D'Aprano wrote:
> Peter Otten wrote:
>
>> Steven D'Aprano wrote:
>
>>> Obviously the easiest way to recover is to exit the current session and
>>> restart it, but as a challenge, can we recover from this state?
>>
>> $ python3
>> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
>> [GCC 4.8.2] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> globals().clear()
>>>>> import that
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> ImportError: __import__ not found
>>>>> object = 1 .__class__.__bases__[0]
>>>>> Quitter = [c for c in object.__subclasses__() if c.__name__ ==
>> "Quitter"][0]
>>>>> __builtins__ = Quitter.__call__.__globals__["__builtins__"]
>>>>> import this
>> The Zen of Python, by Tim Peters
>
> Nicely done!
>

Other than the Zen of Python what has Tim Peters ever done for us? 
Apart from...

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#78746

On 2014-10-01, Peter Otten <__peter__@web.de> wrote:
> Steven D'Aprano wrote:
>
>> Out of curiosity, I ran:
>> 
>> globals().clear()
>> 
>> Oops.
>> 
>> So, with no built-ins available, import no longer works. That makes things
>> rather tricky.
>> 
>> Obviously the easiest way to recover is to exit the current session and
>> restart it, but as a challenge, can we recover from this state?
>
> $ python3
> Python 3.4.0 (default, Apr 11 2014, 13:05:11) 
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> globals().clear()
>>>> import that
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ImportError: __import__ not found
>>>> object = 1 .__class__.__bases__[0]
>>>> Quitter = [c for c in object.__subclasses__() if c.__name__ == 
> "Quitter"][0]
>>>> __builtins__ = Quitter.__call__.__globals__["__builtins__"]
>>>> import this
> The Zen of Python, by Tim Peters
>
> Beautiful is better than ugly.
> Explicit is better than implicit.
> Simple is better than complex.
> Complex is better than complicated.
> Flat is better than nested.
> Sparse is better than dense.
> Readability counts.
> Special cases aren't special enough to break the rules.
> Although practicality beats purity.
> Errors should never pass silently.
> Unless explicitly silenced.
> In the face of ambiguity, refuse the temptation to guess.
> There should be one-- and preferably only one --obvious way to do it.
> Although that way may not be obvious at first unless you're Dutch.
> Now is better than never.
> Although never is often better than *right* now.
> If the implementation is hard to explain, it's a bad idea.
> If the implementation is easy to explain, it may be a good idea.
> Namespaces are one honking great idea -- let's do more of those!


I have funnier one :)

$ python3
Python 3.4.1 (default, Sep 10 2014, 22:21:18)
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> globals().clear()
>>> import this
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  ImportError: __import__ not found
>>> True.__
True.__abs__(           True.__dir__(           True.__format__(
True.__init__(          True.__mul__(           True.__radd__(
True.__rlshift__(       True.__rshift__(        True.__sub__(
True.__add__(           True.__divmod__(        True.__ge__(
True.__int__(           True.__ne__(            True.__rand__(
True.__rmod__(          True.__rsub__(          True.__subclasshook__(
True.__and__(           True.__doc__            True.__getattribute__(
True.__invert__(        True.__neg__(           True.__rdivmod__(
True.__rmul__(          True.__rtruediv__(      True.__truediv__(
True.__bool__(          True.__eq__(            True.__getnewargs__(
True.__le__(            True.__new__(           True.__reduce__(
True.__ror__(           True.__rxor__(          True.__trunc__(
True.__ceil__(          True.__float__(         True.__gt__(
True.__lshift__(        True.__or__(            True.__reduce_ex__(
True.__round__(         True.__setattr__(       True.__xor__(
True.__class__(         True.__floor__(         True.__hash__(
True.__lt__(            True.__pos__(           True.__repr__(
True.__rpow__(          True.__sizeof__(
True.__delattr__(       True.__floordiv__(      True.__index__(
True.__mod__(           True.__pow__(           True.__rfloordiv__(
True.__rrshift__(       True.__str__(
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>>


After invoking autocompletion (.__ and hitting tab),
session has been automagically healed.


-- 
jakubo

[toc] | [prev] | [next] | [standalone]

#78747

On Mon, 06 Oct 2014 09:05:17 +0000, jakubo wrote:

> After invoking autocompletion (.__ and hitting tab), session has been
> automagically healed.


That's ... um ...

I have no words.


Can you confirm that autocompletion modifies globals()? I don't have 3.4 
installed here.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#78748

On Mon, Oct 6, 2014 at 8:08 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Mon, 06 Oct 2014 09:05:17 +0000, jakubo wrote:
>
>> After invoking autocompletion (.__ and hitting tab), session has been
>> automagically healed.
>
>
> That's ... um ...
>
> I have no words.
>
>
> Can you confirm that autocompletion modifies globals()? I don't have 3.4
> installed here.

I can confirm that the trick does work. That's... extremely weird.

ChrisA

[toc] | [prev] | [next] | [standalone]

#78563

On 01/10/2014 17:00, Steven D'Aprano wrote:
> Out of curiosity, I ran:
>
> globals().clear()
>
> in the interactive interpreter. It broke much more than I expected!
> Built-ins were no longer available, and import stopped working.
>
> I expected that global variables would be all lost, but built-ins would
> remain, since they don't live in the global namespace. I was wrong:
>
>>>> globals().clear()
>>>> x = len([])
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> NameError: name 'len' is not defined
>
> The reason, I think, is that CPython has a special __builtins__ global
> variable that the interpreter uses to access built-in functions. What
> *appears* to be happening is that if that __builtins__ global is missing,
> CPython can not access the built-ins.
>
> (Supporting this interpretation: IronPython, like CPython, has a
> __builtins__ global, and behaves the same when the globals() are cleared.
> Jython, which does not have a __builtins__ global, does not.)
>
> Well that's okay, I thought to myself, I'll just import the built-in
> functions:
>
>>>> from builtins import len  # use '__builtin__' in Python 2
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ImportError: __import__ not found
>
> Oops.
>
> So, with no built-ins available, import no longer works. That makes things
> rather tricky.
>
> Obviously the easiest way to recover is to exit the current session and
> restart it, but as a challenge, can we recover from this state?
>
>

We obviously need a Restart Command.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#78564

On 10/1/2014 12:00 PM, Steven D'Aprano wrote:
> Out of curiosity, I ran:
>
> globals().clear()
>
> in the interactive interpreter. It broke much more than I expected!
> Built-ins were no longer available, and import stopped working.

As you discovered, this reduces the interpreter to a pure syntax machine 
with no name bindings -- sort of like a high-level assembler with no 
access to a function library.

> I expected that global variables would be all lost, but built-ins would
> remain, since they don't live in the global namespace. ...
> The reason, I think, is that CPython has a special __builtins__ global
> variable that the interpreter uses to access built-in functions.

When executing statements from Shell, Idle restores __builtins__.
 >>> globals().clear()
 >>> print(dir())
['__builtins__']
 >>> print(__builtins__)
<long listing>

The odd thing is that executing the same code from a file prints [] and 
then raises "NameError: name '__builtins__' is not defined".

> Well that's okay, I thought to myself, I'll just import the built-in
> functions:
>
>>>> from builtins import len  # use '__builtin__' in Python 2
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ImportError: __import__ not found

Most keyword statements are implemented by internal functions not 
accessible from Python except through the statement syntax.  Import is 
an exception.  This is one of the hooks that allow the behavior of 
imports to be modified.  The result is like, for instance, programming 
in C with 'include' disabled.

> Oops.
>
> So, with no built-ins available, import no longer works. That makes things
> rather tricky.
>
> Obviously the easiest way to recover is to exit the current session and
> restart it, but as a challenge, can we recover from this state?


-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#78604

On 01/10/2014 18:00, Steven D'Aprano wrote:

> Out of curiosity, I ran:
>
> globals().clear()
>
> in the interactive interpreter. It broke much more than I expected!
> Built-ins were no longer available, and import stopped working.

Interesting... :D

> Obviously the easiest way to recover is to exit the current session and
> restart it, but as a challenge, can we recover from this state?

 >>> [].__class__.__module__ # It is a string...
'__builtin__'

So, I do not have any idea about how to get a module without importing 
something. Thanks for sharing :)

-- 
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbuttu@oa-cagliari.inaf.it

[toc] | [prev] | [next] | [standalone]

#78606

On 2014-10-01, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> Obviously the easiest way to recover is to exit the current session and
> restart it, but as a challenge, can we recover from this state?

Python apparently _does_ need a "restart command".

-- 
Grant Edwards               grant.b.edwards        Yow! The PILLSBURY DOUGHBOY
                                  at               is CRYING for an END to
                              gmail.com            BURT REYNOLDS movies!!

[toc] | [prev] | [next] | [standalone]

#78607

On Fri, Oct 3, 2014 at 12:07 AM, Grant Edwards <invalid@invalid.invalid> wrote:
> On 2014-10-01, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>
>> Obviously the easiest way to recover is to exit the current session and
>> restart it, but as a challenge, can we recover from this state?
>
> Python apparently _does_ need a "restart command".

Apparently not... you saw how easily Peter recovered :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#78635

Chris Angelico schrieb am 02.10.2014 um 16:12:
> On Fri, Oct 3, 2014 at 12:07 AM, Grant Edwards wrote:
>> On 2014-10-01, Steven D'Aprano wrote:
>>
>>> Obviously the easiest way to recover is to exit the current session and
>>> restart it, but as a challenge, can we recover from this state?
>>
>> Python apparently _does_ need a "restart command".
> 
> Apparently not... you saw how easily Peter recovered :)

Right. All we need is a builtin function for that recovery code.

Stefan

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web