Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Terry Reedy <tjreedy@udel.edu>
Subject: Re: Messing with the GC
Date: Sat, 19 Jan 2013 11:40:28 -0500
References: <alvpvkFthuoU1@mid.uni-berlin.de>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0
In-Reply-To: <alvpvkFthuoU1@mid.uni-berlin.de>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.694.1358613663.2939.python-list@python.org>
Lines: 119
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:37090

On 1/19/2013 9:47 AM, Jens Thoms Toerring wrote:

The code comments mostly answer your questions about what happens or 
does not happen and when. The comments after add more detail.

> import sys
>
> class X( object ) :
>      def __init__( self, parent, cnt ) :
>          print( "In constructor for {0} {1}".format( self, cnt ),
>                 file = sys.stderr )
>          self.parent = parent
>          self.cnt = cnt
>
>      def __del__( self ) :
>          print( "In destructor for {0} {1}".format( self, self.cnt ),
>                 file = sys.stderr )
>
>      def foo( self ) :

At this point, self is self.parent.z, which is to say, self and 
self.parent.z are 2 references to 1 object. The self and self.parent 
object refer to each other and therefore constitute a reference cycle.

>          print( "Before", file = sys.stderr )
>          self.parent.z = X( self.parent, 2 )     # Is this bad?

At this point, self.parent.z is another instance of X, breaking the 
existing reference cycle *and* creating a new one. Now self has just the 
one reference 'self' (plus another non-circular, hidden one, see below#).

>          print( "After", file = sys.stderr )

At this point, self goes out of scope, the CPython reference count goes 
to 0, and the object that was self is deleted. Other implementations 
typically wait longer to delete.

> class Y( object ) :
>      def __init__( self ) :
>          print( "In constructor for {0}".format( self ),
>                 file = sys.stderr )
>          self.z = X( self, 1 )

At this point, self is self.z.parent, where z is an X. This is a 
circular reference: self and self.z reference each other.

>      def __del__( self ) :
>          print( "In destructor for {0}".format( self ),
>                 file = sys.stderr )
>
> Y( ).z.foo( )

Y() creates a reference cycle. z.foo() substitute a different instance 
of X in the cycle but there still is a cycle, so the Y and X objects 
have reference counts of 1. There are no other references to the two 
objects, making them unreachable and 'dead' to the program.

The cyclic reference garbage collector (see the gc module doc) that is 
meant to delete such orphans does not delete them here because of the 
__del__ methods. Since gc was added, __del__ is semi-obsolete. If an 
object might possibly be put in a reference cycle (which is quite easy), 
any code that might have been put in __del__ should go in an explicitly 
called .close() method.

> Have a look at the line with the comment. At this point the
> only reference in existence to the X class instance, of which
> a method is just being executed, goes out of scope.

Nope, the remaining reference, 'self', stays in scope until after the 
function exits. That is when X1 is deleted and the deletion message 
printed.

> Is my assumption about this flawed

Yes

> and there are no potential dangers?

The orphaned two-object cycle constitutes a memory leak.
If you called Y( ).z.foo( ) a million times,
you would have a million useless pairs of objects.
This is why gc was added.

> Y( ).z.foo( )
>
> [perhaps] a temporary second reference is created that keeps the GC
> for removing the X instance...

Function calls (normally) bind argument object to parameter names in the 
function's local namespace. That binding is a temporary reference and 
objects will not disappear in the middle of the call.

# In CPython, at least, there is another internal reference to arguments 
that also disappears when the function returns, allowing the deletion of 
arguments without other references.

 >>> x = 12343252  # random large number to make sure its a new object
 >>> sys.getrefcount(x)
2
 >>> def f(a): print(sys.getrefcount(a))

 >>> f(x)
4

So print(sys.getrefcount(self)) at the top of foo prints 4 (3+1),
rather than 3 (2+1), as one might expect. The +1 is explained in the doc

sys.getrefcount(obj):
     Return the reference count of the object. The count returned is 
generally one higher than you might expect, because it includes the 
(temporary) reference as an argument to getrefcount().

(Why doesn't getrefcount add 2 instead of 1, as f seems to? I don't 
know, but perhaps because it is written in C rather than Python and 
Python code objects are different from C code.)

-- 
Terry Jan Reedy