Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Subject: Reliably call code after object no longer exists or is "unreachable"?
From: Jack Bates <ms419@freezone.co.uk>
To: python-list@python.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Date: Wed, 27 Apr 2011 09:48:42 -0700
Mime-Version: 1.0
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.912.1303940829.9059.python-list@python.org>
Lines: 133
NNTP-Posting-Host: 82.94.164.166
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:4166

In Python, how can you reliably call code - but wait until an object no
longer exists or is "unreachable"?

I want to ensure that some code is called (excluding some exotic
situations like when the program is killed by a signal not handled by
Python) but can't call it immediately. I want to wait until there are no
references to an object - or the only references to the object are from
unreachable reference cycles


#!/usr/bin/env python

class Goodbye:
  def __del__(self):
    print 'Goodbye, world!'

ref =3D Goodbye()


$ ./goodbye
Goodbye, world!
$=20


Python's __del__ or destructor method works (above) - but only in the
absence of reference cycles (below). An object, with a __del__ method,
in a reference cycle, causes all objects in the cycle to be
"uncollectable". This can cause memory leaks and because the object is
never collected, its __del__ method is never called


> Circular references which are garbage are detected when the option
> cycle detector is enabled (it's on by default), but can only be
> cleaned up if there are no Python-level __del__() methods involved.


#!/usr/bin/env python

class Goodbye:
  def __del__(self):
    print 'Goodbye, world!'

class Cycle:
  def __init__(self, cycle):
    self.next =3D cycle
    cycle.next =3D self

Cycle(Goodbye())


$ ./cycle
$=20


In PEP 342 I read that an object, with a __del__ method, referenced by a
cycle but not itself participating in the cycle, doesn't cause objects
to be uncollectable. If the cycle is "collectable" then when it's
eventually collected by the garbage collector, the __del__ method is
called


> If the generator object participates in a cycle, g.__del__() may not
> be called. This is the behavior of CPython's current garbage
> collector. The reason for the restriction is that the GC code needs to
> "break" a cycle at an arbitrary point in order to collect it, and from
> then on no Python code should be allowed to see the objects that
> formed the cycle, as they may be in an invalid state. Objects "hanging
> off" a cycle are not subject to this restriction.


#!/usr/bin/env python

import sys

class Destruct:
  def __init__(self, callback):
    self.__del__ =3D callback

class Goodbye:
  def __init__(self):
    self.destruct =3D Destruct(lambda: sys.stdout.write('Goodbye, world!\n'=
))

class Cycle:
  def __init__(self, cycle):
    self.next =3D cycle
    cycle.next =3D self

Cycle(Goodbye())


$ ./dangle
Goodbye, world!
$=20


However it's *extremely* tricky to ensure that the object with a __del__
method doesn't participate in a cycle, e.g. in the example below, the
__del__ method is never called - I suspect because the object with a
__del__ method is reachable from the global scope, and this forms a
cycle with a frame's f_globals reference? "storing a generator object in
a global variable creates a cycle via the generator frame's f_globals
pointer"


#!/usr/bin/env python

import sys

class Destruct:
  def __init__(self, callback):
    self.__del__ =3D callback

class Goodbye:
  def __init__(self):
    self.destruct =3D Destruct(lambda: sys.stdout.write('Goodbye, world!\n'=
))

class Cycle:
  def __init__(self, cycle):
    self.next =3D cycle
    cycle.next =3D self

ref =3D Cycle(Goodbye())


$ ./global
$=20


Faced with the real potential for reference cycles, how can you reliably
call code - but wait until an object no longer exists or is
"unreachable"?