Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <878usmrijk.fsf@elektro.pacujo.net>
References: <CACwCsY65gpn_Zx78wquE-rmrZu10pu9GYbzWd-Y3ypzHsMb2Yw@mail.gmail.com> <CACwCsY4nxs-4WmKboA9mOwrjQjXKXmaBPTuuq0GSmjzEpA3efQ@mail.gmail.com> <CAPTjJmr=1CW3BP23x2RTmnfxWppUNz_m_jHp=6aBTF0qj3YkVw@mail.gmail.com> <CACwCsY4LCg5Mw-fx_Og93ErJ_N14Vqw7TO2TrEnp3wAdY0POkg@mail.gmail.com> <mailman.7879.1394144925.18130.python-list@python.org> <87ha7bq7ls.fsf@elektro.pacujo.net> <mailman.7880.1394145824.18130.python-list@python.org> <87d2hyrkf4.fsf@elektro.pacujo.net> <mailman.7881.1394148716.18130.python-list@python.org> <878usmrijk.fsf@elektro.pacujo.net>
Date: Fri, 7 Mar 2014 11:11:16 +1100
Subject: Re: script uses up all memory
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.7883.1394151084.18130.python-list@python.org>
Lines: 85
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:67966

On Fri, Mar 7, 2014 at 10:53 AM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Chris Angelico <rosuav@gmail.com>:
>
>> Can you give a useful example of a closure that does create a refloop?
>
> Just the other day, I mentioned the state pattern:
>
>    class MyStateMachine:
>        def __init__(self):
>            sm = self
>
>            class IDLE:
>                def ding(self):
>                    sm.open_door()
>                    sm.state = AT_DOOR()

Yeah, that's an extremely unusual way to do things. Why keep on
instantiating objects when you could just reference functions?

> In general, event-driven programming produces circular references left
> and right, and that might come into wider use with asyncio.

Nope; certainly not with closures. I do a whole lot of event-driven
programming (usually in Pike rather than Python, but they work the
same way in this), and there's no reference loop. Properly-done
event-driven programming should have two basic states: a reference
from some invisible thing that can trigger the event (eg a GUI widget)
to a callable, and a reference from that callable to its state. Once
the trigger is gone, the callable is dropped, its state is dropped,
and everything's cleaned up. You don't usually need a reference inside
the function to that function.

Don't forget, a closure need only hang onto the things it actually
uses. It doesn't need all its locals.

> I suspect generators might create circular references as well.

I doubt it.

>>> def foo(x):
    return ("x"*i for i in range(x))
>>> len([foo(5) for _ in range(1000)])
1000
>>> gc.collect()
0
>>> len([foo(5) for _ in range(1000)])
1000
>>> gc.collect()
0

Again, unless it keeps a reference to itself, there's no loop. It'll
need to hang onto some of its locals, but that's all.

> Any tree data structure with parent references creates cycles.

Yes, but how many of those do you actually have and drop? If you
create a GUI, you generally hold your entire widget tree stably. The
only issue is if you create a parent-child subtree and then drop it.
That shouldn't be being done in a tight loop. Most of the classic data
structures like trees are implemented at the C level, so again, your
code shouldn't be concerning itself with that.

> In fact, I would imagine most OO designs create a pretty tight mesh of
> back-and-forth references.

Examples, please? I can think of a handful of situations where I've
created reference loops, and they're sufficiently rare that I can put
comments against them and explicitly break them. For instance, I have
a "Subwindow" that has a "Connection". My window can have multiple
subwindows, a subwindow may or may not have a connection, and the
connection always references its subwindow. The subw->connection->subw
loop is explicitly broken when the connection is terminated. If the
window chooses to drop a subw, it first checks if there's a connection
(and prompts the user to confirm), and then will explicitly
disconnect, which breaks the refloop (as the connection's terminated).
I did a similar thing at work, again with explicit refloop breakage to
ensure clean removal. Apart from those two cases, I can't think of
anything in the last ten years where I've had a data structure with a
loop in it, where the whole loop could be dropped. (My MUD has a loop,
in that a character exists in a room, and the room keeps track of its
contents; but it's not logical to drop a room with characters in it,
and dropping a character is done by moving it to no-room, which breaks
the refloop.)

ChrisA