Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.albasani.net!news.stack.nl!newsfeed.xs4all.nl!newsfeed3a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
From: Nikolaus Rath <Nikolaus@rath.org>
To: python-list@python.org
Subject: Re: Missing stack frames?
References: <87fvjlv22v.fsf@vostro.rath.org> <CAPTjJmqzgZnziz3+yZb-4bsYtfxT2AG5HQ9Pb3Ya8AWDW8+oHA@mail.gmail.com> <87oay8pobi.fsf@rath.org> <87r433ri00.fsf@handshake.de>
Mail-Copies-To: never
Mail-Followup-To: python-list@python.org
Date: Thu, 05 Jun 2014 19:16:06 -0700
In-Reply-To: <87r433ri00.fsf@handshake.de> (dieter@handshake.de's message of "Thu, 05 Jun 2014 08:24:31 +0200")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.10801.1402020972.18130.python-list@python.org>
Lines: 88
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:72809

dieter <dieter@handshake.de> writes:
[...]
> Someone else already mentioned that the "close" call
> can come from a destructor. Destructors can easily be called
> at not obvious places (e.g. "s =3D C(); ... x =3D [s for s in ...]";
> in this example the list comprehension calls the "C" destructor
> which is not obvious when one looks only locally).
> The destructor calls often have intervening C code (which
> one does not see). However, in your case, I do not see
> why the "cgi" module should cause a destructor call of one
> of your server components.

Paul, dieter, you are my heroes. It was indeed an issue with a
destructor. It turns out that the io.RawIOBase destructor calls
self.close(). If the instance of a derived class is part of a reference
cycle, it gets called on the next routine run of the garbage
collector -- with the stack trace originating at whatever statement was
last executed before the gc run.

The following minimal example reproduces the problem:

#!/usr/bin/env python3
import io
import traceback
import threading

class Container:
    pass

class InnocentVictim(io.RawIOBase):
    def close(self):
        print('close called in %s by:'
              % threading.current_thread().name)
        traceback.print_stack()

def busywork():
    numbers =3D []
    for i in range(500):
        o =3D Container()
        o.l =3D numbers
        numbers.append(o)

        if i % 87 =3D=3D 0:
            numbers =3D []

l =3D [ InnocentVictim() ]
l[0].cycle =3D l
del l

t =3D threading.Thread(target=3Dbusywork)
t.start()
t.join()


If you run this, you could things like:

close called in Thread-1 by:
  File "/usr/lib/python3.4/threading.py", line 888, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "./test.py", line 18, in busywork
    o =3D Container()
  File "./test.py", line 13, in close
    traceback.print_stack()


Ie, a method being called by a thread that doesn't have access to the
object, and without any reference to the call in the source.


I am left wondering:

 - Is there really a point in the RawIOBase destructor calling close?
 - Is there some way to make the call stack for destructors less confusing?

=20=20=20=20
Best,
-Nikolaus

--=20
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2=
=AB