Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Newsgroups: comp.lang.python
Date: Sun, 18 Dec 2011 18:01:50 -0800 (PST)
In-Reply-To: <mailman.3799.1324230769.27778.python-list@python.org>
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=98.248.112.191; posting-account=_aPj8woAAAA9KMWxfZbAmBu6pRusF4_6
References: <18365650.792.1324181855929.JavaMail.geo-discussion-forums@prht13> <mailman.3799.1324230769.27778.python-list@python.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Subject: Re: Debugging a difficult refcount issue.
From: buck <workitharder@gmail.com>
To: comp.lang.python@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org
Precedence: list
Reply-To: comp.lang.python@googlegroups.com
Message-ID: <mailman.3810.1324260120.27778.python-list@python.org>
Lines: 47
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:17488

Thanks Jack. I think printf is what it will come down to. I plan to put a l=
ittle code into PyDict_New to print the id and the line at which it was all=
ocated. Hopefully this will show me all the possible suspects and I can fig=
ure it out from there.

I hope figuring out the file and line-number from within that code isn't to=
o hard.


On Sunday, December 18, 2011 9:52:46 AM UTC-8, Jack Diederich wrote:
> I don't have any great advice, that kind of issue is hard to pin down.
>  That said, do try using a python compile with --with-debug enabled,
> with that you can turn your unit tests on and off to pinpoint where
> the refcounts are getting messed up.  It also causes python to use
> plain malloc()s so valgrind becomes useful.  Worst case add assertions
> and printf()s in the places you think are most janky.
>=20
> -Jack
>=20
> On Sat, Dec 17, 2011 at 11:17 PM, buck <workit...@gmail.com> wrote:
> > I'm getting a fatal python error "Fatal Python error: GC object already=
 tracked"[1].
> >
> > Using gdb, I've pinpointed the place where the error is detected. It is=
 an empty dictionary which is marked as in-use. This is somewhat helpful si=
nce I can reliably find the memory address of the dict, but it does not hel=
p me pinpoint the issue. I was able to find the piece of code that allocate=
s the problematic dict via a malloc/LD_PRELOAD interposer, but that code wa=
s pure python. I don't think it was the cause.
> >
> > I believe that the dict was deallocated, cached, and re-allocated via P=
yDict_New to a C routine with bad refcount logic, then the above error mani=
fests when the dict is again deallocated, cached, and re-allocated.
> >
> > I tried to pinpoint this intermediate allocation with a similar PyDict_=
New/LD_PRELOAD interposer, but that isn't working for me[2].
> >
> > How should I go about debugging this further? I've been completely stuc=
k on this for two days now :(
> >
> > [1] http://hg.python.org/cpython/file/99af4b44e7e4/Include/objimpl.h#l2=
67
> > [2] http://stackoverflow.com/questions/8549671/cant-intercept-pydict-ne=
w-with-ld-preload
> > --
> > http://mail.python.org/mailman/listinfo/python-list