Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.006 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'python.': 0.04; 'python': 0.08; 'dict': 0.09; 'fatal': 0.09; 'figuring': 0.09; 'reply- to:addr:comp.lang.python': 0.09; 'to:addr:comp.lang.python': 0.09; 'url:github': 0.09; 'error:': 0.10; 'debugging': 0.13; 'intermediate': 0.15; 'advice,': 0.16; 'allocates': 0.16; 'cached,': 0.16; 'diederich': 0.16; 'line-number': 0.16; 'pinpoint': 0.16; 'printf': 0.16; 'url:hg': 0.16; 'valgrind': 0.16; 'cc:addr:python-list': 0.16; 'wrote:': 0.18; 'issue.': 0.19; 'cc:no real name:2**0': 0.20; 'memory': 0.21; 'dec': 0.22; 'header :In-Reply-To:1': 0.22; 'figure': 0.23; 'dictionary': 0.23; 'sunday,': 0.23; 'suspects': 0.23; 'worst': 0.23; 'cc:2**0': 0.24; 'there.': 0.24; 'hopefully': 0.24; 'up.': 0.25; 'sat,': 0.25; 'tests': 0.25; 'code': 0.25; 'helpful': 0.26; "i'm": 0.26; 'tried': 0.27; 'url:mailman': 0.28; 'compile': 0.29; 'print': 0.29; 'cc:addr:python.org': 0.29; 'error': 0.29; 'pm,': 0.29; 'enabled,': 0.30; 'further?': 0.30; 'thanks': 0.31; "i've": 0.31; 'does': 0.32; 'pure': 0.32; 'url:listinfo': 0.32; "isn't": 0.33; 'header:User-Agent:1': 0.33; 'received:209.85.160': 0.33; 'object': 0.33; 'too': 0.34; '17,': 0.34; 'allocation': 0.34; 'routine': 0.34; 'stuck': 0.34; 'issue': 0.35; 'url:python': 0.36; 'file': 0.36; 'similar': 0.36; 'two': 0.37; 'but': 0.37; 'received:google.com': 0.37; 'think': 0.37; 'using': 0.38; 'somewhat': 0.38; 'received:209.85': 0.38; 'getting': 0.38; 'put': 0.38; 'url:org': 0.39; 'should': 0.39; 'plain': 0.39; 'help': 0.39; 'received:209': 0.40; 'within': 0.60; 'hope': 0.61; '2011': 0.61; 'address': 0.61; 'your': 0.61; 'kind': 0.61; 'marked': 0.64; 'believe': 0.65; 'piece': 0.66; 'show': 0.67; 'header:Reply-To:1': 0.71; 'reply-to:no real name:2**0': 0.72; 'reply- to:addr:googlegroups.com': 0.74; 'buck': 0.84; 'dict,': 0.84; 'messed': 0.84; 'problematic': 0.84; 'tomorrow.': 0.84; 'url:include': 0.84 Newsgroups: comp.lang.python Date: Mon, 19 Dec 2011 00:09:19 -0800 (PST) In-Reply-To: Complaints-To: groups-abuse@google.com Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=98.248.112.191; posting-account=_aPj8woAAAA9KMWxfZbAmBu6pRusF4_6 References: <18365650.792.1324181855929.JavaMail.geo-discussion-forums@prht13> User-Agent: G2/1.0 X-Google-Web-Client: true MIME-Version: 1.0 Subject: Re: Debugging a difficult refcount issue. From: buck To: comp.lang.python@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: python-list@python.org X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: comp.lang.python@googlegroups.com List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Message-ID: Lines: 54 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1324282889 news.xs4all.nl 6860 [2001:888:2000:d::a6]:58542 X-Complaints-To: abuse@xs4all.nl Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:17503 This is what I came up with: https://gist.github.com/1496028 We'll see if it helps, tomorrow. On Sunday, December 18, 2011 6:01:50 PM UTC-8, buck wrote: > Thanks Jack. I think printf is what it will come down to. I plan to put a= little code into PyDict_New to print the id and the line at which it was a= llocated. Hopefully this will show me all the possible suspects and I can f= igure it out from there. >=20 > I hope figuring out the file and line-number from within that code isn't = too hard. >=20 >=20 > On Sunday, December 18, 2011 9:52:46 AM UTC-8, Jack Diederich wrote: > > I don't have any great advice, that kind of issue is hard to pin down. > > That said, do try using a python compile with --with-debug enabled, > > with that you can turn your unit tests on and off to pinpoint where > > the refcounts are getting messed up. It also causes python to use > > plain malloc()s so valgrind becomes useful. Worst case add assertions > > and printf()s in the places you think are most janky. > >=20 > > -Jack > >=20 > > On Sat, Dec 17, 2011 at 11:17 PM, buck wrote: > > > I'm getting a fatal python error "Fatal Python error: GC object alrea= dy tracked"[1]. > > > > > > Using gdb, I've pinpointed the place where the error is detected. It = is an empty dictionary which is marked as in-use. This is somewhat helpful = since I can reliably find the memory address of the dict, but it does not h= elp me pinpoint the issue. I was able to find the piece of code that alloca= tes the problematic dict via a malloc/LD_PRELOAD interposer, but that code = was pure python. I don't think it was the cause. > > > > > > I believe that the dict was deallocated, cached, and re-allocated via= PyDict_New to a C routine with bad refcount logic, then the above error ma= nifests when the dict is again deallocated, cached, and re-allocated. > > > > > > I tried to pinpoint this intermediate allocation with a similar PyDic= t_New/LD_PRELOAD interposer, but that isn't working for me[2]. > > > > > > How should I go about debugging this further? I've been completely st= uck on this for two days now :( > > > > > > [1] http://hg.python.org/cpython/file/99af4b44e7e4/Include/objimpl.h#= l267 > > > [2] http://stackoverflow.com/questions/8549671/cant-intercept-pydict-= new-with-ld-preload > > > -- > > > http://mail.python.org/mailman/listinfo/python-list