Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.002 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'causing': 0.04; 'languages,': 0.04; 'cpython': 0.05; 'say,': 0.05; 'subject:Python': 0.06; 'affected': 0.07; 'dan': 0.09; 'hiding': 0.09; 'cc:addr:python-list': 0.11; 'python': 0.11; 'bug': 0.12; 'creates': 0.14; 'language.': 0.14; '[*]': 0.16; '__del__': 0.16; 'accesses': 0.16; 'caching': 0.16; 'created.': 0.16; 'elsewhere,': 0.16; 'failure.': 0.16; 'garbage': 0.16; 'higher-level': 0.16; 'invocation': 0.16; 'mutable': 0.16; 'pdb': 0.16; 'wrote:': 0.18; 'all,': 0.19; '(where': 0.19; 'properly': 0.19; 'command': 0.22; 'memory': 0.22; 'python?': 0.22; 'cc:addr:python.org': 0.22; 'print': 0.22; 'subject:problem': 0.24; 'why.': 0.24; 'mon,': 0.24; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; 'source': 0.25; 'header:In-Reply-To:1': 0.27; 'appear': 0.29; 'am,': 0.29; 'tim': 0.29; "doesn't": 0.30; 'message-id:@mail.gmail.com': 0.30; 'code': 0.31; 'python"': 0.31; 'allows': 0.31; 'critical': 0.32; 'languages': 0.32; 'run': 0.32; 'alone': 0.33; 'could': 0.34; 'problem': 0.35; 'subject:with': 0.35; 'received:209.85': 0.35; 'common': 0.35; 'received:209.85.220': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'really': 0.36; 'useful': 0.36; 'example,': 0.37; 'wrong': 0.37; 'received:209': 0.37; 'actions': 0.38; 'depends': 0.38; 'whatever': 0.38; 'anything': 0.39; 'does': 0.39; 'aside': 0.39; 'called': 0.40; 'how': 0.40; 'days': 0.60; 'even': 0.60; 'skip:u 10': 0.60; 'dangerous': 0.60; 'most': 0.60; 'affect': 0.61; 'guarantee': 0.63; 'subject:The': 0.64; 'therefore,': 0.64; 'due': 0.66; 'increase': 0.74; 'protect': 0.79; 'attractive': 0.81; 'potentially': 0.81; 'friends': 0.81; "else's": 0.84; 'ridiculously': 0.84; 'magical': 0.91; '2013': 0.98 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=sEP/99AjmAuhzVxw94IC+/QeLIyW6lcq6ZPV2LAl8MY=; b=et5NlBiq9mNik25TR7/i44mNRDZDBPEFSlZOOA8RReODYvchG5Hgb415IGfGTOJ4XU 1gFAJqhENw0E33l3PgerdazSI+cGujw0PchuXHFRtNR/60IMXxIAgRsgqOxGvk/Qhg3g hKDf05mqg+HsasoJifrzxkyHd23f445v677Dg2+ghRg03nfmNQuT0a6/oB4XmC5QR5C2 YsqsJ0tjvybkA70owhb2k2MvWFQdP9pcZQKjXeEsnj+opy6rpJG2Ea7BQJFW0JF2al45 SZZCGhgzVmbo4mMxa9GlVy8ATFsq6hydxMNbQK47ooQKDTmGt2CEk326CfIEmrW0SzYt GgMw== X-Received: by 10.52.99.168 with SMTP id er8mr13946942vdb.3.1370243201692; Mon, 03 Jun 2013 00:06:41 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <687dea63-84da-4c45-9366-cb5a10665d1f@googlegroups.com> From: Devin Jeanpierre Date: Mon, 3 Jun 2013 03:06:01 -0400 Subject: Re: Python Heisenbugs? (was: Re: PyWart: The problem with "print") To: Dan Sommers Content-Type: text/plain; charset=UTF-8 Cc: python-list@python.org X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 37 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1370243205 news.xs4all.nl 15927 [2001:888:2000:d::a6]:45726 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:46773 On Mon, Jun 3, 2013 at 12:34 AM, Dan Sommers wrote: > On Mon, 03 Jun 2013 13:37:27 +1000, Tim Delaney wrote: > >> With the increase in use of higher-level languages, these days >> Heisenbugs most often appear with multithreaded code that doesn't >> properly protect critical sections, but as you say, with lower-level >> languages uninitialised memory is a common source of them. > > Aside from an I/O caching bug directly affected by calling print or > somefile.write (where somefile is stdout), how else could I create a > Heisenbug in pure Python? The garbage collector can do this, but I think in practice it's ridiculously rare, since __del__ is almost never useful due to its unreliability*. The problem is that the garbage collector can do whatever it wants. For example, in CPython it is called after so many cycles have been created. This allows code and user actions to inadvertently affect the garbage collector, and therefore, the invocation of __del__. If your __del__ does anything that accesses mutable global state also used elsewhere, it's conceivable that the order of someone else's access and __del__'s invocation depends on the GC. One order or the other might be the wrong one which causes a failure. As it happens, the "bt" command in pdb creates a cycle and might trigger the garbage collector, causing __del__ to run immediately, and potentially hiding the failure. This isn't really "pure python" in that Python doesn't even guarantee __del__ is ever called at all, let alone why. It's completely implementation-specific, and not a property of Python the language. -- Devin .. [*] Some people use it as an "unreliable fallback"; this turns their magical autosaving code into an attractive and yet horribly dangerous nuisance. Friends don't let friends use __del__.