Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!cs.uu.nl!news.stack.nl!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.035
X-Spam-Evidence: '*H*': 0.93; '*S*': 0.00; 'preferably': 0.05; 'debugging': 0.07; '"if': 0.09; 'advice?': 0.09; 'restart': 0.09; 'weeks,': 0.09; 'runs': 0.10; 'cares': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'leaks': 0.16; 'reboot': 0.16; 'url:gnu': 0.16; 'hire': 0.16; 'fix': 0.17; 'wrote:': 0.18; 'year,': 0.18; 'finished': 0.19; 'small,': 0.19; 'thu,': 0.19; 'seems': 0.21; '>>>': 0.22; 'memory': 0.22; 'reset': 0.22; 'issue,': 0.24; 'script': 0.25; 'least': 0.26; 'header:In-Reply- To:1': 0.27; 'fixed': 0.29; '[1]': 0.29; "doesn't": 0.30; 'message-id:@mail.gmail.com': 0.30; 'easier': 0.31; "d'aprano": 0.31; 'steven': 0.31; 'probably': 0.32; 'everyone': 0.33; 'running': 0.33; 'fri,': 0.33; 'problem': 0.35; 'advice': 0.35; 'received:209.85': 0.35; 'deadline': 0.35; 'received:209.85.220': 0.35; 'but': 0.35; 'received:google.com': 0.35; '14,': 0.36; "i'll": 0.36; 'url:org': 0.36; 'should': 0.36; 'too': 0.37; 'project': 0.37; 'received:209': 0.37; 'sometimes': 0.38; 'to:addr :python-list': 0.38; 'pm,': 0.38; 'rather': 0.38; 'does': 0.39; 'to:addr:python.org': 0.39; 'enough': 0.39; 'either': 0.39; 'even': 0.60; 'deadline:': 0.60; 'year.': 0.61; 'times': 0.62; 'week,': 0.64; 'more': 0.64; 'within': 0.65; 'periodically': 0.68; 'analysis': 0.75; 'yourself': 0.78; 'hands,': 0.84; 'leak': 0.84; 'maybe,': 0.84; 'much)': 0.84; 'terrible': 0.84; 'url:last': 0.84; 'ports': 0.93; '2013': 0.98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=GYHlRfi8M0tfjwvFnh6tlurX8hIBcDt72he1LonUnmA=; b=mk4fgyxhqyKzKg6jjpcs0//m0Pz2xAdt7UU4AxARR/cIe8af2tPbhoO7ur86mQ63u2 /wJXL33qepeHEOkIyUIrshxRaBgtXiOTERiJtFOzkNpq/SjCqjFg6taqYiQpagPr0+4P E3i9CiqjfDGbDdMGTZ3eesOZzE/yni+EraNXp+bR8V3Uj54vj/ctPxlLd1vNgjqFwkRz E0NCxB/YD4QoACReyZbIORo0HmNtLCRvW/JqsWK0+2iyke+ApFS7qXCKcUy0AYl69LlP fTk8CdFo+jCeEiX+LDhLVJNgciwCJoNZBj644pUZ4er/8HCBDJ0T8kry8UbGKPeaLCmO SHkw==
MIME-Version: 1.0
X-Received: by 10.220.112.76 with SMTP id v12mr607030vcp.63.1371202238090; Fri, 14 Jun 2013 02:30:38 -0700 (PDT)
In-Reply-To: <51ba82b5$0$29997$c3e8da3$5496439d@news.astraweb.com>
References: <09917103-b35e-4728-8fea-bcb4ce2bd1af@googlegroups.com> <mailman.3165.1371104977.3114.python-list@python.org> <slrnkrkacg.2u4.giorgos.tzampanakis@brilliance.eternal-september.org> <51ba82b5$0$29997$c3e8da3$5496439d@news.astraweb.com>
Date: Fri, 14 Jun 2013 19:30:38 +1000
Subject: Re: Debugging memory leaks
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.3279.1371202240.3114.python-list@python.org>
Lines: 41
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1371202240 news.xs4all.nl 16009 [2001:888:2000:d::a6]:46913
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:48101

On Fri, Jun 14, 2013 at 12:40 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Thu, 13 Jun 2013 20:15:42 +0000, Giorgos Tzampanakis wrote:
>
>>> Therefore: if the leak seems to be small, it may be much more advicable
>>> to restart your process periodically (during times where a restart does
>>> not hurt much) rather than try to find (and fix) the leaks. Only when
>>> the leak is large enough that it would force you to too frequent
>>> restarts, a deeper analysis may be advicable (large leaks are easier to
>>> locate as well).
>>
>>
>> Am I the only one who thinks this is terrible advice?
> Sub-optimal, maybe, but terrible? Not even close. Terrible advice would
> be "open up all the ports on your firewall, that will fix it!"
>...
>
> My advice is to give yourself a deadline:
>
> "If I have not found the leak in one week, or found and fixed it in three
> weeks, then I'll probably never fix it and I should just give up and
> apply palliative reboots to work around the problem."
>
> Either that or hire an expert at debugging memory leaks.

It's terrible advice in generality, because it encourages a sloppiness
of thinking: "Memory usage doesn't matter, we'll just instruct people
to reset everything now and then". When you have a problem on your
hands, you always have to set a deadline [1] but sometimes you have to
set the deadline the other way, too: "I'll just reboot it now, but if
it runs out of memory within a week, I *have* to find the problem".
Also, I think everyone should have at least one shot at a project that
has to stay up for multiple months, preferably a year. Even if you
never actually achieve a whole year of uptime, *think* that way. It'll
help you get things into perspective: "If I were running this all
year, that might be an issue, but who cares about a memory leak in a
script that's going to be finished in an hour!".

[1] cf http://www.gnu.org/fun/jokes/last.bug.html

ChrisA