Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
User-Agent: K-9 Mail for Android
In-Reply-To: <slrnkrkacg.2u4.giorgos.tzampanakis@brilliance.eternal-september.org>
References: <09917103-b35e-4728-8fea-bcb4ce2bd1af@googlegroups.com> <mailman.3165.1371104977.3114.python-list@python.org> <slrnkrkacg.2u4.giorgos.tzampanakis@brilliance.eternal-september.org>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----5BAP8JN4YS2BEDC7B9GVFXM793V7XK"
Subject: Re: Debugging memory leaks
From: Steve Simmons <square.steve@gmail.com>
Date: Thu, 13 Jun 2013 22:45:16 +0100
To: Giorgos Tzampanakis <giorgos.tzampanakis@gmail.com>, python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.3232.1371162031.3114.python-list@python.org>
Lines: 60
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:48032

------5BAP8JN4YS2BEDC7B9GVFXM793V7XK
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 8bit

Giorgos Tzampanakis <giorgos.tzampanakis@gmail.com> wrote:

>On 2013-06-13, dieter wrote:
>
>>> ...  Anyway, my real question is how to go about debugging memory
>leak
>>> problems in Python, particularly for a long running server process
>>> written with Twisted. I'm not sure how to use heapy or guppy, and
>>> objgraph doesn't tell me enough to locate the problem.
>>
>> Analysing memory leaks is really difficult: huge amounts of data is
>> involved and usually, it is almost impossible to determine which of
>the
>> mentioned objects are leaked and which are rightfully in use.  In
>> addition, long running Python processes usually have degrading memory
>> use - due to memory fragmentation. There is nothing you can do
>against
>> this.
>>
>> Therefore: if the leak seems to be small, it may be much more
>advicable
>> to restart your process periodically (during times where a restart
>does
>> not hurt much) rather than try to find (and fix) the leaks. Only when
>> the leak is large enough that it would force you to too frequent
>> restarts, a deeper analysis may be advicable (large leaks are easier
>to
>> locate as well).
>
>
>Am I the only one who thinks this is terrible advice?
>
>-- 
>Real (i.e. statistical) tennis and snooker player rankings and ratings:
>http://www.statsfair.com/ 
>-- 
>http://mail.python.org/mailman/listinfo/python-list

No you are not alone.  Ignoring a bug is only sensible if you absolutely understand what is going wrong - and by the time you understand the problem that well, you probably have enough understanding to fix it.  If tools are available (as the OP knows), then learn them and use them to find/fix the bug. 
Steve S

Sent from a Galaxy far far away
------5BAP8JN4YS2BEDC7B9GVFXM793V7XK
Content-Type: text/html;
 charset=utf-8
Content-Transfer-Encoding: 8bit

<html><head/><body><html><head></head><body><div class="gmail_quote">Giorgos Tzampanakis &lt;giorgos.tzampanakis@gmail.com&gt; wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre style="white-space: pre-wrap; word-wrap:break-word; font-family: monospace; margin-top: 0px">On 2013-06-13, dieter wrote:<br /><br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">...  Anyway, my real question is how to go about debugging memory leak<br />problems in Python, particularly for a long running server process<br />written with Twisted. I'm not sure how to use heapy or guppy, and<br />objgraph doesn't tell me enough to locate the problem.</blockquote><br />Analysing memory leaks is really difficult: huge amounts of data is<br />involved and usually, it is almost impossible to determine which of the<br />mentioned objects are leaked and which are rightfully in use.  In<br />addition, long running Python processes usually have degrading memory<br />use - due to memory fragmentation.
There is nothing you can do against<br />this.<br /><br />Therefore: if the leak seems to be small, it may be much more advicable<br />to restart your process periodically (during times where a restart does<br />not hurt much) rather than try to find (and fix) the leaks. Only when<br />the leak is large enough that it would force you to too frequent<br />restarts, a deeper analysis may be advicable (large leaks are easier to<br />locate as well).</blockquote><br /><br />Am I the only one who thinks this is terrible advice?<br /></pre></blockquote></div><br clear="all">No you are not alone.  Ignoring a bug is only sensible if you absolutely understand what is going wrong - and by the time you understand the problem that well, you probably have enough understanding to fix it.  If tools are available (as the OP knows), then learn them and use them to find/fix the bug. <br>
Steve S<br>
<br>
Sent from a Galaxy far far away</body></html></body></html>
------5BAP8JN4YS2BEDC7B9GVFXM793V7XK--