Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!feeder.news-service.com!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <ms419@freezone.co.uk>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.006
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'python,': 0.01; 'python': 0.08; 'append': 0.09; 'googling': 0.09; 'graph': 0.09; 'url:github': 0.09; 'hopefully': 0.15; 'code?': 0.16; 'measured': 0.16; 'naive': 0.16; 'runs,': 0.16; 'subject:memory': 0.16; 'subject:used': 0.16; 'x-mailer:evolution 2.32.3': 0.16; 'once,': 0.18; 'wrote': 0.20; 'memory': 0.21; 'file,': 0.21; 'changed': 0.24; "i'm": 0.27; 'compare': 0.28; 'postfix': 0.30; 'subject:?': 0.31; 'does': 0.32; 'anyone': 0.32; 'actually': 0.33; 'to:addr :python-list': 0.33; 'it?': 0.33; 'filter': 0.34; 'quite': 0.34; 'uses': 0.35; 'hack': 0.37; 'run': 0.37; 'but': 0.37; 'two': 0.37; 'could': 0.38; 'some': 0.38; 'should': 0.38; 'received:192': 0.39; 'header:Mime-Version:1': 0.39; 'else': 0.39; 'to:addr:python.org': 0.39; "i'd": 0.40; 'periodically': 0.67; 'received:184': 0.67; 'revealed': 0.68; 'graphs': 0.84; 'measuring': 0.84; 'url:cookie': 0.84
X-Cloudmark-SP-Filtered: true
X-Cloudmark-SP-Result: v=1.1 cv=t+ZAJ7mrPn0ghCTQRrYVa3vbw5sbuRhC9KBpKnc45f0= c=1 sm=1 a=7CP8D3MB8GoA:10 a=qWYXSweIvAIA:10 a=BLceEmwcHowA:10 a=LYYm-LO_OfoA:10 a=RBKHmHSr9-IA:10 a=IkcTkHD0fZMA:10 a=b0KxtKZxmKbTALECVfpD+Q==:17 a=NEAV23lmAAAA:8 a=Ry9LkYuwdLlMD3ylXdoA:9 a=kY44adlLoAtbSp4YQJcA:7 a=QEXdDO2ut3YA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117
X-IronPort-AV: E=Sophos;i="4.68,245,1312178400"; d="scan'208";a="314359104"
X-reinject: true
Subject: Measure the amount of memory used?
From: Jack Bates <ms419@freezone.co.uk>
To: python-list@python.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Date: Thu, 18 Aug 2011 08:08:20 -0700
Mime-Version: 1.0
X-Mailer: Evolution 2.32.3
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.166.1313680196.27778.python-list@python.org>
Lines: 23
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1313680196 news.xs4all.nl 23856 [2001:888:2000:d::a6]:41416
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:11766

I wrote a content filter for Postfix with Python,
https://github.com/jablko/cookie

It should get started once, and hopefully run for a long time - so I'm
interested in how it uses memory:

 1) How does the amount of memory used change as it runs?

 2) How does the amount of memory used change as I continue to hack on
it, and change the code?

My naive thought was that I'd periodically append to a file, the virtual
memory size from /proc/[pid]/stat and a timestamp. From this a could
make a graph of the amount of memory used as my content filter runs, and
I could compare two graphs to get a clue whether this amount changed as
I continue to hack

 - but some Googling quickly revealed that measuring memory is actually
quite complicated? Neither the virtual memory size nor the "resident set
size" accurately measure the amount of memory used by a process

Has anyone else measured the memory used by a Python program? How did
you do it?