Measure the amount of memory used?

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!feeder.news-service.com!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<ms419@freezone.co.uk>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.006
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'python,': 0.01; 'python': 0.08; 'append': 0.09; 'googling': 0.09; 'graph': 0.09; 'url:github': 0.09; 'hopefully': 0.15; 'code?': 0.16; 'measured': 0.16; 'naive': 0.16; 'runs,': 0.16; 'subject:memory': 0.16; 'subject:used': 0.16; 'x-mailer:evolution 2.32.3': 0.16; 'once,': 0.18; 'wrote': 0.20; 'memory': 0.21; 'file,': 0.21; 'changed': 0.24; "i'm": 0.27; 'compare': 0.28; 'postfix': 0.30; 'subject:?': 0.31; 'does': 0.32; 'anyone': 0.32; 'actually': 0.33; 'to:addr :python-list': 0.33; 'it?': 0.33; 'filter': 0.34; 'quite': 0.34; 'uses': 0.35; 'hack': 0.37; 'run': 0.37; 'but': 0.37; 'two': 0.37; 'could': 0.38; 'some': 0.38; 'should': 0.38; 'received:192': 0.39; 'header:Mime-Version:1': 0.39; 'else': 0.39; 'to:addr:python.org': 0.39; "i'd": 0.40; 'periodically': 0.67; 'received:184': 0.67; 'revealed': 0.68; 'graphs': 0.84; 'measuring': 0.84; 'url:cookie': 0.84
X-Cloudmark-SP-Filtered	true
X-Cloudmark-SP-Result	v=1.1 cv=t+ZAJ7mrPn0ghCTQRrYVa3vbw5sbuRhC9KBpKnc45f0= c=1 sm=1 a=7CP8D3MB8GoA:10 a=qWYXSweIvAIA:10 a=BLceEmwcHowA:10 a=LYYm-LO_OfoA:10 a=RBKHmHSr9-IA:10 a=IkcTkHD0fZMA:10 a=b0KxtKZxmKbTALECVfpD+Q==:17 a=NEAV23lmAAAA:8 a=Ry9LkYuwdLlMD3ylXdoA:9 a=kY44adlLoAtbSp4YQJcA:7 a=QEXdDO2ut3YA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117
X-IronPort-AV	E=Sophos;i="4.68,245,1312178400"; d="scan'208";a="314359104"
X-reinject	true
Subject	Measure the amount of memory used?
From	Jack Bates <ms419@freezone.co.uk>
To	python-list@python.org
Content-Type	text/plain; charset="UTF-8"
Content-Transfer-Encoding	quoted-printable
Date	Thu, 18 Aug 2011 08:08:20 -0700
Mime-Version	1.0
X-Mailer	Evolution 2.32.3
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.166.1313680196.27778.python-list@python.org> (permalink)
Lines	23
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1313680196 news.xs4all.nl 23856 [2001:888:2000:d::a6]:41416
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:11766

Show key headers only | View raw

I wrote a content filter for Postfix with Python,
https://github.com/jablko/cookie

It should get started once, and hopefully run for a long time - so I'm
interested in how it uses memory:

 1) How does the amount of memory used change as it runs?

 2) How does the amount of memory used change as I continue to hack on
it, and change the code?

My naive thought was that I'd periodically append to a file, the virtual
memory size from /proc/[pid]/stat and a timestamp. From this a could
make a graph of the amount of memory used as my content filter runs, and
I could compare two graphs to get a clue whether this amount changed as
I continue to hack

 - but some Googling quickly revealed that measuring memory is actually
quite complicated? Neither the virtual memory size nor the "resident set
size" accurately measure the amount of memory used by a process

Has anyone else measured the memory used by a Python program? How did
you do it?

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Measure the amount of memory used? Jack Bates <ms419@freezone.co.uk> - 2011-08-18 08:08 -0700
  Re: Measure the amount of memory used? John Gordon <gordon@panix.com> - 2011-08-18 15:39 +0000
  Re: Measure the amount of memory used? MrJean1 <mrjean1@gmail.com> - 2011-08-18 19:07 -0700

csiph-web