Re: "High water" Memory fragmentation still a thing?

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin3!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<python-python-list@m.gmane.org>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901; t=1412365878; bh=W4YUPBrmPxZG6gqedFABwVeAXZ300e2J1UupZOMEwZ4=; h=To:From:Subject:Date:References:In-Reply-To:From; b=gfWHThWf8KlUvvF1cNnHOel394PsLXp+EzKHtuavlBjuDpm0U4LXp5JH1xMpsgXfc d6aLIxavgXXTUBFUkuVqnPu4FjJbBRfItRcFh5TpEG9xosQXz3u8TDmNTyKgzngjTp 7IvOyZ3HqHFraVe2bg/CMER1j39x1W+Q09M+iRH4=
X-Spam-Status	OK 0.006
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'memory.': 0.07; 'referring': 0.07; 'assuming': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:skip:f 10': 0.09; 'subject:still': 0.09; 'python': 0.11; 'antoine': 0.16; 'confuse': 0.16; 'from:name:christian heimes': 0.16; 'heap': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'from:addr:python.org': 0.16; 'wrote:': 0.18; 'memory': 0.22; 'header:User-Agent:1': 0.23; 'space.': 0.24; 'tend': 0.24; 'header:X-Complaints-To:1': 0.27; 'header:In-Reply-To:1': 0.27; "doesn't": 0.30; 'os,': 0.31; 'run': 0.32; 'quite': 0.32; 'running': 0.33; 'totally': 0.33; 'common': 0.35; 'something': 0.35; 'but': 0.35; 'subject:?': 0.36; 'virtual': 0.37; 'wrong': 0.37; 'to:addr:python-list': 0.38; 'does': 0.39; 'itself': 0.39; 'subject:" ': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'release': 0.40; 'address': 0.63; 'received:79': 0.64; 'different': 0.65; 'size.': 0.65; 'charset:windows-1252': 0.65; 'physical': 0.72; 'article': 0.77; 'leak.': 0.84; 'pools': 0.84; 'subject:High': 0.84; 'hundred': 0.95
X-Injected-Via-Gmane	http://gmane.org/
To	python-list@python.org
From	Christian Heimes <christian@python.org>
Subject	Re: "High water" Memory fragmentation still a thing?
Date	Fri, 03 Oct 2014 21:51:02 +0200
References	<CANm61jc=pB7n+5kuea6bmYXTbo3hXLT8d4JZx3Yq-TBtgGrFOQ@mail.gmail.com> <loom.20141003T211459-72@post.gmane.org>
Mime-Version	1.0
Content-Type	text/plain; charset=windows-1252
Content-Transfer-Encoding	7bit
X-Gmane-NNTP-Posting-Host	p4fd27685.dip0.t-ipconnect.de
User-Agent	Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2
In-Reply-To	<loom.20141003T211459-72@post.gmane.org>
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.14583.1412365879.18130.python-list@python.org> (permalink)
Lines	15
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1412365879 news.xs4all.nl 2922 [2001:888:2000:d::a6]:36115
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:78663

Show key headers only | View raw

On 03.10.2014 21:16, Antoine Pitrou wrote:
> It is not a leak. It is a quite common pattern of memory fragmentation.
> The article is wrong in assuming that Python doesn't return the memory to
> the OS. Python does return its empty memory pools to the OS, however the OS
> itself may not be able to release that memory, because of heap fragmentation.

The article doesn't state if the writer is referring to virtual memory
or resident set size. For long-running 32bit processes it is quite
common to run out of virtual address space. But that's something totally
different than running out of memory. A 64bit process can have a virtual
address size of several GB but only occupy a few hundred MB of physical
memory. People tend to confuse the meaning of VSZ and RSS and forget
about paging.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: "High water" Memory fragmentation still a thing? Christian Heimes <christian@python.org> - 2014-10-03 21:51 +0200

csiph-web