Re: "High water" Memory fragmentation still a thing?

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<python-python-list@m.gmane.org>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901; t=1412363830; bh=HCsxigiP9OXwMyI57hAigg8c0AFiGQfGGsJNUohJ0Jw=; h=To:From:Subject:Date:References:From; b=DOhzUYduRXInI7ZxdhaZwbGRpvuwscQQJd4QWjyQgs5RVB12tjJjV7ZXkfyhimCKn YMR2Ab9ULRY6YzPVrgNh5Q7Se28P49Bzo0zvUR77cLdACyozo/oJ/45+PrqVlwKtb7 2QD2eApdGLWWmmEUgfu6Cjswe2Bb8KikwXfBMq4A=
X-Spam-Status	OK 0.005
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'assuming': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:skip:f 10': 0.09; 'subject:still': 0.09; 'python': 0.11; 'from:name:antoine pitrou': 0.16; 'heap': 0.16; 'message- id:@post.gmane.org': 0.16; 'received:213.41.240': 0.16; 'received:213.41.240.54': 0.16; 'received:80.91.229.3': 0.16; 'received:charlus.yi.org': 0.16; 'received:plane.gmane.org': 0.16; 'received:yi.org': 0.16; 'extensions': 0.16; 'from:addr:python.org': 0.16; 'memory': 0.22; 'header:User- Agent:1': 0.23; 'exhibit': 0.24; 'versions': 0.24; 'header:X -Complaints-To:1': 0.27; "doesn't": 0.30; 'os,': 0.31; 'question:': 0.31; 'writes:': 0.31; 'quite': 0.32; 'common': 0.35; 'charset:us-ascii': 0.36; 'subject:?': 0.36; 'wrong': 0.37; 'to:addr:python-list': 0.38; 'recent': 0.39; 'does': 0.39; 'itself': 0.39; 'subject:" ': 0.39; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'release': 0.40; 'improved': 0.60; 'article': 0.77; 'antoine.': 0.84; 'leak': 0.84; 'leak.': 0.84; 'pools': 0.84; 'subject:High': 0.84
X-Injected-Via-Gmane	http://gmane.org/
To	python-list@python.org
From	Antoine Pitrou <antoine@python.org>
Subject	Re: "High water" Memory fragmentation still a thing?
Date	Fri, 3 Oct 2014 19:16:57 +0000 (UTC)
References	<CANm61jc=pB7n+5kuea6bmYXTbo3hXLT8d4JZx3Yq-TBtgGrFOQ@mail.gmail.com>
Mime-Version	1.0
Content-Type	text/plain; charset=us-ascii
Content-Transfer-Encoding	7bit
X-Gmane-NNTP-Posting-Host	sea.gmane.org
User-Agent	Loom/3.14 (http://gmane.org/)
X-Loom-IP	213.41.240.54 (Mozilla/5.0 (X11; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0)
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.14582.1412363831.18130.python-list@python.org> (permalink)
Lines	23
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1412363831 news.xs4all.nl 2880 [2001:888:2000:d::a6]:60829
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:78662

Show key headers only | View raw

Hello,

Croepha <croepha <at> gmail.com> writes:
> 
> Question:
> 
> Does python in general terms (apart from extensions or gc manipulation),
exhibit a "high water" type leak of allocated memory in recent python
versions (2.7+)?

It is not a leak. It is a quite common pattern of memory fragmentation.
The article is wrong in assuming that Python doesn't return the memory to
the OS. Python does return its empty memory pools to the OS, however the OS
itself may not be able to release that memory, because of heap fragmentation.

As the article mentions, this was improved (mostly fixed?) in 3.3.

Regards

Antoine.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: "High water" Memory fragmentation still a thing? Antoine Pitrou <antoine@python.org> - 2014-10-03 19:16 +0000

csiph-web