Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #107052

Re: Python garbage collection: not releasing memory to OS!

Path csiph.com!optima2.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!newsreader4.netcologne.de!news.netcologne.de!newsfeed0.kamp.net!newsfeed.kamp.net!fu-berlin.de!uni-berlin.de!not-for-mail
From Dennis Lee Bieber <wlfraed@ix.netcom.com>
Newsgroups comp.lang.python
Subject Re: Python garbage collection: not releasing memory to OS!
Date Fri, 15 Apr 2016 08:24:04 -0400
Organization IISS Elusive Unicorn
Lines 23
Message-ID <mailman.20.1460723049.6324.python-list@python.org> (permalink)
References <215cf2a4-c2fa-41d0-8c49-23b9494234d4@googlegroups.com> <trm1hb16b6slo2qts756iv77kgkls0hgkm@4ax.com>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
X-Trace news.uni-berlin.de Rs+F4cGXTjK6gDO3R02krAwxiDvf7ZnfQSndW1ZLTsDA==
Return-Path <python-python-list@m.gmane.org>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.002
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'subject:Python': 0.05; 'determines': 0.09; 'message-id:@4ax.com': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'spawn': 0.09; 'python': 0.10; 'itself.': 0.11; 'subject:not': 0.11; '2016': 0.16; 'adjacent': 0.16; 'devs': 0.16; 'heap': 0.16; 'received:80.91.229.3': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'reclaim': 0.16; 'run-time.': 0.16; 'system-wide': 0.16; 'memory': 0.17; 'config': 0.18; 'url:home': 0.18; 'windows': 0.20; 'space.': 0.22; 'finished': 0.23; 'header:X-Complaints-To:1': 0.26; 'fri,': 0.27; 'question': 0.27; 'allocated': 0.27; 'idea': 0.28; 'thinks': 0.29; 'objects': 0.29; 'system,': 0.30; 'task': 0.30; "i'd": 0.31; "can't": 0.32; 'addresses': 0.32; 'possibly': 0.32; 'returned': 0.32; 'maybe': 0.33; '-0700': 0.33; 'throughout': 0.34; 'running': 0.34; 'could': 0.35; 'skip:* 20': 0.35; 'there': 0.36; 'to:addr :python-list': 0.36; 'subject:: ': 0.37; 'operating': 0.37; 'received:org': 0.37; 'charset:us-ascii': 0.37; 'virtual': 0.38; 'data': 0.39; 'does': 0.39; 'enough': 0.39; 'to:addr:python.org': 0.40; 'entire': 0.61; 'email addr:gmail.com': 0.62; 'more': 0.63; 'counts': 0.81; '**is': 0.84; 'seldom': 0.84; '"how': 0.91; 'dennis': 0.91; 'different.': 0.91; 'killed': 0.91; 'received:108': 0.93
X-Injected-Via-Gmane http://gmane.org/
X-Gmane-NNTP-Posting-Host adsl-108-73-119-88.dsl.klmzmi.sbcglobal.net
X-Newsreader Forte Agent 6.00/32.1186
X-No-Archive YES
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.21
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID <trm1hb16b6slo2qts756iv77kgkls0hgkm@4ax.com>
X-Mailman-Original-References <215cf2a4-c2fa-41d0-8c49-23b9494234d4@googlegroups.com>
Xref csiph.com comp.lang.python:107052

Show key headers only | View raw

On Fri, 15 Apr 2016 03:25:39 -0700 (PDT), cshintov@gmail.com declaimed the
following:

>So if the celery worker is not killed after its task is finished it is going to keep the memory for itself. I know I can use **max_tasks_per_child** config value to kill the process and spawn a new one. **Is there any other way to return the memory to OS from a python process?.**

	How would you do that from any other process? If it's easy, I'd think
the Python devs would have considered adding it to the run-time.

	IOWs, the question is more likely to be "how does the OS reclaim memory
from running processes". In many operating systems -- it only does that
when cleaning up the entire process space. Memory allocated for the heap
seldom is reclaimed piecemeal as heap objects may be scattered throughout
the allocation and can't be moved.

	No idea how Windows could be different. Possibly it only counts pages
that actually have live data (how it determines that I don't know -- maybe
malloc/free use a system-wide pool of virtual memory so when a large enough
block of adjacent addresses is freed the entire page is returned to the
system, even though the process thinks it is using a contiguous allotment).
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python garbage collection: not releasing memory to OS! cshintov@gmail.com - 2016-04-15 03:25 -0700
  Re: Python garbage collection: not releasing memory to OS! Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-15 08:24 -0400
  Re: Python garbage collection: not releasing memory to OS! Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2016-04-15 14:02 +0100
  Re: Python garbage collection: not releasing memory to OS! Michael Torrie <torriem@gmail.com> - 2016-04-15 07:59 -0600
  Re: Python garbage collection: not releasing memory to OS! Sam <python@net153.net> - 2016-04-15 09:27 -0500

csiph-web