Path: csiph.com!optima2.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!newsreader4.netcologne.de!news.netcologne.de!newsfeed0.kamp.net!newsfeed.kamp.net!fu-berlin.de!uni-berlin.de!not-for-mail From: Dennis Lee Bieber Newsgroups: comp.lang.python Subject: Re: Python garbage collection: not releasing memory to OS! Date: Fri, 15 Apr 2016 08:24:04 -0400 Organization: IISS Elusive Unicorn Lines: 23 Message-ID: References: <215cf2a4-c2fa-41d0-8c49-23b9494234d4@googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Trace: news.uni-berlin.de Rs+F4cGXTjK6gDO3R02krAwxiDvf7ZnfQSndW1ZLTsDA== Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.002 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'subject:Python': 0.05; 'determines': 0.09; 'message-id:@4ax.com': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'spawn': 0.09; 'python': 0.10; 'itself.': 0.11; 'subject:not': 0.11; '2016': 0.16; 'adjacent': 0.16; 'devs': 0.16; 'heap': 0.16; 'received:80.91.229.3': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'reclaim': 0.16; 'run-time.': 0.16; 'system-wide': 0.16; 'memory': 0.17; 'config': 0.18; 'url:home': 0.18; 'windows': 0.20; 'space.': 0.22; 'finished': 0.23; 'header:X-Complaints-To:1': 0.26; 'fri,': 0.27; 'question': 0.27; 'allocated': 0.27; 'idea': 0.28; 'thinks': 0.29; 'objects': 0.29; 'system,': 0.30; 'task': 0.30; "i'd": 0.31; "can't": 0.32; 'addresses': 0.32; 'possibly': 0.32; 'returned': 0.32; 'maybe': 0.33; '-0700': 0.33; 'throughout': 0.34; 'running': 0.34; 'could': 0.35; 'skip:* 20': 0.35; 'there': 0.36; 'to:addr :python-list': 0.36; 'subject:: ': 0.37; 'operating': 0.37; 'received:org': 0.37; 'charset:us-ascii': 0.37; 'virtual': 0.38; 'data': 0.39; 'does': 0.39; 'enough': 0.39; 'to:addr:python.org': 0.40; 'entire': 0.61; 'email addr:gmail.com': 0.62; 'more': 0.63; 'counts': 0.81; '**is': 0.84; 'seldom': 0.84; '"how': 0.91; 'dennis': 0.91; 'different.': 0.91; 'killed': 0.91; 'received:108': 0.93 X-Injected-Via-Gmane: http://gmane.org/ X-Gmane-NNTP-Posting-Host: adsl-108-73-119-88.dsl.klmzmi.sbcglobal.net X-Newsreader: Forte Agent 6.00/32.1186 X-No-Archive: YES X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Mailman-Original-Message-ID: X-Mailman-Original-References: <215cf2a4-c2fa-41d0-8c49-23b9494234d4@googlegroups.com> Xref: csiph.com comp.lang.python:107052 On Fri, 15 Apr 2016 03:25:39 -0700 (PDT), cshintov@gmail.com declaimed the following: >So if the celery worker is not killed after its task is finished it is going to keep the memory for itself. I know I can use **max_tasks_per_child** config value to kill the process and spawn a new one. **Is there any other way to return the memory to OS from a python process?.** How would you do that from any other process? If it's easy, I'd think the Python devs would have considered adding it to the run-time. IOWs, the question is more likely to be "how does the OS reclaim memory from running processes". In many operating systems -- it only does that when cleaning up the entire process space. Memory allocated for the heap seldom is reclaimed piecemeal as heap objects may be scattered throughout the allocation and can't be moved. No idea how Windows could be different. Possibly it only counts pages that actually have live data (how it determines that I don't know -- maybe malloc/free use a system-wide pool of virtual memory so when a large enough block of adjacent addresses is freed the entire page is returned to the system, even though the process thinks it is using a contiguous allotment). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed@ix.netcom.com HTTP://wlfraed.home.netcom.com/