Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #107057

Re: Python garbage collection: not releasing memory to OS!

Show all headers | View raw

On 04/15/2016 04:25 AM, cshintov@gmail.com wrote:
> The input was a 4MB file. Even after returning from the 'fileopen'
> function the 4MB memory was not released. I checked htop output while
> the loop was running, the resident memory stays at 14MB. So unless
> the process is stopped the memory stays with it.

I guess the question is, why is this a problem?  If there are no leaks,
then I confess I don't understand what your concern is.  And indeed you
say it's not leaking as it never rises above 14 MB.

Also there are ways of reading a file without allocating huge amounts of
memory.  Why not read it in in chunks, or in lines.  Take advantage of
Python's generator facilities to process your data.

> So if the celery worker is not killed after its task is finished it
> is going to keep the memory for itself. I know I can use
> **max_tasks_per_child** config value to kill the process and spawn a
> new one. **Is there any other way to return the memory to OS from a
> python process?.**

Have you tried using the subprocess module of python? If I understand it
correctly, this would allow you to run python code as a subprocess
(completely separate process), which would be completely reaped by the
OS when it's finished.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python garbage collection: not releasing memory to OS! cshintov@gmail.com - 2016-04-15 03:25 -0700
  Re: Python garbage collection: not releasing memory to OS! Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2016-04-15 08:24 -0400
  Re: Python garbage collection: not releasing memory to OS! Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2016-04-15 14:02 +0100
  Re: Python garbage collection: not releasing memory to OS! Michael Torrie <torriem@gmail.com> - 2016-04-15 07:59 -0600
  Re: Python garbage collection: not releasing memory to OS! Sam <python@net153.net> - 2016-04-15 09:27 -0500

csiph-web