Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #40948

Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64)

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

In general, it is hard for any process to return the memory the OS allocate
to it back to the OS, short of exiting the whole process.  The only case
that this works reliably is when the process allocates a chunk of memory by
mmap (which is chosen by libc if it malloc or calloc a large chunk of
memory), and that whole chunk is not needed any more.  In that case the
process can munmap it.  Evidently you are not see that in your program.
What you allocate might be too small (so libc choose to allocate it using
another system call "sbrk"), or that the allocated memory also hold other
objects not freed.

If you want to reduce the footprint of a long running program that
periodically allocates a large chunk of memory, the "easiest" solution is
to fork a different process to achieve the computations that needs the
memory.  That way, you can exit the process after you complete the
computation, and at that point all memory allocated to it is guaranteed to
be freed to the OS.

Modules like multiprocessing probably make the idea sufficiently easy to
implement.


On Sat, Mar 9, 2013 at 4:07 PM, Wong Wah Meng-R32813
<r32813@freescale.com>wrote:

>
>
>         If the memory usage is continually growing, you have something
> else that is a problem -- something is holding onto objects. Even if Python
> is not returning memory to the OS, it should be reusing the memory it has
> if objects are being freed.
> --
> [] Yes I have verified my python application is reusing the memory (just
> that it doesn't reduce once it has grown) and my python process doesn't
> have any issue to run even though it is seen taking up more than 2G in
> footprint. My problem is capacity planning on the server whereby since my
> python process doesn't release memory back to the OS, the OS wasn't able to
> allocate memory when a new process is spawn off.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-05 23:45 -0800
  Re: sync databse table based on current directory data without losign previous values Lele Gaifax <lele@metapensiero.it> - 2013-03-06 09:19 +0100
    Re: sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-06 00:57 -0800
      Re: sync databse table based on current directory data without losign previous values Lele Gaifax <lele@metapensiero.it> - 2013-03-06 10:24 +0100
        Re: sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-06 01:41 -0800
          Re: sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-06 01:43 -0800
            Re: sync databse table based on current directory data without losign previous values Bryan Devaney <bryan.devaney@gmail.com> - 2013-03-06 02:15 -0800
            Re: sync databse table based on current directory data without losign previous values Bryan Devaney <bryan.devaney@gmail.com> - 2013-03-06 02:15 -0800
          Re: sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-06 01:43 -0800
          Re: sync databse table based on current directory data without losign previous values Lele Gaifax <lele@metapensiero.it> - 2013-03-06 11:27 +0100
          Re: sync databse table based on current directory data without losign previous values Dave Angel <davea@davea.name> - 2013-03-06 08:31 -0500
          Re: sync databse table based on current directory data without losign previous values Lele Gaifax <lele@metapensiero.it> - 2013-03-06 15:16 +0100
        Re: sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-06 01:41 -0800
    Re: sync databse table based on current directory data without losign previous values Νίκος Γκρ33κ <nikos.gr33k@gmail.com> - 2013-03-06 00:57 -0800
  Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Wong Wah Meng-R32813 <r32813@freescale.com> - 2013-03-06 10:11 +0000
    Re: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Bryan Devaney <bryan.devaney@gmail.com> - 2013-03-06 02:25 -0800
      RE: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Wong Wah Meng-R32813 <r32813@freescale.com> - 2013-03-06 12:31 +0000
      Re: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Dave Angel <davea@davea.name> - 2013-03-06 08:18 -0500
      Re: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Dave Angel <davea@davea.name> - 2013-03-06 08:25 -0500
    Re: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Bryan Devaney <bryan.devaney@gmail.com> - 2013-03-06 02:25 -0800
    Re: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-03-06 23:34 +0000
      RE: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Wong Wah Meng-R32813 <r32813@freescale.com> - 2013-03-07 06:33 +0000
      Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Chris Angelico <rosuav@gmail.com> - 2013-03-07 18:19 +1100
      RE: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Wong Wah Meng-R32813 <r32813@freescale.com> - 2013-03-08 09:08 +0000
      Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-03-08 19:40 -0500
      RE: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Wong Wah Meng-R32813 <r32813@freescale.com> - 2013-03-09 08:07 +0000
        Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Grant Edwards <invalid@invalid.invalid> - 2013-03-09 19:18 +0000
          Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Roy Smith <roy@panix.com> - 2013-03-09 15:04 -0500
            Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Grant Edwards <invalid@invalid.invalid> - 2013-03-09 20:35 +0000
              Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Roy Smith <roy@panix.com> - 2013-03-09 16:44 -0500
                Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Grant Edwards <invalid@invalid.invalid> - 2013-03-11 14:27 +0000
      Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Dave Angel <davea@davea.name> - 2013-03-09 06:02 -0500
      Re: Set x to to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Isaac To <isaac.to@gmail.com> - 2013-03-09 23:02 +0800
  Re: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Terry Reedy <tjreedy@udel.edu> - 2013-03-06 05:59 -0500
  Re: sync databse table based on current directory data without losign previous values Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-03-06 11:52 +0000
  RE: Set x to  to None and del x doesn't release memory in python 2.7.1 (HPUX 11.23, ia64) Wong Wah Meng-R32813 <r32813@freescale.com> - 2013-03-06 12:36 +0000
  Re: sync databse table based on current directory data without losign previous values Chris Angelico <rosuav@gmail.com> - 2013-03-07 00:40 +1100
  Re: sync databse table based on current directory data without losign previous values "Michael Ross" <gmx@ross.cx> - 2013-03-06 15:04 +0100
    Re: sync databse table based on current directory data without losign previous values nagia.retsina@gmail.com - 2013-03-06 08:09 -0800
    Re: sync databse table based on current directory data without losign previous values nagia.retsina@gmail.com - 2013-03-06 08:09 -0800

csiph-web