Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #95390

Re: memory control in Python

Show all headers | View raw

Ping Liu <yanzhipingliu@gmail.com> writes:
> ...
> For small cases, Python works well. But if we consider longer time period.
> then it would fail due to the memory usage issues. We have tested several
> case studies to check the memory use for different time period, including
> 1) 2 hours in one day, 2) 24 hours in one day, 3) 20 days with 24 hours
> each day, as well as 4) 30 days with 24 hours each day. The first 3 cases
> are feasible while the last case gives out the memory error.
>
> When we are testing the forth case, the memory error comes out while
> creating the inequality constraints. The problem size is 1) Aeq: 12 * 26,
> Aineq: 30 * 26; 2) Aeq: 144*268, Aineq:316*268; 3) Aeq: 2880*5284, Aineq:
> 6244*5284; 4) Aeq: 4320 * 7924, Aineq is unknown due to the memory error.
>
> The solver is CPLEX (academic). It turns out that the solver is taking a
> lot of memory as you can see in the memory test report. for the first three
> cases, different memory usage is observed, and it grows up dramatically
> with the increase of the time period. 1) solver memory usage: 25.6 MB, 2)
> 19.5 MB; 3) solver memory usage: 830.0742 MB.

The C implementation of Python (called "CPython") does not use
memory compaction and places most of its objects on the heap.
Those implementations tend to suffer from memory fragmentation in
long running processes with large memory use.

You may consider "Jython" (a Python implementation in "Java") or
"IronPython" (a ".Net" based Python implementation) to get a
Python implementation with memory compaction.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: memory control in Python dieter <dieter@handshake.de> - 2015-08-15 09:21 +0200
  Re: memory control in Python Ping Liu <yanzhipingliu@gmail.com> - 2015-08-17 15:09 -0700
    Re: memory control in Python Chris Angelico <rosuav@gmail.com> - 2015-08-18 10:13 +1000
    Re: memory control in Python dieter <dieter@handshake.de> - 2015-08-18 08:02 +0200
    Re: memory control in Python Laura Creighton <lac@openend.se> - 2015-08-18 08:16 +0200
    Re: memory control in Python Rustom Mody <rustompmody@gmail.com> - 2015-08-18 01:56 -0700
      Re: memory control in Python Laura Creighton <lac@openend.se> - 2015-08-18 11:13 +0200

csiph-web