Path: csiph.com!eternal-september.org!feeder.eternal-september.org!border1.nntp.ams1.giganews.com!nntp.giganews.com!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Mark Lawrence <breamoreboy@yahoo.co.uk>
Subject: Re: memory control in Python
Date: Sat, 15 Aug 2015 19:21:04 +0100
References: <CAM5e_kbbTAEHE=mi0WFXutJ5G2A-UV4wLsAPMcBT5XrCqhr3MA@mail.gmail.com> <87io8hc8z3.fsf@handshake.de> <mqnsrm$mek$1@ger.gmane.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0
In-Reply-To: <mqnsrm$mek$1@ger.gmane.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.32.1439662878.4764.python-list@python.org>
Lines: 48
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:95396

On 15/08/2015 18:28, Terry Reedy wrote:
> On 8/15/2015 3:21 AM, dieter wrote:
>> Ping Liu <yanzhipingliu@gmail.com> writes:
>>> ...
>>> For small cases, Python works well. But if we consider longer time
>>> period.
>>> then it would fail due to the memory usage issues. We have tested
>>> several
>>> case studies to check the memory use for different time period,
>>> including
>>> 1) 2 hours in one day, 2) 24 hours in one day, 3) 20 days with 24 hours
>>> each day, as well as 4) 30 days with 24 hours each day. The first 3
>>> cases
>>> are feasible while the last case gives out the memory error.
>>>
>>> When we are testing the forth case, the memory error comes out while
>>> creating the inequality constraints. The problem size is 1) Aeq: 12 *
>>> 26,
>>> Aineq: 30 * 26; 2) Aeq: 144*268, Aineq:316*268; 3) Aeq: 2880*5284,
>>> Aineq:
>>> 6244*5284; 4) Aeq: 4320 * 7924, Aineq is unknown due to the memory
>>> error.
>>>
>>> The solver is CPLEX (academic). It turns out that the solver is taking a
>>> lot of memory as you can see in the memory test report. for the first
>>> three
>>> cases, different memory usage is observed, and it grows up dramatically
>>> with the increase of the time period. 1) solver memory usage: 25.6
>>> MB, 2)
>>> 19.5 MB; 3) solver memory usage: 830.0742 MB.
>
> Make sure that the solver is using numpy arrays.
>

I doubt that as CPLEX was first released in 1988.

A very quick bit of searching found this 
http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html 
which I believe is the equivalent of the Aeq and Aineq mentioned above. 
  Possibly a better option as must surely be using numpy, but as usual 
there's only one way to find out? :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence