Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #10337 > unrolled thread
| Started by | Ralph Dratman <ralph.dratman@gmail.com> |
|---|---|
| First post | 2012-06-03 09:03 +0000 |
| Last post | 2012-06-07 09:19 +0000 |
| Articles | 4 — 4 participants |
Back to article view | Back to comp.soft-sys.math.mathematica
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: Memory Blowup Issues Ralph Dratman <ralph.dratman@gmail.com> - 2012-06-03 09:03 +0000
Re: Memory Blowup Issues Szabolcs Horvát <szhorvat@gmail.com> - 2012-06-05 08:51 +0000
Re: Memory Blowup Issues David Bailey <dave@removedbailey.co.uk> - 2012-06-06 08:50 +0000
Re: Memory Blowup Issues W Craig Carter <ccarter@mit.edu> - 2012-06-07 09:19 +0000
| From | Ralph Dratman <ralph.dratman@gmail.com> |
|---|---|
| Date | 2012-06-03 09:03 +0000 |
| Subject | Re: Memory Blowup Issues |
| Message-ID | <jqf98n$a65$1@smc.vnet.net> |
Kevin, Thank you for passing along those tips. I usually put $HistoryLength=1 and also turn off "Enable Notebook History Tracking" in the Preferences dialog -- though I'm not sure the latter makes any difference. I just want to re-emphasize how egregious I think the situation with crashing really is. It is unacceptable. I hope everyone is making that absolutely clear to WRI. How can we continue to recommend a program which often crashes? It defies common sense. Ralph On Sat, Jun 2, 2012 at 9:45 AM, Kevin J. McCann <kjm@kevinmccann.com> wrote: > I already have 12Gb; so, that is not a solution. Others have suggested > $HistoryLength = 0, which works. On the other hand, mvary = . did not change > the amount of memory in use at all. > > I guess acceptability of the memory/crash problem depends on your point of > view. If indeed the ability to access previous outputs is important to the > way you work (it is not to me), then I can see why it needs to be there even > though it could lead to a crash in extreme cases like mine. However, the > ability to limit this with $HistoryLength gives flexibility to the user. > > I do not make use of the whole In[] Out[] thing, and, in fact, turn those > labels off in all my notebooks. > > Kevin > > On 6/2/2012 5:48 AM, Ralph Dratman wrote: >> >> Solution: install more memory in your computer? >> >> That answer is sort of a wisecrack, but in real life, my Mac Mini now >> has 8 GB of RAM for the benefit of Mathematica -- yet the program >> still somehow manages to fill up memory and crash. That is why I keep >> glancing at the desktop memory monitor. If memory is almost full, I >> can usually stop Mathematica before things get really serious. >> >> That said, my experience has been that disposing of variables (using >> myvar=.) does help quite a bit. But I am not reading files very often. >> In your case, maybe something about reading a file is making it harder >> to free the memory used. >> >> By the way, in case you might be thinking that this whole crash >> problem is pretty unacceptable behavior for an app that sells for a >> lot of money in 2012.... yeah, you're right, it is. >> >> Ralph >> >> >> >> On Fri, Jun 1, 2012 at 5:18 AM, Kevin J. McCann<kjm@kevinmccann.com> >> wrote: >>> >>> I am dealing with some good sized data sets from satellite data. These >>> data are stored in hdf format. The problem is that I process a sequence >>> of these data sets by Importing and then processing; however, it seems >>> that the memory usage grows with each Import, even if I Remove the data. >>> >>> Here is a somewhat simplified example: >>> >>> MemoryInUse[] (* some data already read in *) >>> 2781549920 >>> >>> Data=Import[fileName,"Data"]; (* import a new data set *) >>> MemoryInUse[] >>> 3302168936 >>> >>> Data=Import[fileName,"Data"];(* import it again *) >>> MemoryInUse[] >>> 3822781920 >>> >>> Remove[Data] (* this has no effect on the memory usage *) >>> MemoryInUse[] >>> 3822787840 >>> >>> As you can see, I will shortly run out of memory. Any ideas? >>> >>> Thanks, >>> >>> Kevin >>> >> >
[toc] | [next] | [standalone]
| From | Szabolcs Horvát <szhorvat@gmail.com> |
|---|---|
| Date | 2012-06-05 08:51 +0000 |
| Message-ID | <jqkhal$4jv$1@smc.vnet.net> |
| In reply to | #10337 |
On 2012.06.03. 11:03, Ralph Dratman wrote: > Kevin, > > Thank you for passing along those tips. I usually put $HistoryLength=1 > and also turn off "Enable Notebook History Tracking" in the > Preferences dialog -- though I'm not sure the latter makes any > difference. > Hello Ralph, "Notebook History Tracking" only affects what is saved with the notebook you are working with, but it does not affect how the kernel works or what it keeps in memory (i.e. doesn't appreciably affect memory use). This option will simply record the times at which who have created or edited each cell. If you open a notebook file with a text editor, you'll see this information in the CellChangeTimes options. Turning it off will slightly reduce notebook sizes and makes version tracking much more convenient. -- Szabolcs Horvát Visit Mathematica.SE: http://mathematica.stackexchange.com/
[toc] | [prev] | [next] | [standalone]
| From | David Bailey <dave@removedbailey.co.uk> |
|---|---|
| Date | 2012-06-06 08:50 +0000 |
| Message-ID | <jqn5jr$hdi$1@smc.vnet.net> |
| In reply to | #10337 |
On 03/06/2012 10:03, Ralph Dratman wrote: > Kevin, > > Thank you for passing along those tips. I usually put $HistoryLength=1 > and also turn off "Enable Notebook History Tracking" in the > Preferences dialog -- though I'm not sure the latter makes any > difference. > > I just want to re-emphasize how egregious I think the situation with > crashing really is. It is unacceptable. I hope everyone is making that > absolutely clear to WRI. How can we continue to recommend a program > which often crashes? It defies common sense. > To be fair to WRI, I think the problem is that you have a very complex piece of software that is partly programmed by users. The kernel does put out a message if it runs out of memory, so it does detect the problem (at least under Windows!), but leaving the software in a well defined and useful state is another matter. The FrontEnd should remain running. It is also possible to run calculations using MemoryConstrained. Assuming the data you are reading in is an array or real numbers (or of integers), you should check if Import returns a packed array, and pack it if necessary - this will save a lot of space. David Bailey http://www.dbaileyconsultancy.co.uk
[toc] | [prev] | [next] | [standalone]
| From | W Craig Carter <ccarter@mit.edu> |
|---|---|
| Date | 2012-06-07 09:19 +0000 |
| Message-ID | <jqprmg$2cm$1@smc.vnet.net> |
| In reply to | #10337 |
Hello,
I find the following function to be a useful way to extract information
for memory management:
checkMemory[]:=Grid@Sort[{ByteCount[ToExpression[#]], HoldForm[#]} &
/@ Names["Global`*"], (First[#1] > First[#2]) &]
This returns a list of symbols ranked by their ByteCount. However, I am
not sure if evaluating checkMemory causes a momentary doubling of memory
for each evaluation of ByteCount=85.
I thought I would pass this along for anyone who might be hunting for
such a thing=85
W Craig Carter
Professor of Materials Science, MIT
On Jun 6, , at Wed Jun 6, 12 @4:52 AM, David Bailey wrote:
> On 03/06/2012 10:03, Ralph Dratman wrote:
>> Kevin,
>>
>> Thank you for passing along those tips. I usually put $HistoryLength=1
>> and also turn off "Enable Notebook History Tracking" in the
>> Preferences dialog -- though I'm not sure the latter makes any
>> difference.
>>
>> I just want to re-emphasize how egregious I think the situation with
>> crashing really is. It is unacceptable. I hope everyone is making that
>> absolutely clear to WRI. How can we continue to recommend a program
>> which often crashes? It defies common sense.
>>
>
>
> To be fair to WRI, I think the problem is that you have a very complex
> piece of software that is partly programmed by users. The kernel does
> put out a message if it runs out of memory, so it does detect the
> problem (at least under Windows!), but leaving the software in a well
> defined and useful state is another matter. The FrontEnd should remain
> running.
>
> It is also possible to run calculations using MemoryConstrained.
>
> Assuming the data you are reading in is an array or real numbers (or of
> integers), you should check if Import returns a packed array, and pack
> it if necessary - this will save a lot of space.
>
> David Bailey
> http://www.dbaileyconsultancy.co.uk
>
[toc] | [prev] | [standalone]
Back to top | Article view | comp.soft-sys.math.mathematica
csiph-web