Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #4476 > unrolled thread
| Started by | Peter Rodenbach <peter.rodenbach@googlemail.com> |
|---|---|
| First post | 2011-08-19 10:39 +0000 |
| Last post | 2011-08-21 09:36 +0000 |
| Articles | 3 — 3 participants |
Back to article view | Back to comp.soft-sys.math.mathematica
Memory when reding in data Peter Rodenbach <peter.rodenbach@googlemail.com> - 2011-08-19 10:39 +0000
Re: Memory when reding in data "Alexey Popkov" <lehin.p@gmail.com> - 2011-08-20 10:22 +0000
Re: Memory when reding in data David Bailey <dave@removedbailey.co.uk> - 2011-08-21 09:36 +0000
| From | Peter Rodenbach <peter.rodenbach@googlemail.com> |
|---|---|
| Date | 2011-08-19 10:39 +0000 |
| Subject | Memory when reding in data |
| Message-ID | <j2lehm$ngv$1@smc.vnet.net> |
Hi everyone.
I am analysing image data and I have the problem that after about 50
images or so Mathematica quits the calculation with the hint out of
memory. The interesting part of the code i wrote is :
(*Import the first image into Maxvals, which will be compared to all \
other images and overwritten if any point is found to be larger*)
SetDirectory[
"E:\\...];
Initialise = OpenRead["s2_260_0000.edf", BinaryFormat -> True];
MaxValues =
BinaryRead["s2_260_0000.edf",
Table["UnsignedInteger16", {512}, {512}]];
counter = Dimensions[MaxValues];
(*Load all experimental images and perform maximum projection*)
MaxProjection[filename_] := {OpenRead[filename, BinaryFormat -> True],
Imagevalues =
BinaryRead[filename, Table["UnsignedInteger16", {512}, {512}]],
For[i = 1, i <= counter[[1]], i++,
For [j = 1, j <= counter[[2]], j++,
If[MaxValues[[i, j]] >= Imagevalues[[i, j]],
MaxValues[[i, j]] = MaxValues[[i, j]],
MaxValues[[i, j]] = Imagevalues[[i, j]]]
]
], Close[Imagevalues]}
MaxProjection[#] & /@ FileNames@"*edf"
So I guess Mathm saves all the data of all the images after each
other. But once the comparision is done with the one MaxProjection
Image I want to have, all the data can be dumped from the memory and
then the next image can be compared with this MaxProjection image.
Does anyone have an idea? Thx.
[toc] | [next] | [standalone]
| From | "Alexey Popkov" <lehin.p@gmail.com> |
|---|---|
| Date | 2011-08-20 10:22 +0000 |
| Message-ID | <j2o1to$4v5$1@smc.vnet.net> |
| In reply to | #4476 |
Try to evaluate
$HistoryLenght=0;
and Clear variables when their values are no longer needed.
"Peter Rodenbach" <peter.rodenbach@googlemail.com> wrote:
news:j2lehm$ngv$1@smc.vnet.net...
> Hi everyone.
>
> I am analysing image data and I have the problem that after about 50
> images or so Mathematica quits the calculation with the hint out of
> memory. The interesting part of the code i wrote is :
>
> (*Import the first image into Maxvals, which will be compared to all \
> other images and overwritten if any point is found to be larger*)
>
> SetDirectory[
> "E:\\...];
>
> Initialise = OpenRead["s2_260_0000.edf", BinaryFormat -> True];
> MaxValues =
> BinaryRead["s2_260_0000.edf",
> Table["UnsignedInteger16", {512}, {512}]];
> counter = Dimensions[MaxValues];
>
> (*Load all experimental images and perform maximum projection*)
>
> MaxProjection[filename_] := {OpenRead[filename, BinaryFormat -> True],
> Imagevalues =
> BinaryRead[filename, Table["UnsignedInteger16", {512}, {512}]],
> For[i = 1, i <= counter[[1]], i++,
> For [j = 1, j <= counter[[2]], j++,
> If[MaxValues[[i, j]] >= Imagevalues[[i, j]],
> MaxValues[[i, j]] = MaxValues[[i, j]],
> MaxValues[[i, j]] = Imagevalues[[i, j]]]
> ]
> ], Close[Imagevalues]}
>
>
> MaxProjection[#] & /@ FileNames@"*edf"
>
> So I guess Mathm saves all the data of all the images after each
> other. But once the comparision is done with the one MaxProjection
> Image I want to have, all the data can be dumped from the memory and
> then the next image can be compared with this MaxProjection image.
>
> Does anyone have an idea? Thx.
>
[toc] | [prev] | [next] | [standalone]
| From | David Bailey <dave@removedbailey.co.uk> |
|---|---|
| Date | 2011-08-21 09:36 +0000 |
| Message-ID | <j2qji4$d28$1@smc.vnet.net> |
| In reply to | #4476 |
On 19/08/2011 11:39, Peter Rodenbach wrote:
> Hi everyone.
>
> I am analysing image data and I have the problem that after about 50
> images or so Mathematica quits the calculation with the hint out of
> memory. The interesting part of the code i wrote is :
>
> (*Import the first image into Maxvals, which will be compared to all \
> other images and overwritten if any point is found to be larger*)
>
> SetDirectory[
> "E:\\...];
>
> Initialise = OpenRead["s2_260_0000.edf", BinaryFormat -> True];
> MaxValues =
> BinaryRead["s2_260_0000.edf",
> Table["UnsignedInteger16", {512}, {512}]];
> counter = Dimensions[MaxValues];
>
> (*Load all experimental images and perform maximum projection*)
>
> MaxProjection[filename_] := {OpenRead[filename, BinaryFormat -> True],
> Imagevalues =
> BinaryRead[filename, Table["UnsignedInteger16", {512}, {512}]],
> For[i = 1, i<= counter[[1]], i++,
> For [j = 1, j<= counter[[2]], j++,
> If[MaxValues[[i, j]]>= Imagevalues[[i, j]],
> MaxValues[[i, j]] = MaxValues[[i, j]],
> MaxValues[[i, j]] = Imagevalues[[i, j]]]
> ]
> ], Close[Imagevalues]}
>
>
> MaxProjection[#]& /@ FileNames@"*edf"
>
> So I guess Mathm saves all the data of all the images after each
> other. But once the comparision is done with the one MaxProjection
> Image I want to have, all the data can be dumped from the memory and
> then the next image can be compared with this MaxProjection image.
>
> Does anyone have an idea? Thx.
>
Your code looks wrong because you open each file with OpenRead and never
close (or use) the stream it produces (I wonder if binary streams open a
large buffer).
Try something like this:
strm=OpenRead[filename, BinaryFormat -> True];
Imagevalues=BinaryRead[strm, Table["UnsignedInteger16", {512}, {512}]];
..........
Close[strm]
Notice that it is the stream that should be closed - not the data!
Also, I would use Module to define functions, rather than just using a
list of expressions { .......}.
With a Module, you can specify certain variables, such as strm, to be
local to the module, and also, you won't return a list of garbage as the
result of the call - which could cause trouble if one of those objects
is big!
David Bailey
http://www.dbaileyconsultancy.co.uk
[toc] | [prev] | [standalone]
Back to top | Article view | comp.soft-sys.math.mathematica
csiph-web