Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #9908
| From | Šerých Jakub <Serych@panska.cz> |
|---|---|
| Newsgroups | comp.soft-sys.math.mathematica |
| Subject | Re: Importing large file into table and calculating takes a long time. How to improve efficiency? |
| Date | 2012-05-01 18:59 +0000 |
| Organization | Steven M. Christensen and Associates, Inc and MathTensor, Inc. |
| Message-ID | <jnpbqb$ahs$1@smc.vnet.net> (permalink) |
I haven't studied your code, but definitely the first error I can see is, that normal asignment is "=" not "==" in Mathematica.
Jakub
> -----Original Message-----
> From: Gangerolv [mailto:ipschka@yahoo.com]
> Sent: Tuesday, May 01, 2012 11:24 AM
> To: mathgroup@smc.vnet.net
> Subject: Importing large file into table and calculating takes a
> long time. How to improve efficiency?
>
> (First my disclaimer, I'm new to mathematica)
>
> I'm importing a file with three values, x,y,z in hexadecimal.
> Sample of the input data: {0., 24, "009d"}, {0., 28, 9}, {0., 28, 99}, {"00dc",
> 27, 98}, {0., 29, 95},...
>
> This set is converted to integer: {0., 36, 157}, {0., 40, 9}, {0., 40, 153}, {220,
> 39, 152}, {0., 41, 149},...
>
> A final vector is calculated using: g==sqrt(x^2 + y^2 + z^2) for each data set.
>
> The data is then plotted.
>
> All of this is good an works great for a small file (2000 data sets). But when I
> try to import and calculate a larger file (over 100k data sets), it seems to take
> forever. Either my methods are not efficient (use of Table), or I'm not using
> correct settings for importing of the data. The file is only 2Mb so I know
> mathematica should be able to handle it.
>
> Here's what I'm doing:
>
> =========================================================================
> dataHex ==
> Import["C:\\Projects\\Mathematica\\test.csv"]
> points == Length[dataHex]
> dataDec == Table[
> ToExpression["16^^" <> #] & /@ {
> ToString[dataHex[[i, 1]]],
> ToString[dataHex[[i, 2]]],
> ToString[dataHex[[i, 3]]]},
> {i, 1, points}]
> dataDecCompl == Table[{
> If[dataDec[[i, 1]] > 32768, dataDec[[i, 1]] - 65536, dataDec[[i, 1]]],
> If[dataDec[[i, 2]] > 32768, dataDec[[i, 2]] - 65536, dataDec[[i, 2]]],
> If[dataDec[[i, 3]] > 32768, dataDec[[i, 2]] - 65536, dataDec[[i, 3]]]},
> {i, 1, points}];
>
> dataG == MovingAverage[Table[
> Sqrt[dataX[[i]]^2 + dataY[[i]]^2 + dataZ[[i]]^2] // N,
> {i, 1, points - filter}],
> 3];
> plotG == ListLinePlot[{dataG}, PlotRange -> All];
> =========================================================================
>
> Any suggestions would be highly appreciated. Using M 8.0.4 on Win7.
>
> Cheers!
>
> jan
>
Back to comp.soft-sys.math.mathematica | Previous | Next | Find similar | Unroll thread
Re: Importing large file into table and calculating takes a long time. How to improve efficiency? Šerých Jakub <Serych@panska.cz> - 2012-05-01 18:59 +0000
csiph-web