Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #10508
| From | Paul E McHale <paulmchale@me.com> |
|---|---|
| Newsgroups | comp.soft-sys.math.mathematica |
| Subject | Structure of "identical" data not equal in size |
| Date | 2012-06-17 08:01 +0000 |
| Organization | Steven M. Christensen and Associates, Inc and MathTensor, Inc. |
| Message-ID | <jrk2s2$8n0$1@smc.vnet.net> (permalink) |
(* First, the internal data and then writing it to CSV *)
m=Table[{i,Sin[i],Cos[i]},{i,1,1000,1.0}];
SetDirectory[NotebookDirectory[]]
Export["Test2.csv",m, "CSV"]
(*Reading it back in *)
mIn = Import["Test2.csv", "CSV"]
(* compare data *)
mIn==m
>> True
(* see memory usage *)
ByteCount[m]
>> 24168
ByteCount[mIn]
>> 144040
(* what is the minimum size using reals *)
1000 * 3 * 8
>> 24000
Actual size: 24168
Read in file data: 144040
(* Size *)
ByteCount[mIn]/ByteCount[m] * 1.0
>> 5.96
--------------
Why is the "same" data taking up 6x the memory after being written to disk and read back in. This is a serious problem as we have large data at work being shared by files and Mathematica is currently the only language that can't read it (C# being the other language).
How can they pass for equal yet internally generated takes much less memory?
Any input welcome as I can only reduce the data with an external editor so I can try to work with it.
Thanks,
Paul
Back to comp.soft-sys.math.mathematica | Previous | Next — Next in thread | Find similar | Unroll thread
Structure of "identical" data not equal in size Paul E McHale <paulmchale@me.com> - 2012-06-17 08:01 +0000 Re: Structure of "identical" data not equal in size Paul E McHale <paulmchale@me.com> - 2012-06-18 09:43 +0000 Re: Structure of "identical" data not equal in size awnl <awnl@gmx-topmail.de> - 2012-06-18 09:46 +0000 Re: Structure of "identical" data not equal in size Szabolcs Horvát <szhorvat@gmail.com> - 2012-06-18 09:47 +0000 Re: Structure of "identical" data not equal in size Szabolcs Horvát <szhorvat@gmail.com> - 2012-06-18 09:47 +0000
csiph-web