Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #5914

Re: Error on importing previously saved data file: Get::bigfile

From Tomas Garza <tgarza10@msn.com>
Newsgroups comp.soft-sys.math.mathematica
Subject Re: Error on importing previously saved data file: Get::bigfile
Date 2011-10-27 10:34 +0000
Organization Steven M. Christensen and Associates, Inc and MathTensor, Inc.
Message-ID <j8bc4i$8pg$1@smc.vnet.net> (permalink)

Show all headers | View raw


Maybe you could use Put (>>) and Get(<<) instead of Save. E.g.
In[1]:= SetDirectory[$TemporaryDirectory];
In[7]:= data = RandomChoice[Range[0, 9], 1000000];
In[8]:= data >> "digits";
In[9]:= ByteCount[data]
Out[9]= 4000168
In[10]:= << digits; // AbsoluteTiming
Out[10]= {2.8584000, Null}
Takes less than 3 seconds to load, so perhaps this approach might be useful. Of course your file is about 150 times as large.
Put by default writes expressions in InputForm.
--Tomas

> Date: Wed, 26 Oct 2011 17:36:04 -0400
> From: mseise@gmail.com
> Subject: Error on importing previously saved data file: Get::bigfile
> To: mathgroup@smc.vnet.net
>
> I'm working in Mathematica 8, 64 bit.
>
> I have saved a data set (Save["filename",data]) - which is about 3.5
> MB on the harddisk (a table with numbers and images,
> ByteCount=610117226).
> When I try to read it again into Memory I get "Get::bigfile: "The file
> "filename" is too large to be read into memory."
>
> I can work around it by splitting the data in smaller junks, but there
> should be a better way since the data set should not be too large
>
> Any ideas?
>
>
> Best regards,
> Matthias Seise
>

Back to comp.soft-sys.math.mathematica | Previous | Next | Find similar | Unroll thread


Thread

Re: Error on importing previously saved data file: Get::bigfile Tomas Garza <tgarza10@msn.com> - 2011-10-27 10:34 +0000

csiph-web