Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #5933 > unrolled thread

Re: Error on importing previously saved data file: Get::bigfile

Started byLeonid Shifrin <lshifr@gmail.com>
First post2011-10-27 10:42 +0000
Last post2011-10-27 10:42 +0000
Articles 1 — 1 participant

Back to article view | Back to comp.soft-sys.math.mathematica

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: Error on importing previously saved data file: Get::bigfile Leonid Shifrin <lshifr@gmail.com> - 2011-10-27 10:42 +0000

#5933 — Re: Error on importing previously saved data file: Get::bigfile

FromLeonid Shifrin <lshifr@gmail.com>
Date2011-10-27 10:42 +0000
SubjectRe: Error on importing previously saved data file: Get::bigfile
Message-ID<j8bchv$8sq$1@smc.vnet.net>
Matthias,

On the page I linked, I provided 2 implementations - for general tables
(first), and
for sparse tables. The first one should be as general as Import[...,
"Table"] itself,
although I did not test extensively.

Cheers,
Leonid


On Thu, Oct 27, 2011 at 10:03 AM, Matthias Seise <mseise@gmx.de> wrote:

> Dear Leonid,
>
> thanks for that link. I did see it before but it seems to work only with
> "normal" tables [only numbers]. I do have image data as well in the list and
> would like to save and load them. Of course I could convert them to a list
> on save and back on loading but It seems quite a lot of trouble for a simple
> read/write task.
>
> BR
> Matthias
>
>
> On 27 October 2011 04:20, Leonid Shifrin <lshifr@gmail.com> wrote:
>
>> Matthias,
>>
>> Check out my implementation of an alternative to Import for "Table" format
>> - it lives
>> here:
>>
>>
>> http://stackoverflow.com/questions/7525782/import-big-files-arrays-with-mathematica/7527064#7527064
>>
>> It may be able to solve your problem.
>>
>> Regards,
>> Leonid
>>
>>
>> On Thu, Oct 27, 2011 at 1:36 AM, Matthias Seise <mseise@gmail.com> wrote:
>>
>>> I'm working in Mathematica 8, 64 bit.
>>>
>>> I have saved a data set (Save["filename",data]) - which is about 3.5
>>> MB on the harddisk (a table with numbers and images,
>>> ByteCount=610117226).
>>> When I try to read it again into Memory I get "Get::bigfile: "The file
>>> "filename" is too large to be read into memory."
>>>
>>> I can work around it by splitting the data in smaller junks, but there
>>> should be a better way since the data set should not be too large
>>>
>>> Any ideas?
>>>
>>>
>>> Best regards,
>>> Matthias Seise
>>>
>>>
>>
>

[toc] | [standalone]


Back to top | Article view | comp.soft-sys.math.mathematica


csiph-web