Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.soft-sys.math.mathematica > #1575

Re: Importing text file as machine-precision array

Show all headers | View raw

On 09/04/2011 12:29, Sidney Cadot wrote:
> Hi all,
>
> I am trying to import a big text file containing non-sparse 2D
> numerical data into Mathematica. My aim is to get an array where the
> numbers are stored as machine-precision numbers. In other words,
> ByteCount[data] should be approximately equal to 8 times the number of
> table elements (assuming machine precision corresponds to 8-byte
> IEEE-754 numbers).
>
> If I do a straight data=Import["file","Table"], the file is parsed
> with all numbers represented as arbitrary precision floating point
> numbers; the data structure is ten times as big as it should be. If I
> do data=N[data] afterwards, the data is still four times as big as I
> would have expected.
>
> What is the appropriate way to accomplish what I want? Unfortunately,
> I cannot seem to find an option to Import[] to do this immediately
> when reading the data.
>
The only way that you will obtain an array as compact as you calculate 
(plus a small overhead) is to use the function Developer`ToPackedArray 
on the result after applying N.

You need to realise that arrays of reals (and some other basic types) 
come in two formats. That packed array format assumes that all elements 
are Real, or whatever. The other format can accommodate any type of 
object in any position, so it is less compact. Mathematica algorithms 
produce the same results on either format, but sometimes (as with 
Import) an array is not packed when it could be.

Packed arrays are also more efficient in terms of speed of processing.

David Bailey
http://www.dbaileyconsultancy.co.uk

Back to comp.soft-sys.math.mathematica | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Importing text file as machine-precision array Sidney Cadot <sidney@jigsaw.nl> - 2011-04-09 11:29 +0000
  Re: Importing text file as machine-precision array David Bailey <dave@removedbailey.co.uk> - 2011-04-09 22:00 +0000
    Re: Importing text file as machine-precision array Sidney Cadot <sidney.cadot@gmail.com> - 2011-04-11 11:05 +0000
      Re: Importing text file as machine-precision array David Bailey <dave@removedbailey.co.uk> - 2011-04-17 11:53 +0000

csiph-web