Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #8583 > unrolled thread

Reading Single and Double reals in Little-endian and Big-endian

Started byezh <dgolber@aol.com>
First post2012-02-17 11:27 +0000
Last post2012-02-26 09:21 +0000
Articles 5 — 3 participants

Back to article view | Back to comp.soft-sys.math.mathematica


Contents

  Reading Single and Double reals in Little-endian and Big-endian ezh <dgolber@aol.com> - 2012-02-17 11:27 +0000
    Re: Reading Single and Double reals in Little-endian and Big-endian "Oleksandr Rasputinov" <oleksandr_rasputinov@ymail.com> - 2012-02-20 07:54 +0000
      Re: Reading Single and Double reals in Little-endian and Big-endian ezh <dgolber@aol.com> - 2012-02-24 06:02 +0000
        Re: Reading Single and Double reals in Little-endian and Big-endian David Bailey <dave@removedbailey.co.uk> - 2012-02-25 06:55 +0000
      Re: Reading Single and Double reals in Little-endian and Big-endian "Oleksandr Rasputinov" <oleksandr_rasputinov@ymail.com> - 2012-02-26 09:21 +0000

#8583 — Reading Single and Double reals in Little-endian and Big-endian

Fromezh <dgolber@aol.com>
Date2012-02-17 11:27 +0000
SubjectReading Single and Double reals in Little-endian and Big-endian
Message-ID<jhldir$ot7$1@smc.vnet.net>
I am running Mathematica 4 on a PC.

I am reading a file (an STL file), which has 32-bit integers and 32-
bit reals in little-endian format.  I can read in the integers
correctly, using

numTriangles = ReadBinary[strm, Int32, ByteOrder ->
LeastSignificantByteFirst];

However, the documentation states (and is correct) that ByteOrder ->
LeastSignificantByteFirst only affects reading integers.  So far, I
have not figured out how to read 32-bit reals ("Single") correctly.

I could read in the 32-bits as bytes, and reverse the order, and build
a 32-bit integer with the correct bits in it to be the desired 32-bit
real.  But how do I get Mathematica to regard those 32-bits as an IEEE
real number?

I'd hate to have to read up on the IEEE format and build the value
myself.

I can't be the first person in theworld to hit this problem!

And why (futile question) doesn't mathematica know about big- and
little-endian for _reals_ as well as for _integers_?

Thanks
Dave Golber

[toc] | [next] | [standalone]


#8645

From"Oleksandr Rasputinov" <oleksandr_rasputinov@ymail.com>
Date2012-02-20 07:54 +0000
Message-ID<jhsu7g$np4$1@smc.vnet.net>
In reply to#8583
On Fri, 17 Feb 2012 11:27:23 -0000, ezh <dgolber@aol.com> wrote:

> I am running Mathematica 4 on a PC.
>
> I am reading a file (an STL file), which has 32-bit integers and 32-
> bit reals in little-endian format.  I can read in the integers
> correctly, using
>
> numTriangles = ReadBinary[strm, Int32, ByteOrder ->
> LeastSignificantByteFirst];
>
> However, the documentation states (and is correct) that ByteOrder ->
> LeastSignificantByteFirst only affects reading integers.  So far, I
> have not figured out how to read 32-bit reals ("Single") correctly.
>
> I could read in the 32-bits as bytes, and reverse the order, and build
> a 32-bit integer with the correct bits in it to be the desired 32-bit
> real.  But how do I get Mathematica to regard those 32-bits as an IEEE
> real number?
>
> I'd hate to have to read up on the IEEE format and build the value
> myself.
>
> I can't be the first person in theworld to hit this problem!
>
> And why (futile question) doesn't mathematica know about big- and
> little-endian for _reals_ as well as for _integers_?
>
> Thanks
> Dave Golber
>

It does. However, this support (at least as concerns BinaryRead) was added  
in Mathematica 5.2 (released in June 2005). So, no, you're not the first  
person in the world to encounter this problem, but you might be the last  
person in the world still to be using Mathematica 4.

Nonetheless, Mathematica 4's Import is documented as accepting the  
ByteOrdering option (taking values of +1 or -1 representing big- and  
little-endian formats respectively); whether this applies to  
floating-point as well as integer values I do not know off-hand (as it has  
been a very long time since I used Mathematica 4) but the implementation  
is certainly different to that in the Utilities`BinaryFiles` package you  
seem to be using, and in more recent versions of Mathematica the support  
is there, so I would encourage you to give it a try.

However, I wonder why you need to specify the byte ordering at all, as STL  
files use little-endian ordering, and so do PCs, so no conversion ought to  
be necessary. Or, indeed, why you have to import the file using  
Utilities`BinaryFiles`ReadBinary to read values from a stream rather than  
Import["file.stl"], which should do the job with much less effort. In  
fact, it seems to me as if you may be using code written for Mathematica 3  
on a Mac (where these issues certainly would need to be dealt with), which  
is in need of updating for Mathematica 4.

[toc] | [prev] | [next] | [standalone]


#8721

Fromezh <dgolber@aol.com>
Date2012-02-24 06:02 +0000
Message-ID<ji794s$hcs$1@smc.vnet.net>
In reply to#8645
Yes, it's an old Mathematica.  I pay for it myself, that's why.

Yes, I need to reverse the byte order.  After all, it's not a PC
storing the stuff read from the file, it's Mathematica.  In any case,
this is not a theoretical statement.  I tried it

Here's the work-around I developed:

strm1= open orginal file for input
strm2= open "reversed' file for output.
copy the strm1 file to the strm2 file, but
At each 4-byte floating number in the input file:
stuff=ReadBinaryList[strm1,Byte,4]
WriteBinary[strm2,Reverse[stuff],ByteConversion -> (ToBytes[#,
IntegerConvert -> Byte] &)]
Now close the files.
The "reversed" file is the same as the original file, but with each 4-
byte floating number in the opposite byte order

Now open the "reversed" file for binary input.
Now
value=ReadBinary[strm,Single];
gets the right value.

[toc] | [prev] | [next] | [standalone]


#8729

FromDavid Bailey <dave@removedbailey.co.uk>
Date2012-02-25 06:55 +0000
Message-ID<jia0kh$196$1@smc.vnet.net>
In reply to#8721
On 24/02/2012 06:02, ezh wrote:
> Yes, it's an old Mathematica.  I pay for it myself, that's why.
>
> Yes, I need to reverse the byte order.  After all, it's not a PC
> storing the stuff read from the file, it's Mathematica.  In any case,
> this is not a theoretical statement.  I tried it
>
> Here's the work-around I developed:
>
> strm1= open orginal file for input
> strm2= open "reversed' file for output.
> copy the strm1 file to the strm2 file, but
> At each 4-byte floating number in the input file:
> stuff=ReadBinaryList[strm1,Byte,4]
> WriteBinary[strm2,Reverse[stuff],ByteConversion ->  (ToBytes[#,
> IntegerConvert ->  Byte]&)]
> Now close the files.
> The "reversed" file is the same as the original file, but with each 4-
> byte floating number in the opposite byte order
>
> Now open the "reversed" file for binary input.
> Now
> value=ReadBinary[strm,Single];
> gets the right value.
>
I wonder if you would be eligible for this license:

http://www.wolfram.com/mathematica/how-to-buy/hobbyist.html

David Bailey
http://www.dbaileyconsultancy.co.uk

[toc] | [prev] | [next] | [standalone]


#8753

From"Oleksandr Rasputinov" <oleksandr_rasputinov@ymail.com>
Date2012-02-26 09:21 +0000
Message-ID<jicti8$em3$1@smc.vnet.net>
In reply to#8645
On Fri, 24 Feb 2012 06:02:04 -0000, ezh <dgolber@aol.com> wrote:

> Yes, I need to reverse the byte order.  After all, it's not a PC
> storing the stuff read from the file, it's Mathematica.

I see. Well, your problem is caused by the Utilities`BinaryFiles` package,  
which reads binary files in a fairly insane way and hard-codes an  
assumption of big-endianness when dealing with reals. This code was  
written over 20 years ago (in 1991) and is now well and truly obsolete. On  
the plus side, the package source (in  
AddOns/StandardPackages/Utilities/BinaryFiles.m) can be edited as you see  
fit, including to add support for little-endian reals. So, for example, in  
the package we find the following definition which provides for reading a  
single precision number:

rb[stream_InputStream,Single,_] :=
   floatvalue @@ {Quotient[#1,128],
         (Mod[#1,128])*2 + Quotient[#2,128],
         ((Mod[#2,128])*256^2 + #3*256 + #4)/(2^23) + 1}& @@
       Read[stream,{Byte,Byte,Byte,Byte}]

Reading the rest of the package source, we note that the third argument  
corresponds to endianness, with True meaning big-endian, and False,  
little-endian. So, we make the needed changes:

rb[stream_InputStream,Single,bigendian_] :=
   floatvalue @@ {Quotient[#1,128],
       (Mod[#1,128])*2 + Quotient[#2,128],
       ((Mod[#2,128])*256^2 + #3*256 + #4)/(2^23) + 1}& @@
     If[bigendian,Identity,Reverse]@Read[stream,{Byte,Byte,Byte,Byte}]

and, just like that, we have support for little-endian single precision  
numbers (or at least, I hope so--I didn't test this). Do the same for  
double precision numbers and you're finished updating the package.

I hope this fixes your problem so that workarounds are no longer needed.  
However, I still think that using Import to start with rather than trying  
to read the file manually would have been easier, as well as avoiding this  
bizarre problem.

[toc] | [prev] | [standalone]


Back to top | Article view | comp.soft-sys.math.mathematica


csiph-web