Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #16137 > unrolled thread
| Started by | Alain Cochard <Alain.Cochard@unistra.fr> |
|---|---|
| First post | 2013-10-29 03:14 +0000 |
| Last post | 2013-11-06 05:21 +0000 |
| Articles | 5 — 2 participants |
Back to article view | Back to comp.soft-sys.math.mathematica
Mathematica not IEEE-754 compliant? Alain Cochard <Alain.Cochard@unistra.fr> - 2013-10-29 03:14 +0000
Re: Mathematica not IEEE-754 compliant? danl@wolfram.com - 2013-10-31 03:45 +0000
Re: Mathematica not IEEE-754 compliant? Alain Cochard <Alain.Cochard@unistra.fr> - 2013-11-02 06:25 +0000
Re: Mathematica not IEEE-754 compliant? danl@wolfram.com - 2013-11-05 04:12 +0000
Re: Mathematica not IEEE-754 compliant? Alain Cochard <Alain.Cochard@unistra.fr> - 2013-11-06 05:21 +0000
| From | Alain Cochard <Alain.Cochard@unistra.fr> |
|---|---|
| Date | 2013-10-29 03:14 +0000 |
| Subject | Mathematica not IEEE-754 compliant? |
| Message-ID | <l4n97h$b25$1@smc.vnet.net> |
Hello.
My understanding is that, when machine-precision numbers are used, Mathematica
follows the IEEE-754 standard, hence results should be identical to
other IEEE-754 compliant software+hardware combinations when the +, -,
*, /, operations are used.
Well, I am aware of the fact that floating point arithmetics can be
tricky and that systems that do follow the IEEE-754 standard may give
different results, but it seems to me it should not be the case here.
More precisely, my understanding is that, given any one of these
arithmetic operations, the machine number provided as the result should
be the machine number closest to the exact result (for the default
rounding mode).
I believe I show below a case where this is not true in Mathematica.
NB: the tests have been done for Mathematica versions 7.0 and 8.0 on two
different GNU/Linux machines. This is for the division of two numbers,
'num' and 'den' (for 'numerator' and 'denominator').
----
(* Infinite precision versions of the numbers: *)
numInfPrec=2^^11100011010111111010100100110001100111111111111101111*2^-3;
denInfPrec=2^^11100011010111111010100100110001100111111111111110111*2^-3;
(* Transform to machine-precision numbers: *)
num=1.*numInfPrec;
den=1.*denInfPrec;
(* The strings of 0s and 1s above start with a '1' and are 53-digit
long, hence the above numbers correspond to regular normalized numbers
of the 64-bit long IEEE-754 format -- indeed, the outputs of
RealDigits[num,2] and RealDigits[den,2] are consistent with the above
numbers. *)
res=num/den; (* Actual computation under testing *)
bin=RealDigits[res,2] (* Corresponding binary representation *)
This gives:
{{1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 0, 1, 1, 0}, 0}
I contend that is this not the machine number closest to the exact
result and that, instead,
1.1111111111111111111111111111111111111111111111110111 * 2^-1 is the
closest number, called below "presumablyClosestInfPrec" for its
infinite precision version; this number differs from the Mathematica number
above by the last binary digit of the mantissa -- it is the next
machine number.
Checking:
---------
(* Infinite precision number corresponding to the result, res: *)
resInfPrec=FromDigits[bin,2];
presumablyClosestInfPrec=
2^^11111111111111111111111111111111111111111111111110111*2^-53;
resExactInfPrec=numInfPrec/denInfPrec;
(* Difference between Mathematica result and exact result: *)
diffMma=N[resInfPrec-resExactInfPrec,3]//InputForm
(* This gives -1.1022302462516`3.*^-16 *)
(* Diff between presumably closest machine number and exact result: *)
diffPresumablyClosest=N[presumablyClosestInfPrec-resExactInfPrec,3]//InputForm
(* This gives 7.9927783736023861873`3.*^-19 *)
This shows that the exact result of the division is between the two
(adjacent) machine numbers and indeed closer to the presumably closest
number.
----
Another way to illustrate the problem:
It just so happens that num=(1/e)-2 and den=(1/e)-1, where e=1. 10^-15.
It is very easy to check that num/den obtained by Mathematica is different
from num/den obtained with another IEEE-754 compliant software (I have
tried with Matlab, Octave, and GNU Fortran):
Mathematica:
e=1. 10^-15;
num=(1/e)-2 ; InputForm[num] (* gives 9.999999999999979*^14 *)
den=(1/e)-1 ; InputForm[den] (* gives 9.999999999999989*^14 *)
res=num/den ; InputForm[res] (* gives 0.9999999999999989 *)
matlab:
e=1e-15;
num=(1/e)-2 % gives 9.999999999999979e+14
den=(1/e)-1 % gives 9.999999999999989e+14
res=num/den % gives 9.999999999999990e-01
num and den are the same for Mathematica and Matlab, while res=num/den differ.
I have also checked that
(*) the 52+1 binary representations of num and den obtained with
Matlab and Octave (with decimal-to-binary converters), and GNU Fortran
(with the b64.64 format), are identical to the binary reresentations
given by Mathematica;
(*) the 52+1 binaray representations of 'res' obtained with Matlab,
Octave, and GNU Fortran, are all identical to the representation of
what I refer above by the "closest machine number".
----
Hopefully, if Mathematica is not faulty here, someone can point out the
mistake I am making.
Thank you.
Alain Cochard
--
EOST (École et Observatoire des Sciences de la Terre)
IPG (Institut de Physique du Globe) | alain.cochard@unistra.fr
5 rue René Descartes [bureau 106] | Phone: +33 (0)3 68 85 50 44
F-67084 Strasbourg Cedex, France | Fax: +33 (0)3 68 85 01 25
[toc] | [next] | [standalone]
| From | danl@wolfram.com |
|---|---|
| Date | 2013-10-31 03:45 +0000 |
| Message-ID | <l4sjo2$pkv$1@smc.vnet.net> |
| In reply to | #16137 |
I'm not convinced that one or the other is incorrect. It could be the case, for example, that 80 bit registers at certain steps are required to get one result whereas 64 bit registers, or storing from 80 to 64 bit doubles in memory, will give the other. Whether or how 80 bit registers get used will depend on settings at compile time for the various programs being used.
One thing I can point out is that the Mathematica result is arguably the better of the two. Once you create 'num' and 'den' as machine numbers, if you now take their bit sequences, make exact rationals, and divide to high precision, then create the bit sequence for the corresponding approximate value, you will recover the 'bin' set of bits, not that of 'presumablyClosestInfPrec'. I show this below.
In[39]:= Take[
RealDigits[
N[FromDigits[RealDigits[num, 2], 2]/
FromDigits[RealDigits[den, 2], 2], 20], 2][[1]], 53] === bin[[1]]
Out[39]= True
Notice that I used N[...,20] to make sure to correctly reconstruct that (exact) quotient to more than 53 correct bits.
Daniel Lichtblau
Wolfram Research
[toc] | [prev] | [next] | [standalone]
| From | Alain Cochard <Alain.Cochard@unistra.fr> |
|---|---|
| Date | 2013-11-02 06:25 +0000 |
| Message-ID | <l525s5$9gt$1@smc.vnet.net> |
| In reply to | #16153 |
Thank you very much for your answer!
danl@wolfram.com writes on Thu 31 Oct 2013 03:45:
> I'm not convinced that one or the other is incorrect. It could be
> the case, for example, that 80 bit registers at certain steps are
> required to get one result whereas 64 bit registers, or storing
> from 80 to 64 bit doubles in memory, will give the other. Whether
> or how 80 bit registers get used will depend on settings at compile
> time for the various programs being used.
Unfortunately, I am not familiar with 80 bit registers. My --perhaps
naive-- understanding comes from the official document "IEEE Standard
for Binary Floating-Point Arithmetic", in which I read that
All conforming implementations of this standard shall provide
operations to add, subtract, multiply, divide, extract the square
root, find the remainder, round to integer in floating-point
format, convert between different floating-point formats, convert
between floating-point and integer formats, convert binary <--->
decimal, and compare. [...] Except for binary <---> decimal
conversion, each of the operations shall be performed as if it
first produced an intermediate result correct to infinite precision
and with unbounded range, and then coerced this intermediate result
to fit in the destination's format (see Sections 4 and 7).
Section 7 is dealing with exceptions, which should not be a concern
here. Also, as far as I understand, the issue I raised is not related
to binary <---> decimal conversion. Section 4.1 is "Round to Nearest"
and says:
An implementation of this standard shall provide round to nearest
as the default rounding mode. In this mode the representable value
nearest to the infinitely precise result shall be delivered;
So I do not understand how, on a system conforming to the standard,
the result of a single multiplication might or might not give the
closest machine number.
> One thing I can point out is that the Mathematica result is
> arguably the better of the two. Once you create 'num' and 'den' as
> machine numbers, if you now take their bit sequences, make exact
> rationals, and divide to high precision, then create the bit
> sequence for the corresponding approximate value, you will recover
> the 'bin' set of bits, not that of 'presumablyClosestInfPrec'. I
> show this below.
>
> In[39]:= Take[
> RealDigits[
> N[FromDigits[RealDigits[num, 2], 2]/
> FromDigits[RealDigits[den, 2], 2], 20], 2][[1]], 53] === bin[[1]]
>
> Out[39]= True
>
> Notice that I used N[...,20] to make sure to correctly reconstruct
> that (exact) quotient to more than 53 correct bits.
Here too, I am probably missing something. If I understand correctly,
you are saying that Mathematica's result is the better of the two because of
internal consistency. But it seems to me that when you take the first
53 digits, you are simply "floor'ing" the number. As, in my exemple,
the ordering is:
Mma_result < exact_result < closest_number
it is no surprise to me that you recover 'bin', but I do not see why
it shows consistency.
Here is another example with
closest_number < exact_result < Mma_result
numInfPrec=2^^11000001101111100011010100110001011111101100101111110*2^-53;
denInfPrec=2^^11100010111100000100110011100111001001000101100111011*2^-53;
num=1.*numInfPrec;
den=1.*denInfPrec;
res=num/den;
bin=RealDigits[res,2]
This gives:
{{1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0,
0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0,
1, 1, 0, 1, 0, 0}, 0}
presumablyClosestInfPrec=
2^^11011010100011011010100001111010100001001001100110011*2^-53
(* This is the machine number immediately before the previous one *)
Check that it is indeed the machine number closest to the exact result:
N[ { numInfPrec/denInfPrec - presumablyClosestInfPrec ,
FromDigits[bin,2] - presumablyClosestInfPrec}, 3]//InputForm
This gives {3.998894021443906958645`3.*^-17,
1.1102230246251565404236`3.*^-16}
Your piece of code:
Take[
RealDigits[
N[FromDigits[RealDigits[num, 2], 2]/
FromDigits[RealDigits[den, 2], 2], 20], 2][[1]], 53] === bin[[1]]
gives "False", while
Take[
RealDigits[
N[FromDigits[RealDigits[num, 2], 2]/
FromDigits[RealDigits[den, 2], 2], 20], 2][[1]], 53] ===
RealDigits[presumablyClosestInfPrec,2,53] [[1]]
gives "True".
Hopefully you can improve my understanding.
With best wishes,
Alain
--
EOST (École et Observatoire des Sciences de la Terre)
IPG (Institut de Physique du Globe) | alain.cochard@unistra.fr
5 rue René Descartes [bureau 106] | Phone: +33 (0)3 68 85 50 44
F-67084 Strasbourg Cedex, France | Fax: +33 (0)3 68 85 01 25
[toc] | [prev] | [next] | [standalone]
| From | danl@wolfram.com |
|---|---|
| Date | 2013-11-05 04:12 +0000 |
| Message-ID | <l59r6o$rd7$1@smc.vnet.net> |
| In reply to | #16158 |
There is a subtlety having to do with Mathematica semantics rather than IEEE arithmetic. n/d is intertpreted, and evaluated, as n*(d^(-1)). If you go through the various computations to check on how your example will behave, I suspect it will come out fine, with the ELP ("error in last place") discrepancy between n*(d^(-1)) and n/d resulting from roundoff error in the process of taking the reciprocal and then multiplying.
To get the division semantics directly in Mathematica one can instead use
Divide[n,d]
In this case your example will give the result you had expected.
I should mention that once the actual arithmetic is performed in Mathematica, when operands are machine doubles then it is done using hardware operations that are invoked from C. If the C compiler used in the build process is IEEE compliant, then those operations will be as well. To the best of my knowledge all compilers we use to build Mathematica are IEEE compliant.
Daniel Lichtblau
Wolfram Research
[toc] | [prev] | [next] | [standalone]
| From | Alain Cochard <Alain.Cochard@unistra.fr> |
|---|---|
| Date | 2013-11-06 05:21 +0000 |
| Message-ID | <l5cjkr$4qa$1@smc.vnet.net> |
| In reply to | #16153 |
danl@wolfram.com writes:
> There is a subtlety having to do with Mathematica semantics rather
> than IEEE arithmetic. n/d is intertpreted, and evaluated, as
> n*(d^(-1)). If you go through the various computations to check on how
> your example will behave, I suspect it will come out fine, with the
> ELP ("error in last place") discrepancy between n*(d^(-1)) and n/d
> resulting from roundoff error in the process of taking the reciprocal
> and then multiplying.
>
> To get the division semantics directly in Mathematica one can instead use
>
> Divide[n,d]
>
> In this case your example will give the result you had expected.
>
> I should mention that once the actual arithmetic is performed in
> Mathematica, when operands are machine doubles then it is done using
> hardware operations that are invoked from C. If the C compiler used in
> the build process is IEEE compliant, then those operations will be as
> well. To the best of my knowledge all compilers we use to build
> Mathematica are IEEE compliant.
That's a subtlety, all right!
Yes, knowing about it, all my results make sense to me:
- d/n is never more that one machine number away from the closest
machine number (more precisely, for d and n random numbers in [1/2, 1],
it gives the closest machine number about 73% of the time and a number
one machine number away from the closest the remaining 27%);
- Divide[d,n] always gives the closest machine number.
I must confess I had not read the documentation for "/"... But the
subtlety does not seem to be documented there. Indeed, in the Divide
page
http://reference.wolfram.com/mathematica/ref/Divide.html
I read that
x/y or Divide[x, y] is equivalent to x y^-1
The Details section does not say more, it seems. I wonder then if there
is a place where one can learn about this kind of subtleties...
Thanks again for your time.
a.
--
EOST (École et Observatoire des Sciences de la Terre)
IPG (Institut de Physique du Globe) | alain.cochard@unistra.fr
5 rue René Descartes [bureau 106] | Phone: +33 (0)3 68 85 50 44
F-67084 Strasbourg Cedex, France | Fax: +33 (0)3 68 85 01 25
[toc] | [prev] | [standalone]
Back to top | Article view | comp.soft-sys.math.mathematica
csiph-web