Re: Floating Point Test Suite

From	Fabian Russell <root@localhost.localdomain>
Newsgroups	sci.physics, comp.os.linux.advocacy
Subject	Re: Floating Point Test Suite
Date	2015-11-12 04:47 +0000
Organization	NewsGuy - Unlimited Usenet $23.95
Message-ID	<pan.2015.11.12.04.47.15@localhost.localdomain> (permalink)
References	<pan.2015.11.11.18.01.38@localhost.localdomain> <pan.2015.11.11.19.58.41@localhost.localdomain> <WJ-dnVQVTK3yldnLnZ2dnUU7-fOdnZ2d@giganews.com>

Cross-posted to 2 groups.

Show all headers | View raw

On Wed, 11 Nov 2015 21:33:03 -0600, Monkey Man wrote:

> 
> I've included the results in this post if you care to have a look. I'd be 
> happy to learn about what this is telling me if you wouldn't mind 
> breaking it down a bit for me.
> 

The test goes back a long way when there many different kinds of
machines in the computing universe and floating point hardware
differed among them.  Today, most hardware is either Intel or AMD,
which contain the same FP hardware, and the test is no longer that
significant.  But since the test is so rigorous, it could still be
interesting to run it to see if everything performs as expected.
Since the Intel/AMD hardware contains essentially two separate and
distinct arithmetic/FP units, the results can differ depending on
which of the two is selected by the compiler.  In this case, however,
I have set the compiler flags to use the newer SIMD FP unit,
which is IEEE754 compliant.

The test is basically a test for IEEE754 compliance, which is standard
nowadays, and is limited to certain aspects of the arithmetic functions
such as correct rounding, guard bits, etc.  If a multiplication suite passes,
then the processor fulfills all IEEE754 requirements for multiplication.
It doesn't sound too impressive, but there are subtle details of digital
arithmetic that can be missed by hardware or by compiler configuration
of that hardware.

IEEE754 compliance is intended to allow all machines to produce the
same results for mathematical computations.  Surprisingly, this did
not always happens, and it still may not happen depending, as mentioned,
on compiler configuration of the hardware or even of the hardware itself.

Yes, the output is atrocious.  I suppose that academics do not strive
for writing code and output that other people can read.  As long as they
can read their own results then they are satisfied.  But the output is little
more than just a statement of addition passed, subtraction passed,
etc.  All the output boils down to basically that.

You have posted the linux.log file, which is sort of a running commentary
during the program execution.  For each test suite, the details are summarized
in the various *.output files.  You can also check those for more information.

In your case all tests passed except for these:

> 
> UCBFAIL indicates problems!
> ../ucbtest-results/ccos_DP.output: ucbtest UCBFAIL in COS(X) at line 444 
> for generic 
> ../ucbtest-results/cpar_DP.output: ucbtest UCBFAIL in ../ucbctest/par.c 
> at line 2035 for double 
> ../ucbtest-results/csin_DP.output: ucbtest UCBFAIL in SIN(X) at line 444 
> for generic 
> 
> only 11 out of 14 show UCBPASS!
>

Now, two of these failures involve the transcendental SINE and COSINE function.
These results are not actually failures.  The reason is that IEEE754 does
not create standards for the SINE or COSINE.  Hardware, or software, is free
to implement them in any way.  But the UCBTEST marks these as failures because
the results for certain inputs to these two functions have exceeded a certain
tolerance.  If you check the ccos_DP.output and csin_DP.output files you should
see something like this:

                [      From     ,        to     )  N.M.E.  P.M.E. NMC {SYM}

COS(X)      ALL [      0.0000000,      7.0000000)  -3.740   1.567 0
 ucbtest UCBFAIL in COS(X) at line 444 for generic

Here, the NME and PME columns indicate NME and PME values of -3.740 and
1.567.

What does this mean?

Error units in FP calculations are expressed in ulp's, or "units in the
last place.'  In double FP precision, an ulp is about 1e-15.  The UCBTEST
expects COSE and SINE errors to be less than 1 ulp.  In your case, the
max errors observed were -3.74 ulps and 1.567 ulps.  Therefore, although
this is not technically an error, it has been flagged as such.

If you check all the other *.output files you will see the same summary
in terms of ulp's.  The tests that pass all had errors that were less
than 1 ulp.

So, everything looks fine except for cpar_DP test.

Unfortunately, this is UNEXPECTED and possibly represent a true error.

First, some background.

This test is actually not a part of the UCBTEST.  It is taken from
the famous PARANOIA test of W. Kahan, who is a big name in the FP
world.  The PARANOIA test examines all facets of FP on a system.

On your machine, something went wrong.  Please check the file cpar_DP
output, which should contain the specific error or defect found.

I've been using Linix since 1998.  Currently, I see no errors with
the PARANOIA test, but in the past I would regularly get certain
failures.

The fault could be with glibc or with the compiler gcc.

Again, please check the cpar_DP.output.

Well, I've been trying to make a very complex story very short
and I know I've missed a lot of detail.  If you need more clarification
I will try to fill in any gaps.

incidentally, this UCBTEST will test all FP formats, that is double
precision, single precision, and quad precision.  I have switched
on only the double precision test since it is usually the only format
used.  But the other two can be switched on by modifying the linux.sh
script.

Note: quad precision is actually "extended" precision on the Intel
microprocessor and is NOT IEEE754 compliant and will fail the test.

Thread

Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 18:01 +0000
  Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 18:09 +0000
  Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 19:58 +0000
    Re: Floating Point Test Suite Chris Ahlstrom <OFeem1987@teleworm.us> - 2015-11-11 15:19 -0500
      Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 20:28 +0000
      Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 20:37 +0000
        Re: Floating Point Test Suite Chris Ahlstrom <OFeem1987@teleworm.us> - 2015-11-11 16:33 -0500
      Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 21:13 +0000
        Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 15:16 -0600
          Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 21:42 +0000
          Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 21:54 +0000
            Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 16:43 -0600
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 23:15 +0000
                Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 17:34 -0600
                Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 17:37 -0600
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 23:47 +0000
                Re: Floating Point Test Suite dvus <dven1@adelphia.net> - 2015-11-12 07:56 -0500
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 19:07 +0000
                Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-12 19:57 -0600
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 00:45 +0000
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 04:55 +0000
                Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-12 09:26 -0600
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 20:56 +0000
                Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-12 20:00 -0600
            Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 16:48 -0600
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-11 23:08 +0000
                Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 17:12 -0600
            Re: Floating Point Test Suite Odd Bodkin <bodkinodd@gmail.com> - 2015-11-11 17:06 -0600
            Re: Floating Point Test Suite dvus <dven1@adelphia.net> - 2015-11-12 07:35 -0500
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 19:29 +0000
    Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-11 21:33 -0600
      Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 04:47 +0000
        Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-12 20:19 -0600
          Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 03:28 +0000
            Re: Floating Point Test Suite Big Fish in a Small Crotch <bigfishinasmallcrotch@myself.com> - 2015-11-12 22:40 -0500
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 03:59 +0000
                Re: Floating Point Test Suite benj <none@gmail.com> - 2015-11-13 00:16 -0500
                Re: Floating Point Test Suite GreyCloud <cumulus@mist.com> - 2015-11-13 15:00 -0700
            Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-13 01:40 -0600
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 08:22 +0000
                Re: Floating Point Test Suite benj <none@gmail.com> - 2015-11-13 12:41 -0500
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 19:16 +0000
                Re: Floating Point Test Suite Apollyon <adravirgo@gmail.com> - 2015-11-13 13:04 -0800
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-14 22:58 +0000
                Re: Floating Point Test Suite Big Fish in a Small Crotch <bigfishinasmallcrotch@myself.com> - 2015-11-14 18:08 -0500
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-14 23:39 +0000
                Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-14 18:26 -0800
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-15 07:07 +0000
                Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-14 23:42 -0800
                Re: Floating Point Test Suite Poutnik <poutnik4nntp@gmail.com> - 2015-11-15 09:49 +0100
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-15 21:01 +0000
                Re: Floating Point Test Suite jimp@specsol.spam.sux.com - 2015-11-15 21:33 +0000
                Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-15 14:24 -0800
                Re: Floating Point Test Suite jimp@specsol.spam.sux.com - 2015-11-15 22:55 +0000
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-15 22:25 +0000
                Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-15 14:59 -0800
                Re: Floating Point Test Suite Poutnik <poutnik4nntp@gmail.com> - 2015-11-15 09:41 +0100
      Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-12 06:24 +0000
        Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-12 20:29 -0600
          Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 03:49 +0000
            Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-12 22:49 -0600
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 04:59 +0000
                Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-13 01:56 -0600
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 08:28 +0000
                Re: Floating Point Test Suite Monkey Man <manm0nk3y@gmail.com> - 2015-11-13 22:24 -0600
                Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-14 04:45 +0000
          Re: June 1986 ? ! Fabian Russell <root@localhost.localdomain> - 2015-11-13 06:19 +0000
            Re: June 1986 ? ! GreyCloud <cumulus@mist.com> - 2015-11-13 15:03 -0700
      Re: MonkeyMan is FabianRussell. Fabian Russell <root@localhost.localdomain> - 2015-11-12 19:44 +0000
    Re: Floating Point Test Suite Yousuf Khan <bbbl67@spammenot.yahoo.com> - 2015-11-13 01:59 -0500
      Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-13 07:43 +0000
        Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-13 15:11 -0800
          Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-14 00:00 +0000
            Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-13 16:31 -0800
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-14 02:26 +0000
                Re: Floating Point Test Suite jimp@specsol.spam.sux.com - 2015-11-14 03:52 +0000
              Re: Floating Point Test Suite Fabian Russell <root@localhost.localdomain> - 2015-11-14 02:46 +0000
                Re: Floating Point Test Suite Timo <timo@physics.uq.edu.au> - 2015-11-13 19:26 -0800
                Re: Floating Point Test Suite benj <none@gmail.com> - 2015-11-14 02:11 -0500
              Re: Floating Point Test Suite noTthaTguY <abu.kuanysh05@gmail.com> - 2015-11-14 15:20 -0800
            Re: Floating Point Test Suite jimp@specsol.spam.sux.com - 2015-11-14 00:47 +0000
  Re: Floating Point Test Suite Big Fish in a Small Crotch <bigfishinasmallcrotch@myself.com> - 2015-11-11 15:28 -0500

csiph-web