Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > sci.physics > #511307 > unrolled thread

What Floating-Point Precisions Would Physicists Prefer?

Back to article view | Back to sci.physics

Contents

  What Floating-Point Precisions Would Physicists Prefer? Quadibloc <jsavard@ecn.ab.ca> - 2015-08-04 20:41 -0700
    Re: What Floating-Point Precisions Would Physicists Prefer? Fabian Russell <root@localhost.localdomain> - 2015-08-05 05:14 +0000
      Re: What Floating-Point Precisions Would Physicists Prefer? Quadibloc <jsavard@ecn.ab.ca> - 2015-08-05 04:38 -0700
        Re: What Floating-Point Precisions Would Physicists Prefer? Fabian Russell <root@localhost.localdomain> - 2015-08-05 14:22 +0000
          nano-M-sets noTthaTguY <abu.kuanysh05@gmail.com> - 2015-08-05 13:14 -0700
            Re: nano-M-sets Fabian Russell <root@localhost.localdomain> - 2015-08-05 21:23 +0000
              Re: nano-M-sets Poutnik <poutnik4nntp@gmail.com> - 2015-08-06 08:16 +0200
                Re: nano-M-sets Fabian Russell <root@localhost.localdomain> - 2015-08-06 17:34 +0000
                  Re: nano-M-sets Quadibloc <jsavard@ecn.ab.ca> - 2015-08-06 11:23 -0700
                    Re: nano-M-sets Fabian Russell <root@localhost.localdomain> - 2015-08-06 18:39 +0000
                      Re: nano-M-sets Poutnik <poutnik4nntp@gmail.com> - 2015-08-06 21:07 +0200
                        Re: nano-M-sets Fabian Russell <root@localhost.localdomain> - 2015-08-06 19:14 +0000
                          Re: nano-M-sets Poutnik <poutnik4nntp@gmail.com> - 2015-08-07 07:41 +0200
                      Re: nano-M-sets Quadibloc <jsavard@ecn.ab.ca> - 2015-08-06 14:17 -0700
                        Re: nano-M-sets Fabian Russell <root@localhost.localdomain> - 2015-08-06 21:36 +0000
                          Re: nano-M-sets Poutnik <poutnik4nntp@gmail.com> - 2015-08-07 08:08 +0200
                  Re: nano-M-sets Poutnik <poutnik4nntp@gmail.com> - 2015-08-06 21:05 +0200
          Re: What Floating-Point Precisions Would Physicists Prefer? Quadibloc <jsavard@ecn.ab.ca> - 2015-08-06 11:18 -0700
            Re: What Floating-Point Precisions Would Physicists Prefer? Fabian Russell <root@localhost.localdomain> - 2015-08-06 19:03 +0000
              Re: What Floating-Point Precisions Would Physicists Prefer? Quadibloc <jsavard@ecn.ab.ca> - 2015-08-06 14:11 -0700
                Re: What Floating-Point Precisions Would Physicists Prefer? Fabian Russell <root@localhost.localdomain> - 2015-08-06 21:51 +0000
                  Re: What Floating-Point Precisions Would Physicists Prefer? Quadibloc <jsavard@ecn.ab.ca> - 2015-08-06 15:18 -0700
                    Re: What Floating-Point Precisions Would Physicists Prefer? Fabian Russell <root@localhost.localdomain> - 2015-08-07 01:11 +0000
                      Re: What Floating-Point Precisions Would Physicists Prefer? noTthaTguY <abu.kuanysh05@gmail.com> - 2015-08-07 10:25 -0700
                      Re: What Floating-Point Precisions Would Physicists Prefer? Quadibloc <jsavard@ecn.ab.ca> - 2015-08-07 11:41 -0700
                      Re: What Floating-Point Precisions Would Physicists Prefer? "hanson" <hanson@quick.net> - 2015-08-07 16:21 -0700
    Re: What Floating-Point Precisions Would Physicists Prefer? Timo <timo@physics.uq.edu.au> - 2015-08-05 04:47 -0700
      Re: What Floating-Point Precisions Would Physicists Prefer? Poutnik <Poutnik4NNTP@gmail.com> - 2015-08-05 14:36 +0200

Page 1 of 2  [1] 2  Next page →

#511307 — What Floating-Point Precisions Would Physicists Prefer?

On my web page, at

http://www.quadibloc.com/arch/perint.htm

I suggest different ways in which a computer system could defy one of the most prevalent conventions observed by nearly every computer today.

I remember reading, somewhere, that there were loud cries of dismay from people solving scientific problems on computers when the IBM System/360 replaced the IBM 7090 - concerning one characteristic of the new machines.

The 36-bit single-precision floating-point format of the 7090 supported a 
precision of eight significant digits - the 32-bit single-precision 
floating-point format of the 360 provided only seven significant digits, and 
perhaps less, because the format used a hexadecimal base, which caused some 
accuracy problems with that format.

So a lot of programs that used single precision got converted to use double 
precision on the new machine.

Also, there is a lot of historical evidence - from scientific pocket 
calculators going all the way back to logarithm tables - that ten digits was 
viewed as the appropriate precision to reach for when high accuracy was needed.

This would mesh well with a 48-bit floating point format, which can give eleven 
digits of precision.

Now, though, maybe some people do need double precision. Double precision on 
the 7090 took up 72 bits, but some were wasted on a second exponent field. The 
STRETCH used a few bits for flag purposes in its 64-bit floats. And the Control 
Data 6600, with 60 bit floats, seemed to have satisfied users.

So, from this historical data, I rationalized my way to the notion that a 
computer which used a 12-bit data element as its basic unit, and which provided 
floats that occupied 3, 4, or 5 of them - 36, 48, or 72 bits - would satisfy 
the needs of scientists better than a computer providing 32 and 64 bit floats.

Maybe I'm mistaken.

And, of course, using COTS equipment has big advantages, and it would be 
difficult to change the word sizes in use on the typical computer.

But I do recall a company making a chip with a large number of small CPUs on 
it... but the small CPUs could only do single-precision floating-point. Well, 
if the technology wasn't ready for that many double-precision CPUs on a chip, 
maybe that many 36-bit float CPUs could still have been squeezed in.

So just maybe getting the sizes of floats right is worth the effort.

Are there sources of informatiion on how much precision is needed for various 
types of scientific computation?

John Savard

[toc] | [next] | [standalone]

#511309

On Tue, 04 Aug 2015 20:41:18 -0700, Quadibloc wrote:

> 
> So, from this historical data, I rationalized my way to the notion that a 
> computer which used a 12-bit data element as its basic unit, and which provided 
> floats that occupied 3, 4, or 5 of them - 36, 48, or 72 bits - would satisfy 
> the needs of scientists better than a computer providing 32 and 64 bit floats.
> 

Machine floating point was in a state of unreliable chaos before the 1985 IEEE
standard was established, and the important aspect of this standard was not bit
size but correct rounding for all arithmetic operations.

In short, it's not precision that's critical but rather the choice of
algorithm.  A case in point being the log1p function that is part of
the standard C math library.  For values of x near 0, computing log(1+x)
directly can cause extreme error in spite of precision, whereas the 
log1p function is written to allow even single precision floats to give
highly accurate results.

The same applies to all computations.  The choice or construction of the
algorithm is most important.

But since most hardware is built for it, double binary precision at 64-bits
is most often recommended (using the correct algorithm).  Although 128-bit
or quad binary precision is available on some hardware there has been no
great need to move to this level for most computing.  The only reason
for using single precision FP (32-bit) over double would be to take advantage
of the increased parallelism available in the SSEx instruction sets,
but this may soon be moot as the newer AVX instructions incorporate
256-bit and 512-bit registers for use with double FP values.

However, I stress the "binary" aspect of current FP hardware.  Binary FP
may be suitable for scientific purposes but a *decimal* floating point
format would be better for financial calculations.  Software decimal FP
libraries are currently available and in the future decimal FP hardware
will likely be developed. 


> 
> Are there sources of informatiion on how much precision is needed for various 
> types of scientific computation?
> 

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Also, Kahan:

http://www.cs.berkeley.edu/~wkahan/

But there is plenty more available through general searches.  Pay particular
attention of the idea of catastrophic cancellation. 

[toc] | [prev] | [next] | [standalone]

#511337

On Tuesday, August 4, 2015 at 11:14:24 PM UTC-6, Fabian Russell wrote:
> On Tue, 04 Aug 2015 20:41:18 -0700, Quadibloc wrote:

> > Are there sources of informatiion on how much precision is needed for various 
> > types of scientific computation?

> http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

That is not what I meant, worthy document though it is.

John Savard

[toc] | [prev] | [next] | [standalone]

#511362

On Wed, 05 Aug 2015 04:38:10 -0700, Quadibloc wrote:

> 
>> http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
> 
> That is not what I meant,
>

I know exactly what you meant, and I also know that precision itself
is a red herring.  For numerical computation the nature of the
problem and solution algorithm are the only things that matter.

The inversion of an ill-conditioned matrix, for example, will produce
extreme error regardless of how much precision one throws at it.  In
this case the solution algorithm makes all the difference.

So forget about precision.  It is essentially meaningless.

[toc] | [prev] | [next] | [standalone]

#511462 — nano-M-sets

also, IEEE-758, -8-something, is inherently choaotic,
because of rounding (yer mini-Mandlebrots

> The inversion of an ill-conditioned matrix, for example, will produce
> extreme error regardless of how much precision one throws at it.  In
> this case the solution algorithm makes all the difference.
> 
> So forget about precision.  It is essentially meaningless.

[toc] | [prev] | [next] | [standalone]

#511472 — Re: nano-M-sets

On Wed, 05 Aug 2015 13:14:24 -0700, noTthaTguY wrote:

> also, IEEE-758, -8-something,
>

That's IEEE-754 2008 revision.

This is probably one of the most significant standards in
the technical world.  Everyone should have a copy on his,
not her, bookshelf.  (Women belong in the kitchen and not
the laboratory.)

But I think that OP is just another brain-dead dilettante.
He's chasing after the "fool's gold" of precision when precision
itself is quite meaningless.

For every computation, bar none, we need to understand thoroughly
the stability and conditioning of the algorithm utilized.

[toc] | [prev] | [next] | [standalone]

#511524 — Re: nano-M-sets

Dne 05/08/2015 v 23:23 Fabian Russell napsal(a):
> On Wed, 05 Aug 2015 13:14:24 -0700, noTthaTguY wrote:
> 
>> also, IEEE-758, -8-something,
>>
> 
> That's IEEE-754 2008 revision.
> 
> This is probably one of the most significant standards in
> the technical world.  Everyone should have a copy on his,
> not her, bookshelf.  (Women belong in the kitchen and not
> the laboratory.)

Prejudices are seldom attributes of a good scientist.
There is a good list of experimental observations
refuting the idea above.

> 
> But I think that OP is just another brain-dead dilettante.
> He's chasing after the "fool's gold" of precision when precision
> itself is quite meaningless.

Putting down the others does not make one smarter,
it only reveals he is not as smart as he would like to pretend.

Yes, precision itself is meaningless,
but sometimes it is very handy.
> 
> For every computation, bar none, we need to understand thoroughly
> the stability and conditioning of the algorithm utilized.
> 
Not only just algorithms,
but also of equations the algorithms try to solve
and of experimental data used for that.

As algorithms may have the best stability and conditioning
from all three of them, like in numerical weather prediction.

-- 
Poutnik ( the Czech word for a wanderer )

[toc] | [prev] | [next] | [standalone]

#511614 — Re: nano-M-sets

On Thu, 06 Aug 2015 08:16:31 +0200, Poutnik wrote:

>> 
>> (Women belong in the kitchen and not the laboratory.)
>>
> 
> Prejudices are seldom attributes of a good scientist.
>

It's not prejudice.  It is SCIENCE.

It is a science of which you have no experience, however.


>
> There is a good list of experimental observations
> refuting the idea above.
> 

As with all sociological "experiments" they are all just
free-floating, groundless nonsense, most of which are
designed to support foregone conclusions.

[toc] | [prev] | [next] | [standalone]

#511621 — Re: nano-M-sets

On Thursday, August 6, 2015 at 11:35:48 AM UTC-6, Fabian Russell wrote:

> It's not prejudice.  It is SCIENCE.

Well, it's true that Marie Curie came to a sad end.

Indeed, though, science does show there are differences between men and women. Even mental and behavioral ones, not just the narrow anatomical ones.

For example, if you visit a dairy farm, you will be cautioned about the dangers 
of approaching intact male cattle (bulls) while intact female cattle pose no 
particular danger. And yet cattle are not known to have been exposed to harmful 
stereotypes of male and female roles through watching too much of the wrong 
kind of television!

Still, I am not aware of any peer-reviewed studies that would go so far as to 
support the sweeping conclusion that "women belong in the kitchen and not the 
laboratory"; recent experience has shown that women can achieve accomplishments 
in many fields considered traditionally male.

John Savard

[toc] | [prev] | [next] | [standalone]

#511623 — Re: nano-M-sets

On Thu, 06 Aug 2015 11:23:51 -0700, Quadibloc wrote:

> 
> science does show there are differences between men and women.
>

A formalized science is not really necessary.

"Anatomy is destiny," as the saying goes.  Women are designed for one
thing only and that is bear and nurture children.  No matter how much
they attempt to override this fundamental instinct it will wiggle
out in some way.  Even the hard-core female CEO still dotes over her
little pet poodle (substitute child) that she would never forsake. 

But the science of psychoanalysis has elucidated quite well the
psychodynamics of a feminist, or anti-women.  These bitches are
mentally disturbed to a high degree and cannot be trusted in any
kind of social context.

Men, on the other hand, are creatures of aggression and war.
Science is derived from war and consequently science is the
natural domain for men and men only.

All men who disagree are brainwashed, emasculated idiots.

[toc] | [prev] | [next] | [standalone]

#511629 — Re: nano-M-sets

Dne 06/08/2015 v 20:39 Fabian Russell napsal(a):
> 
> All men who disagree are brainwashed, emasculated idiots.

This says much more about you than them.

-- 
Poutnik ( the Czech word for a wanderer )

[toc] | [prev] | [next] | [standalone]

#511630 — Re: nano-M-sets

On Thu, 06 Aug 2015 21:07:48 +0200, Poutnik wrote:

>> 
>> All men who disagree are brainwashed, emasculated idiots.
> 
> This says much more about you than them.
>


That's true.  It says that I am a perfectly normal and psychologically
well adjusted male.  I am not deceived by the mental illness known
as feminism.

 

[toc] | [prev] | [next] | [standalone]

#511697 — Re: nano-M-sets

Dne 06/08/2015 v 21:14 Fabian Russell napsal(a):
> On Thu, 06 Aug 2015 21:07:48 +0200, Poutnik wrote:
> 
>>>
>>> All men who disagree are brainwashed, emasculated idiots.
>>
>> This says much more about you than them.
>>
> 
> 
> That's true.  It says that I am a perfectly normal and psychologically
> well adjusted male.  I am not deceived by the mental illness known
> as feminism.
> 

No, it does not say so.
What it says about you, you cannot evaluate by yourself.


-- 
Poutnik ( the Czech word for a wanderer )

[toc] | [prev] | [next] | [standalone]

#511654 — Re: nano-M-sets

On Thursday, August 6, 2015 at 12:39:57 PM UTC-6, Fabian Russell wrote:

> Men, on the other hand, are creatures of aggression and war.
> Science is derived from war and consequently science is the
> natural domain for men and men only.

I do think that there is some truth to the first sentence of that paragraph - so 
that men should be the ones to go into combat, and the majority of our police, 
and so on.

Science, while certainly advanced by war - because the tendency is for rulers 
to fear innovation because it brings change, which might loosen their grip on 
power - is only one of the things that uses it. But war loosens the government 
purse strings as well... between wars, scientific advance is limited to what 
consumer production drives, leading to limited and unbalanced technological 
progress, as we see now: computers advance, everything else stagnates.

But what this has to do with women in the laboratory is unclear.

Male scientists do not tend to fit into the same macho stereotype as front-line 
soldiers or football players. It's a book larnin' type of activity.

John Savard

[toc] | [prev] | [next] | [standalone]

#511660 — Re: nano-M-sets

On Thu, 06 Aug 2015 14:17:11 -0700, Quadibloc wrote:

> 
> Male scientists do not tend to fit into the same macho stereotype as front-line 
> soldiers or football players. It's a book larnin' type of activity.
> 

War is derived from hunting, and hunting involves a constant effort
at reasoning, inferring, predicting, theorizing, and anticipating.
All these activities are naturally present in men and form the
inclination toward formal science.

Women simply do not hunt, in a primitive sense.

Any woman who embodies or espouses these attributes is highly suspect.

All of this has been well established long ago.  It is only the feminists,
and their emasculated male supporters, that remain ignorant and deluded.

 

[toc] | [prev] | [next] | [standalone]

#511700 — Re: nano-M-sets

Dne 06/08/2015 v 23:36 Fabian Russell napsal(a):
> On Thu, 06 Aug 2015 14:17:11 -0700, Quadibloc wrote:
> 
>>
>> Male scientists do not tend to fit into the same macho stereotype as front-line 
>> soldiers or football players. It's a book larnin' type of activity.
>>
> 
> War is derived from hunting, and hunting involves a constant effort
> at reasoning, inferring, predicting, theorizing, and anticipating.
> All these activities are naturally present in men and form the
> inclination toward formal science.
> 
> Women simply do not hunt, in a primitive sense.
> 
> Any woman who embodies or espouses these attributes is highly suspect.
> 
> All of this has been well established long ago.  It is only the feminists,
> and their emasculated male supporters, that remain ignorant and deluded.
> 

This stereotypes were refuted long time ago by neurologists.

Variance in functionality of man brain - the same for woman brain
- is bigger than difference between these 2 groups.


-- 
Poutnik ( the Czech word for a wanderer )

[toc] | [prev] | [next] | [standalone]

#511628 — Re: nano-M-sets

Dne 06/08/2015 v 19:34 Fabian Russell napsal(a):
> On Thu, 06 Aug 2015 08:16:31 +0200, Poutnik wrote:
> 
>>>
>>> (Women belong in the kitchen and not the laboratory.)
>>>
>>
>> Prejudices are seldom attributes of a good scientist.
>>
> 
> It's not prejudice.  It is SCIENCE.
> 
> It is a science of which you have no experience, however.

Another prejudice.
>>
>> There is a good list of experimental observations
>> refuting the idea above.
>>
> 
> As with all sociological "experiments" they are all just
> free-floating, groundless nonsense, most of which are
> designed to support foregone conclusions.
> 
They are not sociological experiments,
but facts from history of science.



-- 
Poutnik ( the Czech word for a wanderer )

[toc] | [prev] | [next] | [standalone]

#511619

On Wednesday, August 5, 2015 at 8:23:16 AM UTC-6, Fabian Russell wrote:

> The inversion of an ill-conditioned matrix, for example, will produce
> extreme error regardless of how much precision one throws at it.  In
> this case the solution algorithm makes all the difference.

Yes. I remember trying to invert a matrix as a first-year computing assignment, 
and getting a bad result. On the other hand, somewhat later, I came up with a 
way to get a good answer, but the method took O(n^5) time, so it was only 
practical for relatively small matrices.

It's true that there is no substitute for doing your numerical analysis before 
programming - but once it _is_ done, precision is hardly irrelevant. It's when 
numerical analysis _isn't_ done that people just throw lots of excess precision 
at the computation and hope for the best.

If you do it right, you don't have to waste bits - and if you have a problem to 
solve that requires massive resources, on the edge of the state of the art, a 
good fit becomes more beneficial.

John Savard

[toc] | [prev] | [next] | [standalone]

#511627

On Thu, 06 Aug 2015 11:18:07 -0700, Quadibloc wrote:

> 
> Yes. I remember trying to invert a matrix as a first-year computing assignment, 
> and getting a bad result. 
>

Well, the matrix has to be ill-conditioned.  The classic example is the
Hilbert matrix:

https://en.wikipedia.org/wiki/Hilbert_matrix


> 
> If you do it right, you don't have to waste bits
>

The problem with precision is that it has historically been
very expensive to go to higher bit lengths.  With today's cheap
hardware this is no longer the case, but the compromise, both
in hardware and in standards, has settled on 64 (or 128) bits
for floating point.

Another area of concern is compiler optimization.  Some compilers,
such as the Intel C/C++ compiler, has a default level of optimization
that is directed toward computational speed.  To achieve this
it will turn on a math processor feature called "denormal flush-to-zero."

Digital floating point numbers are always "normalized," that is,
the first bit of the mantissa is always an implied "1."  However,
numbers that are close to zero cannot be normalized and they must
have a different format, called a "denormal" number.  Calculating
with the denormal format takes a LOT of time and to increase speed
an option exists to just transform, or flush, these numbers to
zero.  Doing this increases computational speed but it can also
reduce accuracy.

Other compiler optimizations can influence accuracy in other ways
as well.

[toc] | [prev] | [next] | [standalone]

#511651

On Thursday, August 6, 2015 at 1:04:55 PM UTC-6, Fabian Russell wrote:
> However,
> numbers that are close to zero cannot be normalized and they must
> have a different format, called a "denormal" number.  Calculating
> with the denormal format takes a LOT of time and to increase speed
> an option exists to just transform, or flush, these numbers to
> zero.  Doing this increases computational speed but it can also
> reduce accuracy.

If calculating with such numbers "takes a lot of time", then such an option, to 
decrease exponent range, is reasonable - and then the reduced exponent range 
must be taken into account when programming.

It doesn't reduce the precision or accuracy of calculations that don't involve 
very small numbers.

However, for such calculations _to_ take a lot of time means that IEEE 754 was 
not implemented correctly on the processor in question. It is possible to do 
better than that.

John Savard

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | sci.physics

csiph-web