Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.databases.filemaker > #2858

Re: "Total of" Rounding Error?

Show all headers | View raw

On 2017-05-10 00:16:31 +0000, David Empson said:
> 
> Speaking as someone who does understand binary number systems (I'm also
> a programmer of 30+ years experience), your point is valid if numbers
> are being stored as floating point, particularly the inaccuracy of
> fractional values: if a floating point number is used to represent
> dollars with a fractional value for cents, then there is no exact
> representation possible for cent values other than .00, .25, .50 and
> .75, because those are the only values from .00 to .99 which have exact
> sums of negative powers of two.
> 
> This is definitely an issue with Excel and other spreadsheets, which are
> based around floating point numbers.
> 
> https://support.microsoft.com/en-us/help/78113/floating-point-arithmetic-may-give-inaccurate-results-in-excel 
> 
<snip>

Which could explain the silly issue I had with an Excel spreadsheet, 
and makes it utterly useless for dealing with numbers ... it's main 
purpose.  :o\

Sounds like a silly engineer's "close enough is good enough" design 
decision, rather than a proper mathematician's design.

Helpful Harry  :o)

Back to comp.databases.filemaker | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

"Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-02 12:52 -0700
  Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-03 09:21 +1200
    Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-06 08:03 -0700
      Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-07 10:41 +1200
        Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-06 19:05 -0700
          Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-07 17:45 +1200
            Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-08 01:22 -0700
              Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-09 09:33 +1200
                Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-09 01:40 -0700
                Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-10 08:56 +1200
                Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-09 16:15 -0700
                Re: "Total of" Rounding Error? dempson@actrix.gen.nz (David Empson) - 2017-05-10 12:16 +1200
                Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-10 13:59 +1200
                Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-10 14:07 +1200
                Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-11 15:09 -0700
                Re: "Total of" Rounding Error? dempson@actrix.gen.nz (David Empson) - 2017-05-12 12:56 +1200
  Re: "Total of" Rounding Error? clkaufmann@gmx.ch (Christoph L. Kaufmann) - 2017-05-04 09:23 +0200
    Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-06 07:42 -0700
  Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-07 19:36 +1200
    Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-08 01:22 -0700
      Re: "Total of" Rounding Error? Helpful Harry <HelpfulHarry@BusyWorking.com> - 2017-05-12 13:36 +1200
  Re: "Total of" Rounding Error? csampson@inetworld.net (Charles H. Sampson) - 2017-05-09 16:04 -0700

csiph-web