Groups > comp.graphics.apps.gnuplot > #3157

Re: incorrect arithmetik?

From	Karl Ratzsch <mail.kfr@gmx.net>
Newsgroups	comp.graphics.apps.gnuplot
Subject	Re: incorrect arithmetik?
Date	2016-01-12 14:19 +0100
Organization	solani.org
Message-ID	<n72ugm$5ls$1@solani.org> (permalink)
References	<n72l7f$865$1@newsserver.rrzn.uni-hannover.de>

Show all headers | View raw

Am 12.01.2016 um 11:40 schrieb Jörg Buchholz:
> I can not found the solution to the following problem:
> 
> b=7.2-6.9
> c=8.2-7.9
> 
> show variables

> b = 0.3
> c = 0.299999999999999
> 
> Why is c not 0.3?

Floating point arithmetics. Most decimal fractions cannot be represented
exactly by a binary float variable.

8.2 is 1000.00110011....
7.2 is 111.11100110011....

If the two errors add up in one direction, the result of your
subtraction will be closer to a value that is in fact off by 1 in the
least significant digit.

  Karl

Back to comp.graphics.apps.gnuplot | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

incorrect arithmetik? Jörg Buchholz <bookwood4new@freenet.de> - 2016-01-12 11:40 +0100
  Re: incorrect arithmetik? Karl Ratzsch <mail.kfr@gmx.net> - 2016-01-12 14:19 +0100
    Re: incorrect arithmetik? Jörg Buchholz <bookwood4news@freenet.de> - 2016-01-12 18:50 +0100
      Re: incorrect arithmetik? Gene Wagenbreth <gwagenbreth@gmail.com> - 2016-02-17 23:30 -0800
  Re: incorrect arithmetik? Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2016-01-12 17:06 +0100

csiph-web