Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.graphics.apps.gnuplot > #3159
| From | Jörg Buchholz <bookwood4news@freenet.de> |
|---|---|
| Newsgroups | comp.graphics.apps.gnuplot |
| Subject | Re: incorrect arithmetik? |
| Date | 2016-01-12 18:50 +0100 |
| Organization | A noiseless patient Spider |
| Message-ID | <n73e83$oa9$1@dont-email.me> (permalink) |
| References | <n72l7f$865$1@newsserver.rrzn.uni-hannover.de> <n72ugm$5ls$1@solani.org> |
On 12.01.2016 14:19, Karl Ratzsch wrote: > Am 12.01.2016 um 11:40 schrieb Jörg Buchholz: >> I can not found the solution to the following problem: >> >> b=7.2-6.9 >> c=8.2-7.9 >> >> show variables > >> b = 0.3 >> c = 0.299999999999999 >> >> Why is c not 0.3? > > Floating point arithmetics. Most decimal fractions cannot be represented > exactly by a binary float variable. > > 8.2 is 1000.00110011.... > 7.2 is 111.11100110011.... > > If the two errors add up in one direction, the result of your > subtraction will be closer to a value that is in fact off by 1 in the > least significant digit. Thanks for the explanation, I think I understand it. Jörg
Back to comp.graphics.apps.gnuplot | Previous | Next — Previous in thread | Next in thread | Find similar
incorrect arithmetik? Jörg Buchholz <bookwood4new@freenet.de> - 2016-01-12 11:40 +0100
Re: incorrect arithmetik? Karl Ratzsch <mail.kfr@gmx.net> - 2016-01-12 14:19 +0100
Re: incorrect arithmetik? Jörg Buchholz <bookwood4news@freenet.de> - 2016-01-12 18:50 +0100
Re: incorrect arithmetik? Gene Wagenbreth <gwagenbreth@gmail.com> - 2016-02-17 23:30 -0800
Re: incorrect arithmetik? Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2016-01-12 17:06 +0100
csiph-web