Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.arch > #4065 > unrolled thread
| Started by | "Skybuck Flying" <Windows7IsOK@DreamPC2006.com> |
|---|---|
| First post | 2011-10-08 02:54 +0200 |
| Last post | 2011-10-13 12:10 +0200 |
| Articles | 20 on this page of 89 — 21 participants |
Back to article view | Back to comp.arch
Infinity + Infinity (or NegInfinity - NegInfinity) "Skybuck Flying" <Windows7IsOK@DreamPC2006.com> - 2011-10-08 02:54 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Robert Myers <rbmyersusa@gmail.com> - 2011-10-07 18:40 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) James Kuyper <jameskuyper@verizon.net> - 2011-10-07 22:39 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) William Elliot <marsh@rdrop.com> - 2011-10-07 19:56 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Jasen Betts <jasen@xnet.co.nz> - 2011-10-08 06:45 +0000
Re: Infinity + Infinity (or NegInfinity - NegInfinity) gordonb.eug6r@burditt.org (Gordon Burditt) - 2011-10-08 02:37 -0500
Re: Infinity + Infinity (or NegInfinity - NegInfinity) William Elliot <marsh@rdrop.com> - 2011-10-08 21:40 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Kaba <kaba@nowhere.com> - 2011-10-09 12:17 +0300
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Jasen Betts <jasen@xnet.co.nz> - 2011-10-08 06:33 +0000
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Skybuck Flying" <Windows7IsOK@DreamPC2006.com> - 2011-10-08 16:41 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) kenney@cix.compulink.co.uk - 2011-10-09 04:01 -0500
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Kaba <kaba@nowhere.com> - 2011-10-08 17:43 +0300
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "christian.bau" <christian.bau@cbau.wanadoo.co.uk> - 2011-10-08 10:20 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Kaba <kaba@nowhere.com> - 2011-10-08 22:35 +0300
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-08 18:55 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Kaba <kaba@nowhere.com> - 2011-10-08 23:03 +0300
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-08 21:27 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) James Kuyper <jameskuyper@verizon.net> - 2011-10-08 17:28 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Robert Myers <rbmyersusa@gmail.com> - 2011-10-08 18:15 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-08 23:23 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) James Kuyper <jameskuyper@verizon.net> - 2011-10-08 21:29 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-09 10:01 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Al Grant <algrant@myrealbox.com> - 2011-10-12 06:14 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-12 15:25 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) James Kuyper <jameskuyper@verizon.net> - 2011-10-12 12:10 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-12 18:41 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) James Kuyper <jameskuyper@verizon.net> - 2011-10-12 15:32 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-12 20:44 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Robert Myers <rbmyersusa@gmail.com> - 2011-10-12 18:11 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Andrew Reilly <areilly---@bigpond.net.au> - 2011-10-13 03:48 +0000
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Paul A. Clayton" <paaronclayton@gmail.com> - 2011-10-12 23:15 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-13 09:12 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Paul A. Clayton" <paaronclayton@gmail.com> - 2011-10-13 00:49 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-13 10:37 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) EricP <ThatWouldBeTelling@thevillage.com> - 2011-10-13 11:04 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) EricP <ThatWouldBeTelling@thevillage.com> - 2011-10-13 17:15 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-14 09:22 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) EricP <ThatWouldBeTelling@thevillage.com> - 2011-10-14 13:37 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Andy \"Krazy\" Glew" <andy@SPAM.comp-arch.net> - 2011-10-13 11:11 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Al Grant <algrant@myrealbox.com> - 2011-10-14 01:04 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-13 09:07 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Al Grant <algrant@myrealbox.com> - 2011-10-13 07:00 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Robert Myers <rbmyersusa@gmail.com> - 2011-10-14 10:54 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Andrew Reilly <areilly---@bigpond.net.au> - 2011-10-14 22:36 +0000
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-15 09:26 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Paul A. Clayton" <paaronclayton@gmail.com> - 2011-10-15 05:39 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-15 14:56 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-17 08:37 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-17 12:03 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-17 11:13 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-17 22:30 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Paul A. Clayton" <paaronclayton@gmail.com> - 2011-10-17 14:47 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-18 11:33 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-18 10:21 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-18 08:16 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-18 14:26 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-18 13:32 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-18 17:47 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-18 17:51 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-19 10:37 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-19 09:36 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-19 11:46 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-19 10:45 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-19 13:11 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-19 12:12 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-19 16:22 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-19 15:16 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Andrew Reilly <areilly---@bigpond.net.au> - 2011-10-15 20:16 +0000
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Robert Myers <rbmyersusa@gmail.com> - 2011-10-15 16:26 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-17 08:14 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) nmm1@cam.ac.uk - 2011-10-17 08:22 +0100
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-17 11:39 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) rpw3@rpw3.org (Rob Warnock) - 2011-10-17 21:00 -0500
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Skybuck Flying" <Windows7IsOK@DreamPC2006.com> - 2011-10-18 05:44 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Andy \"Krazy\" Glew" <andy@SPAM.comp-arch.net> - 2011-10-17 21:35 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Jasen Betts <jasen@xnet.co.nz> - 2011-10-24 02:40 +0000
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-10-27 00:32 +0300
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-27 10:41 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2011-10-27 13:10 +0300
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Terje Mathisen <"terje.mathisen at tmsw.no"> - 2011-10-27 13:53 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Keith Thompson <kst-u@mib.org> - 2011-10-12 12:34 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Al Grant <algrant@myrealbox.com> - 2011-10-13 06:39 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) James Kuyper <jameskuyper@verizon.net> - 2011-10-13 10:34 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Al Grant <algrant@myrealbox.com> - 2011-10-14 00:54 -0700
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Larry Hewitt <larryhwtt@comporium.net> - 2011-10-09 21:35 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Skybuck Flying" <Windows7IsOK@DreamPC2006.com> - 2011-10-10 04:37 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Larry Hewitt <larryhwtt@comporium.net> - 2011-10-10 13:58 -0400
Re: Infinity + Infinity (or NegInfinity - NegInfinity) "Skybuck Flying" <Windows7IsOK@DreamPC2006.com> - 2011-10-13 20:17 +0200
Re: Infinity + Infinity (or NegInfinity - NegInfinity) Arivald <arivald_@AT_interia_DOT_.pl> - 2011-10-13 12:10 +0200
Page 3 of 5 — ← Prev page 1 2 [3] 4 5 Next page →
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-13 09:07 +0100 |
| Message-ID | <j7668h$hso$1@gosset.csi.cam.ac.uk> |
| In reply to | #4165 |
In article <9fn5cjFsr4U1@mid.individual.net>, Andrew Reilly <areilly---@bigpond.net.au> wrote: > >Nick's point (I believe) is that floating point arithmetic is fraught, >because it is fairly fundamentally unlike the mathematics that expression- >based languages appear to be offering. ... Actually, no. It's tricky, but not THAT tricky. My real point is that the C99 and other modern 'improvements' make it appear to the naive as if it is more portable and reliable, but actually introduce ten times as many portability and reliability problems as they solve. Almost everyone who has actually tried it has discovered that, but most people code for one system, run a couple of sets of well-behaved data, and then claim that their program is portable and robust and any failures must be the fault of the compiler! >Clearly you can still have parallelism between independent operations, >but you can't manipulate floating point expressions that look like maths >as though they were maths, at least not without giving up being able to >reason about the answer. Yes, you can. Look at most of the classic numerical analysis books. You have to do it rather differently, and very, very few modern 'computer scientists' would know how to start. Wilkinson and Reinsch "The Algebraic Eigenvalue Problem" is one example of such reasoning that I have used in the past. >Which I think just emphasises your point, too: people doing floating >point need to have a clue... Oh, THERE, I am in 100% agreement. When I teach it, I tell people that the key is to think floating-point, not mathematical real, but that 50+ years of Fortran experience shows that it's not as hard as all that. Inter alia, we used to write code that was both reliable and portable across a range of arithmetics that most people nowadays cannot imagine :-) There ARE some books on how to write robust, portable floating-point code, but all are very old-fashioned. And there were a large number of experiments showing both that the exact details of the arithmetic didn't matter much (though directed versus nearest rounding did), and that relying on the the exact details led to the code being LESS robust, rather than more. Of courses, that's now all regarded as heresy .... Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | Al Grant <algrant@myrealbox.com> |
|---|---|
| Date | 2011-10-13 07:00 -0700 |
| Message-ID | <ee6b3c21-d8be-4c40-a3c7-afb70caba4f3@e37g2000yqa.googlegroups.com> |
| In reply to | #4165 |
On Oct 13, 4:48 am, Andrew Reilly <areilly...@bigpond.net.au> wrote: > Nick does have a good point though, notwithstanding the last couple of > posts in which the authors asserted that C (at least) now *does* specify > a total order of operations within expressions (even ones that appear > ambiguous because of expectations of associativity). I didn't realize > that, and I'm still not sure that I believe them. The point is that there is no difference in C between a+b+c and (a+b)+c. Any order implied by the latter is also implied by the former. The point is that operator precedence in C uniquely defines the "evaluation tree". I don't believe anyone has asserted that there is a total order of all computations within the tree, i.e. between the LHS and RHS of a binary operator. > Nick's point (I believe) is that floating point arithmetic is fraught We all know it that floating-point arithmetic is not the same as real arithmetic. Nick appears to be making an abstruse point, about the relationship between ISO C and IEEE 754, without being able to explain what that point is.
[toc] | [prev] | [next] | [standalone]
| From | Robert Myers <rbmyersusa@gmail.com> |
|---|---|
| Date | 2011-10-14 10:54 -0700 |
| Message-ID | <0660aa82-b446-459d-99b5-441c1588f39e@u13g2000vbx.googlegroups.com> |
| In reply to | #4165 |
On Oct 12, 11:48 pm, Andrew Reilly <areilly...@bigpond.net.au> wrote: > > Nick does have a good point though, notwithstanding the last couple of > posts in which the authors asserted that C (at least) now *does* specify > a total order of operations within expressions (even ones that appear > ambiguous because of expectations of associativity). I didn't realize > that, and I'm still not sure that I believe them. > > Nick's point (I believe) is that floating point arithmetic is fraught, > because it is fairly fundamentally unlike the mathematics that expression- > based languages appear to be offering. <snip> > I haven't even mentioned an IEEE standard here: > it ought to be possible to define a stateless floating point method, > where rounding modes were explicit in the instructions and exceptions > were either defined away as in-band values or made of precice exceptions, > but I've never heard of it being done. > <snip> > Which I think just emphasises your point, too: people doing floating > point need to have a clue... > You make some good points that are worth remembering. I'm not sure, though, that a stateless and platform-independent definition of floating point arithmetic is either necessary or desirable or even possible within the bounds of practical utility. If the floating point details that people spend so much time on here make all that much difference, there is probably something flaky about either the algorithm or the code or both. Given that at least some of the people who spend so much time talking about this issue here probably already understand the claim I just made, I don't understand why this alchemist's pursuit of turning lead into gold continues. Given the choice between hammering it into people that floating point arithmetic is not for the naive or even necessarily replicable from one situation to another, and inventing an arbitrary standard that makes it at least make the same mistake every time, I'd prefer the current chaos to artificial predictability. I could and would and do make the same objection about the correspondence to mathematics that you make for floating point arithmetic to every code that purports to represent mathematics where a differential operator appears in the calculation. I know how to make that correspondence precise and completely unarbitrary, but I am repeatedly told that doing so is simply too expensive. I conclude, as I believe I have said before, that this continuing discussion is an instance of Nazrudin's lost key: looking where there is light. Given that many if not most codes that use floating point arithmetic rely on absurd mathematics (repeatedly differentiating derivatives that are piecewise continuous even only in theory), I just can't understand all the fuss about the last bit (rounding), which idealized properties floating point arithmetic does or does not possess, or what arbitrary thing the hardware does when the algorithm leads to a nonsensical or ambiguous result. Robert.
[toc] | [prev] | [next] | [standalone]
| From | Andrew Reilly <areilly---@bigpond.net.au> |
|---|---|
| Date | 2011-10-14 22:36 +0000 |
| Message-ID | <9frrqsF4kcU1@mid.individual.net> |
| In reply to | #4199 |
On Fri, 14 Oct 2011 10:54:06 -0700, Robert Myers wrote: > If the floating point details that people spend so much time on here > make all that much difference, there is probably something flaky about > either the algorithm or the code or both. Well, yes. I'm afraid that I have that particular rant on speed-dial, because I often find myself having to hose down the desire to produce bit- exact test-suites for inherently unstable (or at least somewhat arbitrary) algorithms that just happen to behave in a particular way in floating point with a particular version of a compiler, with a particular set of command-line options. Believe me, it happens a lot. Cheers, -- Andrew
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-15 09:26 +0200 |
| Message-ID | <a6pnm8-kqj2.ln1@ntp6.tmsw.no> |
| In reply to | #4202 |
Andrew Reilly wrote: > On Fri, 14 Oct 2011 10:54:06 -0700, Robert Myers wrote: > >> If the floating point details that people spend so much time on here >> make all that much difference, there is probably something flaky about >> either the algorithm or the code or both. > > Well, yes. I'm afraid that I have that particular rant on speed-dial, > because I often find myself having to hose down the desire to produce bit- > exact test-suites for inherently unstable (or at least somewhat > arbitrary) algorithms that just happen to behave in a particular way in > floating point with a particular version of a compiler, with a particular > set of command-line options. Believe me, it happens a lot. I would in fact be much _more_ useful to have a mode where rounding was arbitrary/random, and could even impact one or more extra low bits. I.e. if your code doesn't produce effectively the same results under such a regime, your algorithms are almost certainly broken. Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | "Paul A. Clayton" <paaronclayton@gmail.com> |
|---|---|
| Date | 2011-10-15 05:39 -0700 |
| Message-ID | <05e483f7-4330-4ae0-b628-594bed7db169@f36g2000vbm.googlegroups.com> |
| In reply to | #4204 |
[newsgroups trimmed] On Oct 15, 2:26 am, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: [snip] > I would in fact be much _more_ useful to have a mode where rounding was > arbitrary/random, and could even impact one or more extra low bits. > > I.e. if your code doesn't produce effectively the same results under > such a regime, your algorithms are almost certainly broken. Would such rounding be random (i.e., fixed probability of rounding up/down) or probabilistic (i.e., at least some of the insignificant bits used for the probability [e.g., 1.011 + 0.001101 = 1.100 + a five in eight chance of adding .001--probably implemented by adding a random number with only insignificant bits and rounding toward zero/truncating])? Random rounding would seem to fit the perturbing goal better (and might have better mathematical properties), but probabilistic rounding seems interesting in that it includes some of the existing information.
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-15 14:56 +0200 |
| Message-ID | <5hcom8-0gk2.ln1@ntp6.tmsw.no> |
| In reply to | #4205 |
Paul A. Clayton wrote: > [newsgroups trimmed] > On Oct 15, 2:26 am, Terje Mathisen<"terje.mathisen at tmsw.no"> > wrote: > [snip] >> I would in fact be much _more_ useful to have a mode where rounding was >> arbitrary/random, and could even impact one or more extra low bits. >> >> I.e. if your code doesn't produce effectively the same results under >> such a regime, your algorithms are almost certainly broken. > > Would such rounding be random (i.e., fixed probability of rounding > up/down) or probabilistic (i.e., at least some of the insignificant > bits > used for the probability [e.g., 1.011 + 0.001101 = 1.100 + a five in > eight chance of adding .001--probably implemented by adding a > random number with only insignificant bits and rounding toward > zero/truncating])? Random rounding would seem to fit the > perturbing goal better (and might have better mathematical > properties), but probabilistic rounding seems interesting in that it > includes some of the existing information. Cue Nick here: He has certainly done much more work with fp than I have. I do believe him though when he states that many algorithms actually behave better with random rounding that the nearest_or_even default. Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-17 08:37 +0100 |
| Message-ID | <j7glv2$h25$1@gosset.csi.cam.ac.uk> |
| In reply to | #4206 |
In article <5hcom8-0gk2.ln1@ntp6.tmsw.no>, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: >Paul A. Clayton wrote: >> [newsgroups trimmed] >> On Oct 15, 2:26 am, Terje Mathisen<"terje.mathisen at tmsw.no"> >> wrote: >> [snip] >>> I would in fact be much _more_ useful to have a mode where rounding was >>> arbitrary/random, and could even impact one or more extra low bits. >>> >>> I.e. if your code doesn't produce effectively the same results under >>> such a regime, your algorithms are almost certainly broken. >> >> Would such rounding be random (i.e., fixed probability of rounding >> up/down) or probabilistic (i.e., at least some of the insignificant >> bits >> used for the probability [e.g., 1.011 + 0.001101 = 1.100 + a five in >> eight chance of adding .001--probably implemented by adding a >> random number with only insignificant bits and rounding toward >> zero/truncating])? Random rounding would seem to fit the >> perturbing goal better (and might have better mathematical >> properties), but probabilistic rounding seems interesting in that it >> includes some of the existing information. > >Cue Nick here: He has certainly done much more work with fp than I have. > >I do believe him though when he states that many algorithms actually >behave better with random rounding that the nearest_or_even default. There terminology here is problematic, because quite a lot of the modern reinventers have tried changing terminology in order to publish old knowledge as something new. Ignoring the round-off is an ancient technique, dating back from when gates were a major limitation, and typically loses one bit of accuracy relative to nearest rounding. While I could produce code that would behave better, I don't know of any realistic examples. True probabilistic rounding is interesting. It is half a bit less accurate on average, but unbiassed for all distributions of the round-off. And, yes, quite a lot of algorithms behave better (especially accumulation ones). On my long list is to get into a situation where I can run real code in both modes and see how they compare. Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-17 12:03 +0200 |
| Message-ID | <34btm8-k2r2.ln1@ntp6.tmsw.no> |
| In reply to | #4219 |
nmm1@cam.ac.uk wrote: > True probabilistic rounding is interesting. It is half a bit less > accurate on average, but unbiassed for all distributions of the Half a bit? I would have said either a quarter of a bit less accurate: Nearest_or_even has by definition a maximum error of 0.5 ulp, with the average error being half of that, i.e. 0.25 ulp, right? Random would seem to give an average error of 0.5 ulp, also by definition. > round-off. And, yes, quite a lot of algorithms behave better > (especially accumulation ones). On my long list is to get into > a situation where I can run real code in both modes and see how > they compare. How hard would this be to do? On existing hw? With or without modifying the compiler? On x86 you could use a random bitstream to toggle the rounding mode back & forth between -Inf and +Inf before every fp operation, but the overhead would be pretty bad. :-( Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-17 11:13 +0100 |
| Message-ID | <j7gv45$5ue$1@gosset.csi.cam.ac.uk> |
| In reply to | #4222 |
In article <34btm8-k2r2.ln1@ntp6.tmsw.no>, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: > >> True probabilistic rounding is interesting. It is half a bit less >> accurate on average, but unbiassed for all distributions of the > >Half a bit? I would have said either a quarter of a bit less accurate: > >Nearest_or_even has by definition a maximum error of 0.5 ulp, with the >average error being half of that, i.e. 0.25 ulp, right? > >Random would seem to give an average error of 0.5 ulp, also by definition. That's ULPs! I was estimating how many extra bits of representation would be needed to give the same accuracy. Whatever. It's a very small amount, whether worst-case or standard error. >> round-off. And, yes, quite a lot of algorithms behave better >> (especially accumulation ones). On my long list is to get into >> a situation where I can run real code in both modes and see how >> they compare. > >How hard would this be to do? On existing hw? With or without modifying >the compiler? > >On x86 you could use a random bitstream to toggle the rounding mode back >& forth between -Inf and +Inf before every fp operation, but the >overhead would be pretty bad. :-( That would be insane! I originally did it using a Fortran 66 utility AUGMENT that allowed you to add data types, and I converted all arithmetic to those. That could be done more easily nowadays in Fortran (using a Python script to make the code changes), and might even be possible for very clean C or C++. In most serious programs, you would need to add a few fixes, anyway, where the code assumes repeatability. It is much faster to execute than most people think, because it uses no more data, has almost exactly the same access patterns, and most 'floating-point' programs use more actual CPU in general instruction execution than floating-point, anyway. Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-17 22:30 +0200 |
| Message-ID | <hsfum8-v9t2.ln1@ntp6.tmsw.no> |
| In reply to | #4223 |
nmm1@cam.ac.uk wrote: > In article<34btm8-k2r2.ln1@ntp6.tmsw.no>, > Terje Mathisen<"terje.mathisen at tmsw.no"> wrote: >> >>> True probabilistic rounding is interesting. It is half a bit less >>> accurate on average, but unbiassed for all distributions of the >> >> Half a bit? I would have said either a quarter of a bit less accurate: >> >> Nearest_or_even has by definition a maximum error of 0.5 ulp, with the >> average error being half of that, i.e. 0.25 ulp, right? >> >> Random would seem to give an average error of 0.5 ulp, also by definition. > > That's ULPs! I was estimating how many extra bits of representation > would be needed to give the same accuracy. Whatever. It's a very > small amount, whether worst-case or standard error. > >>> round-off. And, yes, quite a lot of algorithms behave better >>> (especially accumulation ones). On my long list is to get into >>> a situation where I can run real code in both modes and see how >>> they compare. >> >> How hard would this be to do? On existing hw? With or without modifying >> the compiler? >> >> On x86 you could use a random bitstream to toggle the rounding mode back >> & forth between -Inf and +Inf before every fp operation, but the >> overhead would be pretty bad. :-( > > That would be insane! Why? > > I originally did it using a Fortran 66 utility AUGMENT that allowed > you to add data types, and I converted all arithmetic to those. That > could be done more easily nowadays in Fortran (using a Python script > to make the code changes), and might even be possible for very clean > C or C++. In most serious programs, you would need to add a few > fixes, anyway, where the code assumes repeatability. Having a one-time pad to control the rounding, you could rerun anything and get the same results. > > It is much faster to execute than most people think, because it uses > no more data, has almost exactly the same access patterns, and most > 'floating-point' programs use more actual CPU in general instruction > execution than floating-point, anyway. You're basically replacing each fp op with a function call, right? Doing it with inline code would be more or less equivalent to hacking the compiler to modify the rounding mode, while a function call that did a regular (default rounded) fp op followed by a random jitter added to/subtracted from the mantissa would not give the same effect as a random rounding direction, would it? Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | "Paul A. Clayton" <paaronclayton@gmail.com> |
|---|---|
| Date | 2011-10-17 14:47 -0700 |
| Message-ID | <e0940e4c-4348-469c-ac08-4c5b27e222bd@u13g2000vbx.googlegroups.com> |
| In reply to | #4229 |
On Oct 17, 3:30 pm, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: [snip] > Doing it with inline code would be more or less equivalent to hacking > the compiler to modify the rounding mode, while a function call that did > a regular (default rounded) fp op followed by a random jitter added > to/subtracted from the mantissa would not give the same effect as a > random rounding direction, would it? If the default rounding mode was round-to-zero (truncate), would not a random addition have the same effect as random rounding? (Probabilistic rounding would be more difficult, I suspect.) I wonder if FMADD would make such easier? (Perhaps using a select instruction to move either a zero or 2 raised to the 53rd power [least significant bit of mantissa, I think] for the multiplicand. I.e., Result_round_random = Result_truncated * Select + Result_truncated)
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-18 11:33 +0200 |
| Message-ID | <1otvm8-5gv2.ln1@ntp6.tmsw.no> |
| In reply to | #4231 |
Paul A. Clayton wrote: > On Oct 17, 3:30 pm, Terje Mathisen<"terje.mathisen at tmsw.no"> > wrote: > [snip] >> Doing it with inline code would be more or less equivalent to hacking >> the compiler to modify the rounding mode, while a function call that did >> a regular (default rounded) fp op followed by a random jitter added >> to/subtracted from the mantissa would not give the same effect as a >> random rounding direction, would it? > > If the default rounding mode was round-to-zero (truncate), > would not a random addition have the same effect as > random rounding? (Probabilistic rounding would be more > difficult, I suspect.) No, you would need to start with round-to-minus-infinity, otherwise positive and negative results would need different handling. > > I wonder if FMADD would make such easier? (Perhaps > using a select instruction to move either a zero or 2 > raised to the 53rd power [least significant bit of mantissa, > I think] for the multiplicand. I.e., Result_round_random = > Result_truncated * Select + Result_truncated) The problem is of course that the random added ULP bit needs to be applied directly to the mantissa, independently of the current exponent. Doing it with an extended precision multiplication might be easier... Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-18 10:21 +0100 |
| Message-ID | <j7jgfh$g79$1@gosset.csi.cam.ac.uk> |
| In reply to | #4246 |
In article <1otvm8-5gv2.ln1@ntp6.tmsw.no>, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: >Paul A. Clayton wrote: >> [snip] >>> Doing it with inline code would be more or less equivalent to hacking >>> the compiler to modify the rounding mode, while a function call that did >>> a regular (default rounded) fp op followed by a random jitter added >>> to/subtracted from the mantissa would not give the same effect as a >>> random rounding direction, would it? >> >> If the default rounding mode was round-to-zero (truncate), >> would not a random addition have the same effect as >> random rounding? (Probabilistic rounding would be more >> difficult, I suspect.) > >No, you would need to start with round-to-minus-infinity, otherwise >positive and negative results would need different handling. > >The problem is of course that the random added ULP bit needs to be >applied directly to the mantissa, independently of the current exponent. > >Doing it with an extended precision multiplication might be easier... Essential, I am afraid. But round-to-zero works perfectly well, and is quite likely to be easier. What you do is to copy the result, clear the 'used' part of the mantissa to zero, set the excess to random bits, and add to the original result. That works independently of architecture or ISA, though its efficiency is VERY variable. That's how I did it on the System/370. And, of course, you need to handle IEEE 754 infinities separately, or they will end up as NaNs - which is rarely a problem. Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-18 08:16 +0100 |
| Message-ID | <j7j94s$f3t$1@gosset.csi.cam.ac.uk> |
| In reply to | #4229 |
In article <hsfum8-v9t2.ln1@ntp6.tmsw.no>, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: > > [ True probabilistic rounding ] > >>> >>> On x86 you could use a random bitstream to toggle the rounding mode back >>> & forth between -Inf and +Inf before every fp operation, but the >>> overhead would be pretty bad. :-( >> >> That would be insane! > >Why? Because you need to inspect the excess before deciding which way to toggle! You can do the job much more easily by adding a U(0,ulp) random value and rounding down (or one of the same sign and towards zero). I don't know how fast toggling modes is at present, but I wouldn't expect it to be optimised. >Having a one-time pad to control the rounding, you could rerun anything >and get the same results. Or a decent pseudo-random number generator. Or a combination. >> It is much faster to execute than most people think, because it uses >> no more data, has almost exactly the same access patterns, and most >> 'floating-point' programs use more actual CPU in general instruction >> execution than floating-point, anyway. > >You're basically replacing each fp op with a function call, right? Logically, yes. But modern languages and compilers allow inlining. >Doing it with inline code would be more or less equivalent to hacking >the compiler to modify the rounding mode, Yes, precisely. > while a function call that did >a regular (default rounded) fp op followed by a random jitter added >to/subtracted from the mantissa would not give the same effect as a >random rounding direction, would it? No. There is no option but to use extended precision when simulating such rounding. Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-18 14:26 +0200 |
| Message-ID | <9s70n8-krv2.ln1@ntp6.tmsw.no> |
| In reply to | #4245 |
nmm1@cam.ac.uk wrote: > In article<hsfum8-v9t2.ln1@ntp6.tmsw.no>, > Terje Mathisen<"terje.mathisen at tmsw.no"> wrote: >> >> [ True probabilistic rounding ] >> >>>> >>>> On x86 you could use a random bitstream to toggle the rounding mode back >>>> & forth between -Inf and +Inf before every fp operation, but the >>>> overhead would be pretty bad. :-( >>> >>> That would be insane! >> >> Why? > > Because you need to inspect the excess before deciding which way > to toggle! You can do the job much more easily by adding a > U(0,ulp) random value and rounding down (or one of the same sign > and towards zero). I don't know how fast toggling modes is at > present, but I wouldn't expect it to be optimised. It is very slow, but generating properly scaled U(0-ulp) random adjustments won't be fast either. :-( >> while a function call that did >> a regular (default rounded) fp op followed by a random jitter added >> to/subtracted from the mantissa would not give the same effect as a >> random rounding direction, would it? > > No. There is no option but to use extended precision when simulating > such rounding. There's always more than one way to do it, in this case up to and including sw modification of the mantissa. I'm thinking that with rounding towards -Inf you only need to add either 0 or 1 ulp, with 50% for each, right? You could do this by taking the result, masking away all mantissa bits and saving it as temp1, then set the ulp bit and save as temp2. Finally you use your random generator/stream to select either temp1 or temp2 to be added and then subtract temp1 afterwards. Does that seem OK? Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-18 13:32 +0100 |
| Message-ID | <j7jrkp$iib$1@gosset.csi.cam.ac.uk> |
| In reply to | #4248 |
In article <9s70n8-krv2.ln1@ntp6.tmsw.no>, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: >> >> I don't know how fast toggling modes is at >> present, but I wouldn't expect it to be optimised. > >It is very slow, but generating properly scaled U(0-ulp) random >adjustments won't be fast either. :-( It's not that slow, if done competently. >There's always more than one way to do it, in this case up to and >including sw modification of the mantissa. That is very true. >I'm thinking that with rounding towards -Inf you only need to add either >0 or 1 ulp, with 50% for each, right? No. The probability of 1 ULP is the ignored excess, expressed in ULPs. >You could do this by taking the result, masking away all mantissa bits >and saving it as temp1, then set the ulp bit and save as temp2. Er, you have to ADD one ULP, not just force the bit! >Finally you use your random generator/stream to select either temp1 or >temp2 to be added and then subtract temp1 afterwards. > >Does that seem OK? With my corrections, yes. Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-18 17:47 +0200 |
| Message-ID | <6lj0n8-2903.ln1@ntp6.tmsw.no> |
| In reply to | #4249 |
nmm1@cam.ac.uk wrote: > In article<9s70n8-krv2.ln1@ntp6.tmsw.no>, > Terje Mathisen<"terje.mathisen at tmsw.no"> wrote: >> You could do this by taking the result, masking away all mantissa bits >> and saving it as temp1, then set the ulp bit and save as temp2. > > Er, you have to ADD one ULP, not just force the bit! > >> Finally you use your random generator/stream to select either temp1 or >> temp2 to be added and then subtract temp1 afterwards. >> >> Does that seem OK? > > With my corrections, yes. Please re-read: The forced ulp bit is randomly added or not. Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | nmm1@cam.ac.uk |
|---|---|
| Date | 2011-10-18 17:51 +0100 |
| Message-ID | <j7kapu$lb9$1@gosset.csi.cam.ac.uk> |
| In reply to | #4251 |
In article <6lj0n8-2903.ln1@ntp6.tmsw.no>, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote: > >>> You could do this by taking the result, masking away all mantissa bits >>> and saving it as temp1, then set the ulp bit and save as temp2. >> >> Er, you have to ADD one ULP, not just force the bit! >> >>> Finally you use your random generator/stream to select either temp1 or >>> temp2 to be added and then subtract temp1 afterwards. >>> >>> Does that seem OK? >> >> With my corrections, yes. > >Please re-read: > >The forced ulp bit is randomly added or not. Aargh! You were assuming a hidden bit and I was assume that there wasn't one! If you don't have one, there is no point in all that faffing around with temp1. Regards, Nick Maclaren.
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <"terje.mathisen at tmsw.no"> |
|---|---|
| Date | 2011-10-19 10:37 +0200 |
| Message-ID | <ase2n8-151.ln1@ntp6.tmsw.no> |
| In reply to | #4254 |
nmm1@cam.ac.uk wrote:
> In article<6lj0n8-2903.ln1@ntp6.tmsw.no>,
> Terje Mathisen<"terje.mathisen at tmsw.no"> wrote:
>>
>>>> You could do this by taking the result, masking away all mantissa bits
>>>> and saving it as temp1, then set the ulp bit and save as temp2.
>>>
>>> Er, you have to ADD one ULP, not just force the bit!
>>>
>>>> Finally you use your random generator/stream to select either temp1 or
>>>> temp2 to be added and then subtract temp1 afterwards.
>>>>
>>>> Does that seem OK?
>>>
>>> With my corrections, yes.
>>
>> Please re-read:
>>
>> The forced ulp bit is randomly added or not.
>
> Aargh! You were assuming a hidden bit and I was assume that there
> wasn't one! If you don't have one, there is no point in all that
> faffing around with temp1.
First of all: Both float and double does have hidden bits...
Second, I really don't get how you would like your code to work, here is
my version:
/* Rounding mode is set globally to -Inf */
double fmul(double a, double b)
{
double prod = a*b;
int64_t temp[2];
/* Keep sign and exponent, clear mantissa (except for hidden bit):
This is the same as the integer binary log
*/
temp[0] = (*(int64_t *) &prod) & 0xfff0000000000000;
/* Same as temp[0] but rounded up to the next ulp value */
temp[1] = temp[0] | 1;
/* Generate the randomly rounded product by adding either
temp[0] or the successor temp[1], then subtract temp[0]
*/
prod += ((double *) temp)[get_random_bit()];
prod -= ((double *) temp)[0];
return prod;
}
The timing is as follows:
Generate prod, store to local stack location: 5-6 cycles
Load as int64_t, mask and save: 3 cycles
Increment and save: 2 cycles
Load either first or second element as double: 1 cycle +
get_random_bit() overhead, but that can be overlapped with all the
previous calculations, so I'll assume it is free.
FADD + FSUB: 8 cycles
I.e the total is about 20 cycles, or 5 times more than a plain FMUL.
Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
Page 3 of 5 — ← Prev page 1 2 [3] 4 5 Next page →
Back to top | Article view | comp.arch
csiph-web