Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.glorb.com!news-out.readnews.com!transit3.readnews.com!panix!not-for-mail From: Grant Edwards Newsgroups: comp.os.linux.development.apps Subject: Re: cast double to short problem Date: Tue, 6 Mar 2012 23:21:04 +0000 (UTC) Organization: PANIX Public Access Internet and UNIX, NYC Lines: 77 Message-ID: References: NNTP-Posting-Host: dsl.comtrol.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Trace: reader1.panix.com 1331076064 15680 64.122.56.22 (6 Mar 2012 23:21:04 GMT) X-Complaints-To: abuse@panix.com NNTP-Posting-Date: Tue, 6 Mar 2012 23:21:04 +0000 (UTC) User-Agent: slrn/pre0.9.9-102 (Linux) Xref: csiph.com comp.os.linux.development.apps:445 On 2012-03-06, Bill M wrote: > I'm having a loss of precision problem when casting from a double to > a short. > > In the following, for dval = 0.3 ... 0.3 can't be represented exactly in base-2 floating point, so you're getting a number that's actually something like 0.29999999999999 > short sval = (short)(dval * 10); > > ... sval = 2. > > But if I convert it like this ... > > double nval = dval * 10; > short sval = (short)nval; > > ... then sval = 3 OK. > as I would expect. Ah, there's your problem: expecting to get 3. You should expect to get either 2 or 3. And you do. :) > Can some gcc expert tell me what is happening here? The main problem is that you have invalid expectations about how floating point operations work. [Trust me, _everybody_ starts out that way when they first try to write programs that use floating point in general and binary FP in particular.] The question about why those two particular examples produce those exact results is probably answered by how the compiler has chosen to optimize things (pre-computing something at compile time or not, what order operations occur, etc.). That might be interesting if you're into compiler internals, but if you're interested in writing programs with floating point that work, it's pretty much moot. > I'm not using any special options to gcc, but maybe that could be an > issue? Nope. That's just how floating point works. What exactly are you trying to accomplish with the above code? http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html http://en.wikipedia.org/wiki/Floating_point http://floating-point-gui.de You can't go wrong with the assumption that _all_ floating point values, constants, and calculations are inexact. If you're calculating something that you think is going to end up being _exactly_ 3.0, you'd better handle the case when it's 2.99999999999 and the case where it's 3.000000000001. There _are_ times where a floating point value will come out exact, but _in_practice_ you can't when that will happen except for a few cases. For example floating point representations of integers are exact when they fall between a pair of fairly large postitive and negative bounds, and results of operations on integer-valued floats will be exact as long as no intermediate values exceed the previously mentioned bounds. In the bad old days you didn't even get to know what those bounds were. These days everybody has standardzied on the IEEE-754 standard for FP (except for a few DSPs and the like), so you do have a pretty good idea what those bounds are. But you're probably better off if you pretend that you don't. -- Grant Edwards grant.b.edwards Yow! I am having FUN... at I wonder if it's NET FUN or gmail.com GROSS FUN?