Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.development.apps > #445

Re: cast double to short problem

Show key headers only | View raw

On 2012-03-06, Bill M <wpmccormick@just_about_everywhere.com> wrote:

> I'm having a loss of precision problem when casting from a double to
> a short.
>
> In the following, for dval = 0.3 ...

0.3 can't be represented exactly in base-2 floating point, so you're
getting a number that's actually something like 0.29999999999999

> short sval = (short)(dval * 10);
>
> ... sval = 2.
>
> But if I convert it like this ...
>
> double nval = dval * 10;
> short sval = (short)nval;
>
> ... then sval = 3

OK.

> as I would expect.

Ah, there's your problem: expecting to get 3.  You should expect to
get either 2 or 3.  And you do. :)

> Can some gcc expert tell me what is happening here?

The main problem is that you have invalid expectations about how
floating point operations work.  [Trust me, _everybody_ starts out
that way when they first try to write programs that use floating point
in general and binary FP in particular.]

The question about why those two particular examples produce those
exact results is probably answered by how the compiler has chosen to
optimize things (pre-computing something at compile time or not, what
order operations occur, etc.).

That might be interesting if you're into compiler internals, but if
you're interested in writing programs with floating point that work,
it's pretty much moot.

> I'm not using any special options to gcc, but maybe that could be an
> issue?

Nope.  That's just how floating point works.

What exactly are you trying to accomplish with the above code?

  http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
  http://en.wikipedia.org/wiki/Floating_point
  http://floating-point-gui.de

You can't go wrong with the assumption that _all_ floating point
values, constants, and calculations are inexact.  If you're
calculating something that you think is going to end up being
_exactly_ 3.0, you'd better handle the case when it's 2.99999999999
and the case where it's 3.000000000001.

There _are_ times where a floating point value will come out exact,
but _in_practice_ you can't when that will happen except for a few
cases.  For example floating point representations of integers are
exact when they fall between a pair of fairly large postitive and
negative bounds, and results of operations on integer-valued floats
will be exact as long as no intermediate values exceed the previously
mentioned bounds.  In the bad old days you didn't even get to know
what those bounds were. These days everybody has standardzied on the
IEEE-754 standard for FP (except for a few DSPs and the like), so you
do have a pretty good idea what those bounds are.  But you're probably
better off if you pretend that you don't.

-- 
Grant Edwards               grant.b.edwards        Yow! I am having FUN...
                                  at               I wonder if it's NET FUN or
                              gmail.com            GROSS FUN?

Back to comp.os.linux.development.apps | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

cast double to short problem Bill M <wpmccormick@just_about_everywhere.com> - 2012-03-06 16:52 -0600
  Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-06 23:21 +0000
    Re: cast double to short problem Bill M <wpmccormick@just_about_everywhere.com> - 2012-03-06 17:36 -0600
      Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-07 00:30 +0000
        Re: cast double to short problem Noob <root@127.0.0.1> - 2012-03-07 11:17 +0100
          Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-07 15:23 +0000
    Re: cast double to short problem "Ersek, Laszlo" <lacos@caesar.elte.hu> - 2012-03-07 19:33 +0100
      Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-07 20:13 +0000
        Re: cast double to short problem Bill M <wpmccormick@just_about_everywhere.com> - 2012-03-08 11:45 -0600

csiph-web