Groups > comp.os.linux.development.apps > #450

Re: cast double to short problem

Path	csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!news.szaf.org!eternal-september.org!feeder.eternal-september.org!mx04.eternal-september.org!.POSTED!not-for-mail
From	"Ersek, Laszlo" <lacos@caesar.elte.hu>
Newsgroups	comp.os.linux.development.apps
Subject	Re: cast double to short problem
Date	Wed, 7 Mar 2012 19:33:45 +0100
Organization	A noiseless patient Spider
Lines	42
Message-ID	<alpine.DEB.2.00.1203071908210.1839@login01.caesar.elte.hu> (permalink)
References	<jj64g2$fb8$1@dont-email.me> <jj6650$fa0$1@reader1.panix.com>
Mime-Version	1.0
Content-Type	TEXT/PLAIN; charset=US-ASCII; format=flowed
Injection-Info	mx04.eternal-september.org; posting-host="XWe3szWNBO98ONSQdCfQZw"; logging-data="22476"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+2fk+x0jRAmJUqGLSWTy2I"
User-Agent	Alpine 2.00 (DEB 1167 2008-08-23)
In-Reply-To	<jj6650$fa0$1@reader1.panix.com>
Cancel-Lock	sha1:wbT6UjKSyH4+bJBPQC4P1Iqf1xk=
Xref	csiph.com comp.os.linux.development.apps:450

Show key headers only | View raw

On Tue, 6 Mar 2012, Grant Edwards wrote:

> On 2012-03-06, Bill M <wpmccormick@just_about_everywhere.com> wrote:
>
>> I'm having a loss of precision problem when casting from a double to
>> a short.
>>
>> In the following, for dval = 0.3 ...
>
> 0.3 can't be represented exactly in base-2 floating point, so you're
> getting a number that's actually something like 0.29999999999999
>
>> short sval = (short)(dval * 10);
>>
>> ... sval = 2.
>>
>> But if I convert it like this ...
>>
>> double nval = dval * 10;
>> short sval = (short)nval;
>>
>> ... then sval = 3
>
> OK.
>
>> as I would expect.
>
> Ah, there's your problem: expecting to get 3.  You should expect to
> get either 2 or 3.  And you do. :)

He should get the same result both times.

- The floating point environment is the same.
- dval starts out the same (as in "object representation").
- 10 (int) is converted to double the same way (with exact value).
- Both double multiplications happen with the same operands (as in 
"object representation").
- Initialization of (~ simple assignment to) a double does not change the 
double product (as in "object representation").
- Conversion to short is the same (truncated toward zero).

Laszlo

Back to comp.os.linux.development.apps | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

cast double to short problem Bill M <wpmccormick@just_about_everywhere.com> - 2012-03-06 16:52 -0600
  Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-06 23:21 +0000
    Re: cast double to short problem Bill M <wpmccormick@just_about_everywhere.com> - 2012-03-06 17:36 -0600
      Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-07 00:30 +0000
        Re: cast double to short problem Noob <root@127.0.0.1> - 2012-03-07 11:17 +0100
          Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-07 15:23 +0000
    Re: cast double to short problem "Ersek, Laszlo" <lacos@caesar.elte.hu> - 2012-03-07 19:33 +0100
      Re: cast double to short problem Grant Edwards <invalid@invalid.invalid> - 2012-03-07 20:13 +0000
        Re: cast double to short problem Bill M <wpmccormick@just_about_everywhere.com> - 2012-03-08 11:45 -0600

csiph-web