Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #20707

Re: Floating Point Representation (Question)

Show all headers | View raw

On 12/26/2012 8:22 AM, Stefan Ram wrote:
>    AFAIK 0.1 in hex is 0x1.(9)ap-4, where »(9)« means an
>    infinite sequence of »9«. However, Java only stores a
>    finite number of 9s:
>
> printf( "%a%n", 0.1 )
> 0x1.999999999999ap-4
>
>    . So, since something /positive/ is missing, Javas
>    representation of 0.1 should be /smaller/ than 0.1, but

The conversion of literals to double uses round-to-nearest, with
round-to-even as tie breaker.

The BigDecimal representations of the two exact double values bracketing
decimal 0.1 are:

0.1000000000000000055511151231257827021181583404541015625
0.09999999999999999167332731531132594682276248931884765625

The first is too big by about 5e-18. The second is too small by about
8e-18. It would be a serious bug in the string to double conversion for
the literal if Java picked the second.

The "a" in "0x1.999999999999ap-4" is the last digit of the hex fraction.

>   Removing more 9s makes the value even larger!
>
> println( new java.math.BigDecimal( 0x1.999999999999ap-4 ));
> 0.1000000000000000055511151231257827021181583404541015625

Moving that "a" further to the left increases its significance, and
therefore the overall value.

Patricia

Back to comp.lang.java.programmer | Previous | Next | Find similar | Unroll thread

Thread

Re: Floating Point Representation (Question) Patricia Shanahan <pats@acm.org> - 2012-12-26 09:40 -0800

csiph-web