Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85617 > unrolled thread

Re: Floating point "g" format not stripping trailing zeros

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-12 16:46 -0700
    Re: Floating point "g" format not stripping trailing zeros Grant Edwards <invalid@invalid.invalid> - 2015-02-13 14:02 +0000
      Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 09:56 -0700
        Re: Floating point "g" format not stripping trailing zeros Grant Edwards <invalid@invalid.invalid> - 2015-02-13 20:33 +0000
          Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 14:01 -0700
          Re: Floating point "g" format not stripping trailing zeros Dave Angel <davea@davea.name> - 2015-02-13 16:15 -0500
            Re: Floating point "g" format not stripping trailing zeros Grant Edwards <invalid@invalid.invalid> - 2015-02-13 21:22 +0000
              Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 14:40 -0700
              Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 14:49 -0700

#85617 — Re: Floating point "g" format not stripping trailing zeros

On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic@gmail.com> wrote:
>> >>>>> from decimal import Decimal as D
>> >>>>> x = D(1)/D(999)
>> >>>>> '{:.15g}'.format(x)
>> >>
>> >> '0.00100100100100100'
> [...]
>> > I'd say it's a bug.  P is 15, you've got 17 digits after the decimal place
>> > and two of those are insignificant trailing zeros.
>>
>> Actually it's the float version that doesn't match the documentation.
>> In the decimal version, sure there are 17 digits after the decimal
>> place there, but the first two -- which are leading zeroes -- would
>> not normally be considered significant.
>
> {:.15g} is supposed to give 15 digits of precision, but with trailing
> zeros removed.

The doc says with "insignificant" trailing zeros removed, not all
trailing zeros.

> For example, '{:.15g}'.format(Decimal('0.5')) should
> yield '0.5', not '0.500000000000000' -- and, it indeed does.  It is
> only for some numbers that trailing zeros are not removed, which looks
> like a bug.  The behavior of floats matches both the documentation and
> other languages using the 'g' decimal format, such as C.

Ah, I see now what's going on here. With floats, there is really no
notion of significant digits. The values 0.5 and 0.50000 are
completely equivalent. With decimals, that's not exactly true; if you
give the decimal a trailing zero then you are telling it that the zero
is significant.

>>> Decimal('0.5')
Decimal('0.5')
>>> Decimal('0.50000')
Decimal('0.50000')
>>> Decimal('0.5').as_tuple()
DecimalTuple(sign=0, digits=(5,), exponent=-1)
>>> Decimal('0.50000').as_tuple()
DecimalTuple(sign=0, digits=(5, 0, 0, 0, 0), exponent=-5)

These are distinct; the decimal knows how many significant digits you
passed it. As a result, these are also distinct:

>>> '{:.4g}'.format(Decimal('0.5'))
'0.5'
>>> '{:.4g}'.format(Decimal('0.50000'))
'0.5000'

Now what happens in your original example of 1/999? The default
decimal context uses 28 digits of precision, so the result of that
calculation will have 28 significant digits in it.

>>> decimal.getcontext().prec
28
>>> Decimal(1) / Decimal(999)
Decimal('0.001001001001001001001001001001')

When you specify the a precision of 15 in your format string, you're
telling it to take the first 15 of those. It doesn't care that the
last couple of those are zeros, because as far as it's concerned,
those digits are significant.

[toc] | [next] | [standalone]

#85642

On 2015-02-12, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic@gmail.com> wrote:
>
>> {:.15g} is supposed to give 15 digits of precision, but with trailing
>> zeros removed.
>
> The doc says with "insignificant" trailing zeros removed, not all
> trailing zeros.

Can somebody explain the difference between "significant" and
"insignificant" tailing zeros to somebody who barely passed his single
numerical methods class?  [Though I have, on occasion, had to tinker
with the innards of SW floating point libraries and could fish a
hardcopy of IEEE-754 out of a filing cabinet if needed.]

-- 
Grant Edwards               grant.b.edwards        Yow! My polyvinyl cowboy
                                  at               wallet was made in Hong
                              gmail.com            Kong by Montgomery Clift!

[toc] | [prev] | [next] | [standalone]

#85652

On Fri, Feb 13, 2015 at 7:02 AM, Grant Edwards <invalid@invalid.invalid> wrote:
> On 2015-02-12, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic@gmail.com> wrote:
>>
>>> {:.15g} is supposed to give 15 digits of precision, but with trailing
>>> zeros removed.
>>
>> The doc says with "insignificant" trailing zeros removed, not all
>> trailing zeros.
>
> Can somebody explain the difference between "significant" and
> "insignificant" tailing zeros to somebody who barely passed his single
> numerical methods class?  [Though I have, on occasion, had to tinker
> with the innards of SW floating point libraries and could fish a
> hardcopy of IEEE-754 out of a filing cabinet if needed.]

Significant digits are within the precision of the calculation.
Writing 1.230 indicates that the fourth digit is known to be zero.
Writing 1.23 outside a context of exact calculation indicates that the
fourth digit is unknown due to insufficient precision.

[toc] | [prev] | [next] | [standalone]

#85657

On 2015-02-13, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, Feb 13, 2015 at 7:02 AM, Grant Edwards <invalid@invalid.invalid> wrote:
>> On 2015-02-12, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>>> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic@gmail.com> wrote:
>>>
>>>> {:.15g} is supposed to give 15 digits of precision, but with trailing
>>>> zeros removed.
>>>
>>> The doc says with "insignificant" trailing zeros removed, not all
>>> trailing zeros.
>>
>> Can somebody explain the difference between "significant" and
>> "insignificant" tailing zeros to somebody who barely passed his single
>> numerical methods class?  [Though I have, on occasion, had to tinker
>> with the innards of SW floating point libraries and could fish a
>> hardcopy of IEEE-754 out of a filing cabinet if needed.]
>
> Significant digits are within the precision of the calculation.
> Writing 1.230 indicates that the fourth digit is known to be zero.
> Writing 1.23 outside a context of exact calculation indicates that the
> fourth digit is unknown due to insufficient precision.

I knew that, but I was asking in the context of float/decimal's
formatting function.  I didn't realize that float and/or decimal had a
"significant digit" property, and therefore possess significant vs.
insignificant trailing zeros when represented in base-10.

-- 
Grant Edwards               grant.b.edwards        Yow! Mr and Mrs PED, can I
                                  at               borrow 26.7% of the RAYON
                              gmail.com            TEXTILE production of the
                                                   INDONESIAN archipelago?

[toc] | [prev] | [next] | [standalone]

#85658

On Fri, Feb 13, 2015 at 1:33 PM, Grant Edwards <invalid@invalid.invalid> wrote:
> On 2015-02-13, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>> On Fri, Feb 13, 2015 at 7:02 AM, Grant Edwards <invalid@invalid.invalid> wrote:
>>> On 2015-02-12, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>>>> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic@gmail.com> wrote:
>>>>
>>>>> {:.15g} is supposed to give 15 digits of precision, but with trailing
>>>>> zeros removed.
>>>>
>>>> The doc says with "insignificant" trailing zeros removed, not all
>>>> trailing zeros.
>>>
>>> Can somebody explain the difference between "significant" and
>>> "insignificant" tailing zeros to somebody who barely passed his single
>>> numerical methods class?  [Though I have, on occasion, had to tinker
>>> with the innards of SW floating point libraries and could fish a
>>> hardcopy of IEEE-754 out of a filing cabinet if needed.]
>>
>> Significant digits are within the precision of the calculation.
>> Writing 1.230 indicates that the fourth digit is known to be zero.
>> Writing 1.23 outside a context of exact calculation indicates that the
>> fourth digit is unknown due to insufficient precision.
>
> I knew that, but I was asking in the context of float/decimal's
> formatting function.  I didn't realize that float and/or decimal had a
> "significant digit" property, and therefore possess significant vs.
> insignificant trailing zeros when represented in base-10.

A Decimal object consists of a sign, a tuple of decimal digits, and an
exponent. If the digits are, e.g, (1, 2, 3, 0, 0, 0), then that would
be equal to the Decimal with the digits (1, 2, 3) and the same sign
and exponent, but the explicit presence of the trailing zeros
indicates their significance. If this doesn't answer your question,
then I'm not really sure what you're asking.

[toc] | [prev] | [next] | [standalone]

#85659

On 02/13/2015 03:33 PM, Grant Edwards wrote:
> On 2015-02-13, Ian Kelly <ian.g.kelly@gmail.com> wrote:

>> Significant digits are within the precision of the calculation.
>> Writing 1.230 indicates that the fourth digit is known to be zero.
>> Writing 1.23 outside a context of exact calculation indicates that the
>> fourth digit is unknown due to insufficient precision.
>
> I knew that, but I was asking in the context of float/decimal's
> formatting function.  I didn't realize that float and/or decimal had a
> "significant digit" property, and therefore possess significant vs.
> insignificant trailing zeros when represented in base-10.
>

Just jumping in here, and somebody please correct me if I mess it up.

Built-in binary floating point (float) has a fixed number of bits for 
mantissa, and separate bits for exponent and sign.  Because of those 
fixed number of bits, no assumption can be made as to how many of them 
are relevant.

On the other hand, the Decimal package has a way that the programmer can 
tell how many digits to use at each stage of the calculation.  So if the 
programmer bothered to set it to the correct precision, the print logic 
(could) use that to decide about trailing zeroes.  I have no idea 
whether it does, but this thread would seem to say it does.

I also have no definite opinion as to whether that's reasonable, or 
whether most calculations are done by setting digits to about twice 
what's needed, and rounding at the end.  I know I did exactly that when 
I wrote a variable length math package to double-check the accuracy of a 
fixed precision package I was developing, 40 years ago  (long before the 
IEEE-754 standard began meetings).  The fixed precision package was 
fast, and used lots of clever(?) shortcuts for speed.  The variable one 
was written very brute force, ran maybe 100 times slower (on another 
machine), but the results could be compared with automatic algorithms. 
For simple arithmetic, not too big a deal, but for transcendentals, the 
error analysis was very important.  For example, the fast algorithm was 
a custom chebyshev, while the reference implementation was Taylor series.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#85660

On 2015-02-13, Dave Angel <davea@davea.name> wrote:
> On 02/13/2015 03:33 PM, Grant Edwards wrote:
>> On 2015-02-13, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>>
>>> Significant digits are within the precision of the calculation.
>>> Writing 1.230 indicates that the fourth digit is known to be zero.
>>> Writing 1.23 outside a context of exact calculation indicates that the
>>> fourth digit is unknown due to insufficient precision.
>>
>> I knew that, but I was asking in the context of float/decimal's
>> formatting function.  I didn't realize that float and/or decimal had a
>> "significant digit" property, and therefore possess significant vs.
>> insignificant trailing zeros when represented in base-10.
>
> Just jumping in here, and somebody please correct me if I mess it up.
>
> Built-in binary floating point (float) has a fixed number of bits for 
> mantissa, and separate bits for exponent and sign.  Because of those 
> fixed number of bits, no assumption can be made as to how many of them 
> are relevant.

Right.

> On the other hand, the Decimal package has a way that the programmer
> can tell how many digits to use at each stage of the calculation.

That's what surpised me.  From TFM:

https://docs.python.org/2/library/decimal.html:

 * The decimal module incorporates a notion of significant places so that
   1.30 + 1.20 is 2.50. The trailing zero is kept to indicate
   significance. This is the customary presentation for monetary
   applications. For multiplication, the “schoolbook” approach uses
   all the figures in the multiplicands. For instance, 1.3 * 1.2 gives
   1.56 while 1.30 * 1.20 gives 1.5600.

> So if the programmer bothered to set it to the correct precision, the
> print logic (could) use that to decide about trailing zeroes.  I have
> no idea whether it does, but this thread would seem to say it does.

It seems to. Pretty cool.

-- 
Grant Edwards               grant.b.edwards        Yow! LOOK!!  Sullen
                                  at               American teens wearing
                              gmail.com            MADRAS shorts and "Flock of
                                                   Seagulls" HAIRCUTS!

[toc] | [prev] | [next] | [standalone]

#85661

On Fri, Feb 13, 2015 at 2:22 PM, Grant Edwards <invalid@invalid.invalid> wrote:
> On 2015-02-13, Dave Angel <davea@davea.name> wrote:
>> On the other hand, the Decimal package has a way that the programmer
>> can tell how many digits to use at each stage of the calculation.
>
> That's what surpised me.  From TFM:
>
> https://docs.python.org/2/library/decimal.html:
>
>  * The decimal module incorporates a notion of significant places so that
>    1.30 + 1.20 is 2.50. The trailing zero is kept to indicate
>    significance. This is the customary presentation for monetary
>    applications. For multiplication, the “schoolbook” approach uses
>    all the figures in the multiplicands. For instance, 1.3 * 1.2 gives
>    1.56 while 1.30 * 1.20 gives 1.5600.

Huh. That approach for multiplication is definitely not what I was
taught in school. I was taught that the number of significant digits
in the product is the lesser of the number of significant digits in
either of the measured multiplicands. So 1.30 * 1.20 would be 1.56,
while 1.3 * 1.2 would just be 1.6. Wikipedia appears to agree with me:

http://en.wikipedia.org/wiki/Significance_arithmetic#Multiplication_and_division_using_significance_arithmetic

Moreover:

>>> D('1.304') * D('1.204')
Decimal('1.570016')
>>> D('1.295') * D('1.195')
Decimal('1.547525')

So 1.30 * 1.20 could be written approximately as 1.56 ± 0.01. Given
that, I don't understand how the trailing zeros in 1.5600 could
possibly be considered significant.

[toc] | [prev] | [next] | [standalone]

#85662

On Fri, Feb 13, 2015 at 2:40 PM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> On Fri, Feb 13, 2015 at 2:22 PM, Grant Edwards <invalid@invalid.invalid> wrote:
>> On 2015-02-13, Dave Angel <davea@davea.name> wrote:
>>> On the other hand, the Decimal package has a way that the programmer
>>> can tell how many digits to use at each stage of the calculation.
>>
>> That's what surpised me.  From TFM:
>>
>> https://docs.python.org/2/library/decimal.html:
>>
>>  * The decimal module incorporates a notion of significant places so that
>>    1.30 + 1.20 is 2.50. The trailing zero is kept to indicate
>>    significance. This is the customary presentation for monetary
>>    applications. For multiplication, the “schoolbook” approach uses
>>    all the figures in the multiplicands. For instance, 1.3 * 1.2 gives
>>    1.56 while 1.30 * 1.20 gives 1.5600.
>
> Huh. That approach for multiplication is definitely not what I was
> taught in school. I was taught that the number of significant digits
> in the product is the lesser of the number of significant digits in
> either of the measured multiplicands. So 1.30 * 1.20 would be 1.56,
> while 1.3 * 1.2 would just be 1.6. Wikipedia appears to agree with me:
>
> http://en.wikipedia.org/wiki/Significance_arithmetic#Multiplication_and_division_using_significance_arithmetic
>
> Moreover:
>
>>>> D('1.304') * D('1.204')
> Decimal('1.570016')
>>>> D('1.295') * D('1.195')
> Decimal('1.547525')
>
> So 1.30 * 1.20 could be written approximately as 1.56 ± 0.01. Given
> that, I don't understand how the trailing zeros in 1.5600 could
> possibly be considered significant.

I guess the point here is that the paragraph isn't really talking
about significance arithmetic; it's explaining how it decides how many
digits to keep in the result. It may be fine for 1.30 * 1.20 to return
1.56, but it would be very confusing if 1.35 * 1.25 returned 1.69
instead of 1.6875. The wording of the paragraph seems misleading,
though.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web