Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85665 > unrolled thread

Re: Floating point "g" format not stripping trailing zeros

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Floating point "g" format not stripping trailing zeros Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-13 23:05 +0000
    Re: Floating point "g" format not stripping trailing zeros Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-02-14 16:22 +1300

#85665 — Re: Floating point "g" format not stripping trailing zeros

On 12/02/2015 23:46, Ian Kelly wrote:
> On Thu, Feb 12, 2015 at 1:23 PM, Hrvoje Nikšić <hniksic@gmail.com> wrote:
>>>>>>>> from decimal import Decimal as D
>>>>>>>> x = D(1)/D(999)
>>>>>>>> '{:.15g}'.format(x)
>>>>>
>>>>> '0.00100100100100100'
>> [...]
>>>> I'd say it's a bug.  P is 15, you've got 17 digits after the decimal place
>>>> and two of those are insignificant trailing zeros.
>>>
>>> Actually it's the float version that doesn't match the documentation.
>>> In the decimal version, sure there are 17 digits after the decimal
>>> place there, but the first two -- which are leading zeroes -- would
>>> not normally be considered significant.
>>
>> {:.15g} is supposed to give 15 digits of precision, but with trailing
>> zeros removed.
>
> The doc says with "insignificant" trailing zeros removed, not all
> trailing zeros.
>
>> For example, '{:.15g}'.format(Decimal('0.5')) should
>> yield '0.5', not '0.500000000000000' -- and, it indeed does.  It is
>> only for some numbers that trailing zeros are not removed, which looks
>> like a bug.  The behavior of floats matches both the documentation and
>> other languages using the 'g' decimal format, such as C.
>
> Ah, I see now what's going on here. With floats, there is really no
> notion of significant digits. The values 0.5 and 0.50000 are
> completely equivalent. With decimals, that's not exactly true; if you
> give the decimal a trailing zero then you are telling it that the zero
> is significant.
>
>>>> Decimal('0.5')
> Decimal('0.5')
>>>> Decimal('0.50000')
> Decimal('0.50000')
>>>> Decimal('0.5').as_tuple()
> DecimalTuple(sign=0, digits=(5,), exponent=-1)
>>>> Decimal('0.50000').as_tuple()
> DecimalTuple(sign=0, digits=(5, 0, 0, 0, 0), exponent=-5)
>
> These are distinct; the decimal knows how many significant digits you
> passed it. As a result, these are also distinct:
>
>>>> '{:.4g}'.format(Decimal('0.5'))
> '0.5'
>>>> '{:.4g}'.format(Decimal('0.50000'))
> '0.5000'
>
> Now what happens in your original example of 1/999? The default
> decimal context uses 28 digits of precision, so the result of that
> calculation will have 28 significant digits in it.
>
>>>> decimal.getcontext().prec
> 28
>>>> Decimal(1) / Decimal(999)
> Decimal('0.001001001001001001001001001001')
>
> When you specify the a precision of 15 in your format string, you're
> telling it to take the first 15 of those. It doesn't care that the
> last couple of those are zeros, because as far as it's concerned,
> those digits are significant.
>

I still think it's a bug as the 'p' being referred to in the OP's 
original message is "The precision is a decimal number indicating how 
many digits should be displayed after the decimal point for a floating 
point value formatted with 'f' and 'F', or before and after the decimal 
point for a floating point value formatted with 'g' or 'G'".  In other 
words is has nothing to do with the precision of the underlying number.


-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [next] | [standalone]

#85670

Mark Lawrence wrote:
> I still think it's a bug as the 'p' being referred to in the OP's 
> original message is "The precision is a decimal number indicating how 
> many digits should be displayed after the decimal point for a floating 
> point value formatted with 'f' and 'F', or before and after the decimal 
> point for a floating point value formatted with 'g' or 'G'".  In other 
> words is has nothing to do with the precision of the underlying number.

But it does say that "insignificant trailing
zeros are removed", not just "trailing zeros
are removed".

However, it's possible that "insignificant"
was just intended to mean "after the decimal
point". The specification is a bit vague.

-- 
Greg

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web