Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85659

Re: Floating point "g" format not stripping trailing zeros

Show key headers only | View raw

On 02/13/2015 03:33 PM, Grant Edwards wrote:
> On 2015-02-13, Ian Kelly <ian.g.kelly@gmail.com> wrote:

>> Significant digits are within the precision of the calculation.
>> Writing 1.230 indicates that the fourth digit is known to be zero.
>> Writing 1.23 outside a context of exact calculation indicates that the
>> fourth digit is unknown due to insufficient precision.
>
> I knew that, but I was asking in the context of float/decimal's
> formatting function.  I didn't realize that float and/or decimal had a
> "significant digit" property, and therefore possess significant vs.
> insignificant trailing zeros when represented in base-10.
>

Just jumping in here, and somebody please correct me if I mess it up.

Built-in binary floating point (float) has a fixed number of bits for 
mantissa, and separate bits for exponent and sign.  Because of those 
fixed number of bits, no assumption can be made as to how many of them 
are relevant.

On the other hand, the Decimal package has a way that the programmer can 
tell how many digits to use at each stage of the calculation.  So if the 
programmer bothered to set it to the correct precision, the print logic 
(could) use that to decide about trailing zeroes.  I have no idea 
whether it does, but this thread would seem to say it does.

I also have no definite opinion as to whether that's reasonable, or 
whether most calculations are done by setting digits to about twice 
what's needed, and rounding at the end.  I know I did exactly that when 
I wrote a variable length math package to double-check the accuracy of a 
fixed precision package I was developing, 40 years ago  (long before the 
IEEE-754 standard began meetings).  The fixed precision package was 
fast, and used lots of clever(?) shortcuts for speed.  The variable one 
was written very brute force, ran maybe 100 times slower (on another 
machine), but the results could be compared with automatic algorithms. 
For simple arithmetic, not too big a deal, but for transcendentals, the 
error analysis was very important.  For example, the fast algorithm was 
a custom chebyshev, while the reference implementation was Taylor series.

-- 
DaveA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-12 16:46 -0700
  Re: Floating point "g" format not stripping trailing zeros Grant Edwards <invalid@invalid.invalid> - 2015-02-13 14:02 +0000
    Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 09:56 -0700
      Re: Floating point "g" format not stripping trailing zeros Grant Edwards <invalid@invalid.invalid> - 2015-02-13 20:33 +0000
        Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 14:01 -0700
        Re: Floating point "g" format not stripping trailing zeros Dave Angel <davea@davea.name> - 2015-02-13 16:15 -0500
          Re: Floating point "g" format not stripping trailing zeros Grant Edwards <invalid@invalid.invalid> - 2015-02-13 21:22 +0000
            Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 14:40 -0700
            Re: Floating point "g" format not stripping trailing zeros Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-13 14:49 -0700

csiph-web