Groups | Search | Server Info | Login | Register

Groups > comp.lang.apl > #3741

Re: Nars floor of a big float

Show all headers | View raw

>>>>> "R" == Rosario19  <Ros@invalid.invalid> writes:

R> In Nars is right that the floor of log10 of 9999999999999.9v with
R> fpc=128 result 13?

R>       30??10 ? 9999999999999.9v
R>  13.000000000000000000000000000000

I don't know nars, you are probably being hit by floating point
precision issues.

with arbitrary precision, the log10 of 9999999999999.9 is:

 +12.999999999999995657055180967460008764615648097801793308999084...

Using ieee 64bit floats, you are likely to get 12.999999999999995,
but with 32bit floats you'll get 13.0.

Even with 64bit floats, rounding issues along the way could lead to a
result of 13 rather than 12.99999999...

Hense the difference in floor()'s output.

-JimC
-- 
James Cloos <cloos@jhcloos.com>
            OpenPGP: https://jhcloos.com/0x997A9F17ED7DAEA6.asc

Back to comp.lang.apl | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Nars floor of a big float Rosario19 <Ros@invalid.invalid> - 2024-02-22 11:34 +0100
  Re: Nars floor of a big float James Cloos <cloos@jhcloos.com> - 2024-02-22 10:44 -0500
    Re: Nars floor of a big float Rosario19 <Ros@invalid.invalid> - 2024-02-22 22:41 +0100
  Re: Nars floor of a big float Bob Smith <bsmith@sudleydeplacespam.com> - 2024-02-22 19:41 -0500

csiph-web