Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #6967 > unrolled thread

Re: float("nan") in set or as key

Back to article view | Back to comp.lang.python

Contents

  Re: float("nan") in set or as key Carl Banks <pavlovevidence@gmail.com> - 2011-06-03 13:27 -0700
    Re: float("nan") in set or as key Chris Angelico <rosuav@gmail.com> - 2011-06-04 06:35 +1000
      Re: float("nan") in set or as key Chris Torek <nospam@torek.net> - 2011-06-05 22:54 +0000
        Re: float("nan") in set or as key Chris Angelico <rosuav@gmail.com> - 2011-06-06 09:13 +1000
          Re: float("nan") in set or as key Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-06 01:21 +0000
            Re: float("nan") in set or as key Chris Torek <nospam@torek.net> - 2011-06-06 01:56 +0000
            Re: float("nan") in set or as key Chris Angelico <rosuav@gmail.com> - 2011-06-06 14:11 +1000
              Re: float("nan") in set or as key Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-06 04:59 +0000
                Re: float("nan") in set or as key Chris Angelico <rosuav@gmail.com> - 2011-06-06 15:10 +1000
    Re: float("nan") in set or as key Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-04 04:54 +0000
      Re: float("nan") in set or as key Ethan Furman <ethan@stoneleaf.us> - 2011-06-03 23:04 -0700
        Re: float("nan") in set or as key Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-04 09:35 +0000
          Re: float("nan") in set or as key Ben Finney <ben+python@benfinney.id.au> - 2011-06-04 20:20 +1000
          Re: float("nan") in set or as key Ethan Furman <ethan@stoneleaf.us> - 2011-06-04 14:28 -0700
          Re: float("nan") in set or as key Robert Kern <robert.kern@gmail.com> - 2011-06-04 16:49 -0500
            Re: float("nan") in set or as key Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-06-05 02:03 +0000
              Re: float("nan") in set or as key Robert Kern <robert.kern@gmail.com> - 2011-06-05 14:44 -0500

#6967 — Re: float("nan") in set or as key

On Wednesday, June 1, 2011 5:53:26 PM UTC-7, Steven D'Aprano wrote:
> On Tue, 31 May 2011 19:45:01 -0700, Carl Banks wrote:
> 
> > On Sunday, May 29, 2011 8:59:49 PM UTC-7, Steven D'Aprano wrote:
> >> On Sun, 29 May 2011 17:55:22 -0700, Carl Banks wrote:
> >> 
> >> > Floating point arithmetic evolved more or less on languages like
> >> > Fortran where things like exceptions were unheard of,
> >> 
> >> I'm afraid that you are completely mistaken.
> >> 
> >> Fortran IV had support for floating point traps, which are "things like
> >> exceptions". That's as far back as 1966. I'd be shocked if earlier
> >> Fortrans didn't also have support for traps.
> >> 
> >> http://www.bitsavers.org/pdf/ibm/7040/C28-6806-1_7040ftnMathSubrs.pdf
> > 
> > Fine, it wasn't "unheard of".  I'm pretty sure the existence of a few
> > high end compiler/hardware combinations that supported traps doesn't
> > invalidate my basic point.
> 
> On the contrary, it blows it out of the water and stomps its corpse into 
> a stain on the ground.

Really?  I am claiming that, even if everyone and their mother thought exceptions were the best thing ever, NaN would have been added to IEEE anyway because most hardware didn't support exceptions.  Therefore the fact that NaN is in IEEE is not any evidence that NaN is a good idea.

You are saying that the existence of one early system that supported exceptions not merely argument against that claim, but blows it out of the water?  Your logic sucks then.

You want to go off arguing that there were good reasons aside from backwards compatibility they added NaN, be my guest.  Just don't go around saying, "Its in IEEE there 4 its a good idear LOL".  Lots of standards have all kinds of bad ideas in them for the sake of backwards compatibility, and when someone goes around claiming that something is a good idea simply because some standard includes it, it is the first sign that they're clueless about what standarization actually is.


> NANs weren't invented as an alternative for 
> exceptions, but because exceptions are usually the WRONG THING in serious 
> numeric work.
> 
> Note the "usually". For those times where you do want to interrupt a 
> calculation just because of an invalid operation, the standard allows you 
> to set a trap and raise an exception.

I don't want to get into an argument over best practices in serious numerical programming, so let's just agree with this point for argument's sake.

Here's the problem: Python is not for serious numerical programming.  Yeah, it's a really good language for calling other languages to do numerical programming, but it's not good for doing serious numerical programming itself.  Anyone with some theoretical problem where NaN is a good idea should already be using modules or separate programs written in C or Fortran.

Casual and lightweight numerical work (which Python is good at) is not a wholly separate problem domain where the typical rules ("Errors should never pass silently") should be swept aside.


[snip]
> You'll note that, out of the box, numpy generates NANs:
> 
> >>> import numpy
> >>> x = numpy.array([float(x) for x in range(5)])
> >>> x/x
> Warning: invalid value encountered in divide
> array([ nan,   1.,   1.,   1.,   1.])

Steven, seriously I don't know what's going through your head.  I'm saying strict adherence to IEEE is not the best idea, and you cite the fact that a library tries to strictly adhere to IEEE as evidence that strictly adhering to IEEE is a good idea.  Beg the question much?


> The IEEE standard supports both use-cases: those who want exceptions to 
> bail out early, and those who want NANs so the calculation can continue. 
> This is a good thing. Failing to support the standard is a bad thing. 
> Despite your opinion, it is anything but obsolete.

There are all kinds of good reasons to go against standards.  "Failing to support the standard is a bad thing" are the words of a fool.  A wise person considers the cost of breaking the standard versus the benefit got.

It's clear tha IEEE's NaN handling is woefully out of place in the philosophy of Python, which tries to be newbie friendly and robust to errors; and Python has no real business trying to perform serious numerical work where (ostensibly) NaNs might find a use.  Therefore, the cost of breaking standard is small, but the benefit significant, so Python would be very wise to break with IEEE in the handling of NaNs.


Carl Banks

[toc] | [next] | [standalone]

#6968

On Sat, Jun 4, 2011 at 6:27 AM, Carl Banks <pavlovevidence@gmail.com> wrote:
> Really?  I am claiming that, even if everyone and their mother thought exceptions were the best thing ever, NaN would have been added to IEEE anyway because most hardware didn't support exceptions.  Therefore the fact that NaN is in IEEE is not any evidence that NaN is a good idea.

Uhh, noob question here. I'm way out of my depth with hardware floating point.

Isn't a signaling nan basically the same as an exception? Which would
imply that the hardware did support exceptions (if it did indeed
support IEEE floating point, which specifies signalling nan)?

Chris Angelico

[toc] | [prev] | [next] | [standalone]

#7057

In article <mailman.2438.1307133316.9059.python-list@python.org>
Chris Angelico  <rosuav@gmail.com> wrote:
>Uhh, noob question here. I'm way out of my depth with hardware
>floating point.
>
>Isn't a signaling nan basically the same as an exception?

Not exactly, but one could think of them as "very similar".

Elsethread, someone brought up the key distinction, which is
that in hardware that implements IEEE arithmetic, you have two
possibilities at pretty much all times:

 - op(args) causes an exception (and therefore does not deliver
   a result), or
 - op(args) delivers a result that may indicate "exception-like
   lack of result".

In both cases, a set of "accrued exceptions" flags accumulates the
new exception, and a set of "most recent exceptions" flags tells
you about the current exception.  A set of "exception enable"
flags -- which has all the same elements as "current" and
"accrued" -- tells the hardware which "exceptional results"
should trap.

A number is "NaN" if it has all-1-bits for its exponent and at
least one nonzero bit in its mantissa.  (All-1s exponent, all-0s
mantissa represents Infinity, of the sign specified by the sign
bit.)  For IEEE double precision floating point, there are 52
mantissa bits, so there are (2^52-1) different NaN bit patterns.
One of those 52 bits is the "please signal on use" bit.

A signalling NaN traps at (more or less -- details vary depending
on FPU architecture) load time.  However, there must necessarily
(for OS and thread-library level context switching) be a method
of saving the FPU state without causing an exception when loading
a NaN bit pattern, even if the NaN has the "signal" bit set.

>Which would imply that the hardware did support exceptions (if it
>did indeed support IEEE floating point, which specifies signalling nan)?

The actual hardware implementations (of which there are many) handle
the niggling details differently.  Some CPUs do not implement
Infinity and NaN in hardware at all, delivering a trap to the OS
on every use of an Inf-or-NaN bit pattern.  The OS then has to
emulate what the hardware specification says (if anything), and
make it look as though the hardware did the job.  Sometimes denorms
are also done in software.

Some implementations handle everything directly in hardware, and
some of those get it wrong. :-)  Often the OS has to fix up some
special case -- for instance, the hardware might trap on every NaN
and make software decide whether the bit pattern was a signalling
NaN, and if so, whether user code should receive an exception.

As I think John Nagle pointed out earlier, sometimes the hardware
does "support" exceptions, but rather loosely, where the hardware
delivers a morass of internal state and a vague indication that
one or more exceptions happened "somewhere near address <A>",
leaving a huge pile of work for software.

In Python, the decimal module gets everything either right or
close-to-right per the (draft? final? I have not kept up with
decimal FP standards) standard.  Internal Python floating point,
not quite so much.
-- 
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W)  +1 801 277 2603
email: gmail (figure it out)      http://web.torek.net/torek/index.html

[toc] | [prev] | [next] | [standalone]

#7059

On Mon, Jun 6, 2011 at 8:54 AM, Chris Torek <nospam@torek.net> wrote:
> A signalling NaN traps at (more or less -- details vary depending
> on FPU architecture) load time.

Load. By this you mean the operation of taking a bit-pattern in RAM
and putting it into a register? So, you can calculate 0/0, get a
signalling NaN, and then save that into a memory variable, all without
it trapping; and then it traps when you next perform an operation on
that number?

Apologies, this is getting quite off-topic and away from Python.

Chris Angelico

[toc] | [prev] | [next] | [standalone]

#7065

On Mon, 06 Jun 2011 09:13:25 +1000, Chris Angelico wrote:

> On Mon, Jun 6, 2011 at 8:54 AM, Chris Torek <nospam@torek.net> wrote:
>> A signalling NaN traps at (more or less -- details vary depending on
>> FPU architecture) load time.
> 
> Load. By this you mean the operation of taking a bit-pattern in RAM and
> putting it into a register? So, you can calculate 0/0, get a signalling
> NaN, and then save that into a memory variable, all without it trapping;
> and then it traps when you next perform an operation on that number?

The intended behaviour is operations on "quiet NANs" should return NANs, 
but operations on "signalling NANs" should cause a trap, which can either 
be ignored, and converted into a quiet NAN, or treated as an exception.

E.g. in Decimal:


>>> import decimal
>>> qnan = decimal.Decimal('nan')  # quiet NAN
>>> snan = decimal.Decimal('snan')  # signalling NAN
>>> 1 + qnan
Decimal('NaN')
>>> 1 + snan
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.1/decimal.py", line 1108, in __add__
    ans = self._check_nans(other, context)
  File "/usr/local/lib/python3.1/decimal.py", line 746, in _check_nans
    self)
  File "/usr/local/lib/python3.1/decimal.py", line 3812, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: sNaN



> Apologies, this is getting quite off-topic and away from Python.

Not at all. I think this is a big myth, that the IEEE-754 standard is 
irrelevant for high-level programming languages. It's not.

The state of the art of floating point is in a poor state. Not anywhere 
near as poor as the bad old days before there was *any* standardization 
at all, things were terrible back then, but ignoring the hard-earned 
lessons of those who lived through the days before the standard is a 
mistake. IEEE-754 is not just for hardware, particularly since now the 
vast majority of machines run hardware which almost completely conforms 
to IEEE-754. The bottleneck now is not hardware, but languages that don't 
treat floating point maths correctly.

http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf


(The article is seven years old now, but as far as I know, the criticisms 
still apply.)


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7066

>> On Mon, Jun 6, 2011 at 8:54 AM, Chris Torek <nospam@torek.net> wrote:
>>> A signalling NaN traps at (more or less -- details vary depending on
>>> FPU architecture) load time.

>On Mon, 06 Jun 2011 09:13:25 +1000, Chris Angelico wrote:
>> Load. By this you mean the operation of taking a bit-pattern in RAM and
>> putting it into a register? So, you can calculate 0/0, get a signalling
>> NaN, and then save that into a memory variable, all without it trapping;
>> and then it traps when you next perform an operation on that number?

I mean, if you think of the FPU as working (in principle) with
either just one or two registers and a load/store architecture, or
a tiny little FPU-stack (the latter is in fact the case for Intel
FPUs), with no optimization, you get a trap when you attempted to
load-up the sNaN value in order to do some operation on it.  For
instance, if x is an sNaN, "y = x + 1" turns into "load x; load
1.0; add; store y" and the trap occurs when you do "load x".

In article <4dec2ba6$0$29996$c3e8da3$5496439d@news.astraweb.com>,
Steven D'Aprano  <steve+comp.lang.python@pearwood.info> wrote:
>The intended behaviour is operations on "quiet NANs" should return NANs, 
>but operations on "signalling NANs" should cause a trap, which can either 
>be ignored, and converted into a quiet NAN, or treated as an exception.
>
>E.g. in Decimal:
>
>>>> import decimal
>>>> qnan = decimal.Decimal('nan')  # quiet NAN
>>>> snan = decimal.Decimal('snan')  # signalling NAN
>>>> 1 + qnan
>Decimal('NaN')
>>>> 1 + snan
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/usr/local/lib/python3.1/decimal.py", line 1108, in __add__
>    ans = self._check_nans(other, context)
>  File "/usr/local/lib/python3.1/decimal.py", line 746, in _check_nans
>    self)
>  File "/usr/local/lib/python3.1/decimal.py", line 3812, in _raise_error
>    raise error(explanation)
>decimal.InvalidOperation: sNaN

Moreover:

   >>> cx = decimal.getcontext()
   >>> cx
   Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999999, Emax=999999999, capitals=1, flags=[], traps=[DivisionByZero, Overflow, InvalidOperation])
   >>> cx.traps[decimal.InvalidOperation] = False
   >>> snan
   Decimal("sNaN")
   >>> 1 + snan
   Decimal("NaN")

so as you can see, by ignoring the InvalidOperation exception, we
had our sNaN converted to a (regular, non-signal-ing, "quiet") NaN,
and 1 + NaN is still NaN.

(I admit that my mental model using "loads" can mislead a bit since:

    >>> cx.traps[decimal.InvalidOperation] = True # restore trapping
    >>> also_snan = snan
    >>>

A simple copy operation is not a "load" in this particular sense,
and on most real hardware, one just uses an ordinary 64-bit integer
memory-copying operation to copy FP bit patterns from one place to
another.)

There is some good information on wikipedia:

    http://en.wikipedia.org/wiki/NaN

(Until I read this, I was not aware that IEEE now recommends that
the quiet-vs-signal bit be 1-for-quiet 0-for-signal. I prefer the
other way around since you can then set memory to all-1-bits if it
contains floating point numbers, and get exceptions if you refer
to a value before seting it.)
-- 
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W)  +1 801 277 2603
email: gmail (figure it out)      http://web.torek.net/torek/index.html

[toc] | [prev] | [next] | [standalone]

#7068

On Mon, Jun 6, 2011 at 11:21 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> The intended behaviour is operations on "quiet NANs" should return NANs,
> but operations on "signalling NANs" should cause a trap, which can either
> be ignored, and converted into a quiet NAN, or treated as an exception.
>
> E.g. in Decimal: [snip]

So does this mean that:

a = 0.0/0.0
b = a + 1

(with signalling NANs) should trap on the second line but not the
first? That's the first "operation on a nan".

> http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf
> (The article is seven years old now, but as far as I know, the criticisms
> still apply.)

Thanks, that's my travel-home literature for tonight! :) I read the
other two articles you sent me (asynchronously), and they're most
interesting. I'm definitely still inclined to avoid any sort of
floating point work if at all possible, but hey, this gives me more
topics to bore people with at parties! (Wait. I never get invited to
parties any more. I think my work on that front is complete.)

Chris Angelico

[toc] | [prev] | [next] | [standalone]

#7070

On Mon, 06 Jun 2011 14:11:03 +1000, Chris Angelico wrote:

> On Mon, Jun 6, 2011 at 11:21 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> The intended behaviour is operations on "quiet NANs" should return
>> NANs, but operations on "signalling NANs" should cause a trap, which
>> can either be ignored, and converted into a quiet NAN, or treated as an
>> exception.
>>
>> E.g. in Decimal: [snip]
> 
> So does this mean that:
> 
> a = 0.0/0.0
> b = a + 1
> 
> (with signalling NANs) should trap on the second line but not the first?
> That's the first "operation on a nan".

Sort of.

Firstly, in order for a = 0.0/0.0 to not trap (not raise an exception), 
you have to tell it not to trap InvalidOperation (and DivideByZero I 
think?). So using Decimal:

>>> import decimal
>>> decimal.getcontext()
Context(prec=9, rounding=ROUND_HALF_UP, Emin=-999999999, Emax=999999999, 
capitals=1, flags=[], traps=[Underflow, Clamped, DivisionByZero, 
Overflow, InvalidOperation])


If we call Decimal(0)/Decimal(0), it will be trapped, which is treated as 
an exception in Python. To get a NAN:


>>> decimal.setcontext(decimal.ExtendedContext)
>>> decimal.getcontext()
Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999999, 
Emax=999999999, capitals=1, flags=[], traps=[])
>>>
>>> D = decimal.Decimal
>>> a = D(0)/D(0)
>>> a
Decimal('NaN')


Note that a flag is set, so you can tell that an exceptional event has 
occurred:


>>> decimal.getcontext()
Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999999, 
Emax=999999999, capitals=1, flags=[InvalidOperation], traps=[])


But the NAN given is a quiet NAN. Doing further operations on it doesn't 
trap:


>>> decimal.getcontext().traps[decimal.InvalidOperation] = 1
>>> decimal.getcontext().flags.clear()
>>> b = a + 1
>>> decimal.getcontext()
Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999999, 
Emax=999999999, capitals=1, flags=[], traps=[InvalidOperation])


However, if you use a signalling NAN, the situation is different. As far 
as I can tell, the only way to get a signalling NAN is to create one 
yourself:


>>> c = D('sNAN')
>>> d = c + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.6/decimal.py", line 1064, in __add__
    ans = self._check_nans(other, context)
  File "/usr/local/lib/python2.6/decimal.py", line 703, in _check_nans
    self)
  File "/usr/local/lib/python2.6/decimal.py", line 3778, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: sNaN


I don't think that there's any way to tell IEEE-754 for operations on 
NANs to return signalling NANs. As I understand it, the idea is:


- if you want exceptions to signal, set the appropriate traps;
- if you want NANs that propagate through your calculation, clear the 
traps and you'll get propagating NANs;
- if you need to detect the presence of a NAN in your calculation, you 
can inspect the flags at any time and take whatever action you want;
- and if you want a signalling NAN, you have to inject it yourself into 
your calculation, and then avoid using it.


I'm lead to believe that signalling NANs were added to satisfy politics, 
but apart from being slightly useful for marking uninitialised memory 
before use, nobody actually uses them in practice.

Wanna see something cool? You can check for inexact arithmetic:

>>> decimal.getcontext().flags
{<class 'decimal.InvalidOperation'>: 1}
>>> D(1)/D(7)
Decimal('0.142857143')
>>> decimal.getcontext().flags
{<class 'decimal.Inexact'>: 1, <class 'decimal.InvalidOperation'>: 1, 
<class 'decimal.Rounded'>: 1}


and trap on it:


>>> decimal.getcontext().traps[decimal.Inexact] = 1
>>> D(1)/D(7)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.6/decimal.py", line 1275, in __truediv__
    return ans._fix(context)
  File "/usr/local/lib/python2.6/decimal.py", line 1632, in _fix
    context._raise_error(Inexact)
  File "/usr/local/lib/python2.6/decimal.py", line 3778, in _raise_error
    raise error(explanation)
decimal.Inexact: None


Not surprisingly, by default that's turned off :)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7071

On Mon, Jun 6, 2011 at 2:59 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Mon, 06 Jun 2011 14:11:03 +1000, Chris Angelico wrote:
>> So does this mean that:
>> (with signalling NANs) should trap on the second line but not the first?

BTW, by "should" I meant "would if Python's float were 100% IEEE-754 compliant".

> I don't think that there's any way to tell IEEE-754 for operations on
> NANs to return signalling NANs. As I understand it, the idea is:
>
>
> - if you want exceptions to signal, set the appropriate traps;
> - if you want NANs that propagate through your calculation, clear the
> traps and you'll get propagating NANs;
> - if you need to detect the presence of a NAN in your calculation, you
> can inspect the flags at any time and take whatever action you want;
> - and if you want a signalling NAN, you have to inject it yourself into
> your calculation, and then avoid using it.

That makes plausible sense, at least. Get traps or propagate NANs.

> I'm lead to believe that signalling NANs were added to satisfy politics,
> but apart from being slightly useful for marking uninitialised memory
> before use, nobody actually uses them in practice.

I'm curious as to what sort of politics led to that.

> Wanna see something cool? You can check for inexact arithmetic:
> Not surprisingly, by default that's turned off :)

Neat. That's going to be off trapping pretty much all the time, but I
can imagine circumstances where you check the status at the end.

ChrisA

[toc] | [prev] | [next] | [standalone]

#6995

On Fri, 03 Jun 2011 13:27:00 -0700, Carl Banks wrote:

> On Wednesday, June 1, 2011 5:53:26 PM UTC-7, Steven D'Aprano wrote:
[...]
>> On the contrary, it blows it out of the water and stomps its corpse
>> into a stain on the ground.
> 
> Really?  I am claiming that, even if everyone and their mother thought
> exceptions were the best thing ever, NaN would have been added to IEEE
> anyway because most hardware didn't support exceptions.

You can claim that the Atlantic Ocean is made of strawberry yoghurt too, 
if you like, but that doesn't make it true.

The standard was written by people who made and used hardware that *did* 
support exceptions (hardware traps). They wrote code in languages that 
supported traps (mostly Fortran). The IEEE-754 standard mandates 
exceptions (not in the sense of Python exceptions, but still exceptions), 
and recommends various exception handling mechanisms, including try/catch.

NANs weren't invented because the standard writers didn't have a way of 
performing exceptions. You are simply *completely wrong* on that claim. 
There are plenty of documents about the IEEE-754 standard, including 
draft copies of it, and interviews with some of the participants. Go do 
some reading before spreading more misapprehensions.



> You are saying that the existence of one early system that supported
> exceptions not merely argument against that claim, but blows it out of
> the water?  Your logic sucks then.

Not one. ALL OF THEM. All of the manufacturers who were involved in the 
IEEE-754 standard had traps: Intel, Cray, DEC, CDC, Apple, and Intel. 
There may have been CPUs at the time that didn't have traps, but they 
weren't used for numeric work and they didn't matter. Traps were a 
standard mechanism used in numeric work.


> You want to go off arguing that there were good reasons aside from
> backwards compatibility they added NaN, be my guest.  Just don't go
> around saying, "Its in IEEE there 4 its a good idear LOL".  Lots of
> standards have all kinds of bad ideas in them for the sake of backwards
> compatibility, and when someone goes around claiming that something is a
> good idea simply because some standard includes it, it is the first sign
> that they're clueless about what standarization actually is.

No, I don't think that supporting NANs is useful merely because it is a 
standard. I've *repeatedly* said that NANs are useful as an alternative 
to exceptions, so don't misrepresent what I say.


[...]
> Here's the problem: Python is not for serious numerical programming.

I disagree. So do the numpy and scipy communities, and sage, and 
matplotlib. So do the Python developers: Python now has a fully IEEE-754 
compliant Decimal implementation. (What I want is floats to be equally 
compliant. I don't care if they default to raising exceptions.)

Despite it's weaknesses, Python is a good alternative to things like 
Mathematica and Matlab (which of course have weaknesses of their own), 
and it's not just me comparing them:

http://vnoel.wordpress.com/2008/05/03/bye-matlab-hello-python-thanks-sage/
http://www.larssono.com/musings/matmatpy/index.html
http://blog.revolutionanalytics.com/2009/07/mathematica-vs-matlab-vs-python.html


> Yeah, it's a really good language for calling other languages to do
> numerical programming, but it's not good for doing serious numerical
> programming itself.  Anyone with some theoretical problem where NaN is a
> good idea should already be using modules or separate programs written
> in C or Fortran.

And since Python is intended to be the glue between these modules, how 
are you supposed to get data containing NANs between these modules unless 
Python supports NANs?

I shouldn't have to fear running a snippet of Python code in case it 
chokes on a NAN. That cripples Python's usefulness as a glue language for 
numeric work.


> Casual and lightweight numerical work (which Python is good at) is not a
> wholly separate problem domain where the typical rules ("Errors should
> never pass silently") should be swept aside.

NANs are not necessarily errors, they're hardly silent, and if you don't 
want NANs, the standard mandates that there be a way to turn them off.



> [snip]
>> You'll note that, out of the box, numpy generates NANs:
>> 
>> >>> import numpy
>> >>> x = numpy.array([float(x) for x in range(5)]) x/x
>> Warning: invalid value encountered in divide array([ nan,   1.,   1.,  
>> 1.,   1.])
> 
> Steven, seriously I don't know what's going through your head.  I'm
> saying strict adherence to IEEE is not the best idea, and you cite the
> fact that a library tries to strictly adhere to IEEE as evidence that
> strictly adhering to IEEE is a good idea.  Beg the question much?

And I'm demonstrating that the people who do serious numeric work stick 
to the standard as much as possible. They do this because the standard is 
proven to be useful, otherwise they would abandon it, or start a new 
standard.


[...]
> It's clear tha IEEE's NaN handling is woefully out of place in the
> philosophy of Python, which tries to be newbie friendly and robust to
> errors; 

NANs are newbie friendly, and robust to errors.

You can't get more newbie friendly than Apple's Hypertalk, sadly 
abandoned. Among Mac users in the late 80s and 90s, Hypertalk, and its 
front end Hypercard, was like software Lego and BASIC rolled into one. 
And it supported NANs from day one.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#6999

Steven D'Aprano wrote:
> NANs are not necessarily errors, they're hardly silent, and if you don't 
> want NANs, the standard mandates that there be a way to turn them off.

So how does one turn them off in standard Python?

~Ethan~

[toc] | [prev] | [next] | [standalone]

#7002

On Fri, 03 Jun 2011 23:04:38 -0700, Ethan Furman wrote:

> Steven D'Aprano wrote:
>> NANs are not necessarily errors, they're hardly silent, and if you
>> don't want NANs, the standard mandates that there be a way to turn them
>> off.
> 
> So how does one turn them off in standard Python?

Turn them off? You have to find a way to turn them on first! What makes 
you think that Python supports IEEE-754 for floats?

By default, Decimal raises exceptions for division by zero.

>>> import decimal
>>> 1/decimal.Decimal(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.1/decimal.py", line 1359, in __rtruediv__
    return other.__truediv__(self, context=context)
  File "/usr/local/lib/python3.1/decimal.py", line 1292, in __truediv__
    return context._raise_error(DivisionByZero, 'x / 0', sign)
  File "/usr/local/lib/python3.1/decimal.py", line 3812, in _raise_error
    raise error(explanation)
decimal.DivisionByZero: x / 0 


To get INF or NAN semantics is easy for decimal:

>>> decimal.setcontext(decimal.ExtendedContext)
>>> 1/decimal.Decimal(0)
Decimal('Infinity')


but impossible for float. The best you can do is subclass float, or 
surround each calculation in a try...except, which defeats the point of 
them.

In general, Python goes to great trouble and expense to avoid generating 
any float INFs or NANs -- and when it does generate them, it's normally 
at the whim of the C maths library and therefore non-portable.

>>> math.sqrt(-1.0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: math domain error

>>> decimal.Decimal(-1).sqrt()
Decimal('NaN')


And sometimes inconsistently so:

>>> math.fsum([1, 2, float('inf'), float('nan')])
nan
>>> math.fsum([1, 2, float('inf'), float('-inf')])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: -inf + inf in fsum



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7003

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> What makes you think that Python supports IEEE-754 for floats?

That would be an easy impression to get from this long rambling thread.
The argument that Python's ‘float’ type is not meant to be anything
*but* an IEEE 754 floating point type has been made several times.

What would you say Python's ‘float’ type is intended to be, if not an
IEEE 754 floating point type?

-- 
 \        “Most people, I think, don't even know what a rootkit is, so |
  `\     why should they care about it?” —Thomas Hesse, Sony BMG, 2006 |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#7022

Steven D'Aprano wrote:
> On Fri, 03 Jun 2011 23:04:38 -0700, Ethan Furman wrote:
> 
>> Steven D'Aprano wrote:
>>> NANs are not necessarily errors, they're hardly silent, and if you
>>> don't want NANs, the standard mandates that there be a way to turn them
>>> off.
>> So how does one turn them off in standard Python?
> 
> Turn them off? You have to find a way to turn them on first! What makes 
> you think that Python supports IEEE-754 for floats?

So if Python doesn't support IEEE-754 for floats, why the big deal about 
NaNs?  Does it have to do with how the NumPy, SciPy, Sage, etc., 
libraries interface with Python?

~Ethan~

[toc] | [prev] | [next] | [standalone]

#7023

On 6/4/11 4:28 PM, Ethan Furman wrote:
> Steven D'Aprano wrote:
>> On Fri, 03 Jun 2011 23:04:38 -0700, Ethan Furman wrote:
>>
>>> Steven D'Aprano wrote:
>>>> NANs are not necessarily errors, they're hardly silent, and if you
>>>> don't want NANs, the standard mandates that there be a way to turn them
>>>> off.
>>> So how does one turn them off in standard Python?
>>
>> Turn them off? You have to find a way to turn them on first! What makes you
>> think that Python supports IEEE-754 for floats?
>
> So if Python doesn't support IEEE-754 for floats, why the big deal about NaNs?

Steven is being a little hyperbolic. Python does not fully conform to all of the 
details of the IEEE-754 specification, though it does conform to most of them. 
In particular, it raises an exception when you divide by 0.0 when the IEEE-754 
specification states that you ought to issue the "divide by zero" or "invalid" 
signal depending on the numerator (and which may be trapped by the user, but not 
by default) and will return either an inf or a NaN value if not trapped. Thus, 
the canonical example of a NaN-returning operation in fully-conforming IEEE-754 
arithmetic, 0.0/0.0, raises an exception in Python. You can generate a NaN by 
other means, namely dividing inf/inf.

One other deviation is the one which you were asking about. The standard does 
say that the "invalid" signal should be issued in most circumstances that 
generate a NaN and that the user should be able to trap that signal. Python 
explicitly disables that mechanism. It used to provide an optional module, 
fpectl, for providing a signal handler for those. However, creating a handler 
for such a low-level signal in a high-level language like Python is inherently 
unsafe, so it is not really supported any more.

The decimal module mostly gets it right. It translates the signals into Python 
exceptions that can be disabled in a particular context.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

[toc] | [prev] | [next] | [standalone]

#7030

On Sat, 04 Jun 2011 16:49:40 -0500, Robert Kern wrote:

> Steven is being a little hyperbolic. Python does not fully conform to
> all of the details of the IEEE-754 specification, though it does conform
> to most of them. 

I'm not sure that "most" is correct, but that depends on how you count 
the details. Let's just say it has partial support and let's not attempt 
to quantify it.

(Which is a big step up from how things were even just a few years ago, 
when there wasn't even a consistent way to create special values like INF 
and NAN. Many thanks to those who did that work, whoever you are!)


> In particular, it raises an exception when you divide
> by 0.0 when the IEEE-754 specification states that you ought to issue
> the "divide by zero" or "invalid" signal depending on the numerator (and
> which may be trapped by the user, but not by default) and will return
> either an inf or a NaN value if not trapped. Thus, the canonical example
> of a NaN-returning operation in fully-conforming IEEE-754 arithmetic,
> 0.0/0.0, raises an exception in Python. You can generate a NaN by other
> means, namely dividing inf/inf.

But it's inconsistent and ad hoc. The guiding philosophy of Python 
floating point maths appears to be:

(1) Python will always generate an exception on any failed operation, and 
never a NAN or INF (I believe I've even seen Guido explicitly state this 
as a design principle);

(2) arithmetic expressions and maths functions will usually, but not 
always, honour NANs and INFs if you provide then as input.

I see this thread being driven by people who have failed to notice that 
(1) already applies, and so pure Python will never give them a NAN they 
didn't explicitly create themselves, but want to remove (2) as well.

Personally I think Python would be a better language if it *always* 
returned NANs and INFs for failed float operations, but I recognise that 
I'm in a minority and that many people will prefer exceptions. Even 
though I think Guido is wrong to believe that exceptions are more newbie 
friendly than NANs (my Hypercard experience tells me differently), I 
accept that opinions differ and I'm happy for exceptions to be the 
default behaviour.

But it makes me rather annoyed when people who know nothing about 
IEEE-754 special values, their uses and justification, come along and 
insist that the only right answer is to throw away what little support 
for them we have.


> One other deviation is the one which you were asking about. The standard
> does say that the "invalid" signal should be issued in most
> circumstances that generate a NaN and that the user should be able to
> trap that signal. Python explicitly disables that mechanism. It used to
> provide an optional module, fpectl, for providing a signal handler for
> those. However, creating a handler for such a low-level signal in a
> high-level language like Python is inherently unsafe, so it is not
> really supported any more.

More unsafe than ctypes?

In any case, I believe that in Python, catching an exception is more or 
less the moral equivalent to trapping a low-level signal.


> The decimal module mostly gets it right. It translates the signals into
> Python exceptions that can be disabled in a particular context.

All I want for Christmas is for floats to offer the same level of 
IEEE-754 support as decimal, only faster. And a pony.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#7056

On 6/4/11 9:03 PM, Steven D'Aprano wrote:
> On Sat, 04 Jun 2011 16:49:40 -0500, Robert Kern wrote:
>
>> Steven is being a little hyperbolic. Python does not fully conform to
>> all of the details of the IEEE-754 specification, though it does conform
>> to most of them.
>
> I'm not sure that "most" is correct, but that depends on how you count
> the details. Let's just say it has partial support and let's not attempt
> to quantify it.

Fair enough. When I said "most", I was really counting in terms of operations 
that are actually performed, i.e. successful ones. If I have two regular 
floating point numbers x and y and add them together, the result is going to be 
what the IEEE-754 standard specifies almost all of the time. Almost every flop 
you actually do in Python will give you the IEEE-754 answer.

Of course, where Python tends to diverge is in the failure modes and error 
conditions. And also of course, the standard has more rules for all of those 
special cases, so saying that it conforms to "most of the rules" is not quite 
right, either.

> (Which is a big step up from how things were even just a few years ago,
> when there wasn't even a consistent way to create special values like INF
> and NAN. Many thanks to those who did that work, whoever you are!)
>
>
>> In particular, it raises an exception when you divide
>> by 0.0 when the IEEE-754 specification states that you ought to issue
>> the "divide by zero" or "invalid" signal depending on the numerator (and
>> which may be trapped by the user, but not by default) and will return
>> either an inf or a NaN value if not trapped. Thus, the canonical example
>> of a NaN-returning operation in fully-conforming IEEE-754 arithmetic,
>> 0.0/0.0, raises an exception in Python. You can generate a NaN by other
>> means, namely dividing inf/inf.
>
> But it's inconsistent and ad hoc. The guiding philosophy of Python
> floating point maths appears to be:
>
> (1) Python will always generate an exception on any failed operation, and
> never a NAN or INF (I believe I've even seen Guido explicitly state this
> as a design principle);

Well, if so, then it doesn't do it very well:

[~]
|2> inf = 1e300 * 1e300

[~]
|3> nan = inf / inf

[~]
|4> inf
inf

[~]
|5> nan
nan

> (2) arithmetic expressions and maths functions will usually, but not
> always, honour NANs and INFs if you provide then as input.
>
> I see this thread being driven by people who have failed to notice that
> (1) already applies, and so pure Python will never give them a NAN they
> didn't explicitly create themselves, but want to remove (2) as well.
 >
> Personally I think Python would be a better language if it *always*
> returned NANs and INFs for failed float operations, but I recognise that
> I'm in a minority and that many people will prefer exceptions. Even
> though I think Guido is wrong to believe that exceptions are more newbie
> friendly than NANs (my Hypercard experience tells me differently), I
> accept that opinions differ and I'm happy for exceptions to be the
> default behaviour.
>
> But it makes me rather annoyed when people who know nothing about
> IEEE-754 special values, their uses and justification, come along and
> insist that the only right answer is to throw away what little support
> for them we have.
>
>
>> One other deviation is the one which you were asking about. The standard
>> does say that the "invalid" signal should be issued in most
>> circumstances that generate a NaN and that the user should be able to
>> trap that signal. Python explicitly disables that mechanism. It used to
>> provide an optional module, fpectl, for providing a signal handler for
>> those. However, creating a handler for such a low-level signal in a
>> high-level language like Python is inherently unsafe, so it is not
>> really supported any more.
>
> More unsafe than ctypes?

More difficult to implement safely and more tempting to do unsafe things. If I 
remember correctly, I don't think you are supposed to allocate new memory inside 
such a signal handler. Of course, that's almost impossible to do in pure Python 
code.

> In any case, I believe that in Python, catching an exception is more or
> less the moral equivalent to trapping a low-level signal.

I agree.

>> The decimal module mostly gets it right. It translates the signals into
>> Python exceptions that can be disabled in a particular context.
>
> All I want for Christmas is for floats to offer the same level of
> IEEE-754 support as decimal, only faster. And a pony.

Hear-hear!

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web