Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #74168 > unrolled thread

NaN comparisons - Call For Anecdotes

Started by"Anders J. Munch" <2014@jmunch.dk>
First post2014-07-08 16:53 +0200
Last post2014-07-09 20:07 -0700
Articles 16 — 7 participants

Back to article view | Back to comp.lang.python


Contents

  NaN comparisons - Call For Anecdotes "Anders J. Munch" <2014@jmunch.dk> - 2014-07-08 16:53 +0200
    Re: NaN comparisons - Call For Anecdotes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-08 17:13 +0000
      Re: NaN comparisons - Call For Anecdotes Chris Angelico <rosuav@gmail.com> - 2014-07-09 03:21 +1000
      Re: NaN comparisons - Call For Anecdotes "Anders J. Munch" <2014@jmunch.dk> - 2014-07-08 21:02 +0200
        Re: NaN comparisons - Call For Anecdotes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-09 00:10 +0000
          Re: NaN comparisons - Call For Anecdotes Terry Reedy <tjreedy@udel.edu> - 2014-07-09 00:57 -0400
            Re: NaN comparisons - Call For Anecdotes Steven D'Aprano <steve@pearwood.info> - 2014-07-09 06:43 +0000
              Re: NaN comparisons - Call For Anecdotes Chris Angelico <rosuav@gmail.com> - 2014-07-09 16:52 +1000
          Re: NaN comparisons - Call For Anecdotes "Anders J. Munch" <2014@jmunch.dk> - 2014-07-09 17:08 +0200
            Re: NaN comparisons - Call For Anecdotes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-09 16:53 +0000
              Re: NaN comparisons - Call For Anecdotes Ian Kelly <ian.g.kelly@gmail.com> - 2014-07-09 11:26 -0600
              Re: NaN comparisons - Call For Anecdotes "Anders J. Munch" <2014@jmunch.dk> - 2014-07-09 19:44 +0200
          Re: NaN comparisons - Call For Anecdotes Chris Angelico <rosuav@gmail.com> - 2014-07-10 01:13 +1000
          Re: NaN comparisons - Call For Anecdotes "Anders J. Munch" <2014@jmunch.dk> - 2014-07-09 18:24 +0200
      Re: NaN comparisons - Call For Anecdotes "Anders J. Munch" <2014@jmunch.dk> - 2014-07-08 21:25 +0200
    Re: NaN comparisons - Call For Anecdotes Rustom Mody <rustompmody@gmail.com> - 2014-07-09 20:07 -0700

#74168 — NaN comparisons - Call For Anecdotes

From"Anders J. Munch" <2014@jmunch.dk>
Date2014-07-08 16:53 +0200
SubjectNaN comparisons - Call For Anecdotes
Message-ID<mailman.11626.1404831235.18130.python-list@python.org>
Most people don't need to deal with NaN's in Python at all,
fortunately. They just don't appear in normal computation, because the
interpreter raises an exception instead.

It happens in my work I come across them quite a lot. I'm writing
software that talks to embedded applications that can contain NaN
values for a variety of reasons - never-initialised storage,
initialise-to-NaN, hardware failures etc.

So when my software reads these values in binary, unpack them using
the struct module, and goes to work. And NaN's are no different from
any other value, it's something to store, compare, display etc.

And that worked fine in my Python 2.4 apps.  Then I upgraded to 2.7
and it broke.  Because 2.7 goes out of it's way to ensure that NaN's
don't compare equal to themselves.

I discovered it when a sanity check told me that two functions,
to_binary and from_binary, weren't each other's inverse, as they were
intended to be.  Apparently,
bitPattern==to_binary(from_binary(bitPattern)) wasn't true for this
particular value of bitPattern.  Of course, the bit pattern in
question was the binary representation for a floating-point NaN.

Panic time! If I can't trust == to return True for (mathematically)
equal objects, that means that every single algorithm I had ever written
that explicitly or implicitly does .__eq__ or .__ne__ comparison was
suspect!

That meant I had 30000 lines of code to review.  Every time there's a
comparison, if there was any chance that either value could be a
float NaN, I would have to change e.g.
    if x==y:
to
    if x==y or (isinstance(x, float) and isinstance(y, float) and
                      math.isnan(x) and math.isnan(y)):
To make it bearable, I could wrap the pattern up in a function and
write
    if my_equal(x,y):
but I would still lose, because the standard library does == and !=
all over the place without calling my_equal.

In the end I came up with this hack: Every time I struct.unpack'd a
float, I check if it's a NaN, and if it is, then I replace it with a
reference to a single, shared, "canonical" NaN. That means that
container objects that skip __equal__ when comparing an object to
itself will work -- e.g. hash keys.

It's half a solution, of course: Any further computation with a NaN
value will change it to a different NaN object, so I still needed to
do explicit NaN-checks in various places.  I'm sure there are still
NaN-related bugs in my code, but right now it's "good enough" - I
haven't seen NaN-related bugs in a good while.

Now, all this bothers me.  Not that I had to do some work to get stuff
to work in an imperfect world.  No, what bothers me is that this
behaviour was explicitly and deliberately put in for no good reason.
The only reason is "standard says so". Not that there's any legal
requirement for Python to follow the IEEE-754 standard. Or for that
matter, for Python's spelling of IEEE-754 comparisons to be "==".

So I make this claim: float.__eq__ implementing IEEE-754 NaN
comparison rules creates real problems for developers. And it has
never, ever, helped anyone do anything.

"Never" is a strong claim, and easily disproven if false: Simply
provide a counterexample.  So that is my challenge: If you have a
program (a pre-existing and useful one, not something artificial
created for this challenge) that benefits from NaN!=NaN and that would
fail if x==x for all float objects x, then please come forward and
show it, and I'll buy you a beer the next time I'm at PyCon.

regards, Anders

[toc] | [next] | [standalone]


#74186

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2014-07-08 17:13 +0000
Message-ID<53bc26ca$0$29995$c3e8da3$5496439d@news.astraweb.com>
In reply to#74168
On Tue, 08 Jul 2014 16:53:47 +0200, Anders J. Munch wrote:

> Most people don't need to deal with NaN's in Python at all, fortunately.
> They just don't appear in normal computation, because the interpreter
> raises an exception instead.
> 
> It happens in my work I come across them quite a lot. I'm writing
> software that talks to embedded applications that can contain NaN values
> for a variety of reasons - never-initialised storage, initialise-to-NaN,
> hardware failures etc.
> 
> So when my software reads these values in binary, unpack them using the
> struct module, and goes to work. And NaN's are no different from any
> other value, it's something to store, compare, display etc.
> 
> And that worked fine in my Python 2.4 apps.

I think you're smoking something funny :-)


[steve@ando ~]$ python2.4
Python 2.4.3 (#1, Jan  9 2013, 06:49:54)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> NAN = float('NAN')
py> NAN == NAN
False


NANs compared unequal in Python 2.4.



[...]
> Now, all this bothers me.  Not that I had to do some work to get stuff
> to work in an imperfect world.  No, what bothers me is that this
> behaviour was explicitly and deliberately put in for no good reason.

Oh, you've read the IEEE-754 standard, and that's what it says? "We're 
going to specify this behaviour for NANs just to annoy people" perhaps?



-- 
Steven

[toc] | [prev] | [next] | [standalone]


#74187

FromChris Angelico <rosuav@gmail.com>
Date2014-07-09 03:21 +1000
Message-ID<mailman.11643.1404840084.18130.python-list@python.org>
In reply to#74186
On Wed, Jul 9, 2014 at 3:13 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
>> Now, all this bothers me.  Not that I had to do some work to get stuff
>> to work in an imperfect world.  No, what bothers me is that this
>> behaviour was explicitly and deliberately put in for no good reason.
>
> Oh, you've read the IEEE-754 standard, and that's what it says? "We're
> going to specify this behaviour for NANs just to annoy people" perhaps?

You're confusing IEEE with Adobe. IEEE is pronounced
"Aiiiiieeeeeeeeeeeeeeee!", but it's Adobe that makes people scream
that.

http://www.theregister.co.uk/2009/04/30/xee_photoshop_psd_code_rant/

ChrisA

[toc] | [prev] | [next] | [standalone]


#74201

From"Anders J. Munch" <2014@jmunch.dk>
Date2014-07-08 21:02 +0200
Message-ID<mailman.11653.1404846131.18130.python-list@python.org>
In reply to#74186
Steven D'Aprano wrote:
> Oh, you've read the IEEE-754 standard, and that's what it says? "We're going 
> to specify this behaviour for NANs just to annoy people" perhaps? 
I was referring to the deliberate choice to enforce IEEE-754 rules in Python. 
There is no force of law that requires Python to do so.

regards, Anders

[toc] | [prev] | [next] | [standalone]


#74219

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2014-07-09 00:10 +0000
Message-ID<53bc8861$0$29995$c3e8da3$5496439d@news.astraweb.com>
In reply to#74201
On Tue, 08 Jul 2014 21:02:09 +0200, Anders J. Munch wrote:

> Steven D'Aprano wrote:
>> Oh, you've read the IEEE-754 standard, and that's what it says? "We're
>> going to specify this behaviour for NANs just to annoy people" perhaps?
> I was referring to the deliberate choice to enforce IEEE-754 rules in
> Python. There is no force of law that requires Python to do so.

There's no force of law that requires Python to enforce reflexivity on 
values where reflexivity does not apply, any more than Python should 
enforce total order on values which aren't ordered (such as complex 
numbers, or sets).

I'm sorry that you happened to (apparently) have a use-case where you 
simultaneously have to deal with NANs but not in a numeric context. But 
from the description of your problem, it seems to me that the obvious 
solution is not to deal with floats until as late as possible. That is, 
your current work-flow (if I have understood it correctly) is:

* gather unpacked floats from some device, as ints
* pack them into floats
* process them in some way which requires reflexivity
* (maybe) much later perform numeric calculations on them


It seems to me that the trivial work-around is:

* gather packed floats from some device, as ints
* process them *as ints* in some way which requires reflexivity
* unpack back into floats
* (maybe) much later perform numeric calculations on them


Although perhaps I don't understand your use-case.


-- 
Steven

[toc] | [prev] | [next] | [standalone]


#74227

FromTerry Reedy <tjreedy@udel.edu>
Date2014-07-09 00:57 -0400
Message-ID<mailman.11672.1404881857.18130.python-list@python.org>
In reply to#74219
On 7/8/2014 8:10 PM, Steven D'Aprano wrote:

> There's no force of law that requires Python to enforce reflexivity on
> values where reflexivity does not apply,

There are laws of logic that make the lack of reflexivity obnoxious when 
putting objects in collections. Python evaded the problem, at least for 
some builtins, by contradicting itself and treating nans as equal to 
themselves in the context of collections.

In 2.x, 'in' was defined in terms of ==, but
 >>> nan=float('nan')
 >>> nl = [nan]
 >>> nan in nl
True
even though nan != the only member of nl.

In 3.x, 'in' was redefined to include 'is' as well as '=='.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]


#74230

FromSteven D'Aprano <steve@pearwood.info>
Date2014-07-09 06:43 +0000
Message-ID<53bce483$0$2746$c3e8da3$76491128@news.astraweb.com>
In reply to#74227
On Wed, 09 Jul 2014 00:57:03 -0400, Terry Reedy wrote:

> On 7/8/2014 8:10 PM, Steven D'Aprano wrote:
> 
>> There's no force of law that requires Python to enforce reflexivity on
>> values where reflexivity does not apply,
> 
> There are laws of logic that make the lack of reflexivity obnoxious when
> putting objects in collections. 

Floating point numbers are not real numbers and do not obey the laws of 
real arithmetic. Once you allow floats, or comparisons between arbitrary 
types, no matter what you do, something is going to break and somebody is 
going to be upset.


> Python evaded the problem, at least for
> some builtins, by contradicting itself and treating nans as equal to
> themselves in the context of collections.

It's not just NANs. It's any non-reflexive type:

py> class AlwaysUnequal:
...     def __eq__(self, other): return False
... 
py> x = AlwaysUnequal()
py> x in [1, 2, x]
True
py> any(x == a for a in [1, 2, x])
False




> In 2.x, 'in' was defined in terms of ==, but
>  >>> nan=float('nan')
>  >>> nl = [nan]
>  >>> nan in nl
> True
> even though nan != the only member of nl.
> 
> In 3.x, 'in' was redefined to include 'is' as well as '=='.

I don't understand this. You've just shown an example from Python 2 where 
'in' uses 'is'. How is that a Python 3 change?


-- 
Steve

[toc] | [prev] | [next] | [standalone]


#74231

FromChris Angelico <rosuav@gmail.com>
Date2014-07-09 16:52 +1000
Message-ID<mailman.11674.1404888755.18130.python-list@python.org>
In reply to#74230
On Wed, Jul 9, 2014 at 4:43 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> I don't understand this. You've just shown an example from Python 2 where
> 'in' uses 'is'. How is that a Python 3 change?

A docs change.

https://docs.python.org/2/reference/expressions.html#not-in
"""
For the list and tuple types, x in y is true if and only if there
exists an index i such that x == y[i] is true.
"""

https://docs.python.org/3/reference/expressions.html#not-in
"""
For container types such as list, tuple, set, frozenset, dict, or
collections.deque, the expression x in y is equivalent to any(x is e
or x == e for e in y).
"""

So the definition of the operator (as stated in the docs) has changed
to match its behaviour. That's definitely a change; the Py2 behaviour
is buggy according to its docs. Should the Py2 docs be updated to
match the Py3 docs, or should it be left as a minor technical
distinction that almost never is important?

ChrisA

[toc] | [prev] | [next] | [standalone]


#74257

From"Anders J. Munch" <2014@jmunch.dk>
Date2014-07-09 17:08 +0200
Message-ID<mailman.11692.1404918498.18130.python-list@python.org>
In reply to#74219
Steven D'Aprano wrote:
> It seems to me that the trivial work-around is:
>
> * gather packed floats from some device, as ints
> * process them *as ints* in some way which requires reflexivity
> * unpack back into floats
> * (maybe) much later perform numeric calculations on them
>
>
> Although perhaps I don't understand your use-case.

Clearly you do not. floats are not ints. I have no idea how you imagine 
processing IEEE-754 floating-point values in int form.

My use case is: Working with IEEE-754 floating-point values. That means storing 
and retrieving them, serialising and transferring them, accepting them as user 
input, printing them, all the usual things you do with values.

And doing so in a way that does not require special handling in algorithms that 
are otherwise generic.
When the same algorithm is capable of dealing with ints, bytestrings, text 
string, tuples, list, dictionaries, time stamps, NoneType's, bools, 
floating-point floats and a thousand other things, then NaNs stand out as the 
values that have special algorithm-breaking magic.

I gave an example of such an algorithm in an earlier reply to Chris.

regards, Anders

[toc] | [prev] | [next] | [standalone]


#74264

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2014-07-09 16:53 +0000
Message-ID<53bd739b$0$29995$c3e8da3$5496439d@news.astraweb.com>
In reply to#74257
On Wed, 09 Jul 2014 17:08:15 +0200, Anders J. Munch wrote:

> Steven D'Aprano wrote:
>> It seems to me that the trivial work-around is:
>>
>> * gather packed floats from some device, as ints * process them *as
>> ints* in some way which requires reflexivity * unpack back into floats
>> * (maybe) much later perform numeric calculations on them
>>
>>
>> Although perhaps I don't understand your use-case.
> 
> Clearly you do not. floats are not ints. 

Of course not. But there is a one-to-one correspondence between a 64-bit 
float and a 64-bit int, and the conversion is lossless in both directions.

When you talked about:

"So when my software reads these values in binary, unpack them using the
struct module, and goes to work."

I assumed that you realised that the 64-bit(?) values you were receiving 
in binary could be interpreted as ints. After all, you have to unpack 
them from some bytes.

Since that's not what you're doing, I have no idea what it is.


> I have no idea how you imagine
> processing IEEE-754 floating-point values in int form.

Cast your 64-bit float into a 64-bit int. Or, if it's a C single rather 
than a double, cast the 32-bit float into a 32-bit int. Now you can 
compare them for equality without carrying about NANs, and without losing 
data. Later, when you're ready to start doing some numeric work on them, 
you cast back to floats. That's the idea I had in mind. Perhaps it 
doesn't match your use-case.


> My use case is: Working with IEEE-754 floating-point values. That means
> storing and retrieving them, serialising and transferring them,
> accepting them as user input, printing them, all the usual things you do
> with values.

But apparently not arithmetic?


> And doing so in a way that does not require special handling in
> algorithms that are otherwise generic.

Ah, well there's your problem. NANs are special, as a signed zeroes and 
INFs. Does it distress you that x + x = x when x is an INF?


> When the same algorithm is capable of dealing with ints, bytestrings,
> text string, tuples, list, dictionaries, time stamps, NoneType's, bools,
> floating-point floats and a thousand other things, 
  ^^^^^^^^^^^^^^^^^^^^^

Obviously not, or you wouldn't be complaining about the inability to 
handle floats.

The hardware devices generating your float data... do they also generate 
ints, bytestrings, text strings, tuples, lists, dicts, time stamps, None, 
bools, and a thousand other things? If not, I wonder why you are 
insisting that you have to handle a *specialised* data type using a 
*generic* algorithm.


> then NaNs stand out
> as the values that have special algorithm-breaking magic.

Well yes. Floats have all sorts of problems. They're horrible really, the 
worst possible way to model real numbers, except for all the other ways.

I'm not unsympathetic to your problem, which is why I proposed two new 
operators, === and !==, and a change to == and !=, in another thread. 
Would == always doing an identity test before calling __eq__ solve your 
problem? If not, what would it take to solve your problem?


-- 
Steven

[toc] | [prev] | [next] | [standalone]


#74266

FromIan Kelly <ian.g.kelly@gmail.com>
Date2014-07-09 11:26 -0600
Message-ID<mailman.11700.1404926836.18130.python-list@python.org>
In reply to#74264
On Wed, Jul 9, 2014 at 10:53 AM, Steven D'Aprano
> Cast your 64-bit float into a 64-bit int. Or, if it's a C single rather
> than a double, cast the 32-bit float into a 32-bit int. Now you can
> compare them for equality without carrying about NANs, and without losing
> data. Later, when you're ready to start doing some numeric work on them,
> you cast back to floats. That's the idea I had in mind. Perhaps it
> doesn't match your use-case.

Unfortunately, it's not that simple:

>>> import math, struct
>>> def float_to_int(x):
...   return struct.unpack('L', struct.pack('d', x))[0]
...
>>> 0.0 == -0.0
True
>>> float_to_int(0.0) == float_to_int(-0.0)
False
>>> nan1 = struct.unpack('d', b'\x00'*6+b'\xf1\x7f')[0]
>>> math.isnan(nan1)
True
>>> nan2 = struct.unpack('d', b'\x00'*6+b'\xf2\x7f')[0]
>>> math.isnan(nan2)
True
>>> float_to_int(nan1) == float_to_int(nan1)
True
>>> float_to_int(nan2) == float_to_int(nan2)
True
>>> float_to_int(nan1) == float_to_int(nan2)
False

[toc] | [prev] | [next] | [standalone]


#74267

From"Anders J. Munch" <2014@jmunch.dk>
Date2014-07-09 19:44 +0200
Message-ID<mailman.11701.1404927863.18130.python-list@python.org>
In reply to#74264
Steven D'Aprano wrote:
> I assumed that you realised that the 64-bit(?) values you were receiving
> in binary could be interpreted as ints. After all, you have to unpack
> them from some bytes.

> Since that's not what you're doing, I have no idea what it is.

Stop obsessing over how NaN's came to exist in my software. That's just context.
The argument is over how those NaNs should behave. Their provenance is not relevant.

>> I have no idea how you imagine
>> processing IEEE-754 floating-point values in int form.
> Cast your 64-bit float into a 64-bit int.

I can construct a bijective mapping between any data structure and a subset of 
the natural numbers, but that has nothing to do with practical programming.
A "cast" value would be impossible to work with.

>
>> My use case is: Working with IEEE-754 floating-point values. That means
>> storing and retrieving them, serialising and transferring them,
>> accepting them as user input, printing them, all the usual things you do
>> with values.
> But apparently not arithmetic?
Of course also arithmetic. I left it out of the list because then you would say 
"hah! if you're doing arithmetic then it's not a generic algorithm". Apparently 
I can't win, you are going to nitpick anything I write.

> Ah, well there's your problem. NANs are special, as a signed zeroes and INFs. 
> Does it distress you that x + x = x when x is an INF?
No.

>> When the same algorithm is capable of dealing with ints, bytestrings,
>> text string, tuples, list, dictionaries, time stamps, NoneType's, bools,
>> floating-point floats and a thousand other things,
>    ^^^^^^^^^^^^^^^^^^^^^
>
> Obviously not, or you wouldn't be complaining about the inability to
> handle floats.

NaNs are not floating-point values. A floating-point value has a sign, an 
exponent and a mantissa.  They are "IEEE 754 floating point" values, though.

> The hardware devices generating your float data... do they also generate
> ints, bytestrings, text strings, tuples, lists, dicts, time stamps, None,
> bools, and a thousand other things? If not, I wonder why you are
> insisting that you have to handle a *specialised* data type using a
> *generic* algorithm.
All the other types are also specialised, for their separate purpose. That 
doesn't make them non-reflexive.


> I'm not unsympathetic to your problem, which is why I proposed two new
> operators, === and !==, and a change to == and !=, in another thread.
> Would == always doing an identity test before calling __eq__ solve your
> problem? If not, what would it take to solve your problem?
It would not solve it. Two bitwise identical NaNs would still compare different.
What would solve the problem is making identical NaNs compare equal.

regards, Anders

[toc] | [prev] | [next] | [standalone]


#74258

FromChris Angelico <rosuav@gmail.com>
Date2014-07-10 01:13 +1000
Message-ID<mailman.11693.1404918816.18130.python-list@python.org>
In reply to#74219
On Thu, Jul 10, 2014 at 1:08 AM, Anders J. Munch <2014@jmunch.dk> wrote:
> Steven D'Aprano wrote:
>>
>> It seems to me that the trivial work-around is:
>>
>> * gather packed floats from some device, as ints
>> * process them *as ints* in some way which requires reflexivity
>> * unpack back into floats
>> * (maybe) much later perform numeric calculations on them
>>
>>
>> Although perhaps I don't understand your use-case.
>
>
> Clearly you do not. floats are not ints. I have no idea how you imagine
> processing IEEE-754 floating-point values in int form.
>
> My use case is: Working with IEEE-754 floating-point values. That means
> storing and retrieving them, serialising and transferring them, accepting
> them as user input, printing them, all the usual things you do with values.
>
> And doing so in a way that does not require special handling in algorithms
> that are otherwise generic.
> When the same algorithm is capable of dealing with ints, bytestrings, text
> string, tuples, list, dictionaries, time stamps, NoneType's, bools,
> floating-point floats and a thousand other things, then NaNs stand out as
> the values that have special algorithm-breaking magic.
>
> I gave an example of such an algorithm in an earlier reply to Chris.

If you need to do bitwise comparisons, then the easiest way is to use
the bitpattern, converted to an integer. A 64-bit float becomes a
64-bit integer. It's then very simple to compare them, and reflexivity
is maintained. At what point do you actually need them to be floats?
What are you really doing with them?

ChrisA

[toc] | [prev] | [next] | [standalone]


#74262

From"Anders J. Munch" <2014@jmunch.dk>
Date2014-07-09 18:24 +0200
Message-ID<mailman.11697.1404923089.18130.python-list@python.org>
In reply to#74219
Chris Angelico:
> If you need to do bitwise comparisons, then the easiest way is to use
> the bitpattern, converted to an integer. A 64-bit float becomes a
> 64-bit integer. It's then very simple to compare them, and reflexivity
> is maintained. At what point do you actually need them to be floats?
> What are you really doing with them?

What does one do with floats? Add, subtract, multipy, divide, display, input, 
store and retrieve to and from various formats.
All the usual stuff. Why would my use be different from anyone elses?

What you and Steven seem to be saying is that I should employ strategies to 
avoid NaNs ever being compared. I'll take that one step further and say that as 
long as NaN!=NaN, everyone should seek to avoid NaNs ever being compared.

regards, Anders

[toc] | [prev] | [next] | [standalone]


#74206

From"Anders J. Munch" <2014@jmunch.dk>
Date2014-07-08 21:25 +0200
Message-ID<mailman.11657.1404847547.18130.python-list@python.org>
In reply to#74186
I wrote:
> Steven D'Aprano wrote:
>> Oh, you've read the IEEE-754 standard, and that's what it says? "We're going 
>> to specify this behaviour for NANs just to annoy people" perhaps? 
> I was referring to the deliberate choice to enforce IEEE-754 rules in Python. 
> There is no force of law that requires Python to do so.
>

And just to be clear, I didn't mean python-dev did this to annoy people either. 
I just meant that the choice made is not supported by any use case, so there's 
no upside to outweigh the problems it creates.

At least I've yet to hear any use case. So far I'm 0 beers in debt.

regards, Anders

[toc] | [prev] | [next] | [standalone]


#74290

FromRustom Mody <rustompmody@gmail.com>
Date2014-07-09 20:07 -0700
Message-ID<35d5eccb-3b25-4083-8001-fa268e8b377c@googlegroups.com>
In reply to#74168
On Tuesday, July 8, 2014 8:23:47 PM UTC+5:30, Anders J. Munch wrote:
> Most people don't need to deal with NaN's in Python at all,
> fortunately. They just don't appear in normal computation, because the
> interpreter raises an exception instead.

> So I make this claim: float.__eq__ implementing IEEE-754 NaN
> comparison rules creates real problems for developers. And it has
> never, ever, helped anyone do anything.

> "Never" is a strong claim, and easily disproven if false: Simply
> provide a counterexample.  So that is my challenge: If you have a

No I dont have a direct answer for your challenge but I have two close analogies.

Nan in floats is analogous (almost isomorphic) to two other areas
- null in RDBMS
- bottom – ⊥ – in denotational semantics 

The denotational semantics example would really clarify the issue.
I have discussed it here in the haskell context:
http://blog.languager.org/2012/08/functional-programming-philosophical.html

http://en.wikibooks.org/wiki/Haskell/Denotational_semantics#.E2.8A.A5_Bottom
is a more general survey

RDBMS nulls is probably a more familiar example.

Lets say you have a dbms of people and one nullable field is 'telephone'.

Now some people have no phones and the original intent of the nullable 
field was to take care of that.

But then in data entry some people who had too many phones and could not
decide which to fill kept the field null!!

Are these two cases equivalent?  Your request amounts to making them so.

Now read your own case:

> It happens in my work I come across them quite a lot. I'm writing
> software that talks to embedded applications that can contain NaN
> values for a variety of reasons - never-initialised storage,
> initialise-to-NaN, hardware failures etc.

How do you decide that all these are equivalent/ 'equal'?

Someone or other somewhere or other will be unhappy!!

My analysis of the problem:

The *letter* of the IEEE standard describes nans (and other exotic beasties)
The *spirit* describes exceptional conditions that arise in floating computations
[I have not read the standard -- just guessing]

The standard was made when programming languages did not have well tried and
tested exception mechanisms.

So with python supporting nans we have two competing exception mechanisms:
- the IEEE one -- nans
- the native python one -- exceptions

Once this is clear it should be clear what the solution should be:
DRY, no redundancy etc.

Since exceptions are pythonic, nans should not be supported.

If we were starting from scratch this would be the way to go.
However given the currently extant support, this would be a downgrade for
people like you -- who would be unhappy!

So intermediate more kludgy solution:

The problem is not with

>>> nan==nan
False

But with

>>> type(nan)
<class 'float'>

So nan is a float but by definition is not a number!!

So float a superset of all numbers??!!

If instead like:
>>> type(None)
<class 'NoneType'>

we had
>>> type(None)
<class 'NanType'>

then normal users would never see it
and people like you could deal with it outside the float-umbrella.

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.python


csiph-web