Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #74232 > unrolled thread

Proposal: === and !=== operators

Back to article view | Back to comp.lang.python

Contents

  Proposal: === and !=== operators Steven D'Aprano <steve@pearwood.info> - 2014-07-09 07:00 +0000
    Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-09 17:21 +1000
      Re: Proposal: === and !=== operators Steven D'Aprano <steve@pearwood.info> - 2014-07-09 09:17 +0000
        Re: Proposal: === and !=== operators Rustom Mody <rustompmody@gmail.com> - 2014-07-09 09:20 -0700
        Re: Proposal: === and !=== operators Ian Kelly <ian.g.kelly@gmail.com> - 2014-07-09 11:50 -0600
        Re: Proposal: === and !=== operators Cameron Simpson <cs@zip.com.au> - 2014-07-10 09:16 +1000
        Re: Proposal: === and !=== operators Johannes Bauer <dfnsonfsduifb@gmx.de> - 2014-07-12 13:54 +0200
          Re: Proposal: === and !=== operators Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-12 16:35 +0000
            Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-13 02:54 +1000
              Re: Proposal: === and !=== operators Roy Smith <roy@panix.com> - 2014-07-12 16:39 -0400
            Re: Proposal: === and !=== operators Johannes Bauer <dfnsonfsduifb@gmx.de> - 2014-07-12 20:14 +0200
              Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-13 09:01 +1000
                Re: Proposal: === and !=== operators Roy Smith <roy@panix.com> - 2014-07-12 19:06 -0400
                  Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-13 09:15 +1000
              Re: Proposal: === and !=== operators Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-13 04:48 +0000
    Re: Proposal: === and !=== operators Cameron Simpson <cs@zip.com.au> - 2014-07-09 18:17 +1000
      Re: Proposal: === and !=== operators Steven D'Aprano <steve@pearwood.info> - 2014-07-09 09:02 +0000
        Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-09 19:23 +1000
    Re: Proposal: === and !=== operators Devin Jeanpierre <jeanpierreda@gmail.com> - 2014-07-09 05:01 -0700
    Re: Proposal: === and !=== operators Roy Smith <roy@panix.com> - 2014-07-09 08:27 -0400
      Re: Proposal: === and !=== operators Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-09 12:48 +0000
        Re: Proposal: === and !=== operators Tim Chase <python.list@tim.thechases.com> - 2014-07-09 13:05 -0500
          Re: Proposal: === and !=== operators Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-10 01:10 +0000
        Re: Proposal: === and !=== operators Ian Kelly <ian.g.kelly@gmail.com> - 2014-07-09 12:31 -0600
        Re: Proposal: === and !=== operators Roy Smith <roy@panix.com> - 2014-07-09 16:47 -0400
    Re: Proposal: === and !=== operators Ethan Furman <ethan@stoneleaf.us> - 2014-07-09 05:43 -0700
    Re: Proposal: === and !=== operators Robert Kern <robert.kern@gmail.com> - 2014-07-09 16:27 +0100
    Re: Proposal: === and !=== operators Alex Burke <alexjeffburke@gmail.com> - 2014-07-10 18:33 +0200
    Re: Proposal: === and !=== operators Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-12 03:30 +0000
      Re: Proposal: === and !=== operators Alan Bawden <alan@scooby-doo.csail.mit.edu> - 2014-07-12 01:07 -0400
        Re: Proposal: === and !=== operators Torsten Bronger <bronger@physik.rwth-aachen.de> - 2014-07-12 08:05 +0200
          Re: Proposal: === and !=== operators Torsten Bronger <bronger@physik.rwth-aachen.de> - 2014-07-12 08:14 +0200
        Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-12 16:06 +1000
        Re: Proposal: === and !=== operators Ethan Furman <ethan@stoneleaf.us> - 2014-07-11 23:11 -0700
        Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-12 16:39 +1000
          Re: Proposal: === and !=== operators Marko Rauhamaa <marko@pacujo.net> - 2014-07-12 10:06 +0300
        Re: Proposal: === and !=== operators Ethan Furman <ethan@stoneleaf.us> - 2014-07-11 23:53 -0700
        Re: Proposal: === and !=== operators Chris Angelico <rosuav@gmail.com> - 2014-07-12 17:25 +1000
        Re: Proposal: === and !=== operators Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-07-12 08:33 +0000

Page 1 of 2  [1] 2  Next page →

#74232 — Proposal: === and !=== operators

At the moment, Python has two (in)equality operators, == and != which 
call __eq__ and __ne__ methods. Some problems with those:


* Many people expect == to always be reflexive (that is, x == x for 
  every x) but classes which customise __eq__ may not be.

* The == operator requires __eq__ to return True or False 
  (or NotImplemented) and raises TypeError if it doesn't, which
  makes it impossible to use == with (say) three-valued or fuzzy
  logic.


I propose: 

* The == operator be redefined to *always* assume reflexivity, that
  is, it first compares the two arguments using `is` before calling
  the __eq__ methods.

* That's a backwards-incompatible change, so you need to enable it
  using "from __future__ import equals" in Python 3.5, and then to
  become the default behaviour in 3.6.

* To support non-reflexive types, allow === and !=== operators, which
  are like == and != except they don't call `is` first.

* The new === and !== operators call __eeq__ and __ene__ (extended
  equal and extended not equal) methods; if they don't exist, they
  fall back on __eq__ and __ne__.

* To support multi-valued logics, === and !== are not required to
  return True or False, they can return anything you like and it is
  up to the caller to ensure that they are sensible.

* Returning NotImplemented from __eeq__ and __ene__ has the same 
  meaning as for __eq__ and __ne__.

* For the avoidance of doubt, `if...elif...else` are not expected to
  be aware of multi-valued logics. No other changes to the language 
  are expected.


Thoughts? Comments?



-- 
Steven

[toc] | [next] | [standalone]

#74234

On Wed, Jul 9, 2014 at 5:00 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> Thoughts? Comments?

First thought: It will just add confusion. Currently, there are small
pockets of confusion surrounding the few cases where something's
non-reflexive, and there are occasional threads on the subject, like
we have now. Adding another pair of equality operators will mean that
everyone has to think "Do I want == or ===?", and we just need to look
at PHP and ECMAScript to see what happens - people pick the wrong
operator and have no end of subtle problems. There will be blog posts
around saying "always use === in Python", or "never use === in
Python", and everyone will get confused about how Python's === is
similar to and/or different from ECMAScript's and/or PHP's, and
ultimately, the only people who win out will be those who get paid to
write blog posts.

I think this is a big fat YAGNI. The two operators will differ in so
few situations that you may as well just define a few cases as "x is y
or x == y" instead of creating thew new operator; because really,
that's all that's happening here. (And the Py3 docs now use that kind
of notation to describe 'in'.) If you want fuzzy logic or three-valued
truth or anything, you probably know the types of the things you're
comparing, so it should be safe to use a method.

ChrisA

[toc] | [prev] | [next] | [standalone]

#74243

On Wed, 09 Jul 2014 17:21:20 +1000, Chris Angelico wrote:

> On Wed, Jul 9, 2014 at 5:00 PM, Steven D'Aprano <steve@pearwood.info>
> wrote:
>> Thoughts? Comments?
> 
> First thought: It will just add confusion. Currently, there are small
> pockets of confusion surrounding the few cases where something's
> non-reflexive, and there are occasional threads on the subject, like we
> have now. 

It's a reoccurring issue that keeps coming up over and over again. Most 
people have no need of NANs, and want them to behave like "normal" 
objects. I'm sympathetic to this idea.

Whenever this comes up, no-one has suggested any non-reflexive values 
other than NANs, SQL NUL, and "Always Compares Unequal", which I suspect 
is more of a toy than an actual useful example. So there are *very, very* 
few people who actually need NANs. 

(But those who do ought to be able to easily get it.)



> Adding another pair of equality operators will mean that
> everyone has to think "Do I want == or ===?", 

I don't think so. Nearly everyone will just use ==, those who want === 
will know they need it.


> and we just need to look
> at PHP and ECMAScript to see what happens - people pick the wrong
> operator and have no end of subtle problems. 

People are already having problems, just listen to Anders. He's 
(apparently) not doing NAN-aware computations on his data, he just wants 
to be able to do something like

this_list_of_floats == that_list_of_floats

without NANs screwing it up. But by the same token, if I want to use NANs 
the way they're supposed to be used, I should still be able to use an 
equals operator (rather than a function or method).


> There will be blog posts
> around saying "always use === in Python", or "never use === in Python",

I doubt that this would even come into the radar of most bloggers.


> and everyone will get confused about how Python's === is similar to
> and/or different from ECMAScript's and/or PHP's, 

Like we get confused over how == is different from Javascript's and 
PHP's? :-)



> and ultimately, the
> only people who win out will be those who get paid to write blog posts.
> 
> I think this is a big fat YAGNI. The two operators will differ in so few
> situations that you may as well just define a few cases as "x is y or x
> == y" instead of creating thew new operator; 

But the problem is, most people will need to us "x is y or x == y" nearly 
everywhere! And that doesn't help with containers:

py> alist = [1.0, 2, float('NAN'), 4]
py> blist = [1, 2.0, float('nan'), 4]
py> alist is blist or alist == blist
False


There ought to be a simple way for people to get alist == blist, while 
still allowing IEEE-754 aware code to work.


> because really, that's all
> that's happening here. (And the Py3 docs now use that kind of notation
> to describe 'in'.) If you want fuzzy logic or three-valued truth or
> anything, you probably know the types of the things you're comparing, so
> it should be safe to use a method.

Ah, forget the fuzzy logic, I was dreaming.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#74261

On Wednesday, July 9, 2014 2:47:40 PM UTC+5:30, Steven D'Aprano wrote:
> On Wed, 09 Jul 2014 17:21:20 +1000, Chris Angelico wrote:

> > wrote:
> >> Thoughts? Comments?
> > First thought: It will just add confusion. Currently, there are small
> > pockets of confusion surrounding the few cases where something's
> > non-reflexive, and there are occasional threads on the subject, like we
> > have now. 

> It's a reoccurring issue that keeps coming up over and over again. Most 
> people have no need of NANs, and want them to behave like "normal" 
> objects. I'm sympathetic to this idea.

-nan from me

 And since you count nans as normal, I am sure you can work out
 how much that amounts to <wink>

[toc] | [prev] | [next] | [standalone]

#74268

On Wed, Jul 9, 2014 at 3:17 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> People are already having problems, just listen to Anders. He's
> (apparently) not doing NAN-aware computations on his data, he just wants
> to be able to do something like
>
> this_list_of_floats == that_list_of_floats
>
> without NANs screwing it up. But by the same token, if I want to use NANs
> the way they're supposed to be used, I should still be able to use an
> equals operator (rather than a function or method).

Well, if we're talking about *lists*, then the comparison operator
already compares identity of individual elements:

>>> nan = float('nan')
>>> l1 = [1.0, 2.0, nan, 4.0]
>>> l2 = [1.0, 2.0, nan, 4.0]
>>> l1 == l2
True

So the comparison "x is y or x == y" could also be written "[x] ==
[y]", without requiring any changes to the language.

I suspect that just adding identity comparison is not sufficient to
solve Anders' problem, though.

> But the problem is, most people will need to us "x is y or x == y" nearly
> everywhere! And that doesn't help with containers:
>
> py> alist = [1.0, 2, float('NAN'), 4]
> py> blist = [1, 2.0, float('nan'), 4]
> py> alist is blist or alist == blist
> False

The only reason this fails is because the two nans are neither
identical nor equal. The proposed == semantics would also fail here.

[toc] | [prev] | [next] | [standalone]

#74284

TL;DR: I've got an alternative proposal at the bottom of this message.

On 09Jul2014 09:17, Steven D'Aprano <steve@pearwood.info> wrote:
>On Wed, 09 Jul 2014 17:21:20 +1000, Chris Angelico wrote:
>> First thought: It will just add confusion. Currently, there are small
>> pockets of confusion surrounding the few cases where something's
>> non-reflexive, and there are occasional threads on the subject, like we
>> have now.
>
>It's a reoccurring issue that keeps coming up over and over again. Most
>people have no need of NANs, and want them to behave like "normal"
>objects. I'm sympathetic to this idea.

But most people also expect float addition to act as though they were Decimals.

I think the real problem here is that "float" is IEEE float and this isn't what 
a naive user imagines.

>Whenever this comes up, no-one has suggested any non-reflexive values
>other than NANs, SQL NUL, and "Always Compares Unequal", which I suspect
>is more of a toy than an actual useful example. So there are *very, very*
>few people who actually need NANs.
>
>(But those who do ought to be able to easily get it.)

I agree people should have a noncumbersome way to get particular behaviours.

I have a (possibly ghastly) alternative suggestion, lower down.

>> Adding another pair of equality operators will mean that
>> everyone has to think "Do I want == or ===?",
>
>I don't think so. Nearly everyone will just use ==, those who want ===
>will know they need it.

Um, I disagree. And I also think that === in Python being different to (say) 
PHP === or JS === will further confuse things, since they are spelt the same.

At the moment Python has "is", which is very simple in concept, and ==, which 
is also conceptually simple (equal values, for the relevant values in the 
object). I agree that "is" not implying "==" is confusing when it happens, but 
that is only for a few types. Regrettably, float is heavily used.

>> and we just need to look
>> at PHP and ECMAScript to see what happens - people pick the wrong
>> operator and have no end of subtle problems.
>
>People are already having problems, just listen to Anders. He's
>(apparently) not doing NAN-aware computations on his data, he just wants
>to be able to do something like
>
>this_list_of_floats == that_list_of_floats
>
>without NANs screwing it up.

I think Ian Kelly has pointed out that this already works, and that therefore 
Andres' problem may be more comoplicated and not fixed by your proposal.

I think, based entirely on my subjective memory of discussion rates on 
python-list and doubtless influences by not hanging our in numeric computing 
lists, that float not being a plain decimal construct causes more confusion and 
surprise.

>But by the same token, if I want to use NANs
>the way they're supposed to be used, I should still be able to use an
>equals operator (rather than a function or method).
>
>> There will be blog posts
>> around saying "always use === in Python", or "never use === in Python",
>
>I doubt that this would even come into the radar of most bloggers.

I have (hazy) memories of seeing plenty of little pseudo-informative magazine 
articles discussing this kind of thing for PHP etc. The very presence of the 
extra very similar operator spawns clarification articles.

[...]
>> I think this is a big fat YAGNI. The two operators will differ in so few
>> situations that you may as well just define a few cases as "x is y or x
>> == y" instead of creating thew new operator;
>
>But the problem is, most people will need to us "x is y or x == y" nearly
>everywhere! And that doesn't help with containers:
>
>py> alist = [1.0, 2, float('NAN'), 4]
>py> blist = [1, 2.0, float('nan'), 4]
>py> alist is blist or alist == blist
>False
>
>There ought to be a simple way for people to get alist == blist, while
>still allowing IEEE-754 aware code to work.

Ok, here is my alternative proposal: dynamic float behaviour selection.

Consider this code snippet:

   with float.behaviour(nan_eq=True):
       ... code here ...

This sets a thread-local behaviour flag on the entire float type and undoes it 
on exit from the context.

This has the following advantages:

   - it is very easy to use, and makes plain that this particular chunk of code has special rules

   - it makes NaN == behaviour as requested in a particular window

   - it can wrap all code called inside the suite

   - because it is thread local it doesn't asynchronously affect other running code

   - it doesn't introduce a new operator
   
   - it affects a tightly constrainted behaviour, and can obviously be extended to other special cases if they arise

The downside is that it could break code depending on NaN being nonreflexive 
_if_ that code is called within the suite.

Personally, I would take this over a new and only-subtly-different-from-== 
"===" operator.

Cheers,
Cameron Simpson <cs@zip.com.au>

Check out Doohan's rear/all wheel slide in the French GP.  Oh yeah.
"I was just mucking around.  I won't do that again." - Mick Doohan

[toc] | [prev] | [next] | [standalone]

#74381

On 09.07.2014 11:17, Steven D'Aprano wrote:

> People are already having problems, just listen to Anders. He's 
> (apparently) not doing NAN-aware computations on his data, he just wants 
> to be able to do something like
> 
> this_list_of_floats == that_list_of_floats

This is a horrible example.

There's no pretty way of saying this: Comparing floats using equals
operators has always and will always be an incredibly dumb idea. The
same applies obviously to containers containing floats.

I also agree with Chris that I think an additional operator will make
things worse than better. It'll add confusion with no tangible benefit.
The current operators might have the deficiency that they're not
relexive, but then again: Why should == be always reflexive while the
other operators aren't? Why should I be able to assume that

x == x -> True

but not

when x < x -> False

If you're arguing from a mathematical/logical standpoint then if you
want the former you'll also have to want the latter.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#74384

On Sat, 12 Jul 2014 13:54:07 +0200, Johannes Bauer wrote:

> On 09.07.2014 11:17, Steven D'Aprano wrote:
> 
>> People are already having problems, just listen to Anders. He's
>> (apparently) not doing NAN-aware computations on his data, he just
>> wants to be able to do something like
>> 
>> this_list_of_floats == that_list_of_floats
> 
> This is a horrible example.
> 
> There's no pretty way of saying this: Comparing floats using equals
> operators has always and will always be an incredibly dumb idea. The
> same applies obviously to containers containing floats.

That's a myth. It simply is not true that you should never compare floats 
with the equals operator, it comes from the dark ages of numeric 
computing prior to IEEE-754.

If you said, "for many purposes, one should not compare floats for 
equality, but should use some sort of fuzzy comparison instead" then I 
would agree with you. But your insistence that equality "always" is wrong 
takes it out of good advice into the realm of superstition.

Quoting Professor William Kahan from the foreword to the "Apple Numerics 
Manual", second edition:

    [B]ecause so many computers in the 1960s and 1970's possessed
    so many different arithmetic anomalies, computational lore has 
    become encumbered with a vast body of superstition purporting 
    to cope with them. One such superstitious rule is "*Never* ask
    whether floating-point numbers are exactly equal."


That was written in 1987, just two years after the introduction of the 
IEEE-754 standard. It is heart-breaking that 26 years later this bogus 
"rule" is still being treated as gospel.

Bruce Dawson has written an awesome series of blog posts dealing with 
floating point issues. In this post:

https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/

he discusses some of the issues with comparing two C floats or doubles 
for equality. If you read the entire post, he emphasises how hard it is 
to compare floats, and gives three methods:

- test whether they differ by an absolute error
- test whether they differ by a relative error
- test whether they differ by a certain number of Units In Last Place

(one method he misses is the one used by Python unittest module, which 
rounds the values before comparing them)

and describes some of the weaknesses of each. In a reply to a comment, he 
warns about using == to compare a float (single precision) and a double. 
But if you keep reading all the way down to this comment:

https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/#comment-9989

he says:

    [T]he default equality comparison absolutely should be 
    true equality. To do otherwise risks madness. I have a 
    post almost ready that uses exact floating-point 
    comparisons to validate math, thus proving that exact 
    comparisons are valid.

    [...] So, standard fuzzy comparison functions would be 
    nice, but the default should remain exact comparisons.


There's one obvious use-case for exact comparison:

"Has this value changed to some other value?"

old = x
x = function(x)
if x != old:
    print "changed!"

is fine. Changing the inequality to some fuzzy comparison is bogus, 
because that means that *some changes will not be detected*.



> I also agree with Chris that I think an additional operator will make
> things worse than better. It'll add confusion with no tangible benefit.
> The current operators might have the deficiency that they're not
> relexive, but then again: Why should == be always reflexive while the
> other operators aren't? 

You're not going to hear me arguing that the non-reflexivity of NANs and 
SQL NULL is a bad idea, although some very smart people have:

http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/

Mathematical equality is reflexive. It is fundamental to the nature of 
numbers and equality that a number is always equal to itself. To the 
degree that floats are supposed to model real numbers, they should obey 
the same laws of real numbers. However, they already fail to obey them, 
so the failure of reflexivity is just one more problem that makes 
floating point such a hard problem. Compared to floating point 
arithmetic, calculus is easy.


> Why should I be able to assume that
> 
> x == x -> True
> 
> but not
> 
> when x < x -> False

Because not all types are ordered:

py> x = 1+3j
py> x < x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: complex() < complex()



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#74385

On Sun, Jul 13, 2014 at 2:35 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> You're not going to hear me arguing that the non-reflexivity of NANs and
> SQL NULL is a bad idea, although some very smart people have:
>
> http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/
>
> Mathematical equality is reflexive. It is fundamental to the nature of
> numbers and equality that a number is always equal to itself. To the
> degree that floats are supposed to model real numbers, they should obey
> the same laws of real numbers. However, they already fail to obey them,
> so the failure of reflexivity is just one more problem that makes
> floating point such a hard problem. Compared to floating point
> arithmetic, calculus is easy.
>

And there are plenty of other laws of real numbers that floats violate
(or, let's be generous, "approximate to rather than following
perfectly"). For instance, adding two positive (non-zero) numbers
should result in a number which is greater than either, but thanks to
rounding, that's not always true. (Not to mention that "infinity"
isn't a number, but it is a floating-point value.) The problem is not
equality comparisons, the problem is the expectation that the rules of
reals apply to floats.

ChrisA

[toc] | [prev] | [next] | [standalone]

#74388

In article <mailman.11777.1405184093.18130.python-list@python.org>,
 Chris Angelico <rosuav@gmail.com> wrote:
 
> And there are plenty of other laws of real numbers that floats violate
> (or, let's be generous, "approximate to rather than following
> perfectly"). For instance, adding two positive (non-zero) numbers
> should result in a number which is greater than either, but thanks to
> rounding, that's not always true.

Not to mention that whole "sum of all the positive integers" thing :-)

[toc] | [prev] | [next] | [standalone]

#74387

On 12.07.2014 18:35, Steven D'Aprano wrote:

> If you said, "for many purposes, one should not compare floats for 
> equality, but should use some sort of fuzzy comparison instead" then I 
> would agree with you. But your insistence that equality "always" is wrong 
> takes it out of good advice into the realm of superstition.

Bullshit. Comparing floats by their representation is *generally* a bad
idea because of portability issues. You don't know if IEEE754 is used to
represent floats on the systems that your code is used on.

You're hairsplitting: when I'd have said "in 99.9% of cases" you'd agree
with me but since I said "always" you disagree. Don't lawyer out.
Comparing binary representation of floats is a crappy idea.

Even more so in the age of cloud computing where your code is executed
on who knows which architecture where the exact same high level
interpretation might lead to vastly different results. Not to mention
high performance computing, where specialized FPUs can numerously be
found which don't give a shit about IEEE754.

Another thing why it's good to NEVER compare floats with regards to
their binary representation: Do you exactly know how your FPU is
configured by your operating system. Do you know that your FPUs on a
multiprocessor system are configured all identically with regards to
754? Rounding modes, etc?

Just don't fall in the pit. Don't compare floats via equals.

>> when x < x -> False
> 
> Because not all types are ordered:
> 
> py> x = 1+3j
> py> x < x
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unorderable types: complex() < complex()

Oh, so then you also don't want refelexivity of equals, I think.
Because, obviously, not all types support comparison for equality:

#!/usr/bin/python3
class Yeah(object):
	def __eq__(self, other):
		raise TypeError("Booya")
Yeah() == Yeah()

You cherrypick your logic and hairsplit in your reasoning. It's not
consistent.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1@speranza.aioe.org>

[toc] | [prev] | [next] | [standalone]

#74390

On Sun, Jul 13, 2014 at 4:14 AM, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
> Bullshit. Comparing floats by their representation is *generally* a bad
> idea because of portability issues. You don't know if IEEE754 is used to
> represent floats on the systems that your code is used on.

No, you don't, but I think you can safely assume that 1.0 == 1.0 on
any system that Python runs on.

ChrisA

[toc] | [prev] | [next] | [standalone]

#74391

In article <mailman.11779.1405206078.18130.python-list@python.org>,
 Chris Angelico <rosuav@gmail.com> wrote:

> On Sun, Jul 13, 2014 at 4:14 AM, Johannes Bauer <dfnsonfsduifb@gmx.de> wrote:
> > Bullshit. Comparing floats by their representation is *generally* a bad
> > idea because of portability issues. You don't know if IEEE754 is used to
> > represent floats on the systems that your code is used on.
> 
> No, you don't, but I think you can safely assume that 1.0 == 1.0 on
> any system that Python runs on.
> 
> ChrisA

But, you can still have:

>>> print x
1.0
>>> print y
1.0
>>> print x == y
False


which, I know, isn't really what you were talking about, but it is part 
of the general confusion of using floats.

[toc] | [prev] | [next] | [standalone]

#74392

On Sun, Jul 13, 2014 at 9:06 AM, Roy Smith <roy@panix.com> wrote:
> But, you can still have:
>
>>>> print x
> 1.0
>>>> print y
> 1.0
>>>> print x == y
> False
>
>
> which, I know, isn't really what you were talking about, but it is part
> of the general confusion of using floats.

This is partly because of the oh-so-handy magic of Python's float
reprs, rounding them off. Can you do the same trick in Python 3, where
the repr rules changed? If so, I would say this is a potential flaw in
the display, although not really a serious one.

But in terms of the OP's complaint, this is still fine, as the state
must have changed for it to be unequal. If you care about equality
differences and NOT about the above change, well, I think I've found
your solution: instead of "x == y", you should use "repr(x) ==
repr(y)" :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#74397

On Sat, 12 Jul 2014 20:14:32 +0200, Johannes Bauer wrote:

> On 12.07.2014 18:35, Steven D'Aprano wrote:
> 
>> If you said, "for many purposes, one should not compare floats for
>> equality, but should use some sort of fuzzy comparison instead" then I
>> would agree with you. But your insistence that equality "always" is
>> wrong takes it out of good advice into the realm of superstition.
> 
> Bullshit. Comparing floats by their representation is *generally* a bad
> idea because of portability issues. You don't know if IEEE754 is used to
> represent floats on the systems that your code is used on.

How many systems do you know of are there that support Python but don't 
support IEEE-754? Here's a thread from 2008 which discusses this:

https://mail.python.org/pipermail/python-dev/2008-February/076680.html


If you're running my code on an implementation of Python without IEEE-754 
floats, then I'm quite happy to say "Sorry guys, that's not my problem, 
you're on your own."

And if you're running an implementation of Python where 1.0 != 1.0, well, 
your system is so broken that there is no hope for it. None what so ever.


> You're hairsplitting: when I'd have said "in 99.9% of cases" you'd agree
> with me

I never said that.

I would not put a percentage to it, because it depends on the context and 
what you are trying to do. For some uses, exact equality is the right 
solution. For others, an absolute epsilon comparison is better. For yet 
others still, a relative error, or a ULP comparison, are better 
solutions. There's no way of putting a percentage to those. You have to 
understand what you are doing, and not just mindlessly follow some 
superstition.

When you mindlessly follow superstition, you end up with bogus warnings 
like this:

https://gcc.gnu.org/ml/gcc/2001-08/msg00853.html



> but since I said "always" you disagree. Don't lawyer out.
> Comparing binary representation of floats is a crappy idea.

Yes. And *not* comparing floats with == is a crappy idea too. *EVERY* 
method of comparing two floats to see if they are the same can break 
under some circumstances. Everything about floats is crappy, except that 
avoiding floats completely is *worse*.

Nevertheless, floats are not magically cursed. They are deterministic, 
for any fixed combination of CPU (or FPU) + machine code, and if you 
understand how floats work, then you can understand when to use exact 
equality and when not to:

http://randomascii.wordpress.com/2012/06/26/doubles-are-not-floats-so-
dont-compare-them/


Using any sort of fuzzy comparison means that you lose transitivity:

if x == y and y == z then x == z

This holds for any sane floating point system, but it doesn't hold with 
fuzzy comparisons. By default, APL uses fuzzy comparisons instead of 
exact equality. Out of the thousands of programming languages ever 
designed, APL is unique, or at most one of a tiny handful of languages, 
which eschews exact float equality. Why do you think that is?

The idea of tolerant comparisons and fuzzy functions is a fundamental 
design feature of APL:

http://www.jsoftware.com/papers/satn23.htm

nevertheless even in APL there are uses for setting the tolerance to zero 
(i.e. to get exact equality). Robert Bernecky gives one such example, and 
writes "In such a search exact comparison is absolutely necessary."


[Aside: I note that despite providing fuzzy comparison functions, and a 
system variable that controls the amount of fuzz, APL merely pushes the 
decision of how much fuzz is appropriate onto the user:

"In general, ⎕ct should be chosen to be the smallest value which is large 
enough to mask common arithmetic errors."

And what about uncommon arithmetic errors, I wonder? But I digress.]


> Even more so in the age of cloud computing where your code is executed
> on who knows which architecture where the exact same high level
> interpretation might lead to vastly different results. 

If so, then that is a bug in the cloud computing platform. Not my 
problem. Complain to the provider.


> Not to mention
> high performance computing, where specialized FPUs can numerously be
> found which don't give a shit about IEEE754.

Why should I support such broken platforms? If I run Python code on some 
horrible platform which only checks the first 8 characters of a string 
for equality "for performance reasons":

if "monumentless" == "monumental":
    print "Your Python is broken"

we'd all agree that the implementation was broken. Failure to meet at 
least the semantics of CPython floats is likewise broken.


> Another thing why it's good to NEVER compare floats with regards to
> their binary representation: Do you exactly know how your FPU is
> configured by your operating system. Do you know that your FPUs on a
> multiprocessor system are configured all identically with regards to
> 754? Rounding modes, etc?
> 
> Just don't fall in the pit. Don't compare floats via equals.

And this is why that advise is purest superstition. If you don't compare 
floats via equals, what are you supposed to do? Compare via an absolute 
epsilon? Which epsilon? Do you know if (x - y) is correctly rounded?

In the bad old days of the 1960's and 70's, there used to be systems 
where x - y would underflow to zero even though x != y. If you're telling 
me that in 2014 I should write my Python code to support such ancient 
machines (machines that don't even have a Python interpreter, I might 
add!) I'm going to just laugh at you. No I don't have to support such 
machines, or their more modern (but equally broken) equivalent.

The same fears of buggy float implementations and misconfigured FPUs that 
you use to argue against floating point == apply to every other technique 
as well. Do you know if (x-y) <= y*err is correctly rounded? What happens 
if x*err overflows? What if x-y overflows? Are you sure that your crappy 
FPU even guarantees that HUGE_NEGATIVE_NUMBER - HUGE_POSITIVE NUMBER will 
return a negative value instead of overflowing to positive number?


>>> when x < x -> False
>> 
>> Because not all types are ordered:
>> 
>> py> x = 1+3j
>> py> x < x
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: unorderable types: complex() < complex()
> 
> Oh, so then you also don't want refelexivity of equals, I think.
> Because, obviously, not all types support comparison for equality:
> 
> #!/usr/bin/python3
> class Yeah(object):
> 	def __eq__(self, other):
> 		raise TypeError("Booya")
> Yeah() == Yeah()
> 
> You cherrypick your logic and hairsplit in your reasoning. It's not
> consistent.

Who says it has to be consistent? Being consistent is a Nice To Have, but 
not a Must Have.

"A foolish consistency is the hobgoblin of little minds, adored by little 
statesmen and philosophers and divines."  -- Ralph Waldo Emerson.


Practicality beats purity. If you want some sort of pure logic language, 
then Python is not the language for you. In Python, there are good, 
useful, practical reasons for the built-in containers to assume 
reflexivity of equality but not of other order comparisons.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#74240

On 09Jul2014 07:00, Steven D'Aprano <steve@pearwood.info> wrote:
>At the moment, Python has two (in)equality operators, == and != which
>call __eq__ and __ne__ methods. Some problems with those:
>
>* Many people expect == to always be reflexive (that is, x == x for
>  every x) but classes which customise __eq__ may not be.

I'm presuming this proposal is fallout from the Nan anecdotes, since NaN != 
Nan?

The language spec is at least up front about it, I thought. It could be 
plainer, but at least it says:

   Furthermore, some types (for example, function objects) support only a 
   degenerate notion of comparison where any two objects of that type are 
   unequal.

which implies nonreflexivity.

Returning to Nan, I had thought it was an explicit design choice in IEEE 
floating point that NaN != NaN so that in (hypothetically) common cases results 
won't accidentally issue truthiness in the vein of propagating evaluation 
errors.

Personally I'd go for Nan == Nan raising a ValueError myself, but that is a 
bikeshed I lack the expertise to paint.

Anyway, I thought it is a design feature that a class can arrange for 
nonreflexivity in ==. Surprising, maybe, but wouldn't use of such a special 
class be known to the user?

Have we got some examples of people using nonreflexive == classes and being 
burnt? Aside from Nan, which I'd argue is a well known special case, or should 
be.

>* The == operator requires __eq__ to return True or False
>  (or NotImplemented) and raises TypeError if it doesn't, which
>  makes it impossible to use == with (say) three-valued or fuzzy
>  logic.

In the Python 3.4.0 docs the __eq__ etc methods have this paragraph:

   A rich comparison method may return the singleton NotImplemented if it does 
   not implement the operation for a given pair of arguments. By convention, 
   False and True are returned for a successful comparison. However, these 
   methods can return any value, so if the comparison operator is used in a 
   Boolean context (e.g., in the condition of an if statement), Python will call 
   bool() on the value to determine if the result is true or false.

and some tests with 2.7.8 and 3.4.1:

     % python
     Python 2.7.8 (default, Jul  3 2014, 06:13:58)
     [GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
     Type "help", "copyright", "credits" or "license" for more information.
     >>> class O(object):
     ...   def __eq__(self, other): return 9
     ...
     >>> o=O()
     >>> o == o
     9

     % python3.4
     Python 3.4.1 (default, May 21 2014, 01:39:38)
     [GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
     Type "help", "copyright", "credits" or "license" for more information.
     >>> class O(object):
     ...   def __eq__(self, other): return 9
     ...
     >>> o=O()
     >>> o == o
     9

I don't see this type constraint you describe.

>I propose:
>
>* The == operator be redefined to *always* assume reflexivity, that
>  is, it first compares the two arguments using `is` before calling
>  the __eq__ methods.

Won't this slow down every == test?

>* That's a backwards-incompatible change, so you need to enable it
>  using "from __future__ import equals" in Python 3.5, and then to
>  become the default behaviour in 3.6.
>
>* To support non-reflexive types, allow === and !=== operators, which
>  are like == and != except they don't call `is` first.
[...]

I don't like the spelling. They seem very easy to misuse as typos of 
conventional == and !=, and not visually very different.

Cheers,
Cameron Simpson <cs@zip.com.au>

55 mph is fast enough to get you killed, but slow enough to make you think
you're safe.    - The Gumball Rally

[toc] | [prev] | [next] | [standalone]

#74242

On Wed, 09 Jul 2014 18:17:23 +1000, Cameron Simpson wrote:

> On 09Jul2014 07:00, Steven D'Aprano <steve@pearwood.info> wrote:
>>At the moment, Python has two (in)equality operators, == and != which
>>call __eq__ and __ne__ methods. Some problems with those:
>>
>>* Many people expect == to always be reflexive (that is, x == x for
>>  every x) but classes which customise __eq__ may not be.
> 
> I'm presuming this proposal is fallout from the Nan anecdotes, since NaN
> != Nan?

There is *yet another* bloody argument going on about NANs on Python-Dev, 
from people who (for the most part) don't do numeric programming but feel 
sure that they know better than William Kahan and the IEEE-754 committee 
that designed it :-(


> The language spec is at least up front about it, I thought. It could be
> plainer, but at least it says:
> 
>    Furthermore, some types (for example, function objects) support only
>    a degenerate notion of comparison where any two objects of that type
>    are unequal.
> 
> which implies nonreflexivity.

Functions inherit the default behaviour of __eq__ from object, which 
falls back on identity:


py> def f(): return 1
... 
py> f == f
True



> Returning to Nan, I had thought it was an explicit design choice in IEEE
> floating point that NaN != NaN so that in (hypothetically) common cases
> results won't accidentally issue truthiness in the vein of propagating
> evaluation errors.

There are various reasons for why the NANs always compare unordered 
(including unequal), but yes it is a deliberate decision. Unfortunately 
people who aren't doing numeric work (and a few who are) don't like it, 
and don't like that it breaks certain "common sense" 



> 
> Personally I'd go for Nan == Nan raising a ValueError myself, but that
> is a bikeshed I lack the expertise to paint.

If we can do other nonsensical comparisons and get False, why treat NANs 
differently?

py> "Hello World" == {2.5: None}
False


For what it's worth, the IEEE-754 standard supports the "exception on 
comparison" model with signalling NANs, unfortunately C99 does not 
support signalling NANs and Java explicitly forbids them.

 
> Anyway, I thought it is a design feature that a class can arrange for
> nonreflexivity in ==. Surprising, maybe, but wouldn't use of such a
> special class be known to the user?

Well yes, but then you have folks like Anders ("NaN comparisons - Call 
For Anecdotes" thread) who didn't know that floats are not reflexive (as 
well as not transitive, associative, or commutative). He's hardly the 
only one -- Stackoverflow appears to get a question asking about NANs 
about three times a week. More broadly, if you're writing a generic 
library which will be used with arbitrary objects, there are very few 
assumptions you can make about them -- nevertheless people do.


> Have we got some examples of people using nonreflexive == classes and
> being burnt? Aside from Nan, which I'd argue is a well known special
> case, or should be.

Yes it should be, and no I don't, but people are *extremely* vehement 
that x == x ought to return True for any x. In my opinion, reflexivity is 
not that important outside of pure mathematics and logic, and people only 
get upset about the lack of it because it goes against their intuition 
about what it means for two things to be equal. But other clever people 
disagree, and even though they're wrong *wink* I'd rather seek a 
compromise that gives everybody what they want.


>>* The == operator requires __eq__ to return True or False
>>  (or NotImplemented) and raises TypeError if it doesn't, which makes it
>>  impossible to use == with (say) three-valued or fuzzy logic.

Hmmm... I could have sworn that == raised an exception if __eq__ returned 
something other than True/False/NotImplemented, but apparently I was 
wrong. Maybe I dreamt it.


> I don't see this type constraint you describe.

Neither do I.


>>I propose:
>>
>>* The == operator be redefined to *always* assume reflexivity, that
>>  is, it first compares the two arguments using `is` before calling the
>>  __eq__ methods.
> 
> Won't this slow down every == test?

Only by a pointer comparison, which is very fast. Compared to the cost of 
looking up __eq__ and calling it, the extra cost will be insignificant, 
and since many objects (small ints, certain strings, etc.) are cached, 
the over-all result will probably be to speed up the average == test.


>>* That's a backwards-incompatible change, so you need to enable it
>>  using "from __future__ import equals" in Python 3.5, and then to
>>  become the default behaviour in 3.6.
>>
>>* To support non-reflexive types, allow === and !=== operators, which
>>  are like == and != except they don't call `is` first.
> [...]
> 
> I don't like the spelling. They seem very easy to misuse as typos of
> conventional == and !=, and not visually very different.

If you can tell the difference between x=y and x==y you should be able to 
also distinguish x===y. But I accept that it's a little sub-optimal.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#74244

On Wed, Jul 9, 2014 at 7:02 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> If you can tell the difference between x=y and x==y you should be able to
> also distinguish x===y. But I accept that it's a little sub-optimal.

I'm not bothered so much by the "which one is this" confusion as the
"which should this be" confusion. With = and ==, it's easy: = for
assignment, == for comparison. But == and === would both be comparison
operators, and almost never would return different values. People
would use the wrong one frequently, and not know why.

ChrisA

[toc] | [prev] | [next] | [standalone]

#74248

On Wed, Jul 9, 2014 at 12:00 AM, Steven D'Aprano <steve@pearwood.info> wrote:
> At the moment, Python has two (in)equality operators, == and != which
> call __eq__ and __ne__ methods. Some problems with those:
>
>
> * Many people expect == to always be reflexive (that is, x == x for
>   every x) but classes which customise __eq__ may not be.
>
> * The == operator requires __eq__ to return True or False
>   (or NotImplemented) and raises TypeError if it doesn't, which
>   makes it impossible to use == with (say) three-valued or fuzzy
>   logic.

Counter-proposal: The second use case doesn't matter that much, and
can be added to == without breaking backwards compatibility
meaningfully. The first case is covered by all sorts of things, but
I'd suggest that the most meaningful way in which nan "is the same as"
nan is that they both behave the same, as objects.

The Python "is" operator returns True when two objects are identical.
Is it a great leap to guarantee, in Python, that equivalent immutable
objects are always identical? Then float('nan') is float('nan') should
always return True. We already implicitly allow that a Python
implementation may choose to do this (e.g. small integers are cached),
but it isn't so hard to guarantee it. (You don't even have to cache
anything. PyPy makes things is-identical even when they are behind the
scenes different objects in memory, IIRC.)

See also http://wiki.ecmascript.org/doku.php?id=harmony:egal , which
is a similar proposal for ES6.

-- Devin

[toc] | [prev] | [next] | [standalone]

#74249

In article <53bce8a3$0$2746$c3e8da3$76491128@news.astraweb.com>,
 Steven D'Aprano <steve@pearwood.info> wrote:

> At the moment, Python has two (in)equality operators, == and != which 
> call __eq__ and __ne__ methods. Some problems with those:
> 
> 
> * Many people expect == to always be reflexive (that is, x == x for 
>   every x) but classes which customise __eq__ may not be.
> 
> * The == operator requires __eq__ to return True or False 
>   (or NotImplemented) and raises TypeError if it doesn't, which
>   makes it impossible to use == with (say) three-valued or fuzzy
>   logic.
> 
> 
> I propose: 
> 
> * The == operator be redefined to *always* assume reflexivity, that
>   is, it first compares the two arguments using `is` before calling
>   the __eq__ methods.
> 
> * That's a backwards-incompatible change, so you need to enable it
>   using "from __future__ import equals" in Python 3.5, and then to
>   become the default behaviour in 3.6.
> 
> * To support non-reflexive types, allow === and !=== operators, which
>   are like == and != except they don't call `is` first.
> 
> * The new === and !== operators call __eeq__ and __ene__ (extended
>   equal and extended not equal) methods; if they don't exist, they
>   fall back on __eq__ and __ne__.
> 
> * To support multi-valued logics, === and !== are not required to
>   return True or False, they can return anything you like and it is
>   up to the caller to ensure that they are sensible.
> 
> * Returning NotImplemented from __eeq__ and __ene__ has the same 
>   meaning as for __eq__ and __ne__.
> 
> * For the avoidance of doubt, `if...elif...else` are not expected to
>   be aware of multi-valued logics. No other changes to the language 
>   are expected.
> 
> 
> Thoughts? Comments?

-1.  This seems like it will just add additional complexity and 
confusion, for very little gain.  We would have *three* ways to compare 
for equality (==, ===, and is).

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web