Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #83924 > unrolled thread

numpy.allclose()

Back to article view | Back to comp.lang.python

Contents

  numpy.allclose() Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-01-18 01:26 +1100
    Re: numpy.allclose() Ian Kelly <ian.g.kelly@gmail.com> - 2015-01-17 11:42 -0700
      Re: numpy.allclose() Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-01-18 14:09 +1100
        Re: numpy.allclose() Ian Kelly <ian.g.kelly@gmail.com> - 2015-01-18 01:24 -0700
          Re: numpy.allclose() Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-01-18 21:25 +1100

#83924 — numpy.allclose()

Can anyone explain the rationale for numpy's allclose() semantics?

help(allclose) says:


allclose(a, b, rtol=1e-05, atol=1e-08)
    Returns True if two arrays are element-wise equal within a tolerance.

    The tolerance values are positive, typically very small numbers.  The
    relative difference (`rtol` * abs(`b`)) and the absolute difference
    `atol` are added together to compare against the absolute difference
    between `a` and `b`.

    [...]

    If the following equation is element-wise True, then allclose returns
    True.

    absolute(`a` - `b`) <= (`atol` + `rtol` * absolute(`b`))



I don't understand why they add the error tolerances together. I can
understand taking the minimum, or the maximum: 

* taking the maximum is equivalent to the rule "numbers are close if they
  differ by no more than EITHER the absolute tolerance OR the relative
  tolerance";

* taking the minimum is equivalent to the rule "numbers are close if they
  differ by no more than BOTH the absolute tolerance AND the relative
  tolerance".


But adding them together doesn't make any sense to me. That leads to the
situation where two values are deemed "close" if you give two tolerances
even though it fails each test individually:

py> numpy.allclose([1.2], [1.0], 0.0, 0.1)  # Fails absolute error test.
False
py> numpy.allclose([1.2], [1.0], 0.1, 0.0)  # Fails relative error test.
False
py> numpy.allclose([1.2], [1.0], 0.1, 0.1)  # Passes!
True




-- 
Steven

[toc] | [next] | [standalone]

#83940

On Sat, Jan 17, 2015 at 7:26 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> I don't understand why they add the error tolerances together. I can
> understand taking the minimum, or the maximum:

The usual idea is that the tolerance is calculated as a relative
value, but an absolute tolerance is used instead when the values get
close to zero, so under normal circumstances the relative tolerance
will dominate and the absolute tolerance will be very small in
comparison. So why add them instead of taking the maximum? I'm not
sure, but I would guess it's probably for efficiency. Determining the
maximum requires the CPU to perform a conditional branch that a simple
addition does not. In any case, this is not unique to numpy.  See
e.g.:

https://books.google.com/books?id=CbH0AwAAQBAJ&pg=PA164
(equation 8.26)

or:

https://books.google.com/books?id=j0rY3D9fx-0C&pg=PA71
(equation 7.437)

On the other hand, Matlab evidently uses the maximum:
http://www.mathworks.com/help/simbio/ref/absolutetolerance.html

> * taking the minimum is equivalent to the rule "numbers are close if they
>   differ by no more than BOTH the absolute tolerance AND the relative
>   tolerance".

This is generally not the desired semantic, and it's not unheard of to
pass 0 for one of the tolerances, in which case the minimum would
clearly be incorrect.

[toc] | [prev] | [next] | [standalone]

#83964

Ian Kelly wrote:

> On Sat, Jan 17, 2015 at 7:26 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> I don't understand why they add the error tolerances together. I can
>> understand taking the minimum, or the maximum:
> 
> The usual idea is that the tolerance is calculated as a relative
> value, but an absolute tolerance is used instead when the values get
> close to zero, so under normal circumstances the relative tolerance
> will dominate and the absolute tolerance will be very small in
> comparison. 

That suggests that approx_equal() should return True if the actual error is
smaller than *either* the absolute or relative error, that is, using the
maximum.


> So why add them instead of taking the maximum? I'm not 
> sure, but I would guess it's probably for efficiency. Determining the
> maximum requires the CPU to perform a conditional branch that a simple
> addition does not.

That's... horrible. 

"Tell me whether these two numbers differ by at most X or Y%."

"Oh, that would take a nanosecond longer than I think is reasonable, so I'm
instead going to tell you whether they differ by some arbitrary Z instead."

I'm guessing that can only have come from the mindset of C/C++ programmers,
where this sort of thing is considered acceptable:

http://blogs.msdn.com/b/oldnewthing/archive/2014/06/27/10537746.aspx

:-)


> In any case, this is not unique to numpy.  See 
> e.g.:
> 
> https://books.google.com/books?id=CbH0AwAAQBAJ&pg=PA164
> (equation 8.26)

Ah, well as soon as you start using GPUs for numerical computation, you're
already living in a state of sin. GPU's attitude to numerical correctness
is best described as "meh, close enough".

Seriously, anyone familiar with the horrible state of numeric computing
prior to IEEE-756 surely cannot look at GPU numerics without having
flashbacks to the bad old days where:

    if x != 0:
        y = 1/x

could crash with a divide by zero!


[...]
>> * taking the minimum is equivalent to the rule "numbers are close if they
>>   differ by no more than BOTH the absolute tolerance AND the relative
>>   tolerance".
> 
> This is generally not the desired semantic, and it's not unheard of to
> pass 0 for one of the tolerances, in which case the minimum would
> clearly be incorrect.

True. In that case, you would have to treat 0 as a special case, which
starts getting messy.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#83970

On Sat, Jan 17, 2015 at 8:09 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> I'm guessing that can only have come from the mindset of C/C++ programmers,
> where this sort of thing is considered acceptable:

Maybe. The journal reference in the second link I posted dates the
practice back to at least 1975, a time predating K&R C, when it was
most notable as the language that Unix was written in. I wouldn't be
surprised if this actually originated from Fortran.

[toc] | [prev] | [next] | [standalone]

#83975

Ian Kelly wrote:

> On Sat, Jan 17, 2015 at 8:09 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> I'm guessing that can only have come from the mindset of C/C++
>> programmers, where this sort of thing is considered acceptable:
> 
> Maybe. The journal reference in the second link I posted dates the
> practice back to at least 1975, a time predating K&R C, when it was
> most notable as the language that Unix was written in. I wouldn't be
> surprised if this actually originated from Fortran.

Ah, the good old days, when computer designers built machines where 1.0*X
could overflow and X-0.0 could underflow.




-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web