Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #44301 > unrolled thread

epiphany

Back to article view | Back to comp.lang.python

Contents

  epiphany Roy Smith <roy@panix.com> - 2013-04-24 19:50 -0400
    Re: epiphany Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-04-25 01:35 +0000
      Re: epiphany Roy Smith <roy@panix.com> - 2013-04-24 21:41 -0400
        Re: epiphany Chris Angelico <rosuav@gmail.com> - 2013-04-25 12:20 +1000
          Re: epiphany Roy Smith <roy@panix.com> - 2013-04-24 22:33 -0400
        Re: epiphany Ethan Furman <ethan@stoneleaf.us> - 2013-04-24 19:28 -0700
      Re: epiphany Ethan Furman <ethan@stoneleaf.us> - 2013-04-24 19:25 -0700
        Re: epiphany Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-04-25 04:32 +0000
          Re: epiphany Roy Smith <roy@panix.com> - 2013-04-25 08:36 -0400
            Re: epiphany Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-04-25 12:52 +0000
              Re: epiphany Roy Smith <roy@panix.com> - 2013-04-25 09:15 -0400
    Re: epiphany 88888 Dihedral <dihedral88888@googlemail.com> - 2013-04-28 14:14 -0700

#44301 — epiphany

I discovered something really neat today.

We've got a system with a bunch of rules.  Each rule is a method which 
returns True or False.  At some point, we need to know if all the rules 
are True.  Complicating things, not all the rules are implemented.  
Those that are not implemented raise NotImplementedError.

We used to have some ugly logic which kept track of which rules were 
active and only evaluated those.

So, here's the neat thing.  It turns out that bool(NotImplemented) 
returns True.  By changing the unimplemented rules from raising 
NotImplementedError to returning NotImplemented, the whole thing becomes:

    return all(r() for r in rules)

[toc] | [next] | [standalone]

#44308

On Wed, 24 Apr 2013 19:50:33 -0400, Roy Smith wrote:

> I discovered something really neat today.
> 
> We've got a system with a bunch of rules.  Each rule is a method which
> returns True or False.  At some point, we need to know if all the rules
> are True.  Complicating things, not all the rules are implemented. Those
> that are not implemented raise NotImplementedError.


NotImplementedError is intended to be raised by abstract base classes to 
indicate a method that must be overridden. I also use it as a place-
holder for functions or methods I haven't actually written yet. I'm not 
sure what semantics you're giving NotImplementedError in your code, but I 
wonder whether a neater solution might be to just use rule = None for 
unimplemented rules, rather than:

def unimplemented():
    raise NotImplementedError

rule = unimplemented

Then your logic for seeing if all rules return true would become:

all(r() for r in rules if r is not None)

and for seeing if all rules return true or are unimplemented:

all(r is None or r() for r in rules)



> We used to have some ugly logic which kept track of which rules were
> active and only evaluated those.

I don't see why you would need anything like that. Reading further on, I 
see that you are counting unimplemented rules as true, for some reason 
which I don't understand. (Knowing nothing of your use-case, I would have 
expected intuitively that unimplemented rules count as not true.) A 
simple helper function will do the job:


def eval(rule):
    try:
        return rule()
    except NotImplementedError:
        return True

everything_is_true = all(eval(r) for r in rules)



No need for complicated ugly logic keeping track of what rules are 
implemented. But if you're worried about the cost of catching those 
exceptions (you've profiled your code, right?) then that's easy with a 
decorator:


def not_implemented(func):
    @functools.wraps(func)
    def inner(*args, **kw):
        raise NotImplementedError
    inner.ni = True
    return inner


# Decorate only the rules you want to be unimplemented.

@not_implemented
def my_rule():
    pass


everything_is_true = all(r() for r in rules if not hasattr(r, 'ni'))



Note that if you could reverse the logic so that unimplemented rules 
count as not true, this will also work:

try:
    everything_is_true = all(r() for r in rules)
except NotImplementedError:
    everything_is_true = False



> So, here's the neat thing.  It turns out that bool(NotImplemented)
> returns True.  By changing the unimplemented rules from raising
> NotImplementedError to returning NotImplemented, the whole thing
> becomes:
> 
>     return all(r() for r in rules)

Objects are supposed to return NotImplemented from special dunder methods 
like __add__, __lt__, etc. to say "I don't know how to implement this 
method for the given argument". Python will then try calling the other 
object's special method. If both objects return NotImplemented, Python 
falls back on whatever default behaviour is appropriate.

So, knowing nothing of your application, I fear that this is an abuse of 
NotImplemented's semantics. If a rule returns NotImplemented, I would 
expect your application to fall back on a different rule. If that's not 
the case, you're using it in a non-standard way that will cause confusion 
for those with expectations of what NotImplemented means.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#44309

In article <5178884b$0$29977$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> I don't see why you would need anything like that. Reading further on, I 
> see that you are counting unimplemented rules as true, for some reason 
> which I don't understand.

The top-level logic we need to enforce is "this configuration doesn't 
violate any rules".

[toc] | [prev] | [next] | [standalone]

#44310

On Thu, Apr 25, 2013 at 11:41 AM, Roy Smith <roy@panix.com> wrote:
> In article <5178884b$0$29977$c3e8da3$5496439d@news.astraweb.com>,
>  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>
>> I don't see why you would need anything like that. Reading further on, I
>> see that you are counting unimplemented rules as true, for some reason
>> which I don't understand.
>
> The top-level logic we need to enforce is "this configuration doesn't
> violate any rules".

Then have your unimplemented rules simply return True. Easy!

ChrisA

[toc] | [prev] | [next] | [standalone]

#44311

In article <mailman.1044.1366856455.3114.python-list@python.org>,
 Chris Angelico <rosuav@gmail.com> wrote:

> On Thu, Apr 25, 2013 at 11:41 AM, Roy Smith <roy@panix.com> wrote:
> > In article <5178884b$0$29977$c3e8da3$5496439d@news.astraweb.com>,
> >  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> >
> >> I don't see why you would need anything like that. Reading further on, I
> >> see that you are counting unimplemented rules as true, for some reason
> >> which I don't understand.
> >
> > The top-level logic we need to enforce is "this configuration doesn't
> > violate any rules".
> 
> Then have your unimplemented rules simply return True. Easy!
> 
> ChrisA

It's nice to have tri-state logic:

* This rule passes

* This rule fails

* This rule was not evaluated

What I've got now expresses that perfectly.

[toc] | [prev] | [next] | [standalone]

#44314

On 04/24/2013 07:20 PM, Chris Angelico wrote:
> On Thu, Apr 25, 2013 at 11:41 AM, Roy Smith <roy@panix.com> wrote:
>> In article <5178884b$0$29977$c3e8da3$5496439d@news.astraweb.com>,
>>   Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>>
>>> I don't see why you would need anything like that. Reading further on, I
>>> see that you are counting unimplemented rules as true, for some reason
>>> which I don't understand.
>>
>> The top-level logic we need to enforce is "this configuration doesn't
>> violate any rules".
>
> Then have your unimplemented rules simply return True. Easy!

And less clear.

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#44312

On 04/24/2013 06:35 PM, Steven D'Aprano wrote:
> Objects are supposed to return NotImplemented from special dunder methods
> like __add__, __lt__, etc. to say "I don't know how to implement this
> method for the given argument". Python will then try calling the other
> object's special method. If both objects return NotImplemented, Python
> falls back on whatever default behaviour is appropriate.
>
> So, knowing nothing of your application, I fear that this is an abuse of
> NotImplemented's semantics. If a rule returns NotImplemented, I would
> expect your application to fall back on a different rule. If that's not
> the case, you're using it in a non-standard way that will cause confusion
> for those with expectations of what NotImplemented means.

Why would you assume some random application is going to deal with NotImplemented the same way the python interpreter 
does?  And even the interpreter isn't consistent -- sometimes it will return false (__eq__) and sometimes it will raise 
an Exception (__add__).

I hardly think it an abuse of NotImplemented to signal something is not implemented when NotImplemented means, um, not 
implemented.

possibly-not-implemented-ly yours,

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#44318

On Wed, 24 Apr 2013 19:25:37 -0700, Ethan Furman wrote:

> On 04/24/2013 06:35 PM, Steven D'Aprano wrote:
>> Objects are supposed to return NotImplemented from special dunder
>> methods like __add__, __lt__, etc. to say "I don't know how to
>> implement this method for the given argument". Python will then try
>> calling the other object's special method. If both objects return
>> NotImplemented, Python falls back on whatever default behaviour is
>> appropriate.
>>
>> So, knowing nothing of your application, I fear that this is an abuse
>> of NotImplemented's semantics. If a rule returns NotImplemented, I
>> would expect your application to fall back on a different rule. If
>> that's not the case, you're using it in a non-standard way that will
>> cause confusion for those with expectations of what NotImplemented
>> means.
> 
> Why would you assume some random application is going to deal with
> NotImplemented the same way the python interpreter does?


Why would you assume that some random application is going to treat x==y 
the same way the Python interpreter does?

Just because you can design your objects to do anything you want doesn't 
mean you should. Breaking conventions carries costs by the mere fact that 
you're breaking conventions. There are established semantics that an 
experienced Python developer will expect for NotImplemented, and doing 
something else risks causing confusion and mistakes.

Or worse, bugs. If there is any chance that a rule might be called in a 
context where the Python interpreter gets to interpret the return result 
before you see it, then returning NotImplemented could lead to difficult 
to debug problems.



> And even the
> interpreter isn't consistent -- sometimes it will return false (__eq__)
> and sometimes it will raise an Exception (__add__).

As I said:

"If both objects return NotImplemented, Python falls back on whatever 
default behaviour is appropriate."

If neither object knows how to compare the other for equality, the 
appropriate behaviour is to treat them as unequal. If neither object 
knows how to add itself to the other, the appropriate behaviour is to 
raise an exception.


> I hardly think it an abuse of NotImplemented to signal something is not
> implemented when NotImplemented means, um, not implemented.

It doesn't just mean "not implemented in general", it has a specific 
meaning: "I don't know what to do here, let the other object handle it".

As I have repeatedly said, I don't know the context of the application, 
but from what little has been described, this part of it doesn't feel to 
me like a good, clean design. I might be wrong, but from the outside it 
feels like the API should be that rules return a three-state logic 
instance:

True, False, Unknown

where Unknown can be trivially created with 

Unknown = object()

The semantics of NotImplementedError is that it is an *error*, and that 
doesn't sound appropriate given the example shown. Why would a rule that 
raises an *error* exception be treated as if it had passed? That's just 
wrong.

The semantics of NotImplemented is that it is a signal for one object to 
say "I don't know how to do this, let somebody else try". That also 
doesn't seem appropriate. There's no sign that Roy's application does the 
equivalent to this:

result = rule()
if result is NotImplemented:
    result = another_rule()
if result is NotImplemented:
    result = some_default


Since rules apparently take no arguments, either:

1) they rely on global state, which is a nasty design; or

2) rules actually have a fixed return result, in which case why make them 
functions in the first place?


Since both possibilities seem stupid, and I do not believe that Roy 
actually is stupid, I suspect that his example over-simplifies the 
situation. But I can't comment on the infinite number of things that his 
code might do, I can only comment on the examples as actually given, and 
as given, I don't think that either NotImplementedError or NotImplemented 
is a clean solution to the problem.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#44332

In article <5178b1db$0$29977$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> The semantics of NotImplemented is that it is a signal for one object to 
> say "I don't know how to do this, let somebody else try".

That's precisely the logic here.  The rule says, "I don't know how to 
tell you if this is OK or not, ask another rule".

> Since rules apparently take no arguments, either:
> 
> 1) they rely on global state, which is a nasty design; or
> 
> 2) rules actually have a fixed return result, in which case why make them 
> functions in the first place?

Yes, rules take arguments.  I elided them from the original description 
since it wasn't germane to what I was trying to show.

> Since both possibilities seem stupid, and I do not believe that Roy 
> actually is stupid,

I am honored that you have such a high opinion of me :-)

Here's what the docs say about NotImplemented:

> This type has a single value. There is a single object with this 
> value. This object is accessed through the built-in name 
> NotImplemented. Numeric methods and rich comparison methods may 
> return this value if they do not implement the operation for the 
> operands provided. (The interpreter will then try the reflected 
> operation, or some other fallback, depending on the operator.) Its 
> truth value is true.

It gives an example of a use by numeric methods.  It doesn't say that's 
the only thing it can be used for.

It also says, "Its truth value is true".  Why would they document that 
fact if you weren't supposed to use it as a boolean operand?

[toc] | [prev] | [next] | [standalone]

#44335

On Thu, 25 Apr 2013 08:36:34 -0400, Roy Smith wrote:

> In article <5178b1db$0$29977$c3e8da3$5496439d@news.astraweb.com>,
>  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> 
>> The semantics of NotImplemented is that it is a signal for one object
>> to say "I don't know how to do this, let somebody else try".
> 
> That's precisely the logic here.  The rule says, "I don't know how to
> tell you if this is OK or not, ask another rule".

Sounds good to me then. It looks like your design is actually much closer 
to what I believe the standard Python semantics are intended to be than 
it appeared at first.



[...]
> Here's what the docs say about NotImplemented:
> 
>> This type has a single value. There is a single object with this value.
>> This object is accessed through the built-in name NotImplemented.
>> Numeric methods and rich comparison methods may return this value if
>> they do not implement the operation for the operands provided. (The
>> interpreter will then try the reflected operation, or some other
>> fallback, depending on the operator.) Its truth value is true.
> 
> It gives an example of a use by numeric methods.  It doesn't say that's
> the only thing it can be used for.

Right. You can do a lot of things in Python, including shooting your foot 
off :-) but that doesn't mean you should. The further away from standard 
Python conventions you get, the more wary you should be. That's all.


> It also says, "Its truth value is true".  Why would they document that
> fact if you weren't supposed to use it as a boolean operand?

You can use *anything* in Python in a boolean context. That's a language 
feature: all objects are either truthy or falsey. As for why it is 
documented for NotImplemented, I guess that's because some people might 
guess that it is falsey, like None.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#44337

In article <51792710$0$29977$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> > It also says, "Its truth value is true".  Why would they document that
> > fact if you weren't supposed to use it as a boolean operand?
> 
> You can use *anything* in Python in a boolean context. That's a language 
> feature: all objects are either truthy or falsey. As for why it is 
> documented for NotImplemented, I guess that's because some people might 
> guess that it is falsey, like None.

That was part of what added the epiphanality to the experience.  My 
first guess was exactly as you say, that bool(NotImplemented) would be 
false.  Once I discovered that it was true, the rest immediately fell 
into place and many lines of code got replaced by the simple:

    return all(r(...) for r in rules)
                  ^
                  |
                  +---- stuff that I'm not showing goes here :-)

[toc] | [prev] | [next] | [standalone]

#44476

Roy Smith於 2013年4月25日星期四UTC+8上午7時50分33秒寫道：
> I discovered something really neat today.
> 
> 
> 
> We've got a system with a bunch of rules.  Each rule is a method which 
> 
> returns True or False.  At some point, we need to know if all the rules 
> 
> are True.  Complicating things, not all the rules are implemented.  
> 
> Those that are not implemented raise NotImplementedError.
> 
> 
> 
> We used to have some ugly logic which kept track of which rules were 
> 
> active and only evaluated those.
> 
> 
> 
> So, here's the neat thing.  It turns out that bool(NotImplemented) 
> 
> returns True.  By changing the unimplemented rules from raising 
> 
> NotImplementedError to returning NotImplemented, the whole thing becomes:
> 
> 
> 
>     return all(r() for r in rules)

  Problems of rules in  Boolean algebra or  the bi-level logic
inference engine in AI were all solved long time ago
in the text book about AI.

There are some variations about the multi-level  or 
the continuous level logic engine with some new phases 
in Fuzzy theory  in the expert system.

A dynamical typed language is better to be used in this kind of 
problems.  
 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web