Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53184 > unrolled thread

Interface and duck typing woes

Back to article view | Back to comp.lang.python

Contents

  Interface and duck typing woes Joe Junior <joe.fbs.junior@gmail.com> - 2013-08-28 18:09 -0300
    Re: Interface and duck typing woes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-28 23:24 +0000
    Re: Interface and duck typing woes Nobody <nobody@nowhere.com> - 2013-08-29 10:41 +0100
      Re: Interface and duck typing woes Joe Junior <joe.fbs.junior@gmail.com> - 2013-08-29 09:40 -0300
        Re: Interface and duck typing woes alex23 <wuwei23@gmail.com> - 2013-08-30 10:14 +1000
        Re: Interface and duck typing woes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-30 02:42 +0000
          Re: Interface and duck typing woes Roy Smith <roy@panix.com> - 2013-08-30 06:35 -0400
            Re: Interface and duck typing woes Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-31 00:13 +0000
              Re: Interface and duck typing woes Ned Batchelder <ned@nedbatchelder.com> - 2013-08-30 20:45 -0400
              Re: Interface and duck typing woes Joshua Landau <joshua@landau.ws> - 2013-09-01 00:18 +0100
              Re: Interface and duck typing woes Roy Smith <roy@panix.com> - 2013-08-31 20:52 -0400
      Re: Interface and duck typing woes Chris Angelico <rosuav@gmail.com> - 2013-08-29 23:07 +1000
      Re: Interface and duck typing woes Joe Junior <joe.fbs.junior@gmail.com> - 2013-08-29 11:11 -0300
    Re: Interface and duck typing woes jussi.santti@ard.fi - 2013-08-29 22:37 -0700

#53184 — Interface and duck typing woes

While designing a simple library, I found myself asking a
philosophical question: to check or not to check the parameter's
interface?

I think that, considering it is Python, the usual answer would be
"no", but here is the situation that got me thinking:

class Flock:

    def __init__(self):
        self.ducks= []

    def do_stuff(self):
        for duck in self.ducks:
            duck.quack()

class Duck:

    def quack(self):
        #quack-quack
        pass

f = Flock()
d = Duck()
f.ducks.append(d)
f.do_stuff()

Ok, no big deal there, the problem is if the user forgets to implement
the quack() method. The stack trace would complain that "duck.quack()"
is wrong, but that can happen hundreds of lines after the user
actually added the object to the Flock, and it can be hard to find out
what is happening and which object is wrong.

Of course I don't want to check isistance(), I like duck typing, but
should I check if hasattr() and callable() before adding to the
container? What is the pythonic way to deal with it? Am I worrying too
much ;-)?

Thanks,

Joe

[toc] | [next] | [standalone]

#53194

On Wed, 28 Aug 2013 18:09:22 -0300, Joe Junior wrote:

> While designing a simple library, I found myself asking a philosophical
> question: to check or not to check the parameter's interface?

The only correct answer to that is, "Yes no maybe".

:-)


> I think that, considering it is Python, the usual answer would be "no",
> but here is the situation that got me thinking:
> 
> class Flock:
> 
>     def __init__(self):
>         self.ducks= []
> 
>     def do_stuff(self):
>         for duck in self.ducks:
>             duck.quack()
> 
> class Duck:
> 
>     def quack(self):
>         #quack-quack
>         pass
> 
> f = Flock()
> d = Duck()
> f.ducks.append(d)
> f.do_stuff()
> 
> Ok, no big deal there, the problem is if the user forgets to implement
> the quack() method. The stack trace would complain that "duck.quack()"
> is wrong, but that can happen hundreds of lines after the user actually
> added the object to the Flock, and it can be hard to find out what is
> happening and which object is wrong.

True, but that's a good case for improving the error message, or using a 
debugger. Here is Flock.do_stuff re-written to give a more verbose/useful 
error message:


    def do_stuff(self):
        for i, duck in enumerate(self.ducks):
             try:
                 duck.quack()
             except AttributeError:
                 raise DuckError(
                         'object %r at index %d has no quack' % (duck, i)
                         )


Okay, seeing the index is useful. But we would have got nearly as much 
information from the AttributeError traceback, minus the index:

py> (42).quack()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute 'quack'

So how much extra work are you prepared to put in to make a rare 
occurrence (passing a magpie instead of a duck) easier to debug? Since 
errors are presumably rare, maybe the answer is "not a lot of extra 
work". But if the consequence of an error is catastrophic ("for want of a 
duck quacking, the nuclear reactor explodes, making the northern 
hemisphere uninhabitable"), maybe the answer is "as much as it takes".

Other questions: what happens if duck.quack is buggy and raises 
AttributeError? A good question, but just how far should we go in 
worrying about things like this? What happens if duck.quack is buggy and 
raises StopIteration? Sometimes the right reaction is to deal with it if 
and when it actually occurs. In other words, wait for the bug report 
before trying to fix it.

("Fix it" may mean telling people "don't do that!".)


> Of course I don't want to check isistance(), I like duck typing, but
> should I check if hasattr() and callable() before adding to the
> container? What is the pythonic way to deal with it? Am I worrying too
> much ;-)?

Yes :-)

Except in the (rare?) case that you aren't worrying enough, in which case 
you can check hasattr and callable up front, or do whatever other tests 
you feel the need to check. It depends on the specific code you are 
writing.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#53231

On Wed, 28 Aug 2013 18:09:22 -0300, Joe Junior wrote:

> Of course I don't want to check isistance(), I like duck typing, but
> should I check if hasattr() and callable() before adding to the container?

That won't tell you if the object has a quack() method but with
incompatible semantics (e.g. wrong number or types of arguments).

> What is the pythonic way to deal with it?

Ignore it. If you want early type checking, use a statically-typed
language.

[toc] | [prev] | [next] | [standalone]

#53242

Well, the main reason for me asking this question here was because of
the Java/C#/Whatever developer in me craving for an Interface for the
container's items, and I noticed that I'm not alone in this. But I was
actually expecting the "We're all consenting adults, here", I guess I
just needed the confirmation :-)

Another reason for this question is that I read some people saying
they wouldn't use python for large projects, and they always point at
the lack of Interfaces as a concern. I actually disagree, but I can
see their point. What do you think?

@Nobody
>> Of course I don't want to check isistance(), I like duck typing, but
>> should I check if hasattr() and callable() before adding to the container?
>That won't tell you if the object has a quack() method but with
>incompatible semantics (e.g. wrong number or types of arguments).

Yeah, didn't think about that, it's kinda swimming upstream!
Definitely it is more hassle than it is worth.

@ChrisA
>Do you believe that you can write code to catch every bug you might
>make? If so, you are naive and probably haven't spent much time
>programming yet :) And if not, then you must acknowledge that bugs
>WILL happen; therefore you will need to cope with them after the
>event. So rather than trying to prevent them all, just improve your
>means of coping, and you'll accomplish the same end with much less
>trouble.

Oh, no! I'm not that presumptuous (or naive)! But what do you mean by
"improve means of coping"? Do you know any article on the subject you
could point me?

@Steven
>> Of course I don't want to check isistance(), I like duck typing, but
>> should I check if hasattr() and callable() before adding to the
>> container? What is the pythonic way to deal with it? Am I worrying too
>> much ;-)?

>Yes :-)

Well, thanks! :-) And thanks for the article.

[toc] | [prev] | [next] | [standalone]

#53265

On 29/08/2013 10:40 PM, Joe Junior wrote:
> Another reason for this question is that I read some people saying
> they wouldn't use python for large projects, and they always point at
> the lack of Interfaces as a concern. I actually disagree, but I can
> see their point. What do you think?

Having worked on large Python projects both with & without interfaces, 
it's definitely possible with either approach. It certainly isn't 
lacking in for support for them:

http://docs.zope.org/zope.interface/

[toc] | [prev] | [next] | [standalone]

#53270

On Thu, 29 Aug 2013 09:40:32 -0300, Joe Junior wrote:

> Well, the main reason for me asking this question here was because of
> the Java/C#/Whatever developer in me craving for an Interface for the
> container's items, and I noticed that I'm not alone in this. But I was
> actually expecting the "We're all consenting adults, here", I guess I
> just needed the confirmation :-)
> 
> Another reason for this question is that I read some people saying they
> wouldn't use python for large projects, and they always point at the
> lack of Interfaces as a concern. I actually disagree, but I can see
> their point. What do you think?

Interfaces aren't a built-in part of the language, but big frameworks 
like Zope and Twisted include them. See for example discussion here:

http://dirtsimple.org/2004/12/python-interfaces-are-not-java.html

In a more ad-hoc manner, there are recipes for interface-like 
functionality. For example, from the Python Cookbook, we have this:

http://code.activestate.com/recipes/52291


It's a myth that Python is entirely opposed to type-checking. Many built-
ins do it. Way back in Python 1.5, Python's creator Guido van Rossum 
wrote an essay describing a way to implement Eiffel-like pre- and post-
condition checking:

http://www.python.org/doc/essays/metaclasses/


These days, it would be relatively simple to implement pre- and post-
condition checking using decorators, and indeed one of the motivating use-
cases for function annotations in Python 3 is to allow such things.

http://www.python.org/dev/peps/pep-3107/

(Function annotations are perhaps the best Python feature that nobody 
uses.)


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#53288

In article <52200699$0$6599$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> These days, it would be relatively simple to implement pre- and post-
> condition checking using decorators, and indeed one of the motivating use-
> cases for function annotations in Python 3 is to allow such things.
> 
> http://www.python.org/dev/peps/pep-3107/
> 
> (Function annotations are perhaps the best Python feature that nobody 
> uses.)

This is awesome.

[toc] | [prev] | [next] | [standalone]

#53316

On Fri, 30 Aug 2013 06:35:47 -0400, Roy Smith wrote:

> In article <52200699$0$6599$c3e8da3$5496439d@news.astraweb.com>,
>  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> 
>> These days, it would be relatively simple to implement pre- and post-
>> condition checking using decorators, and indeed one of the motivating
>> use- cases for function annotations in Python 3 is to allow such
>> things.
>> 
>> http://www.python.org/dev/peps/pep-3107/
>> 
>> (Function annotations are perhaps the best Python feature that nobody
>> uses.)
> 
> This is awesome.


Heh, everybody has one of two reactions: 

"This is awesome!"

"You'll add type checking to my Python code over my dead body!!!"

But I'm still to see a practical use for annotations in real world code. 
Or indeed to think of a use for them other than type checking.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#53319

On 8/30/13 8:13 PM, Steven D'Aprano wrote:
> On Fri, 30 Aug 2013 06:35:47 -0400, Roy Smith wrote:
>
>> In article <52200699$0$6599$c3e8da3$5496439d@news.astraweb.com>,
>>   Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>>
>>> These days, it would be relatively simple to implement pre- and post-
>>> condition checking using decorators, and indeed one of the motivating
>>> use- cases for function annotations in Python 3 is to allow such
>>> things.
>>>
>>> http://www.python.org/dev/peps/pep-3107/
>>>
>>> (Function annotations are perhaps the best Python feature that nobody
>>> uses.)
>> This is awesome.
>
> Heh, everybody has one of two reactions:
>
> "This is awesome!"
>
> "You'll add type checking to my Python code over my dead body!!!"
>
> But I'm still to see a practical use for annotations in real world code.
> Or indeed to think of a use for them other than type checking.
>
>

At PyCon 2007 (I think), Guido was giving a keynote about the features 
coming in Py3k, and he couldn't remember the name "function 
annotations." He said, "what are they called, the things that aren't 
type declarations."

--Ned.

[toc] | [prev] | [next] | [standalone]

#53379

On 31 August 2013 01:13, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Fri, 30 Aug 2013 06:35:47 -0400, Roy Smith wrote:
>
>> In article <52200699$0$6599$c3e8da3$5496439d@news.astraweb.com>,
>>  Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
>>
>>> These days, it would be relatively simple to implement pre- and post-
>>> condition checking using decorators, and indeed one of the motivating
>>> use- cases for function annotations in Python 3 is to allow such
>>> things.
>>>
>>> http://www.python.org/dev/peps/pep-3107/
>>>
>>> (Function annotations are perhaps the best Python feature that nobody
>>> uses.)
>>
>> This is awesome.
>
> Heh, everybody has one of two reactions:
>
> "This is awesome!"
>
> "You'll add type checking to my Python code over my dead body!!!"
>
> But I'm still to see a practical use for annotations in real world code.
> Or indeed to think of a use for them other than type checking.

I occasionally use them for documentation. I think that there some are
cases where the return type (encoded as a string) is as good an indicator
of functionality as a short docstring, so use both.

[toc] | [prev] | [next] | [standalone]

#53390

In article <5221352b$0$6599$c3e8da3$5496439d@news.astraweb.com>,
 Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:

> Heh, everybody has one of two reactions: 
> 
> "This is awesome!" [[i.e. what I said]]
> 
> "You'll add type checking to my Python code over my dead body!!!"

Duck typing is a funny thing.  Sure, I don't have to give you a Duck, I 
just have to give you something that quacks like a Duck.  But, at some 
point, you and I need to agree on what that means.  If you're expecting 
a https://en.wikipedia.org/wiki/Duck and I give you a 
https://en.wikipedia.org/wiki/DUKW, we've had a failure to communicate.

To take a quasi-realistic example, let's say I've got this musical 
masterpiece in my database:

{
   "_id" : ObjectId("4ccb7052e5f37551d479add6"),
   "album" : "My World",
   "album_comp_id" : NumberLong(34732133),
   "artist" : "Justin Bieber",
   "avail_date_aac" : ISODate("1970-01-01T00:00:00Z"),
   "avail_date_mp3" : ISODate("1970-01-01T00:00:00Z"),
   "duration" : 192,
   "genre" : "Pop",
   "mn_comp_id" : NumberLong(34732147),
   "seq_num" : 1297,
   "song_id" : 6544798,
   "title" : "Love Me",
   "track" : -1
}

If I want to ask you, "Please return to me a url from which I can stream 
this song as an mp3", I could look at your Song class and find:

@staticmethod
def get_stream_url(song):
   [some code goes here]

but that would leave me wondering what you mean by "song".  You could 
legitimately mean 6544798, "4ccb7052e5f37551d479add6", 
ObjectId("4ccb7052e5f37551d479add6"), an instance of the Song class 
itself, or possibly even 34732147.  Depending on the context, any of 
those might very well be the right answer.

And, we haven't even gotten to describing what I should expect to get 
back.

So, to clear things up, you had to go and write something in the doc 
string:

@staticmethod
def get_stream_url(song):
   """Song is an instance of class Song.  This will return
   an absolute URL as a string."""

But, why not just embed that information in some way which is both 
compact and machine readable?

Of course, when I say, "Song is an instance of class Song", what I 
really (in full duck typing glory) mean is, "Song is something which has 
a "mn_comp_id" attribute whose value is something which I can pass to 
str() and get back a properly formatted decimal integer string.  So, 
this would work, wouldn't it?

class Platypus:
    def __getattr__(self, name):
        return 34732147

duck = Platypus()
Song.get_stream_url(duck)

Hey, it met the description you gave me, didn't it?  And, sure enough, 
if you do with duck what I expect you will, we will soon hear Justin 
Bieber's, "Love Me" coming out the speaker.  But, in reality, I suspect 
we would quickly get into an argument about just what exactly did you 
mean when you said "Duck".

[toc] | [prev] | [next] | [standalone]

#53244

On Thu, Aug 29, 2013 at 10:40 PM, Joe Junior <joe.fbs.junior@gmail.com> wrote:
> @ChrisA
>>Do you believe that you can write code to catch every bug you might
>>make? If so, you are naive and probably haven't spent much time
>>programming yet :) And if not, then you must acknowledge that bugs
>>WILL happen; therefore you will need to cope with them after the
>>event. So rather than trying to prevent them all, just improve your
>>means of coping, and you'll accomplish the same end with much less
>>trouble.
>
> Oh, no! I'm not that presumptuous (or naive)! But what do you mean by
> "improve means of coping"? Do you know any article on the subject you
> could point me?

Hmm. l don't know of any good articles off-hand. But what I'm talking
about is simply developing the skill of reading exceptions, plus a few
simple things like knowing where it's appropriate to catch-and-log;
sometimes, what that means is actually writing some code to (for
example) email you whenever there's an exception, but more likely it
means writing no code at all, and just looking at STDERR of your live
usage. Works really well for >95% of Python scripts.

The most important thing to consider is: What happens if my code
doesn't run all the way through? Is it safe for this to run part way,
then bomb with an exception? For many scripts, it's pretty easy: fix
the problem and rerun the script, and it'll completely rewrite its
output file. For others, this is a good reason for putting all your
"real data" into a transactional database - you begin a transaction at
the top, don't commit till the end, and if an exception kills your
script, your transaction will be rolled back. I have a system for
patching our database based on a script (written in Pike, not Python,
but the same applies); if I have any sort of critical failure in the
patch script, it'll bomb out as soon as I test it - but since I use
PostgreSQL, all that DDL (eg "ALTER TABLE") is covered by
transactional integrity (which it isn't with MySQL - another reason to
be wary of MySQL), so my patch will be backed out, and I can fix it
and start over. I don't need to have a Look Before You Leap approach
to database changes - I can simply do stuff, and if it crashes, all's
well. (That same script also has a system for catching errors at a
mid-level point that means that the process doesn't terminate when
there's an error; it supports full code reload, so once I fix the
patch, I send the process a SIGHUP and it fetches from disk again.)
*That* is error handling the safe way.

ChrisA

[toc] | [prev] | [next] | [standalone]

#53247

On 29 August 2013 10:07, Chris Angelico <rosuav@gmail.com> wrote:
> Hmm. l don't know of any good articles off-hand. But what I'm talking
> about is simply developing the skill of reading exceptions, plus a few
> simple things like knowing where it's appropriate to catch-and-log;
> sometimes, what that means is actually writing some code to (for
> example) email you whenever there's an exception, but more likely it
> means writing no code at all, and just looking at STDERR of your live
> usage. Works really well for >95% of Python scripts.
>
> The most important thing to consider is: What happens if my code
> doesn't run all the way through? Is it safe for this to run part way,
> then bomb with an exception? For many scripts, it's pretty easy: fix
> the problem and rerun the script, and it'll completely rewrite its
> output file. For others, this is a good reason for putting all your
> "real data" into a transactional database - you begin a transaction at
> the top, don't commit till the end, and if an exception kills your
> script, your transaction will be rolled back. I have a system for
> patching our database based on a script (written in Pike, not Python,
> but the same applies); if I have any sort of critical failure in the
> patch script, it'll bomb out as soon as I test it - but since I use
> PostgreSQL, all that DDL (eg "ALTER TABLE") is covered by
> transactional integrity (which it isn't with MySQL - another reason to
> be wary of MySQL), so my patch will be backed out, and I can fix it
> and start over. I don't need to have a Look Before You Leap approach
> to database changes - I can simply do stuff, and if it crashes, all's
> well. (That same script also has a system for catching errors at a
> mid-level point that means that the process doesn't terminate when
> there's an error; it supports full code reload, so once I fix the
> patch, I send the process a SIGHUP and it fetches from disk again.)
> *That* is error handling the safe way.
>

Oh, I get it! Thanks.

[toc] | [prev] | [next] | [standalone]

#53275

On Thursday, August 29, 2013 12:09:22 AM UTC+3, Joe Junior wrote:
> While designing a simple library, I found myself asking a
> 
> philosophical question: to check or not to check the parameter's
> 
> interface?
> 
Design by contract discipline says: do not.
> 
> 
> I think that, considering it is Python, the usual answer would be
> 
> "no", but here is the situation that got me thinking:
> 
> 
> 
> class Flock:
> 
> 
> 
>     def __init__(self):
> 
>         self.ducks= []
> 
> 
> 
>     def do_stuff(self):
> 
>         for duck in self.ducks:
> 
>             duck.quack()
> 
> 
> 
> class Duck:
> 
> 
> 
>     def quack(self):
> 
>         #quack-quack
> 
>         pass
> 
> 
> 
> f = Flock()
> 
> d = Duck()
> 
> f.ducks.append(d)
> 
> f.do_stuff()
> 
> 
> 
> Ok, no big deal there, the problem is if the user forgets to implement
> 
> the quack() method. The stack trace would complain that "duck.quack()"
> 
> is wrong, but that can happen hundreds of lines after the user
> 
> actually added the object to the Flock, and it can be hard to find out
> 
> what is happening and which object is wrong.
> 
> 
> 
> Of course I don't want to check isistance(), I like duck typing, but
> 
> should I check if hasattr() and callable() before adding to the
> 
> container? What is the pythonic way to deal with it? Am I worrying too
> 
> much ;-)?
> 
> 
> 
> Thanks,
> 
> 
> 
> Joe

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web