Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #12075 > unrolled thread

is there any principle when writing python function

Started bysmith jack <thinke365@gmail.com>
First post2011-08-23 19:59 +0800
Last post2011-08-27 23:51 -0500
Articles 20 on this page of 47 — 21 participants

Back to article view | Back to comp.lang.python


Contents

  is there any principle when writing python function smith jack <thinke365@gmail.com> - 2011-08-23 19:59 +0800
    Re: is there any principle when writing python function Peter Otten <__peter__@web.de> - 2011-08-23 14:20 +0200
      Re: is there any principle when writing python function Roy Smith <roy@panix.com> - 2011-08-23 08:56 -0400
    Re: is there any principle when writing python function Mel <mwilson@the-wire.com> - 2011-08-23 08:53 -0400
    Re: is there any principle when writing python function Roy Smith <roy@panix.com> - 2011-08-23 08:55 -0400
    Re: is there any principle when writing python function Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-08-23 15:00 +0200
    Re: is there any principle when writing python function Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-24 01:22 +1000
      Re: is there any principle when writing python function Terry Reedy <tjreedy@udel.edu> - 2011-08-23 14:29 -0400
        Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-23 13:22 -0700
        Re: is there any principle when writing python function Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-24 11:44 +1000
    Re: is there any principle when writing python function Seebs <usenet-nospam@seebs.net> - 2011-08-23 16:53 +0000
    Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-23 10:02 -0700
      Re: is there any principle when writing python function alex23 <wuwei23@gmail.com> - 2011-08-23 20:05 -0700
      Re: is there any principle when writing python function alex23 <wuwei23@gmail.com> - 2011-08-23 20:08 -0700
        Re: is there any principle when writing python function Red John <redjohn367@gmail.com> - 2011-08-24 16:29 -0700
    Re: is there any principle when writing python function ting@thsu.org - 2011-08-25 22:20 -0700
      Re: is there any principle when writing python function Roy Smith <roy@panix.com> - 2011-08-26 07:15 -0400
        Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-26 08:20 -0700
          Re: is there any principle when writing python function John Gordon <gordon@panix.com> - 2011-08-26 15:40 +0000
            Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-26 11:05 -0700
              Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-27 07:45 +1000
                Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-26 15:26 -0700
                Re: is there any principle when writing python function Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-27 11:26 +1000
                  Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-27 11:37 +1000
                Re: is there any principle when writing python function Roy Smith <roy@panix.com> - 2011-08-27 12:41 -0400
                  Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-28 02:57 +1000
                  Re: is there any principle when writing python function Emile van Sebille <emile@fenx.com> - 2011-08-27 10:27 -0700
                    Re: is there any principle when writing python function Ben Finney <ben+python@benfinney.id.au> - 2011-08-28 07:57 +1000
                      Re: is there any principle when writing python function Emile van Sebille <emile@fenx.com> - 2011-08-27 15:21 -0700
                        Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-27 16:01 -0700
                        Re: is there any principle when writing python function Roy Smith <roy@panix.com> - 2011-08-27 19:09 -0400
                      Re: is there any principle when writing python function Stephen Hansen <me+list/python@ixokai.io> - 2011-08-27 16:27 -0700
                  Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-28 03:31 +1000
                    Re: is there any principle when writing python function Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-28 06:27 +1000
                      Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-28 06:38 +1000
                      Re: is there any principle when writing python function Roy Smith <roy@panix.com> - 2011-08-27 17:09 -0400
          Re: is there any principle when writing python function Tobiah <tobiah@teranews.com> - 2011-08-26 08:48 -0700
            Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-27 02:10 +1000
              Re: is there any principle when writing python function Neil Cerutti <neilc@norwich.edu> - 2011-08-29 14:52 +0000
                Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-30 04:20 +1000
                  Re: is there any principle when writing python function Neil Cerutti <neilc@norwich.edu> - 2011-08-29 18:40 +0000
                    Re: is there any principle when writing python function Chris Angelico <rosuav@gmail.com> - 2011-08-30 05:02 +1000
                    For some value of “sing” (was: is there any principle when writing python function) Ben Finney <ben+python@benfinney.id.au> - 2011-08-30 08:17 +1000
                    Re: is there any principle when writing python function Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2011-08-29 23:20 -0700
            Re: is there any principle when writing python function Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-27 04:16 +1000
              Re: is there any principle when writing python function rantingrick <rantingrick@gmail.com> - 2011-08-26 15:37 -0700
    Re: is there any principle when writing python function harrismh777 <harmar@member.fsf.org> - 2011-08-27 23:51 -0500

Page 1 of 3  [1] 2 3  Next page →


#12075 — is there any principle when writing python function

Fromsmith jack <thinke365@gmail.com>
Date2011-08-23 19:59 +0800
Subjectis there any principle when writing python function
Message-ID<mailman.346.1314100765.27778.python-list@python.org>
i have heard that function invocation in python is expensive, but make
lots of functions are a good design habit in many other languages, so
is there any principle when writing python function?
for example, how many lines should form a function?

[toc] | [next] | [standalone]


#12076

FromPeter Otten <__peter__@web.de>
Date2011-08-23 14:20 +0200
Message-ID<j305uo$pmd$1@solani.org>
In reply to#12075
smith jack wrote:

> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?
> for example, how many lines should form a function?

Five ;)

[toc] | [prev] | [next] | [standalone]


#12079

FromRoy Smith <roy@panix.com>
Date2011-08-23 08:56 -0400
Message-ID<roy-B2DFE7.08565723082011@news.panix.com>
In reply to#12076
In article <j305uo$pmd$1@solani.org>, Peter Otten <__peter__@web.de> 
wrote:

> smith jack wrote:
> 
> > i have heard that function invocation in python is expensive, but make
> > lots of functions are a good design habit in many other languages, so
> > is there any principle when writing python function?
> > for example, how many lines should form a function?
> 
> Five ;)

Five is right out.

[toc] | [prev] | [next] | [standalone]


#12077

FromMel <mwilson@the-wire.com>
Date2011-08-23 08:53 -0400
Message-ID<j307ss$7hc$1@speranza.aioe.org>
In reply to#12075
smith jack wrote:

> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?

It's hard to discuss in the abstract.  A function should perform a 
recognizable step in solving the program's problem.  If you prepared to 
write your program by describing each of several operations the program 
would have to perform, then you might go on to plan a function for each of 
the described operations.  The high-level functions can then be analyzed, 
and will probably lead to functions of their own.

Test-driven development encourages smaller functions that give you a better 
granularity of testing.  Even so, the testable functions should each perform 
one meaningful step of a more general problem.

> for example, how many lines should form a function?
Maybe as few as one.

def increase (x, a):
    return x+a

is kind of stupid, but a more complicated line

def expand_template (bitwidth, defs):
	'''Turn Run-Length-Encoded list into bits.'''
	return np.array (sum (([bit]*(count*bitwidth) for count, bit in 
defs), []), np.int8)

is the epitome of intelligence.  I wrote it myself.  Even increase might be 
useful:

def increase (x, a):
    return x + a * application_dependent_quantity

`increase` has become a meaningful operation in the imaginary application 
we're discussing.


For an upper bound, it's harder to say.  If you read to the end of a 
function and can't remember how it started, or what it did in between, it's 
too big.  If you're reading on your favourite screen, and the end and the 
beginning are more than one page-scroll apart, it might be too big.  If it's 
too big, factoring it into sub-steps and making functions of some of those 
sub-steps is the fix.

	Mel.

[toc] | [prev] | [next] | [standalone]


#12078

FromRoy Smith <roy@panix.com>
Date2011-08-23 08:55 -0400
Message-ID<roy-DE4E2A.08551923082011@news.panix.com>
In reply to#12075
In article <mailman.346.1314100765.27778.python-list@python.org>,
 smith jack <thinke365@gmail.com> wrote:

> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?
> for example, how many lines should form a function?

Enough lines to do what the function needs to do, but no more.

Seriously, break up your program into functions based on logical 
groupings, and whatever makes your code easiest to understand.  When 
you're all done, if your program is too slow, run it under the profiler.  
Use the profiling results to indicate which parts need improvement.

It's very unlikely that function call overhead will be a significant 
issue.  Don't worry about stuff like that unless the profiler shows its 
a bottleneck.  Don't try to guess what's slow.  My guesses are almost 
always wrong.  Yours will be too.

If your program runs fast enough as it is, don't even bother with the 
profiler.  Be happy that you've got something useful and move on to the 
next thing you've got to do.

[toc] | [prev] | [next] | [standalone]


#12080

FromUlrich Eckhardt <ulrich.eckhardt@dominolaser.com>
Date2011-08-23 15:00 +0200
Message-ID<m9kci8-ob6.ln1@satorlaser.homedns.org>
In reply to#12075
smith jack wrote:
> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?
> for example, how many lines should form a function?

Don't compromise the design and clarity of your code just because you heard 
some rumors about performance. Also, for any performance question, please 
consult a profiler.

Uli

-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

[toc] | [prev] | [next] | [standalone]


#12089

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2011-08-24 01:22 +1000
Message-ID<4e53c5ab$0$29996$c3e8da3$5496439d@news.astraweb.com>
In reply to#12075
smith jack wrote:

> i have heard that function invocation in python is expensive, 

It's expensive, but not *that* expensive. Compare:

[steve@sylar ~]$ python3.2 -m timeit 'x = "abc".upper()'
1000000 loops, best of 3: 0.31 usec per loop
[steve@sylar ~]$ python3.2 -m timeit -s 'def f():
return "abc".upper()' 'f()'
1000000 loops, best of 3: 0.53 usec per loop

So the function call is nearly as expensive as this (very simple!) sample
code. But in absolute terms, that's not very expensive at all. If we make
the code more expensive:

[steve@sylar ~]$ python3.2 -m timeit '("abc"*1000)[2:995].upper().lower()'
10000 loops, best of 3: 32.3 usec per loop
[steve@sylar ~]$ python3.2 -m timeit -s 'def f(): return ("abc"*1000
[2:995].upper().lower()' 'f()'
10000 loops, best of 3: 33.9 usec per loop

the function call overhead becomes trivial.

Cases where function call overhead is significant are rare. Not vanishingly
rare, but rare enough that you shouldn't worry about them.


> but make 
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?
> for example, how many lines should form a function?

About as long as a piece of string.

A more serious answer: it should be exactly as long as needed to do the
smallest amount of work that makes up one action, and no longer or shorter.

If you want to maximise the programmer's efficiency, a single function
should be short enough to keep the whole thing in your short-term memory at
once. This means it should consist of no more than seven, plus or minus
two, chunks of code. A chunk may be a single line, or a few lines that
together make up a unit, or if the lines are particularly complex, *less*
than a line.

http://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two
http://www.codinghorror.com/blog/2006/08/the-magical-number-seven-plus-or-minus-two.html

(Don't be put off by the use of the term "magical" -- there's nothing
literally magical about this. It's just a side-effect of the way human
cognition works.)

Anything longer than 7±2 chunks, and you will find yourself having to scroll
backwards and forwards through the function, swapping information into your
short-term memory, in order to understand it.

Even 7±2 is probably excessive: I find that I'm most comfortable with
functions that perform 4±1 chunks of work. An example from one of my
classes:

    def find(self, prefix):
        """Find the item that matches prefix."""
        prefix = prefix.lower()  # Chunk #1
        menu = self._cleaned_menu  # Chunk #2
        for i,s in enumerate(menu, 1):  # Chunk #3
            if s.lower().startswith(prefix):
                return i
        return None  # Chunk #4

So that's three one-line chunks and one three-line chunk.



-- 
Steven

[toc] | [prev] | [next] | [standalone]


#12111

FromTerry Reedy <tjreedy@udel.edu>
Date2011-08-23 14:29 -0400
Message-ID<mailman.365.1314124224.27778.python-list@python.org>
In reply to#12089
On 8/23/2011 11:22 AM, Steven D'Aprano wrote:

> Even 7±2 is probably excessive: I find that I'm most comfortable with
> functions that perform 4±1 chunks of work. An example from one of my
> classes:
>
>      def find(self, prefix):
>          """Find the item that matches prefix."""
>          prefix = prefix.lower()  # Chunk #1
>          menu = self._cleaned_menu  # Chunk #2
>          for i,s in enumerate(menu, 1):  # Chunk #3
>              if s.lower().startswith(prefix):
>                  return i
>          return None  # Chunk #4
>
> So that's three one-line chunks and one three-line chunk.

In terms of different functions performed (see my previous post), I see
   attribute lookup
   assignment
   enumerate
   sequence unpacking
   for-looping
   if-conditioning
   lower
   startswith
   return
That is 9,  which is enough.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]


#12114

Fromrantingrick <rantingrick@gmail.com>
Date2011-08-23 13:22 -0700
Message-ID<b0e51032-b476-40d4-aa50-bab0866a1786@t9g2000vbs.googlegroups.com>
In reply to#12111
On Aug 23, 1:29 pm, Terry Reedy <tjre...@udel.edu> wrote:

> In terms of different functions performed (see my previous post), I see
>    attribute lookup
>    assignment
>    enumerate
>    sequence unpacking
>    for-looping
>    if-conditioning
>    lower
>    startswith
>    return
> That is 9,  which is enough.


attribute lookup -> inspection
assignment -> ditto
enumerate -> enumeration
sequence unpacking -> parallel assignment
for-looping -> cycling
if-conditioning -> logic
lower -> mutation (don't try to argue!)
startswith -> boolean-logic
return -> exiting (although all exits require an entrance!)
omitted: documenting, referencing, -presumptuousness-

pedantic-ly yours, rr
;-)

[toc] | [prev] | [next] | [standalone]


#12126

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2011-08-24 11:44 +1000
Message-ID<4e54576e$0$29980$c3e8da3$5496439d@news.astraweb.com>
In reply to#12111
Terry Reedy wrote:

> On 8/23/2011 11:22 AM, Steven D'Aprano wrote:
> 
>> Even 7±2 is probably excessive: I find that I'm most comfortable with
>> functions that perform 4±1 chunks of work. An example from one of my
>> classes:
>>
>>      def find(self, prefix):
>>          """Find the item that matches prefix."""
>>          prefix = prefix.lower()  # Chunk #1
>>          menu = self._cleaned_menu  # Chunk #2
>>          for i,s in enumerate(menu, 1):  # Chunk #3
>>              if s.lower().startswith(prefix):
>>                  return i
>>          return None  # Chunk #4
>>
>> So that's three one-line chunks and one three-line chunk.
> 
> In terms of different functions performed (see my previous post), I see
>    attribute lookup
>    assignment
>    enumerate
>    sequence unpacking
>    for-looping
>    if-conditioning
>    lower
>    startswith
>    return
> That is 9,  which is enough.


I think we have broad agreement, but we're counting different things.
Analogy: you're counting atoms, I'm grouping atoms into molecules and
counting them.

It's a little like phone numbers: it's not an accident that we normally
group phone numbers into groups of 2-4 digits:

011 23 4567 8901

In general, people can more easily memorise four chunks of four digits (give
or take) than one chunk of 13 digits: 0112345678901.



-- 
Steven

[toc] | [prev] | [next] | [standalone]


#12106

FromSeebs <usenet-nospam@seebs.net>
Date2011-08-23 16:53 +0000
Message-ID<slrnj57lb8.2s52.usenet-nospam@guild.seebs.net>
In reply to#12075
On 2011-08-23, smith jack <thinke365@gmail.com> wrote:
> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?

Lots of them.  None of them have to do with performance.

> for example, how many lines should form a function?

Between zero (which has to be written "pass") and a few hundred.  Usually
closer to the lower end of that range.  Occasionally outside it.

Which is to say:  This is the wrong question.

Let us give you the two laws of software optimization.

Law #1:  Don't do it.

If you try to optimize stuff, you will waste a ton of time doing things that,
it turns out, are unimportant.

Law #2: (Experts only.)  Don't do it yet.

You don't know enough to "optimize" this yet.

Write something that does what it is supposed to do and which you understand
clearly.  See how it looks.  If it looks like it is running well enough,
STOP.  You are done.

Now, if it is too slow, and you are running it on real data, NOW it is time
to think about why it is slow.  And the solution there is not to read abstract
theories about your language, but to profile it -- actually time execution and
find out where the time goes.

I've been writing code, and making it faster, for some longish period of time.
I have not yet ever in any language found cause to worry about function call
overhead.

-s
-- 
Copyright 2011, all wrongs reversed.  Peter Seebach / usenet-nospam@seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
I am not speaking for my employer, although they do rent some of my opinions.

[toc] | [prev] | [next] | [standalone]


#12107

Fromrantingrick <rantingrick@gmail.com>
Date2011-08-23 10:02 -0700
Message-ID<1ba7b65b-04b5-4b90-899a-f1a152d56023@fe21g2000vbb.googlegroups.com>
In reply to#12075
On Aug 23, 6:59 am, smith jack <thinke...@gmail.com> wrote:
> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?
> for example, how many lines should form a function?

Everyone here who is suggesting that function bodies should be
confined to ANY length is an idiot. The length of a functions code
block is inconsequential. Don't worry if it too small or too big. It's
not the size that matters, it's the motion of the sources ocean!

A good function can be one line, or a hundred lines. Always use
comments to clarify code and NEVER EVER create more functions only for
the sake of short function bodies, WHY, because all you do is move
confusion OUT OF the function body and INTO the module/class body.

"""Energy can neither be created nor be destroyed: it can only be
transformed from one state to another"""

http://en.wikipedia.org/wiki/Conservation_of_energy
https://sites.google.com/site/thefutureofpython/

[toc] | [prev] | [next] | [standalone]


#12128

Fromalex23 <wuwei23@gmail.com>
Date2011-08-23 20:05 -0700
Message-ID<47c4c2ac-a6ea-4575-ba4f-473ca8ba4c6e@x21g2000prd.googlegroups.com>
In reply to#12107
rantingrick <rantingr...@gmail.com> wrote:
> Everyone here who is suggesting that function bodies should be
> confined to ANY length is an idiot.

Or, more likely, is the sort of coder who has worked with other coders
in the past and understands the value of readable code.

> Don't worry if it too small or too big. It's
> not the size that matters, it's the motion of the sources ocean!

If only you spent as much time actually thinking about what you're
saying as trying to find 'clever' ways to say it...

> Always use
> comments to clarify code and NEVER EVER create more functions only for
> the sake of short function bodies

This is quite likely the worst advice you've ever given. I can only
assume you've never had to refactor the sort of code you're advocating
here.

[toc] | [prev] | [next] | [standalone]


#12129

Fromalex23 <wuwei23@gmail.com>
Date2011-08-23 20:08 -0700
Message-ID<e4e6ef96-e5b1-43de-af3f-dfbcf1f8a2b3@g8g2000prn.googlegroups.com>
In reply to#12107
rantingrick <rantingr...@gmail.com> wrote:
> https://sites.google.com/site/thefutureofpython/

"Very soon I will be hashing out a specification for python 4000."

AHAHAHAHAhahahahahahahAHAHAHAHahahahahaaaaaaa. So rich. Anyone willing
to bet serious money we won't see this before 4000AD?

"Heck even our leader seems as a captain too drunk with vanity to
care; and our members like a ship lost at sea left to sport of every
troll-ish wind!"

Quite frankly, you're a condescending, arrogant blow-hard that this
community would be better off without.

"We must constantly strive to remove multiplicity from our systems;
lest it consumes us!"

s/multiplicity/rantingrick/ and I'm in full agreement.

[toc] | [prev] | [next] | [standalone]


#12165

FromRed John <redjohn367@gmail.com>
Date2011-08-24 16:29 -0700
Message-ID<d6fcf365-3e4f-410b-be8f-92ae8c112064@j5g2000vbk.googlegroups.com>
In reply to#12129
> "We must constantly strive to remove multiplicity from our systems;
> lest it consumes us!"
>
> s/multiplicity/rantingrick/ and I'm in full agreement.

QFT

[toc] | [prev] | [next] | [standalone]


#12202

Fromting@thsu.org
Date2011-08-25 22:20 -0700
Message-ID<c2fe3168-92b1-46a1-a176-0914f0ba9579@19g2000vbv.googlegroups.com>
In reply to#12075
On Aug 23, 7:59 am, smith jack <thinke...@gmail.com> wrote:
> i have heard that function invocation in python is expensive, but make
> lots of functions are a good design habit in many other languages, so
> is there any principle when writing python function?
> for example, how many lines should form a function?

My suggestion is to think how you would test the function, in order to
get 100% code coverage. The parts of the function that are difficult
to test, those are the parts that you want to pull out into their own
separate function.

For example, a block of code within a conditional statement, where the
test condition cannot be passed in, is a prime example of a block of
code that should be pulled out into a separate function.

Obviously, there are times where this is not practical - exception
handling comes to mind - but that should be your rule of thumb. If a
block of code is hard to test, pull it out into it's own function, so
that it's easier to test.
--
// T.Hsu

[toc] | [prev] | [next] | [standalone]


#12210

FromRoy Smith <roy@panix.com>
Date2011-08-26 07:15 -0400
Message-ID<roy-957AF6.07155226082011@news.panix.com>
In reply to#12202
In article 
<c2fe3168-92b1-46a1-a176-0914f0ba9579@19g2000vbv.googlegroups.com>,
 ting@thsu.org wrote:

> On Aug 23, 7:59 am, smith jack <thinke...@gmail.com> wrote:
> > i have heard that function invocation in python is expensive, but make
> > lots of functions are a good design habit in many other languages, so
> > is there any principle when writing python function?
> > for example, how many lines should form a function?
> 
> My suggestion is to think how you would test the function, in order to
> get 100% code coverage.

I'm not convinced 100% code coverage is an achievable goal for any major 
project.  I was once involved in a serious code coverage program.  We 
had a large body of code (100's of KLOC of C++) which we were licensing 
to somebody else.  The customer was insisting that we do code coverage 
testing and set a standard of something like 80% coverage.

There was a dedicated team of about 4 people working on this for the 
better part of a year.  They never came close to 80%.  More like 60%, 
and that was after radical surgery to eliminate dead code and branches 
that couldn't be reached.  The hard parts are testing the code that 
deals with unusual error conditions caused by interfaces to the external 
world.

The problem is, it's just damn hard to simulate all the different kinds 
of errors that can occur.  This was network intensive code.  Every call 
that touches the network can fail in all sorts of ways that are near 
impossible to simulate.  We also had lots of code that tried to deal 
with memory exhaustion.  Again, that's hard to simulate.

I'm not saying code coverage testing is a bad thing.  Many of the issues 
I mention above could have been solved with additional abstraction 
layers, but that adds complexity of its own.  Certainly, designing a 
body of code to be testable from the get-go is a far superior to trying 
to retrofit tests to an existing code base (which is what we were doing).

> The parts of the function that are difficult
> to test, those are the parts that you want to pull out into their own
> separate function.
> 
> For example, a block of code within a conditional statement, where the
> test condition cannot be passed in, is a prime example of a block of
> code that should be pulled out into a separate function.

Maybe.  In general, it's certainly true that a bunch of smallish 
functions, each of which performs exactly one job, is easier to work 
with than a huge ball of spaghetti code.  On the other hand, interfaces 
are a common cause of bugs.  When you pull a hunk of code out into its 
own function, you create a new interface.  Sometimes that adds 
complexity (and bugs) of its own.

> Obviously, there are times where this is not practical - exception
> handling comes to mind - but that should be your rule of thumb. If a
> block of code is hard to test, pull it out into it's own function, so
> that it's easier to test.

In general, that's good advice.  You'll also usually find that code 
which is easy to test is also easy to understand and easy to modify.

[toc] | [prev] | [next] | [standalone]


#12221

Fromrantingrick <rantingrick@gmail.com>
Date2011-08-26 08:20 -0700
Message-ID<7b47ca17-d3f1-4d91-91d1-98421e8708cd@ea4g2000vbb.googlegroups.com>
In reply to#12210
On Aug 26, 6:15 am, Roy Smith <r...@panix.com> wrote:

> Maybe.  In general, it's certainly true that a bunch of smallish
> functions, each of which performs exactly one job, is easier to work
> with than a huge ball of spaghetti code.  

Obviously you need to google the definition of "spaghetti code". When
you move code out of one function and create another function you are
contributing to the "spaghetti-ness" of the code. Think of plate of
spaghetti and how the noodles are all intertwined and without order.
Likewise when you go to one function and have to follow the trial of
one or more helper functions you are creating a twisting and unordered
progression of code -- sniff-sniff, do you smell what i smell?

Furthermore: If you are moving code out of one function to ONLY be
called by that ONE function then you are a bad programmer and should
have your editor taken away for six months. You should ONLY create
more func/methods if those func/methods will be called from two or
more places in the code. The very essence of func/meths is the fact
that they are reusable.

It might still be spaghetti under that definition (of which ALL OOP
code actually is!) however it will be as elegant as spaghetti can be.

> On the other hand, interfaces
> are a common cause of bugs.  When you pull a hunk of code out into its
> own function, you create a new interface.  Sometimes that adds
> complexity (and bugs) of its own.

Which is it? You cannot have it both ways. You're straddling the fence
here like a dirty politician. Yes, this subject IS black and white!

[toc] | [prev] | [next] | [standalone]


#12225

FromJohn Gordon <gordon@panix.com>
Date2011-08-26 15:40 +0000
Message-ID<j38epo$t57$1@reader1.panix.com>
In reply to#12221
In <7b47ca17-d3f1-4d91-91d1-98421e8708cd@ea4g2000vbb.googlegroups.com> rantingrick <rantingrick@gmail.com> writes:

> Furthermore: If you are moving code out of one function to ONLY be
> called by that ONE function then you are a bad programmer and should
> have your editor taken away for six months. You should ONLY create
> more func/methods if those func/methods will be called from two or
> more places in the code. The very essence of func/meths is the fact
> that they are reusable.

That's one very important aspect of functions, yes.  But there's another:
abstraction.

If I'm writing a module that needs to fetch user details from an LDAP
server, it might be worthwhile to put all of the LDAP-specific code in
its own method, even if it's only used once.  That way the main module
can just contain a line like this:

  user_info = get_ldap_results("cn=john gordon,ou=people,dc=company,dc=com")

The main module keeps a high level of abstraction instead of descending
into dozens or even hundreds of lines of LDAP-specific code.

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon@panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"

[toc] | [prev] | [next] | [standalone]


#12238

Fromrantingrick <rantingrick@gmail.com>
Date2011-08-26 11:05 -0700
Message-ID<ac67bf41-0543-4d13-a015-8b5d2baeb4f8@t7g2000vbv.googlegroups.com>
In reply to#12225
On Aug 26, 10:40 am, John Gordon <gor...@panix.com> wrote:
> In <7b47ca17-d3f1-4d91-91d1-98421e870...@ea4g2000vbb.googlegroups.com> rantingrick <rantingr...@gmail.com> writes:
>
> > Furthermore: If you are moving code out of one function to ONLY be
> > called by that ONE function then you are a bad programmer and should
> > have your editor taken away for six months. You should ONLY create
> > more func/methods if those func/methods will be called from two or
> > more places in the code. The very essence of func/meths is the fact
> > that they are reusable.
>
> That's one very important aspect of functions, yes.  But there's another:
> abstraction.
> [...]
> The main module keeps a high level of abstraction instead of descending
> into dozens or even hundreds of lines of LDAP-specific code.

Exactly. I am not arguing against creating intuitive and
simplistically elegant interfaces. I mean, lists *could* have only one
method called apply(process, *args, **kw) which takes an argument like
("append", "value") or ("index", 42) and has a long block of logic to
handle the inputs however that would be a horrible interface.

So in that respect i agree. We must weigh the entire interface from an
empirical perspective. However i can be sure of one point: As you
increase the number of methods you also increase the mental load
required to understand that particular interface.

An interface with a small number of methods will not suffer too
terribly from one or two extra methods however at some point more
methods just equals more confusion. It is a delicate balancing act
that many programmers are not agile enough to juggle elegantly.

Take for instance the interface for Grep, Search, and Replace dialogs
in the idlelib which span two separate modules and have a mind numbing
number of methods for such remedial things as creating buttons and
entrys. All three dialogs look very similar and share many
similarities.

Now take a look at MY simple ONE module solution. It has JUST enough
methods and NOT a single more! Yes the create widgets method is fairly
long (weighing in at 80+ lines with comments!) however all of this
code needs to be contained in ONE and ONLY one method. Heck if i
wanted to get pendantic i could replace the five cb_*() methods with
partials however MY interface is so intuitive there is no need.

############################################################
# START CODE
############################################################
class FindReplaceDialog(object):
    def __init__(self, textbox):
        [...]

    def create_widgets(self, type_):
        # Create toplevel dialog window.
        [...]
        # Create widgets belonging to both
        # search AND replace dialogs dialogs.
        [...]
        if type_ == 'replace':
            # Add widgets unique to replace
            # dialogs.
            [...]
        elif type_ == 'grep':
            # Add widgets unique to grep
            # dialogs.
            [...]
        # Load any initial values and states.
        [...]

    def show(self, type_='find'):
        self.create_widgets(type_)
        # Do any initial setup.

    def close(self, event=None):
        # destroy the dialog.

    def find_again(self, event=None):
        # Event callback bound to textbox.

    def find(self, target):
        # Search the buffer for target and
        # hilight if found.

    def replace(self, action='replace'):
        # Fetch the old and new strings and
        # mediate the work depending on the
        # action.
        [...]
        if action == 'replace+find':
            [...]
        elif action == 'replaceall':
            [...]

    def grep():
        [...]

    def cb_grepbutton(self, event=None):
        self.grep(target.entry.get())

    def cb_findbutton(self, event=None):
        self.find(target.entry.get())

    def cb_replacebutton(self):
        self.replace(action='replace')

    def cb_replacefindbutton(self):
        self.replace(action='replace+find')

    def cb_replaceallbutton(self):
        self.replace(action='replaceall')
############################################################
# END CODE
############################################################

Now look at the three modules in idlelib (Grep Dialog, Search Dialog,
and Replace Dialog) and ask yourself which is cleaner? Which is more
intuiitve? Which is more professional? Which would you rather debug?

*scholl-bell-rings*

[toc] | [prev] | [next] | [standalone]


Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | comp.lang.python


csiph-web