Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #19379 > unrolled thread

Distributing methods of a class across multiple files

Back to article view | Back to comp.lang.python

Contents

  Distributing methods of a class across multiple files lh <lhughes42@gmail.com> - 2012-01-24 19:54 -0800
    Re: Distributing methods of a class across multiple files Roy Smith <roy@panix.com> - 2012-01-24 23:05 -0500
    Re: Distributing methods of a class across multiple files Cameron Simpson <cs@zip.com.au> - 2012-01-25 17:15 +1100
    Re: Distributing methods of a class across multiple files "Frank Millman" <frank@chagford.com> - 2012-01-25 10:26 +0200
    Re: Distributing methods of a class across multiple files Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-01-25 11:10 +0000
    Re: Distributing methods of a class across multiple files Jean-Michel Pichavant <jeanmichel@sequans.com> - 2012-01-25 12:49 +0100
      Re: Distributing methods of a class across multiple files lh <lhughes42@gmail.com> - 2012-01-25 07:19 -0800
        Re: Distributing methods of a class across multiple files Neil Cerutti <neilc@norwich.edu> - 2012-01-25 15:42 +0000
        Re: Distributing methods of a class across multiple files Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-01-25 11:53 -0500
          Re: Distributing methods of a class across multiple files Roy Smith <roy@panix.com> - 2012-01-26 09:11 -0500
            Re: Distributing methods of a class across multiple files Chris Angelico <rosuav@gmail.com> - 2012-01-27 01:41 +1100
        Re: Distributing methods of a class across multiple files Chris Angelico <rosuav@gmail.com> - 2012-01-26 20:25 +1100

#19379 — Distributing methods of a class across multiple files

Is this possible please?  I have done some searching but it is hard to
narrow down Google searches to this question. What I would like to do
is, for example:
1) define a class Foo in file test.py... give it some methods
2) define a file test2.py which contains a set of methods that are
methods of class Foo defined in test.py.  I can import Foo obviously
but it isn't clear to me how to identify the methods in test2.py to be
methods of class Foo defined in test.py (normally I would just indent
them "def"'s under the class but the class isn't textually in
test2.py).

In short I would like to distribute code for one class across multiple
files so a given file doesn't get ridiculously long.

Thank you,
 Luke

[toc] | [next] | [standalone]

#19381

In article 
<569a94a3-cd84-449b-b0c1-80348014aac6@i10g2000pbl.googlegroups.com>,
 lh <lhughes42@gmail.com> wrote:

> Is this possible please?  I have done some searching but it is hard to
> narrow down Google searches to this question. What I would like to do
> is, for example:
> 1) define a class Foo in file test.py... give it some methods
> 2) define a file test2.py which contains a set of methods that are
> methods of class Foo defined in test.py.  I can import Foo obviously
> but it isn't clear to me how to identify the methods in test2.py to be
> methods of class Foo defined in test.py (normally I would just indent
> them "def"'s under the class but the class isn't textually in
> test2.py).
> 
> In short I would like to distribute code for one class across multiple
> files so a given file doesn't get ridiculously long.

The student asks the master, "How long is a file?"

The master replies, "Just long enough to hold its contents".

"But, what if my class is ridiculously long?  Won't the file that 
contains it also be ridiculously long?"

"Just as one cannot count the grains of sand on a beach, neither can one 
count the number of kilobytes one can buy for a few cents at your local 
office supply store"

And the student was enlightened and stopped thinking the whole world was 
Java or C++.

[toc] | [prev] | [next] | [standalone]

#19385

On 24Jan2012 19:54, lh <lhughes42@gmail.com> wrote:
| Is this possible please?  I have done some searching but it is hard to
| narrow down Google searches to this question. What I would like to do
| is, for example:
| 1) define a class Foo in file test.py... give it some methods
| 2) define a file test2.py which contains a set of methods that are
| methods of class Foo defined in test.py.  I can import Foo obviously
| but it isn't clear to me how to identify the methods in test2.py to be
| methods of class Foo defined in test.py (normally I would just indent
| them "def"'s under the class but the class isn't textually in
| test2.py).
| 
| In short I would like to distribute code for one class across multiple
| files so a given file doesn't get ridiculously long.

You may need to define "ridiculously long". What's your objecttion to
a long file? What specific difficulties does it cause? I'm not arguing
here that all programs should be in a single monolithic file, just that
breaking things up just on size is a rather arbitrary basis.

If methods supply a particular type of functionality you can write a
mixin class, like this:

  from fooey import FooeyClass
  from bloohey import BlooheyCLass

  class Foo(object, FooeyClass, BlooheyClass):
    ... core methods of Foo here ...

and then put methods involving Fooeyness in the class definition of
FooeyClass and so forth. But unless you intend to reuse FooeyClass to
augument other classes or FooeyClass really is a well defined standalone
piece of functionality, this is probably more contortation than it is
worth.

Cheers,
-- 
Cameron Simpson <cs@zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Hello, my name is Yog-Sothoth, and I'll be your eldritch horror today.
        - Heather Keith <hkeith+@andrew.cmu.edu>

[toc] | [prev] | [next] | [standalone]

#19388

"lh" <lhughes42@gmail.com> wrote:
> Is this possible please?  I have done some searching but it is hard to
> narrow down Google searches to this question. What I would like to do
> is, for example:
> 1) define a class Foo in file test.py... give it some methods
> 2) define a file test2.py which contains a set of methods that are
> methods of class Foo defined in test.py.  I can import Foo obviously
> but it isn't clear to me how to identify the methods in test2.py to be
> methods of class Foo defined in test.py (normally I would just indent
> them "def"'s under the class but the class isn't textually in
> test2.py).
>
> In short I would like to distribute code for one class across multiple
> files so a given file doesn't get ridiculously long.
>

I take the point of the other responders that it is not a normal thing to 
do, but I had a few long but rarely used methods which I wanted to move out 
of the main file just to keep the main file tidier. I came up with this 
solution, and it seems to work.

In test2.py -

    def long_method_1():
        pass

    def long_method2():
        pass

In test.py -

    import test2

    class Foo:
        long_method_1 = test2.long_method_1
        long_method_2 = test2.long_method_2

Then in Foo I can refer to self.long_method_1().

HTH

Frank Millman

[toc] | [prev] | [next] | [standalone]

#19394

On Tue, 24 Jan 2012 19:54:24 -0800, lh wrote:

> Is this possible please?  I have done some searching but it is hard to
> narrow down Google searches to this question. What I would like to do
> is, for example:
> 1) define a class Foo in file test.py... give it some methods 
> 2) define a file test2.py which contains a set of methods that are
> methods of class Foo defined in test.py.

Technically, yes, this is possible, but you shouldn't need to do it. 
Needing to split a single class across multiple files is a sign of bad 
design. If the class is that huge, then it probably does too many things 
and should be broken up into multiple classes and then reassembled using 
composition.



[...]
> In short I would like to distribute code for one class across multiple
> files so a given file doesn't get ridiculously long.

What do you call "ridiculously long"? 

One of the largest modules in the Python standard library is decimal. I 
consider decimal to be about as big as a single module should get: over 
5000 lines of code. Any larger, and you should consider splitting it into 
a package with multiple files. 

But note that those 5000 lines include over 500 lines of comments, 
details documentation, plenty of blank lines, 14 public classes, 3 public 
functions, and at least 17 private functions or classes. The main class, 
decimal.Decimal, is about 2800 lines of code.

If your class is smaller than that, I don't think you need to worry about 
splitting it.

But let's suppose you really do have a good reason to split the class 
into multiple files. And not just "because that's how I'd do it in Java".

Suppose you have a class Spam, and it has two methods, spam() and eggs(). 
You want to split the methods into different files. (Perhaps you want to 
win a bet.) There are three main possibilities:

(1) Inheritance.

(2) Composition or delegation.

(3) Dynamic code injection.


Let's start with inheritance.

In module a.py, create a class:

class EggMixin:
    def eggs(self):
        c = self.colour
        print("%s eggs go well with %s spam" % (c, c))

Notice that as a mixin, EggsMixin isn't required to provide the 
self.colour attribute.

Technically it isn't necessary for this to be a mixin, but it is probably 
the best design.

Now in module main.py, create the Spam class you really want, using 
multiple inheritance to inherit from the "real" superclass and the mixin:

import a
class Spam(SpamParent, a.EggMixin):
    def __init__(self, colour='green'):
        print("Spam spam spam LOVELY SPAM!!!")
        self.colour = colour
    def spam(self, n):
        return "spam!"*n


By the way, SpamParent is optional. If you don't need it, just leave it 
out.


Now, on to composition. First, let's redesign the egg class in a.py:

class Egg:
    def __init__(self, owner):
        self.owner = owner
    def eggs(self):
        c = self.owner.colour
        print("%s eggs go well with %s spam" % (c, c))


And in main.py:

import a
class Spam(object):
    def __init__(self, colour='green'):
        print("Spam spam spam LOVELY SPAM!!!")
        self.colour = colour
        # Create an Egg that has this Spam instance as the owner.
        egg = a.Egg(owner=self)
        # And store it for later use.
        self._egg = egg
    def spam(self, n):
        return "spam!"*n
    def eggs(self):
        # Delegate to the saved egg.
        return self._egg.eggs()


Last but not least, lets try dynamic code injection. In a.py, create a 
function using a placeholder self parameter:


def eggs(self):
    c = self.colour
    print("%s eggs go well with %s spam" % (c, c))
    

And here is your main module:


import a
class Spam(object):
    def __init__(self, colour='green'):
        print("Spam spam spam LOVELY SPAM!!!")
        self.colour = colour
    def spam(self, n):
        return "spam!"*n

# Add the extra method that we want.
Spam.eggs = a.eggs


So now you have three ways of doing something that shouldn't be done :)


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#19396

lh wrote:
> Is this possible please?  I have done some searching but it is hard to
> narrow down Google searches to this question. What I would like to do
> is, for example:
> 1) define a class Foo in file test.py... give it some methods
> 2) define a file test2.py which contains a set of methods that are
> methods of class Foo defined in test.py.  I can import Foo obviously
> but it isn't clear to me how to identify the methods in test2.py to be
> methods of class Foo defined in test.py (normally I would just indent
> them "def"'s under the class but the class isn't textually in
> test2.py).
>
> In short I would like to distribute code for one class across multiple
> files so a given file doesn't get ridiculously long.
>
> Thank you,
>  Luke
>   

If the file is ridiculously long, could be that the class has a 
ridiculous number of methods. If you spread your class into multiple 
files, you will have a ridiculous number of tiny files with ridiculous 
names.

My 2 cents : keep 1 class in 1 file, and keep your scopes consistent.

JM

[toc] | [prev] | [next] | [standalone]

#19401

First, thanks for all the thoughtful replies. I am grateful.
Second, I figured I'd get a lot of judgement about how I really
shouldn't be doing this. Should have pre-empted it :-) oh well. There
is a place IMHO for filename as another structuring element to help
humans in search. Also it can be convenient to have two people who are
working on methods for one class that have different directions to
have different files (even if modern tools can handle distinct edits
on the same file by multiple folks).
Third, length. Well 5000 lines eh... I'm nowhere near that guess I can
stick with one file.
 Steven thanks especially for your thorough reply about alternatives.
Again I really appreciate the quality replies made here. very
impressed. I had not thought of the mixin approach in particular. I
was just hoping there was something like "extend class Foo" I could
put at the top of the file and then keep writing methods indented
below it. But I will think about some of these alternatives...

Humbly,
  Luke

[toc] | [prev] | [next] | [standalone]

#19402

On 2012-01-25, lh <lhughes42@gmail.com> wrote:
> First, thanks for all the thoughtful replies. I am grateful.
> Second, I figured I'd get a lot of judgement about how I really
> shouldn't be doing this. Should have pre-empted it :-) oh well.
> There is a place IMHO for filename as another structuring
> element to help humans in search. Also it can be convenient to
> have two people who are working on methods for one class that
> have different directions to have different files (even if
> modern tools can handle distinct edits on the same file by
> multiple folks).

Of the three solutions Steven presented, the latter two leave
very strong coupling between the code in your separate files.
This makes working with the files independently impractical.
Stick with mixin classes and pay heed to the Law of Demeter if
you want to de-couple them enough to work on independently.

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#19405

On Wed, 25 Jan 2012 07:19:16 -0800 (PST), lh <lhughes42@gmail.com>
wrote:

>Third, length. Well 5000 lines eh... I'm nowhere near that guess I can
>stick with one file.

	Original Pascal (primarily a teaching language) did not support
separate compilation nor "include" files. Everything was done within one
source file. (Often with a one-pass recursive descent parser)

	The old convention I'd learned was to keep functions down to a
(printer) page (classical 6 lines per inch, 11" high, tractor feed -- so
about 60 lines per function -- possibly extend to a second page if
really needed.

	I'd think even a large complex class definition could fit each
method into something of that size. Okay, initialization may be long
(especially for GUI layouts), but the operations shouldn't be that
complex (in length; doing an orbit propagation with a J2 geopotential
model may be complex numerically, but as I recall it can fit into less
than two pages of text -- in FORTRAN 77)
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#19489

In article <mailman.5080.1327510460.27778.python-list@python.org>,
 Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:

> 	The old convention I'd learned was to keep functions down to a
> (printer) page (classical 6 lines per inch, 11" high, tractor feed -- so
> about 60 lines per function -- possibly extend to a second page if
> really needed.

The generalization of that is "keep a function small enough that you can 
see it all at once".  In the days of line printers and green-bar, that 
meant about 60 lines.  As we moved to glass ttys, that started to mean 
24 lines.  These days, with most people having large high-res monitors, 
the acceptable limit (by that standard) is growing again.  I can easily 
get 60 lines on my laptop screen.

On the other hand, back in the neolithic age, when line printers and 
mainframes roamed the world, function calls were expensive.  People were 
encouraged to write larger functions to avoid function calls.  Now, 
function calls are cheap.  Both because optimizing compilers and better 
hardware support have made them cheap, and because hardware in general 
has gotten so fast nobody cares about it anymore (nor should they).

So, I'd say the driving principle should be that a function should do 
one thing.  Every function should have an elevator talk.  You should be 
able to get on an elevator with a function and when you ask it, "So, 
what do you do?", it should be able to explain itself before you get to, 
"Sorry, that's my floor".  If it takes longer than that to explain, then 
it's probably several functions fighting to get out.

[toc] | [prev] | [next] | [standalone]

#19492

On Fri, Jan 27, 2012 at 1:11 AM, Roy Smith <roy@panix.com> wrote:
> So, I'd say the driving principle should be that a function should do
> one thing.  Every function should have an elevator talk.  You should be
> able to get on an elevator with a function and when you ask it, "So,
> what do you do?", it should be able to explain itself before you get to,
> "Sorry, that's my floor".  If it takes longer than that to explain, then
> it's probably several functions fighting to get out.

Agreed, that's definitely a good rule of thumb. But there's still some
value in keeping the code length down too. In a program I maintain at
work (in C++, but the same holds true), some functions have a perfect
elevator talk, yet are quite a few screenfuls of code - this is not
really advisable. For instance, one function is "handle responses to
previously-sent requests". It incorporates all the code for handling
requests of type X, of type Y, and of type Z. If I were writing this
from scratch in Python, I would probably have X, Y, and Z all
separated out, and the checkresponses function would simply be a
dispatcher.

ChrisA

[toc] | [prev] | [next] | [standalone]

#19480

On Thu, Jan 26, 2012 at 2:19 AM, lh <lhughes42@gmail.com> wrote:
> Third, length. Well 5000 lines eh... I'm nowhere near that guess I can
> stick with one file.

Of all the source files I have at work, the largest is about that, 5K
lines. It gets a little annoying at times (rapid deployment requires
GCC to do its magic on that file), but it's quite manageable. I'd say
you can go even further than that with Python, quite happily.

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web