Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #32637 > unrolled thread

How to improve the usability of nested packages

Back to article view | Back to comp.lang.python

Contents

  How to improve the usability of nested packages Michael Schwarz <michi.schwarz@gmail.com> - 2012-11-02 17:11 +0100
    Re: How to improve the usability of nested packages Rouslan Korneychuk <rouslank@msn.com> - 2012-11-05 23:02 -0500

#32637 — How to improve the usability of nested packages

I need some guidance on how to structure a large library with lots of
packages and sub-packages. Using the library should be as effortless
as possible. It's your average beast of a catch-all for sharing code
across applications at our company. Let's call it "the_library". In my
attempt to structure the library, I was following two principles:

- The complete library is contained in a single package. This is to
avoid polluting the top-level namespace.
- Only modules and sub-packages directly under the top-level package
should be imported directly. This means that any class or function in
the library is accessed using the same qualified name everywhere
inside the library or the application. This makes moving code around
easier.

Following this, using a module from the library is pretty
straight-forward. A typical file in the application code could start
with:

from the_library import sip, rtp, sdp

This works from any module or script in the library or application.
Then I decided to split the "sip" module into smaller modules, e.g.
"message", "transaction", "dialog", all contained in a package named
"sip". Ideally, an application would still import the sip package
using the import above and then, for example, access the "DialogID"
class using "sip.dialog.DialogID". Currently this is only possible
when also explicitly importing the "dialog" module:

from the_library import sip
import the_library.sip.dialog

This is ugly and seems unnecessary to me as, for example, having all
the modules in the "sip" package available using a single import would
not pollute the local namespace. So I tried to enable this by
importing all the modules in the "sip" package from the package's
"__init__.py":

from . import message, transaction, dialog

… which doesn't work. Some of the modules reference other modules in
the same package. I'm not talking about cyclic references, but, for
example, the "dialog" module uses the "transaction" module. The
problem is that the "dialog" module uses the same mechanism shown
above to import the other modules from it's package. This means that
modules and packages are imported in this order:

- Application code executes "from the_library import sip"
- the_library/__init__.py is executed. No imports here.
- the_library/sip/__init__.py executes "from . import [...], dialog"
- the_library/sip/dialog.py executes "from the_library import sip"

During the last import a problem arises: The module object for the
package "the_library" does not yet have a "sip" member (as it is still
executing the import) and so the import fails. It is still possible to
import the "transaction" module directly from the "dialog" module
using:

from . import transaction

But this would make the "transaction" module available under a
different qualified name as anywhere else (where it's accessed using
"sip.transaction").

What path would you take to circumvent this problem? Would you break
the rule that any module should be accessed using the same way, no
matter from where it is accessed, or would you maybe structure the
library entirely different?

Thanks for any suggestions!

Michael

[toc] | [next] | [standalone]

#32795

On 11/02/2012 12:11 PM, Michael Schwarz wrote:
> … which doesn't work. Some of the modules reference other modules in
> the same package. I'm not talking about cyclic references, but, for
> example, the "dialog" module uses the "transaction" module. The
> problem is that the "dialog" module uses the same mechanism shown
> above to import the other modules from it's package. This means that
> modules and packages are imported in this order:
>
> - Application code executes "from the_library import sip"
> - the_library/__init__.py is executed. No imports here.
> - the_library/sip/__init__.py executes "from . import [...], dialog"
> - the_library/sip/dialog.py executes "from the_library import sip"


In a way, you do have a cyclic reference. If you think of "import sip" 
as "from sip import __init__ as sip", it should become apparent.


Anyway, I see two ways around this. One is to rename the package and 
create a module under the_library with the old package name that imports 
what you want. To keep using the same names, every module in the package 
can import siblings like so (where sippkg is the new package name):
   import the_library.sippkg.transaction
   sip = the_library.sippkg


The second way is to not use import at all to load sibling modules 
(except in __init__.py), and instead use:
   sip = sys.modules['the_library.sip']

This will work as long as you are careful to load the modules from 
__init__.py in the correct order, where each module's dependencies must 
appear before the module, in the import list.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web