Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #80840 > unrolled thread

caught in the import web again

Started by"Charles T. Smith" <cts.private.yahoo@gmail.com>
First post2014-11-15 22:52 +0000
Last post2014-11-18 09:13 +0000
Articles 20 on this page of 21 — 11 participants

Back to article view | Back to comp.lang.python


Contents

  caught in the import web again "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2014-11-15 22:52 +0000
    Re: caught in the import web again Ben Finney <ben+python@benfinney.id.au> - 2014-11-16 10:43 +1100
    Re: caught in the import web again Denis McMahon <denismfmcmahon@gmail.com> - 2014-11-16 01:11 +0000
    Re: caught in the import web again dieter <dieter@handshake.de> - 2014-11-16 08:14 +0100
      Re: caught in the import web again "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2014-11-16 08:53 +0000
        Re: caught in the import web again Chris Angelico <rosuav@gmail.com> - 2014-11-16 20:20 +1100
        Re: caught in the import web again dieter <dieter@handshake.de> - 2014-11-17 08:08 +0100
          Re: caught in the import web again "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2014-11-17 20:07 +0000
            Re: caught in the import web again Dave Angel <davea@davea.name> - 2014-11-17 17:17 -0500
            Re: caught in the import web again Ian Kelly <ian.g.kelly@gmail.com> - 2014-11-17 15:52 -0700
            Re: caught in the import web again Dave Angel <davea@davea.name> - 2014-11-17 20:20 -0500
            Re: caught in the import web again Ian Kelly <ian.g.kelly@gmail.com> - 2014-11-18 08:28 -0700
            Re: caught in the import web again Dave Angel <davea@davea.name> - 2014-11-18 14:09 -0500
            Re: caught in the import web again Chris Angelico <rosuav@gmail.com> - 2014-11-19 07:18 +1100
            Re: caught in the import web again Dave Angel <davea@davea.name> - 2014-11-18 17:58 -0500
        Re: caught in the import web again Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-18 09:45 +1100
          Re: caught in the import web again Rick Johnson <rantingrickjohnson@gmail.com> - 2014-11-17 18:17 -0800
            Re: caught in the import web again Chris Angelico <rosuav@gmail.com> - 2014-11-18 13:25 +1100
            Re: caught in the import web again Steven D'Aprano <steve@pearwood.info> - 2014-11-18 07:13 +0000
          Re: caught in the import web again Michael Torrie <torriem@gmail.com> - 2014-11-18 00:00 -0700
            Re: caught in the import web again "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2014-11-18 09:13 +0000

Page 1 of 2  [1] 2  Next page →


#80840 — caught in the import web again

From"Charles T. Smith" <cts.private.yahoo@gmail.com>
Date2014-11-15 22:52 +0000
Subjectcaught in the import web again
Message-ID<m48lfh$n88$1@dont-email.me>
Now, I'm getting these errors:

  ImportError: cannot import name ...

and

  AttributeError: 'module' object has no attribute ...

(what is 'module'?)

Is there a way to resolve this without having to restructure my code 
every couple of days?

I thought using imports of the form:

  from module import symbol

was the "right way" to avoid these hassles...

TIA
cts

www.creative-telcom-solutions.de

[toc] | [next] | [standalone]


#80841

FromBen Finney <ben+python@benfinney.id.au>
Date2014-11-16 10:43 +1100
Message-ID<mailman.15859.1416094993.18130.python-list@python.org>
In reply to#80840
"Charles T. Smith" <cts.private.yahoo@gmail.com> writes:

> Now, I'm getting these errors:

Please reduce the problem to a minimal, complete example demonstrating
the behaviour <URL:http://sscce.org/> so that you can show us exactly
what's happening.

>   AttributeError: 'module' object has no attribute ...
>
> (what is 'module'?)

The type of the object which doesn't have the attribute. It's saying
you're tring to access an attribute on a particular object, and that
object (which is of type ‘module’, hence it is a “'module' object”)
doesn't have that attribute.

> Is there a way to resolve this without having to restructure my code 
> every couple of days?

Not knowing what the actual problem is, it's difficult to say.

Please come up with a (contrived, if you like) minimal clear example of
what's happening – i.e. make it from scratch, without irrelevant parts
from the rest of your program, but ensure it still does what you're
confused by – and present it here.

-- 
 \          “It's a terrible paradox that most charities are driven by |
  `\     religious belief.… if you think altruism without Jesus is not |
_o__)          altruism, then you're a dick.” —Tim Minchin, 2010-11-28 |
Ben Finney

[toc] | [prev] | [next] | [standalone]


#80843

FromDenis McMahon <denismfmcmahon@gmail.com>
Date2014-11-16 01:11 +0000
Message-ID<m48tj9$1uu$4@dont-email.me>
In reply to#80840
On Sat, 15 Nov 2014 22:52:33 +0000, Charles T. Smith wrote:

> Now, I'm getting these errors:
> 
>   ImportError: cannot import name ...
> 
> and
> 
>   AttributeError: 'module' object has no attribute ...

It would be useful to know what you're actually trying to import and what 
the complete error messages are. For example, are these imports of code 
that you've written yourself, or part of the standard library modules, or 
third party modules.

-- 
Denis McMahon, denismfmcmahon@gmail.com

[toc] | [prev] | [next] | [standalone]


#80858

Fromdieter <dieter@handshake.de>
Date2014-11-16 08:14 +0100
Message-ID<mailman.15871.1416122062.18130.python-list@python.org>
In reply to#80840
"Charles T. Smith" <cts.private.yahoo@gmail.com> writes:
> Now, I'm getting these errors:
>
>   ImportError: cannot import name ...
>
> and
>
>   AttributeError: 'module' object has no attribute ...
>
> (what is 'module'?)
>
> Is there a way to resolve this without having to restructure my code 
> every couple of days?
>
> I thought using imports of the form:
>
>   from module import symbol
>
> was the "right way" to avoid these hassles...

I have not noticed your previous posts regarding import problems --
thus, I may repeats things already said.

Usually, Python imports are straight forward and do not lead to
surprises. There are two exceptions: recursive imports and
imports in separate threads. Let's look at the problem areas in turn.

Recursive imports. In this case you have module "A" which imports module "B"
(maybe indirectly via a sequence of intervening imports) which imports
module "A". In this, "import" means either "import ..." or
"from ... import ..." (it does not matter). When module "B" tries
to import module "A", then "A" is not yet complete (as during the
execution of "A"'s initialization code, it started to import "B"
(maybe indirectly) and the following initialization code has not yet
been executed). As a consequence, you may get "AttributeError"
(or "ImportError") when you try to access things from "A".
To avoid problems like this, try to avoid recursive imports.
One way to do this, are local imports -- i.e. imports inside
a function. This way, the import happens when the functions is called
which (hopefully) happens after module initialization.

Imports and threads. The import machinery changes global data structures
(e.g. "sys.modules"). To protect those changes, it uses a lock, held
while an import is in progress. When, during an import, a separate
thread tries to import something, it blocks - waiting for the import
lock to be released. In case, the importing thread waits on this
thread, then the system deadlocks.
To avoid this: do not start threads during imports.

[toc] | [prev] | [next] | [standalone]


#80864

From"Charles T. Smith" <cts.private.yahoo@gmail.com>
Date2014-11-16 08:53 +0000
Message-ID<m49omd$924$1@dont-email.me>
In reply to#80858
On Sun, 16 Nov 2014 08:14:05 +0100, dieter wrote:

> "Charles T. Smith" <cts.private.yahoo@gmail.com> writes:
>> Now, I'm getting these errors:
>>
>>   ImportError: cannot import name ...
>>
>> and
>>
>>   AttributeError: 'module' object has no attribute ...
>>
>> (what is 'module'?)
>>
>> Is there a way to resolve this without having to restructure my code
>> every couple of days?
>>
>> I thought using imports of the form:
>>
>>   from module import symbol
>>
>> was the "right way" to avoid these hassles...
> 
> I have not noticed your previous posts regarding import problems --
> thus, I may repeats things already said.
> 
> Usually, Python imports are straight forward and do not lead to
> surprises. There are two exceptions: recursive imports and imports in
> separate threads. Let's look at the problem areas in turn.
> 
> Recursive imports. In this case you have module "A" which imports module
> "B" (maybe indirectly via a sequence of intervening imports) which
> imports module "A". In this, "import" means either "import ..." or "from
> ... import ..." (it does not matter). When module "B" tries to import
> module "A", then "A" is not yet complete (as during the execution of
> "A"'s initialization code, it started to import "B" (maybe indirectly)
> and the following initialization code has not yet been executed). As a
> consequence, you may get "AttributeError" (or "ImportError") when you
> try to access things from "A". To avoid problems like this, try to avoid
> recursive imports. One way to do this, are local imports -- i.e. imports
> inside a function. This way, the import happens when the functions is
> called which (hopefully) happens after module initialization.


Yes, we're talking about recursive imports here.  It's a complex, object-
oriented system with big classes and little classes that are strongly 
interrelated.  I can get the imports configured properly so everything 
works but if I make a little change to the code, then suddenly all my 
imports are broken and I must painstakingly go through the whole 
structure, reorganizing them.

Are others equally frustrated by this or is there a trick or principle 
that I'm missing.  At this point, I guess the way I'll have to proceed is 
to put every class in its own file, no matter how small.  Hopefully that 
takes care of the problem.

I compared perl's object-oriented philosophy with python's and didn't 
like how perl *requires* you to put everything into its own file.  Now it 
looks like python does, too, implicitly.


cts

www.creative-telcom-solutions.de

[toc] | [prev] | [next] | [standalone]


#80868

FromChris Angelico <rosuav@gmail.com>
Date2014-11-16 20:20 +1100
Message-ID<mailman.15878.1416129659.18130.python-list@python.org>
In reply to#80864
On Sun, Nov 16, 2014 at 7:53 PM, Charles T. Smith
<cts.private.yahoo@gmail.com> wrote:
> Yes, we're talking about recursive imports here.  It's a complex, object-
> oriented system with big classes and little classes that are strongly
> interrelated.  I can get the imports configured properly so everything
> works but if I make a little change to the code, then suddenly all my
> imports are broken and I must painstakingly go through the whole
> structure, reorganizing them.

So reorganize *once*, such that there are no recursive imports.
Reorganize your code so things don't actually import one another.
Remember, you can pass an instance of some class to an instance of
another class that has no knowledge of the first class - like this:

# file1.py

class Foo:
    def some_method(self, some_obj):
        some_obj.some_other_method()

# file2.py

class Bar:
    def some_other_method(self):
        return 42

# main.py

import file1, file2

f=file1.Foo()
b=file2.Bar()
f.some_method(b)


This works just fine, and neither file1 nor file2 needs to import the
other. Recursive imports are almost never necessary.

ChrisA

[toc] | [prev] | [next] | [standalone]


#80961

Fromdieter <dieter@handshake.de>
Date2014-11-17 08:08 +0100
Message-ID<mailman.15947.1416208135.18130.python-list@python.org>
In reply to#80864
"Charles T. Smith" <cts.private.yahoo@gmail.com> writes:
> ...
> Are others equally frustrated by this or is there a trick or principle 
> that I'm missing.  At this point, I guess the way I'll have to proceed is 
> to put every class in its own file, no matter how small.  Hopefully that 
> takes care of the problem.

Recursive imports are inherently difficult. It is best if
a careful design avoids it (note that a cycle free dependency
structure is much easier to handle than a cyclic one -- not only
for Python).

If you think such a cycle free module structure is impossible
in your case, then try to delay module related operations:

   *  consider local imports (imports inside a function).
      They delay the import until the function is called rather
      than execute it at module import time.

   *  prefer "import <module>" over "from <module> import ..."
      This may delay the module access; perhaps (but not
      necessarily) sufficiently such that the accessed object
      is already defined.

Neither of these approaches is complete - you may still get
cyclic import dependencies and related problems. Usually, you
need a careful module dependency design to avoid cyclic dependencies
altogether.
      

> I compared perl's object-oriented philosophy with python's and didn't 
> like how perl *requires* you to put everything into its own file.  Now it 
> looks like python does, too, implicitly.

If you put everything in a single file, then you do not get
recursive imports (but you may still have other problems related to
cyclic dependencies). Thus, Python's difficulties with recursive
imports do not force you to "put everything in its own file".

If you think about it, you will recognize that cyclic dependencies are
inherently difficult (e.g. "chicken - egg" type problems)
and that the way Python handles cyclic
import dependencies is rather natural (and understandable).

[toc] | [prev] | [next] | [standalone]


#80981

From"Charles T. Smith" <cts.private.yahoo@gmail.com>
Date2014-11-17 20:07 +0000
Message-ID<m4dki7$fn6$1@dont-email.me>
In reply to#80961
On Mon, 17 Nov 2014 08:08:40 +0100, dieter wrote:

> "Charles T. Smith" <cts.private.yahoo@gmail.com> writes:
>> ...
>> Are others equally frustrated by this or is there a trick or principle
>> that I'm missing.  At this point, I guess the way I'll have to proceed
>> is to put every class in its own file, no matter how small.  Hopefully
>> that takes care of the problem.
> 
> Recursive imports are inherently difficult. It is best if a careful
> design avoids it (note that a cycle free dependency structure is much
> easier to handle than a cyclic one -- not only for Python).
> 
> If you think such a cycle free module structure is impossible in your
> case, then try to delay module related operations:
> 
>    *  consider local imports (imports inside a function).
>       They delay the import until the function is called rather than
>       execute it at module import time.
> 
>    *  prefer "import <module>" over "from <module> import ..."
>       This may delay the module access; perhaps (but not necessarily)
>       sufficiently such that the accessed object is already defined.
> 
> Neither of these approaches is complete - you may still get cyclic
> import dependencies and related problems. Usually, you need a careful
> module dependency design to avoid cyclic dependencies altogether.
>       
>       
>> I compared perl's object-oriented philosophy with python's and didn't
>> like how perl *requires* you to put everything into its own file.  Now
>> it looks like python does, too, implicitly.
> 
> If you put everything in a single file, then you do not get recursive
> imports (but you may still have other problems related to cyclic
> dependencies). Thus, Python's difficulties with recursive imports do not
> force you to "put everything in its own file".
> 
> If you think about it, you will recognize that cyclic dependencies are
> inherently difficult (e.g. "chicken - egg" type problems) and that the
> way Python handles cyclic import dependencies is rather natural (and
> understandable).


Well, I guess that's the definitive answer... the tips for delaying 
import are good, I'll try to leverage them.

I was hoping there would be a way to have python postpone evaluation 
similar to C's forward references.

[toc] | [prev] | [next] | [standalone]


#80984

FromDave Angel <davea@davea.name>
Date2014-11-17 17:17 -0500
Message-ID<mailman.15962.1416262622.18130.python-list@python.org>
In reply to#80981
"Charles T. Smith" <cts.private.yahoo@gmail.com> Wrote in message:

> Well, I guess that's the definitive answer... the tips for delaying
> import are good, I'll try to leverage them.
> 
> I was hoping there would be a way to have python postpone evaluation
> similar to C's forward references.
> 

In a module that might get tangled in a cycle, avoid global code
 that depends on other modules.  Instead of putting such
 initialization at top level, put inside a function that gets
 called after all suspect imports are completed. (That function
 has global keywords ). 

Similarly avoid class attribute initialization at class level, if
 such initialization references higher level modules.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]


#80986

FromIan Kelly <ian.g.kelly@gmail.com>
Date2014-11-17 15:52 -0700
Message-ID<mailman.15963.1416264801.18130.python-list@python.org>
In reply to#80981
On Mon, Nov 17, 2014 at 3:17 PM, Dave Angel <davea@davea.name> wrote:
> "Charles T. Smith" <cts.private.yahoo@gmail.com> Wrote in message:
>
>> Well, I guess that's the definitive answer... the tips for delaying
>> import are good, I'll try to leverage them.
>>
>> I was hoping there would be a way to have python postpone evaluation
>> similar to C's forward references.
>>
>
> In a module that might get tangled in a cycle, avoid global code
>  that depends on other modules.  Instead of putting such
>  initialization at top level, put inside a function that gets
>  called after all suspect imports are completed. (That function
>  has global keywords ).

If the problem is that one of those modules would import the current
module, then "after all suspect imports are completed" basically means
after the current module has finished importing. So what would be
responsible for calling such a function?

[toc] | [prev] | [next] | [standalone]


#80989

FromDave Angel <davea@davea.name>
Date2014-11-17 20:20 -0500
Message-ID<mailman.15965.1416273652.18130.python-list@python.org>
In reply to#80981
Ian Kelly <ian.g.kelly@gmail.com> Wrote in message:
> On Mon, Nov 17, 2014 at 3:17 PM, Dave Angel <davea@davea.name> wrote:

>> In a module that might get tangled in a cycle, avoid global code
>>  that depends on other modules.  Instead of putting such
>>  initialization at top level, put inside a function that gets
>>  called after all suspect imports are completed. (That function
>>  has global keywords ).
> 
> If the problem is that one of those modules would import the current
> module, then "after all suspect imports are completed" basically means
> after the current module has finished importing. So what would be
> responsible for calling such a function?
> 

If one builds a set of modules that cannot avoid recursive
 imports, then one can ask the code that imports it to make a call
 after importing. Or all that is needed to avoid exposing the mess
 is that the top level not be part of the loops.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]


#81006

FromIan Kelly <ian.g.kelly@gmail.com>
Date2014-11-18 08:28 -0700
Message-ID<mailman.15978.1416324563.18130.python-list@python.org>
In reply to#80981
On Mon, Nov 17, 2014 at 6:20 PM, Dave Angel <davea@davea.name> wrote:
> Ian Kelly <ian.g.kelly@gmail.com> Wrote in message:
>> On Mon, Nov 17, 2014 at 3:17 PM, Dave Angel <davea@davea.name> wrote:
>
>>> In a module that might get tangled in a cycle, avoid global code
>>>  that depends on other modules.  Instead of putting such
>>>  initialization at top level, put inside a function that gets
>>>  called after all suspect imports are completed. (That function
>>>  has global keywords ).
>>
>> If the problem is that one of those modules would import the current
>> module, then "after all suspect imports are completed" basically means
>> after the current module has finished importing. So what would be
>> responsible for calling such a function?
>>
>
> If one builds a set of modules that cannot avoid recursive
>  imports, then one can ask the code that imports it to make a call
>  after importing. Or all that is needed to avoid exposing the mess
>  is that the top level not be part of the loops.

This would make the module less reusable though, since the first other
module to import it in one program might still be used in another
program but not be the first other module to import it. Or I suppose
for simplicity it could be required that all importers call the
initialization function, and have the function return immediately if
it's been called previously. However, I don't think there's any way to
do this without ending up with smelly code.

[toc] | [prev] | [next] | [standalone]


#81009

FromDave Angel <davea@davea.name>
Date2014-11-18 14:09 -0500
Message-ID<mailman.15981.1416337787.18130.python-list@python.org>
In reply to#80981
Ian Kelly <ian.g.kelly@gmail.com> Wrote in message:
> On Mon, Nov 17, 2014 at 6:20 PM, Dave Angel <davea@davea.name> wrote:
>> Ian Kelly <ian.g.kelly@gmail.com> Wrote in message:
>>> On Mon, Nov 17, 2014 at 3:17 PM, Dave Angel <davea@davea.name> wrote:
>>
>>>> In a module that might get tangled in a cycle, avoid global code
>>>>  that depends on other modules.  Instead of putting such
>>>>  initialization at top level, put inside a function that gets
>>>>  called after all suspect imports are completed. (That function
>>>>  has global keywords ).
>>>
>>> If the problem is that one of those modules would import the current
>>> module, then "after all suspect imports are completed" basically means
>>> after the current module has finished importing. So what would be
>>> responsible for calling such a function?
>>>
>>
>> If one builds a set of modules that cannot avoid recursive
>>  imports, then one can ask the code that imports it to make a call
>>  after importing. Or all that is needed to avoid exposing the mess
>>  is that the top level not be part of the loops.
> 
> This would make the module less reusable though, since the first other
> module to import it in one program might still be used in another
> program but not be the first other module to import it. Or I suppose
> for simplicity it could be required that all importers call the
> initialization function, and have the function return immediately if
> it's been called previously. However, I don't think there's any way to
> do this without ending up with smelly code.
> 

The smelly code happened when the designer "can't" break the
 cycle. I understood the question to be "then what." Two-stage
 initialization is sometimes "needed" in any language and I was
 just showing it's possible in Python. I once worked on (and then
 fixed) a build system that could not complete a build from clean.
 It needed some pieces from a previous build in order to get to
 the point where it was ready to build those pieces. Recursive
 depencies at compile and link time.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]


#81011

FromChris Angelico <rosuav@gmail.com>
Date2014-11-19 07:18 +1100
Message-ID<mailman.15982.1416341919.18130.python-list@python.org>
In reply to#80981
On Wed, Nov 19, 2014 at 6:09 AM, Dave Angel <davea@davea.name> wrote:
> I once worked on (and then
>  fixed) a build system that could not complete a build from clean.
>  It needed some pieces from a previous build in order to get to
>  the point where it was ready to build those pieces. Recursive
>  depencies at compile and link time.

Sometimes the solution to that is to add intermediate files to the
source code archive. Pike can't be built from source control unless
you already have a reasonably recent Pike installed, but the exported
.tar.gz archive includes all the generated files, so you can build
that, then go to the absolute latest from source control. There are
always ways around a circular dependency.

ChrisA

[toc] | [prev] | [next] | [standalone]


#81018

FromDave Angel <davea@davea.name>
Date2014-11-18 17:58 -0500
Message-ID<mailman.15984.1416351515.18130.python-list@python.org>
In reply to#80981
Chris Angelico <rosuav@gmail.com> Wrote in message:
> On Wed, Nov 19, 2014 at 6:09 AM, Dave Angel <davea@davea.name> wrote:
>> I once worked on (and then
>>  fixed) a build system that could not complete a build from clean.
>>  It needed some pieces from a previous build in order to get to
>>  the point where it was ready to build those pieces. Recursive
>>  depencies at compile and link time.
> 
> Sometimes the solution to that is to add intermediate files to the
> source code archive. Pike can't be built from source control unless
> you already have a reasonably recent Pike installed, but the exported
> .tar.gz archive includes all the generated files, so you can build
> that, then go to the absolute latest from source control. There are
> always ways around a circular dependency.
> 

Exactly. In the case I was talking about there was a somewhat
 related problem.  The build broke every time,  and the fixups
 were very manual.  Nobody even knew if the results depended on
 the order of applying the fixups. The build lady spent about two
 days each week fixing stuff up.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]


#80985

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2014-11-18 09:45 +1100
Message-ID<546a7aa0$0$12995$c3e8da3$5496439d@news.astraweb.com>
In reply to#80864
Charles T. Smith wrote:

> Yes, we're talking about recursive imports here.  It's a complex, object-
> oriented system with big classes and little classes that are strongly
> interrelated.

Well, there's your problem right there. You're working with a complex,
highly coupled code-base. Alarms bells should be ringing in your head. That
sort of highly-interdependent code base is a nightmare to maintain at the
best of times. Recursive imports are just the start of the problem.

Circular dependencies are not just a problem in Python, they are a problem
throughout most of software design.

https://en.wikipedia.org/wiki/Acyclic_dependencies_principle

http://stackoverflow.com/questions/1897537/why-are-circular-references-considered-harmful


> I can get the imports configured properly so everything 
> works but if I make a little change to the code, then suddenly all my
> imports are broken and I must painstakingly go through the whole
> structure, reorganizing them.

The problems you are having with recursive imports are the symptom of a
deeper design problem. If you can, you should try to refactor your code so
as to have fewer mutual dependencies and less coupling.

 
> Are others equally frustrated by this or is there a trick or principle
> that I'm missing.  

Probably, and yes. Personally, I try very hard to avoid recursive imports as
a matter of principle, even when I can manage them safely, so personally
I'm never frustrated by them. But there are a few tricks you can use to
reduce the pain a little. Think of these as pain-killers. All they are
doing is masking the symptoms of an underlying problem, namely excessive
coupling.

You can often delay imports to when a function or method is called. Although
this is not "best practice", it does have its uses. Instead of this
recursive import:

# spam relies on this module, but this module relies on spam
import spam
class K:
    def __init__(self):
        self.eggs = spam.stuff()


you can write this:

class K:
    def __init__(self):
        import spam
        self.eggs = spam.stuff()


That delays importing spam until after this module is already fully
initialised. If you're worried about efficiency, don't be, only the very
first import of spam will be expensive, the others will be very fast.

Another way to avoid recursive imports is to merge the code from both
modules into a single module. Note that this is the complete opposite of
the conclusion you draw below.

Google for "avoiding circular imports python" (and similar) and you will
find lots of discussions about this. Leave out the "python" and you will
find the same thing for other languages.


> At this point, I guess the way I'll have to proceed is 
> to put every class in its own file, no matter how small.  Hopefully that
> takes care of the problem.

I don't think that's going to help. If anything, it will make it worse.
Suppose you have two classes, A and B, which require each other:

class A:
    def __init__(self):
        self.thing = B()

class B(A): pass


There's a circular dependency right there, but no import is needed so it is
comparatively easy to manage. Now move them into separate modules, and
you've turned a manageable circular dependency within a single module into
a painful recursive import.

Python is not Java, nor Perl, and if you're putting every class into its own
file, you are doing it wrong.


> I compared perl's object-oriented philosophy with python's and didn't
> like how perl *requires* you to put everything into its own file.  Now it
> looks like python does, too, implicitly.

You are mistaken about that.



-- 
Steven

[toc] | [prev] | [next] | [standalone]


#80990

FromRick Johnson <rantingrickjohnson@gmail.com>
Date2014-11-17 18:17 -0800
Message-ID<5ed27643-a8af-469b-a065-3e211998ad62@googlegroups.com>
In reply to#80985
On Monday, November 17, 2014 4:46:05 PM UTC-6, Steven D'Aprano wrote:
> [...]
> Python is not Java, nor Perl, and if you're putting every
> class into its own file, you are doing it wrong.

Stop making these gross generalizations. Just because Java
*REQUIRES* that you only have one class per file does not
mean that having one class per file is *BAD*, or *WRONG* --
quite the contrary! You might as well join the endless hordes
that parrot off such ridiculous bile as:

    * All republicans are racists!

or:

    * All democrats are whiners!

or:

    * All politicians are filthy nasty liars!

...that last one might be true though!

There are times when i have only one class in a file, but
there are other times when i have multiple classes per file

    BUT NOT MANY TIMES!
    
    AND NOT MANY CLASSES!

The number of classes per file is a very subjective matter
indeed, but in my opinion, the number should be kept as low
as possible, because, as the number of lines in a file
increases, your mental grasp of the contents decreases. I
don't care who you *THINK* you are, but your mental focus is
*NOT* infinite!

One of the problems inherent in "large file editing" is the
fact that since you are dealing with multiple namespaces,
there is great possibilities of different namespaces re-using
the same symbols. So for instance, if your searching in
"class A" for "symbolX", and "symbolX" also exists in "class
M", you're going to find yourself 1000 lines away from the
scope you intended to search, and now you have to trog back
up to your original spot!

    BOY I LOVE TROGGING!

YES, YES, i know some of you have fancy editing tools with
"scoped search" abilities, but even *IF* you use such a
tool, you still must define the scope in some way --by
selecting a block of text or choosing from a predefined set
of scopes-- but in any event, you're just doing "new work"
(scoped search rules) to avoid doing "old work" (flipping
between windows), and although there is arguably no net gain
or loss between the two techniques *DIRECTLY*, the person
trying to wrangle a large file is more likely to loose
focus.

    IT'S NOT A ZERO SUM GAME, SO *YOU* LOOSE!
    
The "one class per file rule" handles the "scoped search"
*IMPLICITLY* for me. When i'm in a file that contains only
one class, i know that i could never acidentally edit the
wrong symbol in *ANOTHER* class, and I don't need to define a
search area, no, all i need to do to "search", is to, well, 
*SEARCH*. 

    WHAT A NOVEL IDEA!

Combining multiple classes into a single file is like
combining multiple drinking glasses into a single unit.
Imagine a wine glass, a beer mug, and a soda can joined with
a tight fitting rubber-band -- sure, you can carry the "unit"
comfortably with one hand, and you might impress your drunk
friends at the local pub with your new "beverage dispenser",
however, try drinking from the beer mug without getting
drenched with wine or sticky soda.

And even if you *DID* manage to drink the beer without
spilling the other two in the process, you would have to
expend so much *EXTRA* mental focus during the process that
you've undermined the usefulness of the combination.
Most times, the simplest solution is the best.

    JUST DRINK FROM ONE GLASS AT A TIME YOU IDIOT!

[toc] | [prev] | [next] | [standalone]


#80991

FromChris Angelico <rosuav@gmail.com>
Date2014-11-18 13:25 +1100
Message-ID<mailman.15967.1416277529.18130.python-list@python.org>
In reply to#80990
On Tue, Nov 18, 2014 at 1:17 PM, Rick Johnson
<rantingrickjohnson@gmail.com> wrote:
>     BOY I LOVE TROGGING!

For consistency, you should say GOVE there.

ChrisA

[toc] | [prev] | [next] | [standalone]


#80996

FromSteven D'Aprano <steve@pearwood.info>
Date2014-11-18 07:13 +0000
Message-ID<546af1b6$0$12998$c3e8da3$5496439d@news.astraweb.com>
In reply to#80990
On Mon, 17 Nov 2014 18:17:13 -0800, Rick Johnson wrote:

> BOY I LOVE TROGGING!

Yes, we've noticed.

http://www.urbandictionary.com/define.php?term=trogging




-- 
Steven

[toc] | [prev] | [next] | [standalone]


#80995

FromMichael Torrie <torriem@gmail.com>
Date2014-11-18 00:00 -0700
Message-ID<mailman.15970.1416294062.18130.python-list@python.org>
In reply to#80985
On 11/17/2014 03:45 PM, Steven D'Aprano wrote:

> Circular dependencies are not just a problem in Python, they are a problem
> throughout most of software design.

Personally I find that duck typing eliminates a lot of the circular
dependency problems.  Class A doesn't necessarily have to know about
Class B to work with instances of Class B.  All that has to be known is
what methods are going to be available on those instances.  The
interface pattern can help here if you wanted a bit more safety (like
what Zope provides).  Either way it breaks up the dependency cycle nicely.

If Class A needs to instantiate instances of Class B (which depends on
Class A), then it could call a factory method to do so, perhaps set up
by a third party after both modules have been imported.

There are lots of methods of avoiding the issue entirely.

In the C or C++ world, I find that signalling frameworks can also be
used to break a dependency cycle.  For example, if Class A needs to call
a method on an instance of Class B (which in turn depends on Class A),
that could be done instead as a signal, which is later connected to
Class B's method after everything is successfully instantiated.  Also
makes testing a bit easier, because you can simply hook up the signals
to a test harness.

[toc] | [prev] | [next] | [standalone]


Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python


csiph-web