Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #83303

Re: pickle, modules, and ImportErrors

Show key headers only | View raw

On Wed, Jan 7, 2015 at 1:12 PM, John Ladasky <john_ladasky@sbcglobal.net> wrote:
> Do I need to "import my_svr.model as model" then?  Adding that line changes nothing. I get the exact same "ImportError: No module named 'model'".
>
> Likewise for "import my_svr", "from my_svr import *", or even "from my_svr.model import SVRModel".
>
> It is clear that I'm failing to understand something important.
>
> I do not have any circular import dependencies; however, some of the files in my package do need to import definitions from files earlier in my data pipeline.  In order to make everything work inside the module, as well as making a parent-folder "import my_svr" work from a iPython,  I find myself needing to use statements like these inside my training.py program:
>
>
> try:
>     from model import *
>     from sampling import *
> except ImportError:
>     from .model import *
>     from .sampling import *
>
>
> This bothers me.  I don't know whether it is correct usage.  I don't know whether it is causing my remaining ImportError problem.

It sounds like you have import path confusion. The first imports there
are absolute imports. If the absolute paths of the named modules are
just "model" and "sampling", then that would be correct. However, it
sounds like these modules are part of a "my_svr" package, so the
absolute paths of these modules are actually "my_svr.model" and
"my_svr.sampling". In that case, the the absolute path that you're
trying to use for the import is wrong. Either fully specify the module
path, or use the relative import you have below.

Now, why might those absolute imports appear to work even though the
paths are incorrect? Well, it sounds like you may be running a file
inside the package as your main script. This causes some problems.

1) Python doesn't realize that the script it's running is part of a
package. It just calls that module '__main__', and if something else
happens to import the module later by its real name, then you'll end
up loading a second copy of the module by that name, which can lead to
confusing behavior.

2) Python implicitly adds the directory containing the script to the
front of the sys.path list. Since the directory containing the package
is presumably already on the sys.path, this means that the internal
modules now appear in sys.path *twice*, with two different names:
"my_svr.model" and "model" refer to the same source file. Python
doesn't realize this however, and so they don't refer to the same
module. As a result, importing both "my_svr.model" and "model" will
again result in two separate copies of the module being loaded, with
two different names.

It sounds like your pickle file was created from an object from the
"model" module. This works fine when the sys.path is set such that
"model" is the absolute path of a module, i.e. when you run your main
script within the package. When running an external main script,
sys.path doesn't get set that way, and so it can't find the "model"
module.

The solution to this is to avoid executing a module within a package
as your main script. Run a script outside the package and have it
import and call into the package instead. An alternative is to run the
module using the -m flag of the Python executable; this fixes problem
2 above but not problem 1.

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

pickle, modules, and ImportErrors John Ladasky <john_ladasky@sbcglobal.net> - 2015-01-07 12:12 -0800
  Re: pickle, modules, and ImportErrors Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-01-07 14:55 -0600
    Re: pickle, modules, and ImportErrors John Ladasky <john_ladasky@sbcglobal.net> - 2015-01-07 16:23 -0800
      Re: pickle, modules, and ImportErrors Chris Angelico <rosuav@gmail.com> - 2015-01-08 12:06 +1100
  Re: pickle, modules, and ImportErrors Ian Kelly <ian.g.kelly@gmail.com> - 2015-01-07 14:01 -0700

csiph-web