Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #11356 > unrolled thread

Relative import from script with same name as package

Back to article view | Back to comp.lang.python

Contents

  Relative import from script with same name as package "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-08-13 21:39 +0000
    Re: Relative import from script with same name as package "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-08-13 23:55 +0000
      Re: Relative import from script with same name as package Chris Angelico <rosuav@gmail.com> - 2011-08-14 01:07 +0100
    Re: Relative import from script with same name as package Oktay Şafak <oktaysafak@superonline.com> - 2011-08-14 12:57 +0300
      Re: Relative import from script with same name as package "OKB (not okblacke)" <brenNOSPAMbarn@NObrenSPAMbarn.net> - 2011-08-14 18:48 +0000
        Re: Relative import from script with same name as package Oktay Şafak <oktaysafak@superonline.com> - 2011-08-15 01:56 +0300

#11356 — Relative import from script with same name as package

    	I'm using Python 2.6.5.  I have a directory structure like this:

thetest/
    	__init__.py
    	thetest.py
    	theother.py

__init__.py is an empty file.  theother.py contains a function foo().  
The package is accessible from sys.path, so that if I open the 
interpreter and do "import thetest" or "from thetest import thetest" or 
"import thetest.thetest", it works fine.

Inside thetest.py I have code like this:

###
from __future__ import absolute_import

if __name__ == "__main__" and __package__ is None:
	import thetest
	__package__ = "thetest"

from .theother import foo
###

    	Note that I need the "import thetest" line to avoid a "parent 
module not loaded" error, as described here: 
http://stackoverflow.com/questions/2943847/nightmare-with-relative-
imports-how-does-pep-366-work  

    	If I run foo.py directly, I receive a traceback like this:

Traceback (most recent call last):
  File "C:\...\thetest\thetest.py", line 4, in <module>
    import thetest
  File "C:\...\thetest\thetest.py", line 11, in <module>
    from .theother import foo
ValueError: Attempted relative import in non-package

    	It appears that Python is reading "import thetest" as importing 
thetest.py (the same file that is currently being run).  When it tries 
to run that file a second time, the relative import fails.

    	But why?  That __future__ import is supposed to make absolute 
imports the default, so why is "import thetest" importing thetest.py 
instead of the package called thetest?  The absolute import should make 
it look in sys.path first and not try to import from the script 
directory, right?

    	If I change the outer directory name and change the code in 
thetest.py to match, it works fine.  But I shouldn't have to do this.  
How can I get relative imports to work correctly when running a script 
whose filename is the same as that of the directory (and thus the 
package) in which it resides?

-- 
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is
no path, and leave a trail."
	--author unknown

[toc] | [next] | [standalone]

#11357

OKB (not okblacke) wrote:

>          But why?  That __future__ import is supposed to make
>          absolute 
> imports the default, so why is "import thetest" importing
> thetest.py instead of the package called thetest?  The absolute
> import should make it look in sys.path first and not try to import
> from the script directory, right?
> 
>          If I change the outer directory name and change the code
>          in 
> thetest.py to match, it works fine.  But I shouldn't have to do
> this.  How can I get relative imports to work correctly when
> running a script whose filename is the same as that of the
> directory (and thus the package) in which it resides?

    	After a bit more googling I discovered the answer here: 
http://stackoverflow.com/questions/1959188/absolute-import-failing-in-
subpackage-that-shadows-a-stdlib-package-name

    	The deal is that sys.path by default has the empty string as the 
first element, which tells Python to look first in the directory of the 
script being executed.  This is unfortunate, but can worked around this 
way:

import sys
sys.path = sys.path[1:] + ['']

(That is, move the current directory to the end of the search path 
instead of the beginning.)

-- 
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is
no path, and leave a trail."
	--author unknown

[toc] | [prev] | [next] | [standalone]

#11358

On Sun, Aug 14, 2011 at 12:55 AM, OKB (not okblacke)
<brenNOSPAMbarn@nobrenspambarn.net> wrote:
> sys.path = sys.path[1:] + ['']
>
> (That is, move the current directory to the end of the search path
> instead of the beginning.)
>

Or, equivalently:

sys.path.append(sys.path.pop(0))

ChrisA

[toc] | [prev] | [next] | [standalone]

#11381

14.08.2011 00:39, OKB (not okblacke) yazmış:
>      	I'm using Python 2.6.5.  I have a directory structure like this:
>
> thetest/
>      	__init__.py
>      	thetest.py
>      	theother.py
>
> __init__.py is an empty file.  theother.py contains a function foo().
> The package is accessible from sys.path, so that if I open the
> interpreter and do "import thetest" or "from thetest import thetest" or
> "import thetest.thetest", it works fine.

It should.

>
> Inside thetest.py I have code like this:
>
> ###
> from __future__ import absolute_import
>
> if __name__ == "__main__" and __package__ is None:
> 	import thetest
> 	__package__ = "thetest"
>
> from .theother import foo
> ###
>
...
>
>      	If I run foo.py directly, I receive a traceback like this:
>

Wait! What's foo.py? I guess you mean thetest.py as you say "(the same 
file that is currently being run)" below.

> Traceback (most recent call last):
>    File "C:\...\thetest\thetest.py", line 4, in<module>
>      import thetest
>    File "C:\...\thetest\thetest.py", line 11, in<module>
>      from .theother import foo
> ValueError: Attempted relative import in non-package
>
>      	It appears that Python is reading "import thetest" as importing
> thetest.py (the same file that is currently being run).  When it tries
> to run that file a second time, the relative import fails.
>

No, there is no such thing happening. Read the error message more 
carefully: the error happens when your code reaches the line "from 
.theother import foo", and it fails because you are trying to execute an 
"explicit" relative import statement (with leading dot notation) as 
introduced by PEP 328. What you see is perfectly expected behaviour as 
explained in detail in the PEP because the python interpreter can only 
make sense of that statement if that code is *imported* for use by code 
that resides *outside* the package. That error message is what you see 
when you try to *run* a package member module which uses explicit 
relative imports. Let me try to explain a bit further:

You say that your package is accessible from sys.path. So let's say you 
fire up your interpreter and type

	import thetest.thetest

Here the interpreter first imports the *package* thetest and then the 
module *thetest.py*, therefore it knows that the module is a member of 
the package. And when executing the code in the module thetest.py, it 
can make sense of the leading dot in "import .theother", it can see that 
the leading dot means the packege thetest and find the right module 
inside that package.

But when you run thetest.py directly, the interpreter has no idea what 
the leading dot means because it has not imported any package 
beforehand. So it complains about that.
  >      	But why?  That __future__ import is supposed to make absolute
> imports the default, so why is "import thetest" importing thetest.py
> instead of the package called thetest?

This is not what is happening.

> The absolute import should make
> it look in sys.path first and not try to import from the script
> directory, right?

As I explained above, that's not the error but I think this sentence 
shows your misunderstanding of this relative imports issue. Never mind, 
I found the documentation very confusing at first and took me a great 
deal of effort to finally "get it". Let me share my "distilled wisdom" 
about this:

- First of all, never perform your experiments about relative/absolute 
imports using IDLE! I guess IDLE has its own path manipulation that 
doesn't do *the right thing*.

- "Absolute import" means importing from sys.path only.

- "Relative import" means a "package member" module importing another 
"package member", using a not fully qualified name (i.e X.Y.a importing 
X.Y.b as import b, instead of using import X.Y.b). When Python saw 
(before PEP 328) "import b" or "from b import ..." *in a package 
member*, it first imported the local b if available. (This business of a 
package member importing another package member is called relative 
import, only) Since this had the same notation (syntax) with other (non 
package member) imports, intra-package imports were considered 
"ambiguous" when the module to be imported had the same name with a 
module reachable from sys.path: it was not immediately clear which one 
was meant for untrained eyes. Also, it had the potential of shadowing a 
stdlib module: in that case no clean method of getting at the stdlib 
module was available.

- PEP 328 is about giving "relative imports" a new syntax so that this 
ambiguity no longer exists and the possibility of shadowing a stdlib 
module is removed altogether. The new syntax is called "*explicit* 
relative imports". The former method can thus be called "implicit 
relative imports" or "ambiguous relative imports"

- BIG SOURCE OF CONFUSION: Relative imports (both implicit and explicit) 
is all about imports *inside* packages. When you have 2 scripts app.py 
and string.py sitting in a *non-package* folder (they are called 
top-level modules), that folder is always the first entry in sys.path 
when you run either of those modules directly, so "import string" in 
app.py will always import the string.py in that folder, not the stdlib 
one. (But not when you try with IDLE!) In this case, if you really want 
the standard module, you must manipulate sys.path before your import 
statement (or change the name of your local string.py). THE PEP (328) 
HAS NO EFFECT IN THAT CASE!

- So, "relative imports" is not a concept introduced by PEP 328, it 
means "importing a package member from another member of that package" 
and it existed well before PEP 328. But its syntax was ambiguous so it 
was clarified by PEP 328 by giving it a special syntax (the leading dot 
notation). But that change introduced many small gotchas, including the 
one you are seeing: you can't directly run a package member module if 
you have used this new syntax (leading dot notation) in it. It works 
fine only when you import the package members for use by a module 
outside of that package. This of course brings difficulties with testing 
but it is another story.

I think you will now find it much easier to get what PEP 328 is all about.

Oktay

[toc] | [prev] | [next] | [standalone]

#11408

=?UTF-8?B?T2t0YXkgxZ5hZmFr?= wrote:

>>           It appears that Python is reading "import thetest" as
>>           importing 
>> thetest.py (the same file that is currently being run).  When it
>> tries to run that file a second time, the relative import fails.
>>
> 
> No, there is no such thing happening. Read the error message more 
> carefully: the error happens when your code reaches the line "from 
> .theother import foo", and it fails because you are trying to
> execute an "explicit" relative import statement (with leading dot
> notation) as introduced by PEP 328. What you see is perfectly
> expected behaviour as explained in detail in the PEP because the
> python interpreter can only make sense of that statement if that
> code is *imported* for use by code that resides *outside* the
> package. That error message is what you see when you try to *run* a
> package member module which uses explicit relative imports. Let me
> try to explain a bit further: 

    	Yes, such a thing was happening.  (I described the fix in an 
answer to my own post.)  You should read PEP 366 to understand what I 
was talking about, and what the __package__ variable does.

-- 
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is
no path, and leave a trail."
	--author unknown

[toc] | [prev] | [next] | [standalone]

#11427

14.08.2011 21:48, OKB (not okblacke) yazmış:
> =?UTF-8?B?T2t0YXkgxZ5hZmFr?= wrote:
>
>>>            It appears that Python is reading "import thetest" as
>>>            importing
>>> thetest.py (the same file that is currently being run).  When it
>>> tries to run that file a second time, the relative import fails.
>>>
>>
>> No, there is no such thing happening. Read the error message more
>> carefully: the error happens when your code reaches the line "from
>> .theother import foo", and it fails because you are trying to
>> execute an "explicit" relative import statement (with leading dot
>> notation) as introduced by PEP 328. What you see is perfectly
>> expected behaviour as explained in detail in the PEP because the
>> python interpreter can only make sense of that statement if that
>> code is *imported* for use by code that resides *outside* the
>> package. That error message is what you see when you try to *run* a
>> package member module which uses explicit relative imports. Let me
>> try to explain a bit further:
>
>      	Yes, such a thing was happening.  (I described the fix in an
> answer to my own post.)  You should read PEP 366 to understand what I
> was talking about, and what the __package__ variable does.
>

Hmmm, I have overlooked the middle part of your code. Now I get what you 
mean. Yes that is happening, you are right and I have given the source 
of that behaviour in my previous post:

> - BIG SOURCE OF CONFUSION: Relative imports (both implicit and explicit) is all about imports *inside* packages. When you have 2 scripts app.py and string.py sitting in a *non-package* folder (they are called top-level modules), that folder is always the first entry in sys.path when you run either of those modules directly, so "import string" in app.py will always import the string.py in that folder, not the stdlib one. (But not when you try with IDLE!) In this case, if you really want the standard module, you must manipulate sys.path before your import statement (or change the name of your local string.py). THE PEP (328) HAS NO EFFECT IN THAT CASE!

In your case, you are not using the package by importing its members 
from code outside the package but you are running them directly. So, 
your files thetest.py and theother.py are now *top-level* modules for 
the interpreter, not package members. So as per PEP 366, you need to add 
that boilerplate to set the __package__ attribute.

But the "from __future__ import absolute_import" statement has no effect 
on the "import thetest" statement when you run the file directly: then 
thetest.py is a top level module, and when the interpreter encounters 
the statement "import thetest" *it is already absolute import* (whether 
you issed the __future__ statement or not). It still is in Python 2.7. 
PEP 328 has no effect in this case. Remember that absolute import means 
importing from sys.path only. In this case, thetest.py's folder is 
always the first entry in sys.path so it tries to import itself first. 
You would also see the same behaviour if your "import thetest" statement 
was in theother.py, it would import thetest.py rather than the package. 
So as I have said before and quoted above, in this case you have to 
manipulate sys.path, which you have found out yourself.

So what is this "absolute import" thing that is introduced by PEP 328 
then? It is all about import statements in package members when they are 
imported from code outside the package. Before PEP 328, if theother.py 
contained an "import thetest" statement and if you fired up your 
interpreter and issued

	import thetest.theother

that "import thetest" in theother.py would find thetest.py. It was 
considered to be a(n implicit) package-relative import. After PEP 328, 
if you add to the top of theother.py or issue at the interpreter

	from __future__ import absolute_import

and then issue

	import thetest.theother

the "import thetest" statement in theother.py will find the *package* 
thetest because sys.path is given priority over the package dir by the 
__future__ statement. So the PEP has effectively overthrown the 
convention of treating such imports as special. They are not treated as 
special anymore and receive the same treatment as other imports. This is 
one aspect of the PEP.
The PEP also defines a new notation (leading dot) for those who want the 
sibling thetest.py, which is another aspect.

But when you run the theother.py directly, the PEP and the __future__ 
statement has NO EFFECT. The "import thetest" statement in theother.py 
is not seen as intra package import by the interpreter so it is already 
absolute import (uses sys.path) So it finds thetest.py because it is in 
the same folder as theother.py, which is the first folder in sys.path. 
Many people are confused and think that this is relative import. It's 
not. It is already absolute import (from sys.path) and the PEP has 
nothing to do with it. Now we can answer your critical question:

> But why?  That __future__ import is supposed to make absolute
> imports the default, so why is "import thetest" importing thetest.py
> instead of the package called thetest?

Bacause you are running it directly and the PEP and the __future__ 
statement does not apply here.

I hope it is clear now.

-- 
Oktay Şafak

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web