Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #19729 > unrolled thread

Re: changing sys.path

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: changing sys.path Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-02-01 17:47 +0000
    Re: changing sys.path Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-02-02 00:51 +0000
      Re: changing sys.path Andrea Crotti <andrea.crotti.0@gmail.com> - 2012-02-02 10:03 +0000
        Re: changing sys.path jmfauth <wxjmfauth@gmail.com> - 2012-02-02 05:45 -0800

#19729 — Re: changing sys.path

On 02/01/2012 05:13 PM, Eric Snow wrote:
> On Wed, Feb 1, 2012 at 9:15 AM, Andrea Crotti<andrea.crotti.0@gmail.com>  wrote:
>> So suppose I want to modify the sys.path on the fly before running some code
>> which imports from one of the modules added.
>>
>> at run time I do
>> sys.path.extend(paths_to_add)
>>
>> but it still doesn't work and I get an import error.
> Make sure you are adding to sys.path the directories that your
> packages/modules are in, and not the actual package directories.
> During import Python looks for modules/packages _in_ each of the
> directories on sys.path, but not _at_ those directories.

Yes sure I do this..
>
>> If I take these paths and add them to site-packages/my_paths.pth
>> everything works, but at run-time the paths which I actually see before
>> importing are exactly the same.
> You mean sys.path looks exactly the same in the two cases?
>
> -eric

Yes they are exactly the same, because in that file I just write exactly 
the same list,
but when modifying it at run-time it doesn't work, while if at the 
application start
there is this file everything works correctly...

That's what really puzzles me.. What could that be then?

[toc] | [next] | [standalone]

#19763

On Wed, 01 Feb 2012 17:47:22 +0000, Andrea Crotti wrote:

> Yes they are exactly the same, because in that file I just write exactly
> the same list,
> but when modifying it at run-time it doesn't work, while if at the
> application start
> there is this file everything works correctly...
> 
> That's what really puzzles me.. What could that be then?


Are you using IDLE or WingIDE or some other IDE which may not be 
honouring sys.path? If so, that's a BAD bug in the IDE.

Are you changing the working directory manually, by calling os.chdir? If 
so, that could be interfering with the import somehow. It shouldn't, but 
you never know...

Are you adding absolute paths or relative paths?

You say that you get an ImportError, but that covers a lot of things 
going wrong. Here's a story. Could it be correct? I can't tell because 
you haven't posted the traceback.

When you set site-packages/my_paths.pth you get a sys path that looks 
like ['a', 'b', 'fe', 'fi', 'fo', 'fum']. You then call "import spam" 
which locates b/spam.py and everything works.

But when you call sys.path.extend(['a', 'b']) you get a path that looks 
like ['fe', 'fi', 'fo', 'fum', 'a', 'b']. Calling "import spam" locates 
some left over junk file, fi/spam.py or fi/spam.pyc, which doesn't 
import, and you get an ImportError.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#19783

On 02/02/2012 12:51 AM, Steven D'Aprano wrote:
> On Wed, 01 Feb 2012 17:47:22 +0000, Andrea Crotti wrote:
>
>> Yes they are exactly the same, because in that file I just write exactly
>> the same list,
>> but when modifying it at run-time it doesn't work, while if at the
>> application start
>> there is this file everything works correctly...
>>
>> That's what really puzzles me.. What could that be then?
>
> Are you using IDLE or WingIDE or some other IDE which may not be
> honouring sys.path? If so, that's a BAD bug in the IDE.
> Are you changing the working directory manually, by calling os.chdir? If
> so, that could be interfering with the import somehow. It shouldn't, but
> you never know...
>
> Are you adding absolute paths or relative paths?

No, no and absolute paths..

>
> You say that you get an ImportError, but that covers a lot of things
> going wrong. Here's a story. Could it be correct? I can't tell because
> you haven't posted the traceback.
>
> When you set site-packages/my_paths.pth you get a sys path that looks
> like ['a', 'b', 'fe', 'fi', 'fo', 'fum']. You then call "import spam"
> which locates b/spam.py and everything works.
>
> But when you call sys.path.extend(['a', 'b']) you get a path that looks
> like ['fe', 'fi', 'fo', 'fum', 'a', 'b']. Calling "import spam" locates
> some left over junk file, fi/spam.py or fi/spam.pyc, which doesn't
> import, and you get an ImportError.
>
>
And no the problem is not that I already checked inspecting at run-time..
This is the traceback and it might be related to the fact that it runs 
from the
.exe wrapper generated by setuptools:

Traceback (most recent call last):
   File "c:\python25\scripts\dev_main-script.py", line 8, in <module>
     load_entry_point('psi.devsonly==0.1', 'console_scripts', 'dev_main')()
   File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py", 
line 152, in main
     Develer(ns).full_run()
   File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py", 
line 86, in full_run
     run(project_name, test_only=self.ns.test_only)
   File "h:\git_projs\psi\psi.devsonly\psi\devsonly\environment.py", 
line 277, in run
     from psi.devsonly.run import Runner
   File "h:\git_projs\psi\psi.devsonly\psi\devsonly\run.py", line 7, in 
<module>
     from psi.workbench.api import Workbench, set_new_dev_main
ImportError: No module named workbench.api


Another thing which might matter is that I'm launching Envisage 
applications, which
heavily rely on the use of entry points, so I guess that if something is 
not in the path
the entry point is not loaded automatically (but it can be forced I 
guess somehow).

I solved in another way now, since I also need to keep a dev_main.pth in 
site-packages
to make Eclipse happy, just respawning the same process on ImportError works
already perfectly..

[toc] | [prev] | [next] | [standalone]

#19793

On 2 fév, 11:03, Andrea Crotti <andrea.crott...@gmail.com> wrote:
> On 02/02/2012 12:51 AM, Steven D'Aprano wrote:
>
>
>
> > On Wed, 01 Feb 2012 17:47:22 +0000, Andrea Crotti wrote:
>
> >> Yes they are exactly the same, because in that file I just write exactly
> >> the same list,
> >> but when modifying it at run-time it doesn't work, while if at the
> >> application start
> >> there is this file everything works correctly...
>
> >> That's what really puzzles me.. What could that be then?
>
> > Are you using IDLE or WingIDE or some other IDE which may not be
> > honouring sys.path? If so, that's a BAD bug in the IDE.
> > Are you changing the working directory manually, by calling os.chdir? If
> > so, that could be interfering with the import somehow. It shouldn't, but
> > you never know...
>
> > Are you adding absolute paths or relative paths?
>
> No, no and absolute paths..
>
>
>
> > You say that you get an ImportError, but that covers a lot of things
> > going wrong. Here's a story. Could it be correct? I can't tell because
> > you haven't posted the traceback.
>
> > When you set site-packages/my_paths.pth you get a sys path that looks
> > like ['a', 'b', 'fe', 'fi', 'fo', 'fum']. You then call "import spam"
> > which locates b/spam.py and everything works.
>
> > But when you call sys.path.extend(['a', 'b']) you get a path that looks
> > like ['fe', 'fi', 'fo', 'fum', 'a', 'b']. Calling "import spam" locates
> > some left over junk file, fi/spam.py or fi/spam.pyc, which doesn't
> > import, and you get an ImportError.
>
> And no the problem is not that I already checked inspecting at run-time..
> This is the traceback and it might be related to the fact that it runs
> from the
> .exe wrapper generated by setuptools:
>
> Traceback (most recent call last):
>    File "c:\python25\scripts\dev_main-script.py", line 8, in <module>
>      load_entry_point('psi.devsonly==0.1', 'console_scripts', 'dev_main')()
>    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py",
> line 152, in main
>      Develer(ns).full_run()
>    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py",
> line 86, in full_run
>      run(project_name, test_only=self.ns.test_only)
>    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\environment.py",
> line 277, in run
>      from psi.devsonly.run import Runner
>    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\run.py", line 7, in
> <module>
>      from psi.workbench.api import Workbench, set_new_dev_main
> ImportError: No module named workbench.api
>
> Another thing which might matter is that I'm launching Envisage
> applications, which
> heavily rely on the use of entry points, so I guess that if something is
> not in the path
> the entry point is not loaded automatically (but it can be forced I
> guess somehow).
>
> I solved in another way now, since I also need to keep a dev_main.pth in
> site-packages
> to make Eclipse happy, just respawning the same process on ImportError works
> already perfectly..



There is something strange here. I can not figure
out how correct code will fail with the sys.path.
It seems to me, the lib you are using is somehow not
able to recognize its own structure ("his own sys.path").

Idea. Are you sure you are modifying the sys.path at
the right place, understand at the right time
when Python processes?

I'm using this sys.path tweaking at run time very often;
eg to test or to run different versions of the same lib
residing in different dirs, and this, in *any* dir and
independently of *any* .pth file.

jmf

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web