Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #15798 > unrolled thread

How to insert my own module in front of site eggs?

Back to article view | Back to comp.lang.python

Contents

  How to insert my own module in front of site eggs? Roy Smith <roy@panix.com> - 2011-11-16 20:57 -0500
    Re: How to insert my own module in front of site eggs? Marc Christiansen <usenet@solar-empire.de> - 2011-11-17 10:48 +0100
      Re: How to insert my own module in front of site eggs? Roy Smith <roy@panix.com> - 2011-11-17 09:09 -0500
        Re: How to insert my own module in front of site eggs? Duncan Booth <duncan.booth@invalid.invalid> - 2011-11-17 15:04 +0000
          Re: How to insert my own module in front of site eggs? Roy Smith <roy@panix.com> - 2011-11-17 20:38 -0500
        Re: How to insert my own module in front of site eggs? Paul Rudin <paul.nospam@rudin.co.uk> - 2011-11-17 17:05 +0000
          Re: How to insert my own module in front of site eggs? Roy Smith <roy@panix.com> - 2011-11-17 20:36 -0500
            Re: How to insert my own module in front of site eggs? alex23 <wuwei23@gmail.com> - 2011-11-17 18:58 -0800
              Re: How to insert my own module in front of site eggs? Hans Mulder <hansmu@xs4all.nl> - 2011-11-18 11:52 +0100
            Re: How to insert my own module in front of site eggs? Paul Rudin <paul.nospam@rudin.co.uk> - 2011-11-18 05:54 +0000
            Re: How to insert my own module in front of site eggs? "spartan.the" <spartan.the@gmail.com> - 2011-11-20 12:44 -0800

#15798 — How to insert my own module in front of site eggs?

I'm trying to use a custom version of mongoengine.  I cloned the git 
repo and put the directory on my PYTHONPATH, but python is still 
importing the system's installed version.  Looking at sys.path, it's 
obvious why:

$ echo $PYTHONPATH 
/home/roy/songza:/home/roy/lib/mongoengine

>>> pprint.pprint(sys.path)
['',
 '/usr/local/lib/python2.6/dist-packages/selenium-2.0a5-py2.6.egg',
 '/usr/local/lib/python2.6/dist-packages/unittest2-0.5.1-py2.6.egg',
 
'/usr/local/lib/python2.6/dist-packages/pymongo-1.9-py2.6-linux-x86_64.eg
g',
 '/usr/local/lib/python2.6/dist-packages/virtualenv-1.5.2-py2.6.egg',
 '/usr/local/lib/python2.6/dist-packages/mongoengine-0.5.2-py2.6.egg',
 '/home/roy/songza',
 '/home/roy/lib/mongoengine',
 '/usr/lib/python2.6',
 '/usr/lib/python2.6/plat-linux2',
 '/usr/lib/python2.6/lib-tk',
 '/usr/lib/python2.6/lib-old',
 '/usr/lib/python2.6/lib-dynload',
 '/usr/lib/python2.6/dist-packages',
 '/usr/lib/python2.6/dist-packages/PIL',
 '/usr/lib/python2.6/dist-packages/gst-0.10',
 '/usr/lib/pymodules/python2.6',
 '/usr/lib/python2.6/dist-packages/gtk-2.0',
 '/usr/lib/pymodules/python2.6/gtk-2.0',
 '/usr/local/lib/python2.6/dist-packages']

The system eggs come before my path.  I found 
http://mail.python.org/pipermail/distutils-sig/2006-July/006520.html in 
the archives; it explains that eggs come before PYTHONPATH, but doesn't 
explain how to get around this problem.  I emphatically agree with 
Michael Bayer who said:

> I cant think of a  possible scenario where a path would explicitly
> exist in PYTHONPATH,  non-egg or egg, where the user would still like the
> system-wide installation to take precedence

So, is there any way to get my local copy of mongoengine loaded instead 
of the system egg?  I could probably import sys, and do an egg-ectomy on 
sys.path before importing mongoengine, but that's too gross to 
contemplate.

[toc] | [next] | [standalone]

#15814

Roy Smith <roy@panix.com> wrote:
> I'm trying to use a custom version of mongoengine.  I cloned the git 
> repo and put the directory on my PYTHONPATH, but python is still 
> importing the system's installed version.  Looking at sys.path, it's 
> obvious why:
> 
> $ echo $PYTHONPATH 
> /home/roy/songza:/home/roy/lib/mongoengine
> 
>>>> pprint.pprint(sys.path)
> ['',
> '/usr/local/lib/python2.6/dist-packages/selenium-2.0a5-py2.6.egg',
> '/usr/local/lib/python2.6/dist-packages/unittest2-0.5.1-py2.6.egg',
> 
> '/usr/local/lib/python2.6/dist-packages/pymongo-1.9-py2.6-linux-x86_64.eg
> g',
> '/usr/local/lib/python2.6/dist-packages/virtualenv-1.5.2-py2.6.egg',
> '/usr/local/lib/python2.6/dist-packages/mongoengine-0.5.2-py2.6.egg',
> '/home/roy/songza',
> '/home/roy/lib/mongoengine',
[...]

> The system eggs come before my path.  I found 
> http://mail.python.org/pipermail/distutils-sig/2006-July/006520.html in 
> the archives; it explains that eggs come before PYTHONPATH, but doesn't 
> explain how to get around this problem.  I emphatically agree with 
> Michael Bayer who said:
> 
>> I cant think of a  possible scenario where a path would explicitly
>> exist in PYTHONPATH,  non-egg or egg, where the user would still like the
>> system-wide installation to take precedence
> 
> So, is there any way to get my local copy of mongoengine loaded instead 
> of the system egg?  I could probably import sys, and do an egg-ectomy on 
> sys.path before importing mongoengine, but that's too gross to 
> contemplate.

The only way I found is to edit the easy_install.pth file and comment
the two lines starting with "import sys". You'll have to do that every
time you install/upgrade an egg via easy_install (and maybe setuptools).
In your case the right file should be
/usr/local/lib/python2.6/dist-packages/easy_install.pth

BTW: You could try pip (http://www.pip-installer.org/) instead of
easy_install, it doesn't mess with sys.path.

Ciao
Marc

[toc] | [prev] | [next] | [standalone]

#15819

In article <1s1fp8-6la.ln1@pluto.solar-empire.de>,
 Marc Christiansen <usenet@solar-empire.de> wrote:

> > So, is there any way to get my local copy of mongoengine loaded instead 
> > of the system egg?  I could probably import sys, and do an egg-ectomy on 
> > sys.path before importing mongoengine, but that's too gross to 
> > contemplate.
> 
> The only way I found is to edit the easy_install.pth file and comment
> the two lines starting with "import sys". You'll have to do that every
> time you install/upgrade an egg via easy_install (and maybe setuptools).
> In your case the right file should be
> /usr/local/lib/python2.6/dist-packages/easy_install.pth
> 
> BTW: You could try pip (http://www.pip-installer.org/) instead of
> easy_install, it doesn't mess with sys.path.

But, you're talking about installers.  I'm talking about if I've already 
got something installed, how do I force one particular python process to 
pull in a local copy of a module in preference to the installed one?

In some cases, I own the machine and can make changes to /usr/local/lib 
if I want to.  But what about on a shared machine?  I don't want to (or 
perhaps can't) play with what's in /usr/local/lib just to make my stuff 
load first.

[toc] | [prev] | [next] | [standalone]

#15820

Roy Smith <roy@panix.com> wrote:

> In some cases, I own the machine and can make changes to /usr/local/lib 
> if I want to.  But what about on a shared machine?  I don't want to (or 
> perhaps can't) play with what's in /usr/local/lib just to make my stuff 
> load first.
> 

Have you considered running your code in a virtualenv?
http://pypi.python.org/pypi/virtualenv

-- 
Duncan Booth http://kupuguy.blogspot.com

[toc] | [prev] | [next] | [standalone]

#15844

In article <Xns9FA0995C66F1Fduncanbooth@127.0.0.1>,
 Duncan Booth <duncan.booth@invalid.invalid> wrote:

> Roy Smith <roy@panix.com> wrote:
> 
> > In some cases, I own the machine and can make changes to /usr/local/lib 
> > if I want to.  But what about on a shared machine?  I don't want to (or 
> > perhaps can't) play with what's in /usr/local/lib just to make my stuff 
> > load first.
> > 
> 
> Have you considered running your code in a virtualenv?
> http://pypi.python.org/pypi/virtualenv

I do that for some projects.  In fact, I suspect I will do that for all 
furture projects that I start from scratch.  For this particular one, 
there's a lot of stuff already installed in the system that I need.

[toc] | [prev] | [next] | [standalone]

#15824

Roy Smith <roy@panix.com> writes:


> But, you're talking about installers.  I'm talking about if I've already 
> got something installed, how do I force one particular python process to 
> pull in a local copy of a module in preference to the installed one?
>
> In some cases, I own the machine and can make changes to /usr/local/lib 
> if I want to.  But what about on a shared machine?  I don't want to (or 
> perhaps can't) play with what's in /usr/local/lib just to make my stuff 
> load first.

Maybe I'm missing something - but if I want to do this I just mess about
with sys.path at the top of my python script/program. Always seems to
work... is there a situation in which it doesn't?

[toc] | [prev] | [next] | [standalone]

#15843

In article <874ny2fzn6.fsf@no-fixed-abode.cable.virginmedia.net>,
 Paul Rudin <paul.nospam@rudin.co.uk> wrote:

> Roy Smith <roy@panix.com> writes:
> 
> 
> > But, you're talking about installers.  I'm talking about if I've already 
> > got something installed, how do I force one particular python process to 
> > pull in a local copy of a module in preference to the installed one?
> >
> > In some cases, I own the machine and can make changes to /usr/local/lib 
> > if I want to.  But what about on a shared machine?  I don't want to (or 
> > perhaps can't) play with what's in /usr/local/lib just to make my stuff 
> > load first.
> 
> Maybe I'm missing something - but if I want to do this I just mess about
> with sys.path at the top of my python script/program. Always seems to
> work... is there a situation in which it doesn't?

What if the first import of a module is happening inside some code you 
don't have access to?

It just seems mind-boggling to me that PYTHONPATH doesn't preempt 
everything else.

[toc] | [prev] | [next] | [standalone]

#15854

On Nov 18, 11:36 am, Roy Smith <r...@panix.com> wrote:
> What if the first import of a module is happening inside some code you
> don't have access to?

No import will happen until you import something. As long as you
change sys.path before you do, all subsequent imports will use that
path.

[toc] | [prev] | [next] | [standalone]

#15882

On 18/11/11 03:58:46, alex23 wrote:
> On Nov 18, 11:36 am, Roy Smith<r...@panix.com>  wrote:
>> What if the first import of a module is happening inside some code you
>> don't have access to?

> No import will happen until you import something.

That would be the case if you use the '-S' command line option.

Otherwise, site.py is imported before you get a chance to alter
sys.path, and by default site.py imports other modules.

-- HansM

[toc] | [prev] | [next] | [standalone]

#15874

Roy Smith <roy@panix.com> writes:

> In article <874ny2fzn6.fsf@no-fixed-abode.cable.virginmedia.net>,
>  Paul Rudin <paul.nospam@rudin.co.uk> wrote:
>
>> 
>> Maybe I'm missing something - but if I want to do this I just mess about
>> with sys.path at the top of my python script/program. Always seems to
>> work... is there a situation in which it doesn't?
>
> What if the first import of a module is happening inside some code you 
> don't have access to?

If you change sys.path first - before you do any imports - then any
other imports will surely come from the first thing on sys.path (unless
something else you import also changes sys.path)?

[toc] | [prev] | [next] | [standalone]

#15965

On Nov 18, 3:36 am, Roy Smith <r...@panix.com> wrote:
...
> It just seems mind-boggling to me that PYTHONPATH doesn't preempt
> everything else.

I was surprised too, but this issue is widespread. Best explained
here:
https://bugs.launchpad.net/tahoe-lafs/+bug/821000/comments/0
Quote:

"
The crux of the problem is that the Python documentation says:
  "The PYTHONPATH variable can be set to a list of paths that will be
added to the beginning of sys.path."
http://docs.python.org/install/index.html#modifying-python-s-search-path
This is true with standard distutils, false with setuptools, false
with distribute...
"

Phillip J. Eby (setuptools guy) was religious about "easy installed"
packages over anything else to comment on this issue 3 years ago:

"
This behavior change is unacceptable, as it makes it impossible to
ensure that
an easy_install-ed package takes precedence over an existing,
distutils-installed version of the same package in the same PYTHONPATH
directory.

Please note that that is the intended default behavior of
easy_install: if you
wish to override it, you should use --multi-version, and explicitly
select the
egg(s) to be added to sys.path instead.
"
(http://bugs.python.org/setuptools/issue53)

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web