Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #35113 > unrolled thread

Virtualenv loses context

Back to article view | Back to comp.lang.python

Contents

  Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 06:14 -0800
    Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 06:23 -0800
    Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 06:28 -0800
    Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 06:30 -0800
      Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 06:38 -0800
        Re: Virtualenv loses context Hans Mulder <hansmu@xs4all.nl> - 2012-12-19 20:21 +0100
          Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 11:33 -0800
          Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-20 04:50 -0800
            Re: Virtualenv loses context Ian Kelly <ian.g.kelly@gmail.com> - 2012-12-20 11:38 -0700
      Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 06:42 -0800
        Re: Virtualenv loses context rhythmicdevil@gmail.com - 2012-12-19 07:06 -0800

#35113 — Virtualenv loses context

I am somewhat new to Python and virtualenv. I have setup a virtualenv with no site packages under the assumption that all 3rd party packages will be installed into the site-packages directory for the virtualenv. Another assumption I have about virtualenv is that it's completely self contained.

When I first setup the virtualenv, install my packages and then install my application both of those assumptions seem to be correct. However, at some point I start getting ImportErrors. The be clear, I tested the application and it was all working correctly, the ImportErrors seem to show up at some random point that I cannot reproduce. 

I have a requirements file that I made with 'pip freeze'. I try to use this to reinstall the packages but I get errors saying that I cannot install to the location. The location is the global site-packages directory on my system. I have the virtualenv activated while installing so I dont know why this is happening.

Due to something about Centos6 I cant install M2Crypto using pip and have to do it manually. This gives me the same types of errors:

(botnet_etl)[swright@localhost M2Crypto]$ ./fedora_setup.sh install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/lib64/python2.6/site-packages/test-easy-inst
all-29133.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/lib64/python2.6/site-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  http://peak.telecommunity.com/EasyInstall.html

Please make the appropriate changes for your system and try again.




The only fix I have managed for this problem so far is to reinstall the entire virtualenv from scratch. This works fine for a while and I have to repeat the process. I would think that I was doing something wrong if the ImportErrors showed up every time I attempt to run the app but the fact that they show up later leads me to believe something funky is going on.




Os = Centos6
Python = 2.6.6


Thanks
Steve

[toc] | [next] | [standalone]

#35116

I wrote this little test script to prove my virtualenv is reading from global site packages:

Script Content
from distutils.sysconfig import get_python_lib
print (get_python_lib())

Script Output
(botnet_etl)[swright@localhost app]$ python test.py 
/usr/lib/python2.6/site-packages
(botnet_etl)[swright@localhost app]$ 


Please note that (botnet_etl) is the name of the virtualenv

[toc] | [prev] | [next] | [standalone]

#35118

This may have something to do with it. I create the virtualenv in:
/home/swright/workspace/botnet_etl/

After I install my app the directory structure looks like this:
(botnet_etl)[swright@localhost botnet_etl]$ ll
total 32
drwxrwxr-x 4 swright swright 4096 Dec 19 09:21 app
drwxrwxr-x 3 swright swright 4096 Dec 18 10:34 bin
drwxrwxr-x 9 swright swright 4096 Dec 18 10:24 build
drwxrwxr-x 3 swright swright 4096 Dec 18 10:34 include
drwxrwxr-x 4 swright swright 4096 Dec 18 10:34 lib
-rw-rw-r-- 1 swright swright 1973 Dec 17 15:35 README
-rw-rw-r-- 1 swright swright  109 Dec 17 15:35 stable-req.txt
drwxrwxr-x 3 swright swright 4096 Dec 17 15:35 utils


When I run the following command the path is not showing up which is probably what is causing the problem. I have to reinstall from scratch to see if this is different at install time. 

But, am I supposed to create virtualenv in specific location in the file system? That seems a little odd.

[toc] | [prev] | [next] | [standalone]

#35120

This may have something to do with it. I create the virtualenv in: 
/home/swright/workspace/botnet_etl/ 

After I install my app the directory structure looks like this: 
(botnet_etl)[swright@localhost botnet_etl]$ ll 
total 32 
drwxrwxr-x 4 swright swright 4096 Dec 19 09:21 app 
drwxrwxr-x 3 swright swright 4096 Dec 18 10:34 bin 
drwxrwxr-x 9 swright swright 4096 Dec 18 10:24 build 
drwxrwxr-x 3 swright swright 4096 Dec 18 10:34 include 
drwxrwxr-x 4 swright swright 4096 Dec 18 10:34 lib 
-rw-rw-r-- 1 swright swright 1973 Dec 17 15:35 README 
-rw-rw-r-- 1 swright swright  109 Dec 17 15:35 stable-req.txt 
drwxrwxr-x 3 swright swright 4096 Dec 17 15:35 utils 


When I run the following command the path is not showing up which is probably what is causing the problem. Clearly the site packages for my virtualenv are not listed. I have to reinstall from scratch to see if this is different at install time. 

(botnet_etl)[swright@localhost app]$ python -m site
sys.path = [
    '/home/swright/workspace/botnet_etl/app',
    '/usr/lib/python2.6/site-packages/pymongo-2.3-py2.6-linux-x86_64.egg',
    '/usr/lib/python2.6/site-packages/pip-1.2.1-py2.6.egg',
    '/usr/lib64/python26.zip',
    '/usr/lib64/python2.6',
    '/usr/lib64/python2.6/plat-linux2',
    '/usr/lib64/python2.6/lib-tk',
    '/usr/lib64/python2.6/lib-old',
    '/usr/lib64/python2.6/lib-dynload',
    '/usr/lib64/python2.6/site-packages',
    '/usr/lib64/python2.6/site-packages/gst-0.10',
    '/usr/lib64/python2.6/site-packages/gtk-2.0',
    '/usr/lib64/python2.6/site-packages/webkit-1.0',
    '/usr/lib/python2.6/site-packages',
    '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info',
]
USER_BASE: '/home/swright/.local' (exists)
USER_SITE: '/home/swright/.local/lib/python2.6/site-packages' (doesn't exist)
ENABLE_USER_SITE: True





But, am I supposed to create virtualenv in specific location in the file system? That seems a little odd

[toc] | [prev] | [next] | [standalone]

#35121

Just installed a brand new virtualenv along with two packages. Ran this and I got nothing:

(venvtest)[swright@localhost venvtest]$ python -m site
(venvtest)[swright@localhost venvtest]$ 

I expected to have at least one path in sys.path

[toc] | [prev] | [next] | [standalone]

#35150

On 19/12/12 15:38:01, rhythmicdevil@gmail.com wrote:
> Just installed a brand new virtualenv along with two packages. Ran this and I got nothing:
> 
> (venvtest)[swright@localhost venvtest]$ python -m site
> (venvtest)[swright@localhost venvtest]$ 
>
> I expected to have at least one path in sys.path

For some reason, that doesn't work with python2.7.
I guess that's a bug in 2.7.

One variant that does work, is:

    python -c 'import site; site._script();'

Another would be:

    python /usr/lib/python2.7/site.py

If you think you may have accidentally deactivated your
virtualenv, you can do:

    python -c 'import sys; print sys.executable;'

Your prompt suggests that "venvtest" is active.
However, it's possible to deactivate a virtualenv without
resetting the prompt; you may have unintentionally done that.
If that's the problem, you can solve it by sourcing the
"activate" script again.


Hope this helps,

-- HansM

[toc] | [prev] | [next] | [standalone]

#35151

Thanks for the information Hans, I will double check that stuff. I am positive however that the environment was active. In addition that fact that I get different responses from python -m site from the botnet_etl virtual environment before I blew it away and after I rebuilt it really concerns me.

[toc] | [prev] | [next] | [standalone]

#35193

Brought my laptop out of hibernation to do some work this morning. I attempted to run one of my ETLs and got the following error. I made no changes since it was running yesterday.



[swright@localhost app]$ python etl_botnet_meta.py --mode dev -f
Traceback (most recent call last):
  File "etl_botnet_meta.py", line 9, in <module>
    from make_zip import EtlMakeZip
  File "/home/swright/workspace/botnet_etl/app/make_zip.py", line 1, in <module>
    import M2Crypto, os, time, datetime, json, hashlib, base64, zipfile
ImportError: No module named M2Crypto
[swright@localhost app]$ python -m site
sys.path = [
    '/home/swright/workspace/botnet_etl/app',
    '/usr/lib/python2.6/site-packages/pymongo-2.3-py2.6-linux-x86_64.egg',
    '/usr/lib/python2.6/site-packages/pip-1.2.1-py2.6.egg',
    '/usr/lib64/python26.zip',
    '/usr/lib64/python2.6',
    '/usr/lib64/python2.6/plat-linux2',
    '/usr/lib64/python2.6/lib-tk',
    '/usr/lib64/python2.6/lib-old',
    '/usr/lib64/python2.6/lib-dynload',
    '/usr/lib64/python2.6/site-packages',
    '/usr/lib64/python2.6/site-packages/gst-0.10',
    '/usr/lib64/python2.6/site-packages/gtk-2.0',
    '/usr/lib64/python2.6/site-packages/webkit-1.0',
    '/usr/lib/python2.6/site-packages',
    '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info',
]
USER_BASE: '/home/swright/.local' (exists)
USER_SITE: '/home/swright/.local/lib/python2.6/site-packages' (doesn't exist)
ENABLE_USER_SITE: True
[swright@localhost app]$

[toc] | [prev] | [next] | [standalone]

#35211

On Thu, Dec 20, 2012 at 5:50 AM,  <rhythmicdevil@gmail.com> wrote:
> Brought my laptop out of hibernation to do some work this morning. I attempted to run one of my ETLs and got the following error. I made no changes since it was running yesterday.
>
>
>
> [swright@localhost app]$ python etl_botnet_meta.py --mode dev -f

Are you sure you activated the virtual env?  I see no indication of it
in your prompt.

[toc] | [prev] | [next] | [standalone]

#35123

Just installed a brand new virtualenv along with two packages. Ran this and I got nothing: 

(venvtest)[swright@localhost venvtest]$ python -m site 
(venvtest)[swright@localhost venvtest]$ 

I expected to have at least one path in sys.path 



[swright@localhost workspace]$ virtualenv --no-site-packages venvtest
New python executable in venvtest/bin/python
Installing setuptools............done.
Installing pip...............done.


[swright@localhost venvtest]$ source bin/activate
(venvtest)[swright@localhost venvtest]$ python -m site
(venvtest)[swright@localhost venvtest]$ 


(venvtest)[swright@localhost venvtest]$ pip install -r stable-req.txt 
Downloading/unpacking pycrypto==2.6 (from -r stable-req.txt (line 1))
  Downloading pycrypto-2.6.tar.gz (443kB): 443kB downloaded
  Running setup.py egg_info for package pycrypto
Downloading/unpacking pymongo==2.4 (from -r stable-req.txt (line 2))
  Downloading pymongo-2.4.tar.gz (273kB): 273kB downloaded
  Running setup.py egg_info for package pymongo
Downloading/unpacking yolk==0.4.3 (from -r stable-req.txt (line 3))
  Downloading yolk-0.4.3.tar.gz (86kB): 86kB downloaded
  Running setup.py egg_info for package yolk
    warning: no files found matching '*.txt' under directory 'tests'
    warning: no files found matching '*.conf' under directory 'docs'
    warning: no files found matching '*.css_t' under directory 'docs'
    warning: no files found matching 'indexsidebar.html' under directory 'docs'
    warning: no files found matching 'tests/test_cli.py'
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python
2.6/site-packages/setuptools-0.6c11-py2.6.egg (from yolk==0.4.3->-r stable-req.txt (
line 3))
... lots more text...




(venvtest)[swright@localhost venvtest]$ pwd
/home/swright/workspace/venvtest
(venvtest)[swright@localhost venvtest]$ ll lib/python2.6/site-packages/
total 372
drwxrwxr-x  2 swright swright   4096 Dec 19 09:34 bson
drwxrwxr-x 10 swright swright   4096 Dec 19 09:34 Crypto
-rw-rw-r--  1 swright swright    237 Dec 19 09:32 easy-install.pth
drwxrwxr-x  2 swright swright   4096 Dec 19 09:34 gridfs
drwxrwxr-x  4 swright swright   4096 Dec 19 09:32 pip-1.2.1-py2.6.egg
drwxrwxr-x  2 swright swright   4096 Dec 19 09:34 pycrypto-2.6-py2.6.egg-info
drwxrwxr-x  2 swright swright   4096 Dec 19 09:34 pymongo
drwxrwxr-x  2 swright swright   4096 Dec 19 09:34 pymongo-2.4-py2.6.egg-info
-rw-r--r--  1 swright swright 333447 Dec  5 09:13 setuptools-0.6c11-py2.6.egg
-rw-rw-r--  1 swright swright     30 Dec 19 09:32 setuptools.pth
drwxrwxr-x  3 swright swright   4096 Dec 19 09:34 yolk
drwxrwxr-x  2 swright swright   4096 Dec 19 09:34 yolk-0.4.3-py2.6.egg-info
(venvtest)[swright@localhost venvtest]$ 

[toc] | [prev] | [next] | [standalone]

#35127

So I reinstalled the virtualenv for my project from scratch. Application runs as expected. Here are the notes that I took while installing. Interestingly the command 'python -m site' produces no output now. Notice that before I reinstalled the virtualenv I got a bunch of paths from that command. I am going to reboot my machine to see if that does something. (must be used to windows)


[swright@localhost workspace]$ virtualenv botnet_etl 
New python executable in botnet_etl/bin/python
Installing setuptools............done.
Installing pip...............done.


[swright@localhost workspace]$ cd botnet_etl/
[swright@localhost botnet_etl]$ source bin/activate
(botnet_etl)[swright@localhost botnet_etl]$ 


(botnet_etl)[swright@localhost botnet_etl]$ python -m site
(botnet_etl)[swright@localhost botnet_etl]$

(botnet_etl)[swright@localhost botnet_etl]$ pip install -r stable-req.txt 
--- lots of text follows this for compiling and installing

Install M2Crypto manually:
Installed /home/swright/workspace/botnet_etl/lib/python2.6/site-packages/M2Crypto-0.21.1-py2.6-linux-x86_64.egg
Processing dependencies for M2Crypto==0.21.1
Finished processing dependencies for M2Crypto==0.21.1

Install PySVN manually:
(botnet_etl)[swright@localhost M2Crypto]$ cp -r ~/.virtualenvs/myvirtualenv/lib/python2.6/site-packages/pysvn ~/workspace/botnet_etl/lib/python2.6/site-packages/.



(botnet_etl)[swright@localhost botnet_etl]$ python -m site
(botnet_etl)[swright@localhost botnet_etl]$

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web