Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #95937 > unrolled thread

Porting Python Application to a new linux machine

Back to article view | Back to comp.lang.python

Contents

  Porting Python Application to a new  linux machine Heli Nix <hemla21@gmail.com> - 2015-09-03 07:32 -0700
    Re: Porting Python Application to a new linux machine Joel Goldstick <joel.goldstick@gmail.com> - 2015-09-03 11:27 -0400
    Re: Porting Python Application to a new linux machine Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-09-03 17:31 +0200
      Re: Porting Python Application to a new linux machine Chris Angelico <rosuav@gmail.com> - 2015-09-04 01:39 +1000
      Re: Porting Python Application to a new linux machine Chris Angelico <rosuav@gmail.com> - 2015-09-04 01:55 +1000
      Re: Porting Python Application to a new linux machine Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-09-03 18:23 +0200
        Re: Porting Python Application to a new linux machine Chris Angelico <rosuav@gmail.com> - 2015-09-04 02:49 +1000
          Re: Porting Python Application to a new linux machine Luca Menegotto <otlucaDELETE@DELETEyahoo.it> - 2015-09-03 19:29 +0200
            Re: Porting Python Application to a new linux machine Chris Angelico <rosuav@gmail.com> - 2015-09-04 03:33 +1000
    Re: Porting Python Application to a new linux machine Laura Creighton <lac@openend.se> - 2015-09-04 00:26 +0200
    Re: Porting Python Application to a new  linux machine Christian Gollwitzer <auriocus@gmx.de> - 2015-09-04 07:43 +0200
      Re: Porting Python Application to a new  linux machine Grant Edwards <invalid@invalid.invalid> - 2015-09-04 13:16 +0000
    PyInstaller+ Python3.5 (h5py import error) Heli Nix <hemla21@gmail.com> - 2015-09-23 09:20 -0700
      Re: PyInstaller+ Python3.5 (h5py import error) Christian Gollwitzer <auriocus@gmx.de> - 2015-09-23 20:07 +0200
        Re: PyInstaller+ Python3.5 (h5py import error) Heli Nix <hemla21@gmail.com> - 2015-09-24 02:58 -0700
          Re: PyInstaller+ Python3.5 (h5py import error) Laura Creighton <lac@openend.se> - 2015-09-24 13:11 +0200
            Re: PyInstaller+ Python3.5 (h5py import error) Hedieh E <hemla21@gmail.com> - 2015-09-25 01:53 -0700

#95937 — Porting Python Application to a new linux machine

Dear all, 

I have my python scripts that use several python libraries such as h5py, pyside, numpy.... 

In Windows I have an installer that will install python locally on user machine and so my program gets access to this local python and runs successfully. 

How can I do this in Linux ? ( I want to install python plus my program on the user machine.) I do not want to use the user´s python or to install python on the user´s machine on root. 

Thanks in Advance for your help, 

[toc] | [next] | [standalone]

#95941 — Re: Porting Python Application to a new linux machine

On Thu, Sep 3, 2015 at 10:32 AM, Heli Nix <hemla21@gmail.com> wrote:
> Dear all,
>
> I have my python scripts that use several python libraries such as h5py, pyside, numpy....
>
> In Windows I have an installer that will install python locally on user machine and so my program gets access to this local python and runs successfully.
>
> How can I do this in Linux ? ( I want to install python plus my program on the user machine.) I do not want to use the user´s python or to install python on the user´s machine on root.
>
> Thanks in Advance for your help,
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Look into virtualenv or virtualenvwrapper.  It will let you load a
local python engine along with local copies of the modules you need to
run your application.  Pip is the weapon of choice to load the
libraries, or pip3 with python 3.x

-- 
Joel Goldstick
http://joelgoldstick.com

[toc] | [prev] | [next] | [standalone]

#95943 — Re: Porting Python Application to a new linux machine

Il 03/09/2015 16:32, Heli Nix ha scritto:

> How can I do this in Linux ?

As far as I know, in general a Linux distro comes with Python already 
installed.
All you have to do is check if the installed version matches your needs.
Tipically, you'll find Python 2.7; however, I know there are distros 
with Python3.x as default (Fedora?)

-- 
Ciao!
Luca

[toc] | [prev] | [next] | [standalone]

#95945 — Re: Porting Python Application to a new linux machine

On Fri, Sep 4, 2015 at 1:31 AM, Luca Menegotto
<otlucaDELETE@deleteyahoo.it> wrote:
> Il 03/09/2015 16:32, Heli Nix ha scritto:
>
>> How can I do this in Linux ?
>
>
> As far as I know, in general a Linux distro comes with Python already
> installed.
> All you have to do is check if the installed version matches your needs.
> Tipically, you'll find Python 2.7; however, I know there are distros with
> Python3.x as default (Fedora?)

Also Ubuntu. If you want to work across multiple Linux distros, the
easiest way is to tell people to install either "python2" or "python3"
using their system package manager, and then use that.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95950 — Re: Porting Python Application to a new linux machine

On Fri, Sep 4, 2015 at 1:53 AM, Nick Sarbicki <nick.a.sarbicki@gmail.com> wrote:
> Is 3.x the default on ubuntu now? My 14.10 is still 2.7. Although it does
> have python3 installed.

I'm not sure. I think I read somewhere that the newest Ubuntus would
ship with python3 preinstalled, but python2 not (though of course it'd
be just an apt-get away). Maybe I'm wrong, and that's still in the
future, but certainly it's the intention.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95956 — Re: Porting Python Application to a new linux machine

Il 03/09/2015 17:53, Nick Sarbicki ha scritto:
> Is 3.x the default on ubuntu now? My 14.10 is still 2.7. Although it
> does have python3 installed.

I've checked my Ubuntu 15.04, and the default is 2.7.9.
There is also Python3 (3.4.3), but sorry, I can't remember if I've 
manually installed it or not.

-- 
Ciao!
Luca

[toc] | [prev] | [next] | [standalone]

#95963 — Re: Porting Python Application to a new linux machine

On Fri, Sep 4, 2015 at 2:23 AM, Luca Menegotto
<otlucaDELETE@deleteyahoo.it> wrote:
> Il 03/09/2015 17:53, Nick Sarbicki ha scritto:
>>
>> Is 3.x the default on ubuntu now? My 14.10 is still 2.7. Although it
>> does have python3 installed.
>
>
> I've checked my Ubuntu 15.04, and the default is 2.7.9.
> There is also Python3 (3.4.3), but sorry, I can't remember if I've manually
> installed it or not.

If you mean that typing "python" runs 2.7, then that's PEP 394 at
work. For compatibility reasons, 'python' doesn't ever run Python 3.
(At least, not any time soon.) The question is more: What comes
installed on a fresh system? Anything can be dragged in as a
dependency of some other package, but a normal Ubuntu desktop
installation won't depend on Python 2 for anything. Or at least,
that's the plan; I don't know whether it's been accomplished yet or
not.

ChrisA

[toc] | [prev] | [next] | [standalone]

#95965 — Re: Porting Python Application to a new linux machine

Il 03/09/2015 18:49, Chris Angelico ha scritto:

> If you mean that typing "python" runs 2.7, then that's PEP 394 at
> work. For compatibility reasons, 'python' doesn't ever run Python 3.

Please forgive me, Il make it clearer.
I'm pretty shure that Ubuntu 15.04 comes with Python 2.7.
I don't remember if Python 3 was preinstalled or if I had to install it 
manually.

-- 
Ciao!
Luca

[toc] | [prev] | [next] | [standalone]

#95966 — Re: Porting Python Application to a new linux machine

On Fri, Sep 4, 2015 at 3:29 AM, Luca Menegotto
<otlucaDELETE@deleteyahoo.it> wrote:
> Il 03/09/2015 18:49, Chris Angelico ha scritto:
>
>> If you mean that typing "python" runs 2.7, then that's PEP 394 at
>> work. For compatibility reasons, 'python' doesn't ever run Python 3.
>
>
> Please forgive me, Il make it clearer.
> I'm pretty shure that Ubuntu 15.04 comes with Python 2.7.
> I don't remember if Python 3 was preinstalled or if I had to install it
> manually.

Okay. I don't run any current Ubuntu anywhere, so I don't know. And I
can't even find back the page now where the plans were being
discussed; best I can find is this, about a year out of date now:

https://wiki.ubuntu.com/Python/3

ChrisA

[toc] | [prev] | [next] | [standalone]

#95976 — Re: Porting Python Application to a new linux machine

In a message of Thu, 03 Sep 2015 07:32:55 -0700, Heli Nix writes:
>Dear all, 
>
>I have my python scripts that use several python libraries such as h5py, pyside, numpy.... 
>
>In Windows I have an installer that will install python locally on user machine and so my program gets access to this local python and runs successfully. 
>
>How can I do this in Linux ? ( I want to install python plus my program on the user machine.) I do not want to use the user´s python or to install python on the user´s machine on root. 
>
>Thanks in Advance for your help, 

There are several approaches here.  One is to get your users to
run things in a virtualenv.

see:https://virtualenv.pypa.io/en/latest/
and
https://virtualenvwrapper.readthedocs.org/en/latest/

This works really well, but you have to have users who are
capable of setting up a virtualenv in the first place.
You will still run into problems of 'my shared library is
different from your shared library'.

YOu can also use PyInstaller (which you may have used to make
windows binaries) to make linux ones.  I've never done this, only
made windows ones -- but that is what it says on the label.
https://github.com/pyinstaller/pyinstaller/wiki

I think you will still have to have a set of different files to
download for different linux distributions, but I could be wrong
about that.

And if that problem is unacceptable, then you need docker.
https://www.docker.com/

I've just started playing with it, and I think it is really neat,
but it is too soon for me to have any clue what the problems/tradeoffs
are with it.

Laura

[toc] | [prev] | [next] | [standalone]

#96009

Am 03.09.15 um 16:32 schrieb Heli Nix:
> I have my python scripts that use several python libraries such as
> h5py, pyside, numpy....
>
> In Windows I have an installer that will install python locally on
> user machine and so my program gets access to this local python and
> runs successfully.
>
> How can I do this in Linux ? ( I want to install python plus my
> program on the user machine.) I do not want to use the user´s python
> or to install python on the user´s machine on root.

Another variant is the use of pyinstaller. It can generate a single 
directory with a copy of Python and all needed libraries. You can copy 
that to a different machine, and often it works - unless libc or some 
very basic library is different. Beware that this pulls in half of your 
system, so you'll end up with ~100 MB.

	Christian

[toc] | [prev] | [next] | [standalone]

#96022

On 2015-09-04, Christian Gollwitzer <auriocus@gmx.de> wrote:
> Am 03.09.15 um 16:32 schrieb Heli Nix:
>
>> I have my python scripts that use several python libraries such as
>> h5py, pyside, numpy....
>>
>> In Windows I have an installer that will install python locally on
>> user machine and so my program gets access to this local python and
>> runs successfully.
>>
>> How can I do this in Linux ? ( I want to install python plus my
>> program on the user machine.) I do not want to use the user´s python
>> or to install python on the user´s machine on root.
>
> Another variant is the use of pyinstaller. It can generate a single 
> directory with a copy of Python and all needed libraries. You can copy 
> that to a different machine, and often it works - unless libc or some 
> very basic library is different. Beware that this pulls in half of your 
> system, so you'll end up with ~100 MB.

As an end-user of a number of largish Python applications on Linux, I
don't think any of them use anything like pyinstaller (and I would not
be very happy if they did -- I've likely got almost all of the
required libraries already installed, and I don't need another copy of
all that stuff on my machine that then has to be backed up).

The normal way to distribute even large Python apps with a lot of
required libraries is either as just the Python sources with a
'setup.py' file or as a package that tells the system what
dependancies and libraries are required.  If you don't want to ship
bare sources, the "right" way to distribute a Python app for Linux is
as an .rpm, .ebuild, or .deb.

-- 
Grant Edwards               grant.b.edwards        Yow! An Italian is COMBING
                                  at               his hair in suburban DES
                              gmail.com            MOINES!

[toc] | [prev] | [next] | [standalone]

#97043 — PyInstaller+ Python3.5 (h5py import error)

Dear all, 

Thanks a lot for your replies. Very helpful. I have already done some trials with Virtualenv, but PyInstaller is much closer to the idea of an installer you can pass to someone. 

 I have been using development version of PyInstaller in order to be able to use it with my script written with Python versin 3.3.5. 

I started with a very simple script just to test. I use the following command to create the distribution folder. 

pyinstaller test.py

my script contains the following few lines and it runs ok on my own machine.

import numpy as np
import h5py

a=np.arange(10)
print(a)
inputFiles="test.h5"
with h5py.File(inputFiles, 'w') as inputFileOpen:
  pass

I am getting the following error related to importing h5py. 

test returned -1
Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "/usr/lib/python3.3/site-packages/PyInstaller-3.0.dev2-py3.3.egg/PyInstaller/loader/pyimod03_importers.py", line 311, in load_module
  File "/usr/lib64/python3.3/site-packages/h5py/__init__.py", line 23, in <module>
  File "/usr/lib/python3.3/site-packages/PyInstaller-3.0.dev2-py3.3.egg/PyInstaller/loader/pyimod03_importers.py", line 493, in load_module
  File "h5r.pxd", line 21, in init h5py._conv (/tmp/pip_build_root/h5py/h5py/_conv.c:6563)
  File "/usr/lib/python3.3/site-packages/PyInstaller-3.0.dev2-py3.3.egg/PyInstaller/loader/pyimod03_importers.py", line 493, in load_module
  File "_objects.pxd", line 12, in init h5py.h5r (/tmp/pip_build_root/h5py/h5py/h5r.c:2708)
  File "/usr/lib/python3.3/site-packages/PyInstaller-3.0.dev2-py3.3.egg/PyInstaller/loader/pyimod03_importers.py", line 493, in load_module
  File "_objects.pyx", line 1, in init h5py._objects (/tmp/pip_build_root/h5py/h5py/_objects.c:6407)
ImportError: No module named 'h5py.defs'

If I modify my script to 

import numpy as np
import h5py
a=np.arange(10)
print(a)

 then, the created exectuable will run successfully on other linux machines.  Does anybody have any idea why I am getting the following h5py import error?

My spec file also looks like this:

# -*- mode: python -*-

block_cipher = None


a = Analysis(['test.py'],
             pathex=['/home/albert/test'],
             binaries=None,
             datas=None,
             hiddenimports=[],
             hookspath=None,
             runtime_hooks=None,
             excludes=None,
             win_no_prefer_redirects=None,
             win_private_assemblies=None,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='test',
          debug=False,
          strip=None,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=None,
               upx=True,
               name='test')


Thank you very much in Advance for your help, 

[toc] | [prev] | [next] | [standalone]

#97046 — Re: PyInstaller+ Python3.5 (h5py import error)

Am 23.09.15 um 18:20 schrieb Heli Nix:
> Dear all,
>
> Thanks a lot for your replies. Very helpful. I have already done some trials with Virtualenv, but PyInstaller is much closer to the idea of an installer you can pass to someone.
>
>   I have been using development version of PyInstaller in order to be able to use it with my script written with Python versin 3.3.5.
>
> I started with a very simple script just to test. I use the following command to create the distribution folder.
>
> pyinstaller test.py
>
> my script contains the following few lines and it runs ok on my own machine.
>
> import numpy as np
> import h5py
>
> a=np.arange(10)
> print(a)
> inputFiles="test.h5"
> with h5py.File(inputFiles, 'w') as inputFileOpen:
>    pass
>
> I am getting the following error related to importing h5py.
>
> [...]
> ImportError: No module named 'h5py.defs'

pyinstaller guesses from the code which modules are imported. It looks 
like if h5py imports a module h5py.defs, which is missing. For some 
programs, you need to support pyinstaller with additional information, 
especially if modules are loaded at runtime. Try:


pyinstaller --hidden-import=h5py.defs test.py

> If I modify my script to
>
> import numpy as np
> import h5py
> a=np.arange(10)
> print(a)

This is another hint: obviously h5py defers module loading until you 
first really open a HDF5 file. There pyinstaller has no means to find 
this out.

	Christian

[toc] | [prev] | [next] | [standalone]

#97067 — Re: PyInstaller+ Python3.5 (h5py import error)

Thanks Christian, 

It turned out that h5py.defs was not the only hidden import that I needed to add. 

I managed to get it working with the follwoing command adding 4 hidden imports. 


pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils  --hidden-import=h5py.h5ac --hidden-import=h5py._proxy  test.py


is there anyway that you can use to add all h5py submodules all together?

Thanks, 

[toc] | [prev] | [next] | [standalone]

#97069 — Re: PyInstaller+ Python3.5 (h5py import error)

In a message of Thu, 24 Sep 2015 02:58:35 -0700, Heli Nix writes:
>Thanks Christian, 
>
>It turned out that h5py.defs was not the only hidden import that I needed to add. 
>
>I managed to get it working with the follwoing command adding 4 hidden imports. 
>
>
>pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils  --hidden-import=h5py.h5ac --hidden-import=h5py._proxy  test.py
>
>
>is there anyway that you can use to add all h5py submodules all together?
>
>Thanks, 
>

Yes.  You can use a hook file.
see: https://pythonhosted.org/PyInstaller/#using-hook-files

Laura

[toc] | [prev] | [next] | [standalone]

#97121 — Re: PyInstaller+ Python3.5 (h5py import error)

On Thursday, September 24, 2015 at 1:12:31 PM UTC+2, Laura Creighton wrote:
> In a message of Thu, 24 Sep 2015 02:58:35 -0700, Heli Nix writes:
> >Thanks Christian, 
> >
> >It turned out that h5py.defs was not the only hidden import that I needed to add. 
> >
> >I managed to get it working with the follwoing command adding 4 hidden imports. 
> >
> >
> >pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils  --hidden-import=h5py.h5ac --hidden-import=h5py._proxy  test.py
> >
> >
> >is there anyway that you can use to add all h5py submodules all together?
> >
> >Thanks, 
> >
> 
> Yes.  You can use a hook file.
> see: https://pythonhosted.org/PyInstaller/#using-hook-files
> 
> Laura

Thanks Laura, 
Very Useful, 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web