Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #7220 > unrolled thread

Application libraries private, Distutils metadata available for console scripts and introspection

Back to article view | Back to linux.debian.maint.python

Contents

  Application libraries private, Distutils metadata available for console scripts and introspection Ben Finney <ben+debian@benfinney.id.au> - 2015-09-01 05:00 +0200
    Re: Application libraries private, Distutils metadata available for  console scripts and introspection Robert Collins <robertc@robertcollins.net> - 2015-09-01 06:50 +0200
      Re: Application libraries private, Distutils metadata available for console scripts and introspection Ben Finney <ben+debian@benfinney.id.au> - 2015-09-01 07:40 +0200
        Re: Application libraries private, Distutils metadata available for  console scripts and introspection Robert Collins <robertc@robertcollins.net> - 2015-09-01 09:40 +0200
    Re: Application libraries private, Distutils metadata available for console scripts and introspection Ben Finney <ben+debian@benfinney.id.au> - 2015-09-01 08:10 +0200
      Re: Application libraries private, Distutils metadata available for  console scripts and introspection Piotr Ożarowski <piotr@debian.org> - 2015-09-01 12:20 +0200
    Re: Application libraries private, Distutils metadata available for console scripts and introspection Ben Finney <ben+debian@benfinney.id.au> - 2015-09-02 10:40 +0200
      Re: Application libraries private, Distutils metadata available for  console scripts and introspection Robert Collins <robertc@robertcollins.net> - 2015-09-02 11:10 +0200
        Re: Application libraries private, Distutils metadata available for  console scripts and introspection Karsten Hilbert <Karsten.Hilbert@gmx.net> - 2015-09-02 11:20 +0200
          Re: Application libraries private, Distutils metadata available for  console scripts and introspection Robert Collins <robertc@robertcollins.net> - 2015-09-02 13:30 +0200
          Re: Application libraries private, Distutils metadata available for  console scripts and introspection Barry Warsaw <barry@debian.org> - 2015-09-02 15:50 +0200
        Re: Application libraries private, Distutils metadata available for console scripts and introspection Ben Finney <ben+debian@benfinney.id.au> - 2015-09-02 12:50 +0200
          Re: Application libraries private, Distutils metadata available for  console scripts and introspection Robert Collins <robertc@robertcollins.net> - 2015-09-02 13:30 +0200
        Re: Application libraries private, Distutils metadata available for  console scripts and introspection Barry Warsaw <barry@debian.org> - 2015-09-02 15:50 +0200
      Re: Application libraries private, Distutils metadata available for  console scripts and introspection Piotr Ożarowski <piotr@debian.org> - 2015-09-03 00:20 +0200
      Re: Application libraries private, Distutils metadata available for console scripts and introspection Ben Finney <ben+debian@benfinney.id.au> - 2015-11-27 00:00 +0100
        Re: Application libraries private, Distutils metadata available for  console scripts and introspection Piotr Ożarowski <piotr@debian.org> - 2015-11-27 00:30 +0100
        Re: Application libraries private, Distutils metadata available for  console scripts and introspection Barry Warsaw <barry@debian.org> - 2015-11-30 17:30 +0100

#7220 — Application libraries private, Distutils metadata available for console scripts and introspection

Howdy all,

How can I specify to Pybuild that an application should have its modules
all in a private namespace, but have the Distutils metadata also
available to `pkg_resources` queries?

I install its libraries to an application-specific space with
`PYBUILD_INSTALL_ARGS = --install-lib=/usr/share/foo/`::

    $ find /usr/share/FooApp/
    /usr/share/FooApp/
    /usr/share/FooApp/lorem
    /usr/share/FooApp/lorem/__init__.py
    /usr/share/FooApp/lorem/dolor.py
    /usr/share/FooApp/lorem/sit.py
    /usr/share/FooApp/lorem/amet.py
    /usr/share/FooApp/FooApp-1.2.3.egg-info
    /usr/share/FooApp/FooApp-1.2.3.egg-info/PKG-INFO
    /usr/share/FooApp/FooApp-1.2.3.egg-info/requires.txt
    /usr/share/FooApp/FooApp-1.2.3.egg-info/entry_points.txt
    /usr/share/FooApp/FooApp-1.2.3.egg-info/not-zip-safe
    /usr/share/FooApp/FooApp-1.2.3.egg-info/dependency_links.txt
    /usr/share/FooApp/FooApp-1.2.3.egg-info/top_level.txt
    /usr/share/FooApp/ipsum
    /usr/share/FooApp/ipsum/__init__.py
    /usr/share/FooApp/ipsum/consecteur.py
    /usr/share/FooApp/ipsum/adipiscing.py
    /usr/share/FooApp/ipsum/elit.py

(The actual code base is a fork of ‘dput’ to modernise its packaging
<URL:https://notabug.org/bignose/dput/> and Python idioms.)

The application's Python package is managed like other operating system
files, but is not in the general Python namespace for other programs to
import. Good.

The application has “console scripts” defined in the Distutils
`entry_points` mapping:

    $ cat ./setup.py
    […]
        entry_points={
            'console_scripts': [
                "foo=FooApp.foo:main",
                ],
            },
    […]

which installs command-line programs at `/usr/bin/foo`, for example.
Good.

The distribution metadata (the `FooApp-1.2.3.egg-info` directory) is
also installed to the private directory though, which makes it invisible
to `pkg_resources`:

    $ /usr/bin/foo
    Traceback (most recent call last):
      File "/usr/bin/foo", line 5, in <module>
        from pkg_resources import load_entry_point
      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3084, in <module>
        @_call_aside
      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3070, in _call_aside
        f(*args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3097, in _initialize_master_working_set
        working_set = WorkingSet._build_master()
      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 651, in _build_master
        ws.require(__requires__)
      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 952, in require
        needed = self.resolve(parse_requirements(requirements))
      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 839, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The FooApp==1.2.3' distribution was not found and is required by the application

Other queries to `pkg_resources` for this application's distribution,
for example to get the distribution version or homepage URL, will also
fail.

How can I have Pybuild specify to Distutils that the application's
library modules should be installed away from the public namespace and
not available for general import, but also that the `pkg_resources`
functionality should find the distribution metadata where expected?

-- 
 \     “When I was a kid I used to pray every night for a new bicycle. |
  `\    Then I realised that the Lord doesn't work that way so I stole |
_o__)                   one and asked Him to forgive me.” —Emo Philips |
Ben Finney

[toc] | [next] | [standalone]

#7221 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

[Multipart message — attachments visible in raw view] — view raw

PKG resources should find it anywhere in the python path. I'd the path
correct within the all processes?
On 1 Sep 2015 2:53 pm, "Ben Finney" <ben+debian@benfinney.id.au> wrote:

> Howdy all,
>
> How can I specify to Pybuild that an application should have its modules
> all in a private namespace, but have the Distutils metadata also
> available to `pkg_resources` queries?
>
> I install its libraries to an application-specific space with
> `PYBUILD_INSTALL_ARGS = --install-lib=/usr/share/foo/`::
>
>     $ find /usr/share/FooApp/
>     /usr/share/FooApp/
>     /usr/share/FooApp/lorem
>     /usr/share/FooApp/lorem/__init__.py
>     /usr/share/FooApp/lorem/dolor.py
>     /usr/share/FooApp/lorem/sit.py
>     /usr/share/FooApp/lorem/amet.py
>     /usr/share/FooApp/FooApp-1.2.3.egg-info
>     /usr/share/FooApp/FooApp-1.2.3.egg-info/PKG-INFO
>     /usr/share/FooApp/FooApp-1.2.3.egg-info/requires.txt
>     /usr/share/FooApp/FooApp-1.2.3.egg-info/entry_points.txt
>     /usr/share/FooApp/FooApp-1.2.3.egg-info/not-zip-safe
>     /usr/share/FooApp/FooApp-1.2.3.egg-info/dependency_links.txt
>     /usr/share/FooApp/FooApp-1.2.3.egg-info/top_level.txt
>     /usr/share/FooApp/ipsum
>     /usr/share/FooApp/ipsum/__init__.py
>     /usr/share/FooApp/ipsum/consecteur.py
>     /usr/share/FooApp/ipsum/adipiscing.py
>     /usr/share/FooApp/ipsum/elit.py
>
> (The actual code base is a fork of ‘dput’ to modernise its packaging
> <URL:https://notabug.org/bignose/dput/> and Python idioms.)
>
> The application's Python package is managed like other operating system
> files, but is not in the general Python namespace for other programs to
> import. Good.
>
> The application has “console scripts” defined in the Distutils
> `entry_points` mapping:
>
>     $ cat ./setup.py
>     […]
>         entry_points={
>             'console_scripts': [
>                 "foo=FooApp.foo:main",
>                 ],
>             },
>     […]
>
> which installs command-line programs at `/usr/bin/foo`, for example.
> Good.
>
> The distribution metadata (the `FooApp-1.2.3.egg-info` directory) is
> also installed to the private directory though, which makes it invisible
> to `pkg_resources`:
>
>     $ /usr/bin/foo
>     Traceback (most recent call last):
>       File "/usr/bin/foo", line 5, in <module>
>         from pkg_resources import load_entry_point
>       File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 3084, in <module>
>         @_call_aside
>       File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 3070, in _call_aside
>         f(*args, **kwargs)
>       File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 3097, in _initialize_master_working_set
>         working_set = WorkingSet._build_master()
>       File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 651, in _build_master
>         ws.require(__requires__)
>       File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 952, in require
>         needed = self.resolve(parse_requirements(requirements))
>       File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 839, in resolve
>         raise DistributionNotFound(req, requirers)
>     pkg_resources.DistributionNotFound: The FooApp==1.2.3' distribution
> was not found and is required by the application
>
> Other queries to `pkg_resources` for this application's distribution,
> for example to get the distribution version or homepage URL, will also
> fail.
>
> How can I have Pybuild specify to Distutils that the application's
> library modules should be installed away from the public namespace and
> not available for general import, but also that the `pkg_resources`
> functionality should find the distribution metadata where expected?
>
> --
>  \     “When I was a kid I used to pray every night for a new bicycle. |
>   `\    Then I realised that the Lord doesn't work that way so I stole |
> _o__)                   one and asked Him to forgive me.” —Emo Philips |
> Ben Finney
>
>

[toc] | [prev] | [next] | [standalone]

#7222

Robert Collins <robertc@robertcollins.net> writes:

> PKG resources should find it anywhere in the python path.

That's exactly the point, though. The packages are private to this
application, and should not be available for import by other programs.

On the other hand, the Distutils metadata (the ‘foo-1.2.3.egg-info’
directory) should be available for the program.

What I'd like is for the distribution to be installed such that:

* its modules are private (‘--install-lib=/usr/share/FooApp/’), and

* its distribution metadata is discoverable (‘--install-metadata=???’
  except there doesn't seem to be such an option).

> I'd the path correct within the all processes?

I don't know. I would not expect to need to alter how the program is run
in order for it to find its own distribution metadata.

-- 
 \          “When I get real bored, I like to drive downtown and get a |
  `\         great parking spot, then sit in my car and count how many |
_o__)                    people ask me if I'm leaving.” —Steven Wright |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#7224 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On 1 September 2015 at 17:35, Ben Finney <ben+debian@benfinney.id.au> wrote:
> Robert Collins <robertc@robertcollins.net> writes:
>
>> PKG resources should find it anywhere in the python path.
>
> That's exactly the point, though. The packages are private to this
> application, and should not be available for import by other programs.

Right - I did not suggest they should be.

> On the other hand, the Distutils metadata (the ‘foo-1.2.3.egg-info’
> directory) should be available for the program.

How is this the other hand? The Distutils data should be available to
and only to the programs that are allowed to access the packages.

> What I'd like is for the distribution to be installed such that:
>
> * its modules are private (‘--install-lib=/usr/share/FooApp/’), and
>
> * its distribution metadata is discoverable (‘--install-metadata=???’
>   except there doesn't seem to be such an option).

The distribution metadata is located via the python path; whatever
mechanism is making FooApp accessible to the program *should* also
make the metadata accessible, as long as:

>> I'd the path correct within the all processes?
>
> I don't know. I would not expect to need to alter how the program is run
> in order for it to find its own distribution metadata.

That is one of two options for what is happening to make packages in
/usr/share/FooApp/ accessible. Confirming it (or denying it) will let
us debug the issue.

-Rob

-- 
Robert Collins <rbtcollins@hp.com>
Distinguished Technologist
HP Converged Cloud

[toc] | [prev] | [next] | [standalone]

#7223

Ben Finney <ben+debian@benfinney.id.au> writes:

> The application has “console scripts” defined in the Distutils
> `entry_points` mapping:
>
>     $ cat ./setup.py
>     […]
>         entry_points={
>             'console_scripts': [
>                 "foo=FooApp.foo:main",
>                 ],
>             },
>     […]
>
> which installs command-line programs at `/usr/bin/foo`, for example.
> Good.

In an earlier message, a workaround was offered specifically for the
case of user commands (“scripts”): don't use entry points, instead make
symlinks from ‘/usr/share/FooApp/lorem/dolor.py’ to ‘/usr/bin/foo’.

That isn't sufficient, for several reasons:

* A ‘console_script’ entry point specifies a specific function within
  the module. Running the module file as ‘__main__’ isn't always a
  replacement for that.

* Distutils clobbers the execute bit on module files when installing
  them, so telling Debhelper to just make a symlink still doesn't result
  in an executable command.

* This doesn't address the problem that ‘pkg_resources’ still can't find
  the application's distribution metadata.

  Other queries to `pkg_resources` for this application's distribution,
  for example to get the distribution version or homepage URL, will also
  fail.

Would it be correct design for Pybuild to place Python libraries in one
location and the Distutils metadata in a different location; the former
private, the latter available at least to the application itself?

I don't quite know how that would work, but it sounds like what I'd
expect if we're going to have sensible “application private modules in
‘/usr/share/FooApp/’ where they aren't public” behaviour.

-- 
 \     “As scarce as truth is, the supply has always been in excess of |
  `\                                       the demand.” —Josh Billings |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#7225 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

it just works, are you sure the script you're invoking is in the right
dir?

| mkdir -p /tmp/foo/bar /tmp/foo/bar.egg-info
| cat >/tmp/foo/bar/__init__.py <<EOF
| def run():
|         print('it works!')
| EOF
| 
| cat >/tmp/foo/bar.egg-info/entry_points.txt <<EOF
| [console_scripts]
| run = bar:run
| EOF
| 
| 
| cat > /tmp/foo/run <<EOF
| #! /usr/bin/python3
| 
| __requires__ = 'bar'
| import sys
| from pkg_resources import load_entry_point
| 
| if __name__ == '__main__':
|     sys.exit(
|        load_entry_point('bar', 'console_scripts', 'run')()
|     )
| 
| EOF
| 
| chmod +x /tmp/foo/run
| /tmp/foo/run
| ln -s /tmp/foo/run /tmp/baz
| /tmp/baz

outputs "it works!"
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#7227

Ben Finney <ben+debian@benfinney.id.au> writes:

> How can I specify to Pybuild that an application should have its
> modules all in a private namespace, but have the Distutils metadata
> also available to `pkg_resources` queries?

I've made a real, minimal example of a Debian package of a Python
distribution, with the properties I've described.

* The ‘console_scripts’ entry point is used to specify command line
  programs, ‘lorem’ and ‘ipsum’, using that Distutils feature. Those are
  the correct names for the commands, so I don't want to mess with that.

* The ‘--install-lib’ option is used in ‘PYBUILD_INSTALL_ARGS’ to place
  the Python packages in an application-private directory,
  ‘/usr/share/foo-app/’. The point of that is so they won't be on the
  Python module search path.

* The ‘--install-scripts’ option is *not* used, because Distutils places
  the constructed scripts in the correct directory (‘/usr/bin/’) by
  default.

* The script names are correct, but are identical to names of
  directories in the ‘/usr/share/foo-app/’ top level. That's
  another good reason not to use the ‘--install-scripts’ option. It is
  also normal and expected, and shouldn't be a problem because the
  scripts will not exist there; they belong in ‘/usr/bin/’.

* The console scripts constructed by Distutils call the function
  ‘pkg_resources.load_entry_point’. That requires that the Distutils
  metadata (the ‘FooApp-1.2.3.egg-info/’ directory) be available to that
  function.

  According to Robert's earlier message, that means the Distutils
  metadata file needs to be not in the application's private directory,
  but in a directory on the Python module search path. That seems odd to
  me, since this is not amodule import being done.

What happens is that the Distutils metadata also gets hidden away in the
private directory. Then the ‘pkg_resources.load_entry_point’ function
tries to find the distribution, and can't because it's not in the Python
module search path.


So what i'm looking for is a way to tell Distutils at install time:

* Install the Python libraries to this specific private path that isn't
  part of the Python module search path” (‘--install-lib’).

* Ensure the Distutils metadata can be found by these specific console
  scripts (and, ideally, not by others) when they run. Either:

  * Construct the console scripts, at install time, to load the
    Distutils metadata at run time from that same private directory
    where Distutils installed it.

  Or:

  * Construct the Distutils metadata, at install time, such that it
    points to the private directory for these libraries. Put this
    metadata where it will be found at run time.

* Install the scripts to the default, public script install location.

That facility would ideally be an option I can just add to
‘PYBUILD_INSTALL_ARGS’.


Here is the output from the example. I can provide the files if anyone
wants to experiment.

=====
$ pwd
/home/bignose/Projects/debian/foo-app-1.2.3

$ find .
.
./setup.py
./lorem
./lorem/amet.py
./lorem/dolor.py
./lorem/sit.py
./lorem/__init__.py
./ipsum
./ipsum/elit.py
./ipsum/adipiscing.py
./ipsum/consecteur.py
./ipsum/__init__.py
./debian
./debian/compat
./debian/rules
./debian/changelog
./debian/control

$ cat ./setup.py
from setuptools import (setup, find_packages)

setup(
        name="FooApp",
        version="1.2.3",
        packages=find_packages(),

        entry_points={
            'console_scripts': [
                "lorem = FooApp.lorem.dolor:main",
                "ipsum = FooApp.ipsum.consecteur:main",
                ],
            },
        )

$ cat ./debian/rules 
#! /usr/bin/make -f

export PYBUILD_INSTALL_ARGS ?= --install-lib=/usr/share/foo-app/

%:
        dh $@ --with=python2 --buildsystem=pybuild

$ ./debian/rules clean
dh clean --with=python2 --buildsystem=pybuild
   dh_testdir -O--buildsystem=pybuild
   dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:170: python2.7 setup.py clean 
running clean
removing '/home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it
   dh_clean -O--buildsystem=pybuild

$ fakeroot ./debian/rules binary
dh binary --with=python2 --buildsystem=pybuild
   dh_testroot -O--buildsystem=pybuild
   dh_prep -O--buildsystem=pybuild
   dh_auto_install -O--buildsystem=pybuild
I: pybuild base:170: /usr/bin/python setup.py install --root /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app --install-lib=/usr/share/foo-app/
running install
running build
running build_py
running install_lib
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/ipsum/elit.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/ipsum/adipiscing.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/ipsum/consecteur.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/ipsum/__init__.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/lorem/amet.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/lorem/dolor.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/lorem/sit.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_2.7/build/lorem/__init__.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/elit.py to elit.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/adipiscing.py to adipiscing.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/consecteur.py to consecteur.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/__init__.py to __init__.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/amet.py to amet.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/dolor.py to dolor.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/sit.py to sit.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/__init__.py to __init__.pyc
running install_egg_info
running egg_info
creating FooApp.egg-info
writing FooApp.egg-info/PKG-INFO
writing top-level names to FooApp.egg-info/top_level.txt
writing dependency_links to FooApp.egg-info/dependency_links.txt
writing entry points to FooApp.egg-info/entry_points.txt
writing manifest file 'FooApp.egg-info/SOURCES.txt'
reading manifest file 'FooApp.egg-info/SOURCES.txt'
writing manifest file 'FooApp.egg-info/SOURCES.txt'
Copying FooApp.egg-info to /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/FooApp-1.2.3.egg-info
running install_scripts
Installing ipsum script to /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/bin
Installing lorem script to /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/bin
   dh_installdocs -O--buildsystem=pybuild
   dh_installchangelogs -O--buildsystem=pybuild   
   dh_python2 -O--buildsystem=pybuild
   dh_perl -O--buildsystem=pybuild
   dh_link -O--buildsystem=pybuild
   dh_compress -O--buildsystem=pybuild
   dh_fixperms -O--buildsystem=pybuild
   dh_installdeb -O--buildsystem=pybuild
   dh_gencontrol -O--buildsystem=pybuild
   dh_md5sums -O--buildsystem=pybuild
   dh_builddeb -O--buildsystem=pybuild
dpkg-deb: building package 'foo-app' in '../foo-app_1.2.3-1_all.deb'.

$ sudo dpkg -i ../foo-app_1.2.3-1_all.deb
(Reading database ... 812074 files and directories currently installed.)
Preparing to unpack .../debian/foo-app_1.2.3-1_all.deb ...
Unpacking foo-app (1.2.3-1) over (1.2.3-1) ...
Setting up foo-app (1.2.3-1) ...

$ dpkg --listfiles foo-app
/.
/usr
/usr/bin
/usr/bin/ipsum
/usr/bin/lorem
/usr/share
/usr/share/doc
/usr/share/doc/foo-app
/usr/share/doc/foo-app/changelog.Debian.gz
/usr/share/python
/usr/share/python/runtime.d
/usr/share/python/runtime.d/foo-app.rtupdate
/usr/share/foo-app
/usr/share/foo-app/FooApp-1.2.3.egg-info
/usr/share/foo-app/FooApp-1.2.3.egg-info/top_level.txt
/usr/share/foo-app/FooApp-1.2.3.egg-info/dependency_links.txt
/usr/share/foo-app/FooApp-1.2.3.egg-info/entry_points.txt
/usr/share/foo-app/FooApp-1.2.3.egg-info/PKG-INFO
/usr/share/foo-app/ipsum
/usr/share/foo-app/ipsum/elit.py
/usr/share/foo-app/ipsum/adipiscing.py
/usr/share/foo-app/ipsum/consecteur.py
/usr/share/foo-app/ipsum/__init__.py
/usr/share/foo-app/lorem
/usr/share/foo-app/lorem/amet.py
/usr/share/foo-app/lorem/dolor.py
/usr/share/foo-app/lorem/sit.py
/usr/share/foo-app/lorem/__init__.py

$ cat /usr/bin/lorem
#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'FooApp==1.2.3','console_scripts','lorem'
__requires__ = 'FooApp==1.2.3'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('FooApp==1.2.3', 'console_scripts', 'lorem')()
    )

$ /usr/bin/lorem
Traceback (most recent call last):
  File "/usr/bin/lorem", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3084, in <module>
    @_call_aside
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3070, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3097, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 651, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 952, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 839, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'FooApp==1.2.3' distribution was not found and is required by the application

=====

-- 
 \        “Absurdity, n. A statement or belief manifestly inconsistent |
  `\            with one's own opinion.” —Ambrose Bierce, _The Devil's |
_o__)                                                Dictionary_, 1906 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#7228 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On 2 September 2015 at 20:35, Ben Finney <ben+debian@benfinney.id.au> wrote:
> Ben Finney <ben+debian@benfinney.id.au> writes:

>   According to Robert's earlier message, that means the Distutils
>   metadata file needs to be not in the application's private directory,
>   but in a directory on the Python module search path. That seems odd to
>   me, since this is not amodule import being done.

So this is perhaps the disconnect: I did not say the metadata should
move: it should be in the private directory - it has to be adjacent to
the packages/modules its describing, since if it is present but the
package/module is not that is at the very best confusing for tools
like pip.

That private directory must be on the python search path when running
the application (or the modules can't be imported).

-Rob


-- 
Robert Collins <rbtcollins@hp.com>
Distinguished Technologist
HP Converged Cloud

[toc] | [prev] | [next] | [standalone]

#7229 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On Wed, Sep 02, 2015 at 09:06:01PM +1200, Robert Collins wrote:

> > Ben Finney <ben+debian@benfinney.id.au> writes:
> 
> >   According to Robert's earlier message, that means the Distutils
> >   metadata file needs to be not in the application's private directory,
> >   but in a directory on the Python module search path. That seems odd to
> >   me, since this is not amodule import being done.
> 
> So this is perhaps the disconnect: I did not say the metadata should
> move: it should be in the private directory - it has to be adjacent to
> the packages/modules its describing, since if it is present but the
> package/module is not that is at the very best confusing for tools
> like pip.
> 
> That private directory must be on the python search path

Maybe I'm a bit dense but isn't that what OP very much
DOESN'T want to _happen_ ?

Are you, by implication, saying that that's not _possible_ ?

> when running the application (or the modules can't be imported).

That reason doesn't hold -- the application can be written to
(or told to) "know" where to look and manipulate sys.path
accordingly (regardless of whether that's considered a good
idea or not).

Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346

[toc] | [prev] | [next] | [standalone]

#7232 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On 2 September 2015 at 21:19, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
> On Wed, Sep 02, 2015 at 09:06:01PM +1200, Robert Collins wrote:
>
>> > Ben Finney <ben+debian@benfinney.id.au> writes:
>>
>> >   According to Robert's earlier message, that means the Distutils
>> >   metadata file needs to be not in the application's private directory,
>> >   but in a directory on the Python module search path. That seems odd to
>> >   me, since this is not amodule import being done.
>>
>> So this is perhaps the disconnect: I did not say the metadata should
>> move: it should be in the private directory - it has to be adjacent to
>> the packages/modules its describing, since if it is present but the
>> package/module is not that is at the very best confusing for tools
>> like pip.
>>
>> That private directory must be on the python search path
>
> Maybe I'm a bit dense but isn't that what OP very much
> DOESN'T want to _happen_ ?

They don't want to have to alter the PYTHONPATH variable as far as I
can tell, but that is only one of a number of mechanisms to get a
directory on the search path.

> Are you, by implication, saying that that's not _possible_ ?

I suspect so.

>> when running the application (or the modules can't be imported).
>
> That reason doesn't hold -- the application can be written to
> (or told to) "know" where to look and manipulate sys.path
> accordingly (regardless of whether that's considered a good
> idea or not).

If sys.path is altered by the application to put the private directory
on the search path, then my assertion (that it needs to be on the
path) would be satisfied, and I'd expect it to work.

-Rob


-- 
Robert Collins <rbtcollins@hp.com>
Distinguished Technologist
HP Converged Cloud

[toc] | [prev] | [next] | [standalone]

#7233 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On Sep 02, 2015, at 11:19 AM, Karsten Hilbert wrote:

>> That private directory must be on the python search path
>
>Maybe I'm a bit dense but isn't that what OP very much
>DOESN'T want to _happen_ ?

Not on the default sys.path, sure.  But it has to get on sys.path at some
point before the application starts to import private modules.  How and where
does that happen?  (I.e. is it before or after the entry point tries to get
loaded?)

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#7230

Robert Collins <robertc@robertcollins.net> writes:

> That private directory must be on the python search path when running
> the application (or the modules can't be imported).

Not at all. The application's private modules are imported only via
relative imports.

-- 
 \       “Free thought is a necessary, but not a sufficient, condition |
  `\                                       for democracy.” —Carl Sagan |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#7231 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On 2 September 2015 at 22:45, Ben Finney <ben+debian@benfinney.id.au> wrote:
> Robert Collins <robertc@robertcollins.net> writes:
>
>> That private directory must be on the python search path when running
>> the application (or the modules can't be imported).
>
> Not at all. The application's private modules are imported only via
> relative imports.

Relative imports still consult the various python module search
mechanisms, which is the top level path, + per package paths (but to
import the highest of those packages you have to import the first
member from the root... which is on the top level path).

-Rob

-- 
Robert Collins <rbtcollins@hp.com>
Distinguished Technologist
HP Converged Cloud

[toc] | [prev] | [next] | [standalone]

#7234 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On Sep 02, 2015, at 09:06 PM, Robert Collins wrote:

>So this is perhaps the disconnect: I did not say the metadata should
>move: it should be in the private directory - it has to be adjacent to
>the packages/modules its describing, since if it is present but the
>package/module is not that is at the very best confusing for tools
>like pip.
>
>That private directory must be on the python search path when running
>the application (or the modules can't be imported).

It may be a problem of code execution order.   Where does the sys.path
manipulation happen to add the private application path to sys.path?  If that
happens after the wrapper script tries to load the entry point, it will go
boom in the way Ben is describing.  sys.path has to be hacked before that, but
I don't think the entry point wrapper creation will do that automatically.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#7235 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

[Multipart message — attachments visible in raw view] — view raw

> * The ‘--install-scripts’ option is *not* used, because Distutils places
>   the constructed scripts in the correct directory (‘/usr/bin/’) by
>   default.

and that's why it doesn't work. As I already suggested in my previous mail,
installing scripts into private directory and symlinking them in
/usr/bin is the cleanest solution IMHO
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#7915

Piotr Ożarowski <piotr@debian.org> writes:

> As I already suggested in my previous mail, installing scripts into
> private directory and symlinking them in /usr/bin is the cleanest
> solution IMHO

Thank you. I have come to a compromise solution on this which builds on
that suggestion.


Ben Finney <ben+debian@benfinney.id.au> writes:

> * The ‘console_scripts’ entry point is used to specify command line
>   programs, ‘lorem’ and ‘ipsum’, using that Distutils feature. Those are
>   the correct names for the commands, so I don't want to mess with that.

Main point of compromise: Change the installed names of the commands.

Second point of compromise: Make an application namespace package.


Distutils (and Python generally) makes it highly awkward to have an
importable module *also* be an executable program.

Python's import mechanism makes it infeasible to have a module file
named without a suffix; this makes it infeasible to name a command file
by Unix conventions of having no suffix for the implementation language.

There are many approaches to address this, which I won't detail here,
but suffice it to say that none of them work smoothly.

So the compromise is:

* Patch upstream's ‘setup.py’ to rename the command file to something
  like ‘execute-lorem’, so that it can be installed alongside the
  ‘lorem’ package directory.

* Create a symlink in the application-private directory, making an
  application namespace from which to import the private modules.

* Create a public symlink, ‘/usr/bin/lorem’, to the private
  ‘execute-lorem’ command file.

> * The ‘--install-lib’ option is used in ‘PYBUILD_INSTALL_ARGS’ to place
>   the Python packages in an application-private directory,
>   ‘/usr/share/foo-app/’. The point of that is so they won't be on the
>   Python module search path.

This remains the same; the ‘--install-lib’ option is used to place the
application-private packages in a private directory. That's the whole
point of this endeavour, and everything else is a workaround for
Python's awkwardness to deal with this.

The programs will still need to be importable, but we want to protect
against a conflict with some other publicly-installed Python library.

So we create a namespace ‘FooApp’ specifically for the application, with
a symlink internal to the application's own library directory.

> * The ‘--install-scripts’ option is *not* used, because Distutils places
>   the constructed scripts in the correct directory (‘/usr/bin/’) by
>   default.

We must now install the program command files to the same location, in
order that the application-private packages can be imported by those
programs.

So the ‘--install-scripts’ option is used, naming the same
application-private directory as for the ‘--install-lib’ option.

> * The script names are correct, but are identical to names of
>   directories in the ‘/usr/share/foo-app/’ top level. That's
>   another good reason not to use the ‘--install-scripts’ option. It is
>   also normal and expected, and shouldn't be a problem because the
>   scripts will not exist there; they belong in ‘/usr/bin/’.

Because the command files cannot keep their upstream name and live in
the same location as package directories with the same names, we need to
patch upstream's ‘setup.py’.

Patch upstream's ‘setup.py’ so the ‘console_scripts’ names differ from
the package directory names. For clarity of association and purpose, I
chose to name them ‘execute-lorem’ and ‘execute-ipsum’.

Patch the target of the ‘console_scripts’ entry points, so that those
entry points are preceded by the ‘FooApp’ namespace package.

>   According to Robert's earlier message, that means the Distutils
>   metadata file needs to be not in the application's private
>   directory, but in a directory on the Python module search path. That
>   seems odd to me, since this is not amodule import being done.

Thanks to everyone for explaining this to me; it's a messy hack, and I
do wish the Python Packaging Authority patience and strength in solving
this better than the mess we have now.

> Here is the output from the example. I can provide the files if anyone
> wants to experiment.

Here is an updated session to show how this works:

=====
$ pwd
/home/bignose/Projects/debian/foo-app-1.2.3

$ find .
.
./setup.py
./lorem
./lorem/amet.py
./lorem/dolor.py
./lorem/sit.py
./lorem/__init__.py
./ipsum
./ipsum/elit.py
./ipsum/adipiscing.py
./ipsum/consecteur.py
./ipsum/__init__.py
./debian
./debian/compat
./debian/rules
./debian/foo-app.links
./debian/changelog
./debian/control

$ cat ./setup.py
from setuptools import (setup, find_packages)

setup(
        name="FooApp",
        version="1.2.3",
        packages=find_packages(),

        entry_points={
            'console_scripts': [
                "execute-lorem = FooApp.lorem.dolor:main",
                "execute-ipsum = FooApp.ipsum.consecteur:main",
                ],
            },  
        )   


    debhelper (>= 9)

$ cat ./debian/rules 
#! /usr/bin/make -f

PACKAGE_NAME = foo-app

package_share_dir = /usr/share/${PACKAGE_NAME}
export PYBUILD_INSTALL_ARGS ?= \
        --install-lib=${package_share_dir}/ \
        --install-scripts=${package_share_dir}/

%:
        dh $@ --with=python3 --buildsystem=pybuild

$ cat ./debian/foo-app.links 
# foo-app.links
# Symbolic links to create for the ‘foo-app’ package.

# Make an application-private package name for the libraries.
/usr/share/foo-app /usr/share/foo-app/FooApp

# Make public symlinks for the installed commands.
/usr/share/foo-app/execute-lorem /usr/bin/lorem
/usr/share/foo-app/execute-ipsum /usr/bin/ipsum

$ ./debian/rules clean
dh clean --with=python3 --buildsystem=pybuild
   dh_testdir -O--buildsystem=pybuild
   dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:184: python3.4 setup.py clean
running clean
removing '/home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.4' does not exist -- can't clean it
   dh_clean -O--buildsystem=pybuild

$ fakeroot ./debian/rules binary
dh binary --with=python3 --buildsystem=pybuild
   dh_testdir -O--buildsystem=pybuild
   dh_auto_configure -O--buildsystem=pybuild
I: pybuild base:184: python3.4 setup.py config
running config
   dh_auto_build -O--buildsystem=pybuild
I: pybuild base:184: /usr/bin/python3 setup.py build
running build
running build_py
creating /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum
copying ipsum/elit.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum
copying ipsum/adipiscing.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum
copying ipsum/consecteur.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum
copying ipsum/__init__.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum
creating /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem
copying lorem/amet.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem
copying lorem/dolor.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem
copying lorem/sit.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem
copying lorem/__init__.py -> /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem
   dh_auto_test -O--buildsystem=pybuild
I: pybuild base:184: cd /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build; python3.4 -m unittest discover -v

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
   dh_testroot -O--buildsystem=pybuild
   dh_prep -O--buildsystem=pybuild
   dh_auto_install -O--buildsystem=pybuild
I: pybuild base:184: /usr/bin/python3 setup.py install --root /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app --install-lib=/usr/share/foo-app/ --install-scripts=/usr/share/foo-app/
running install
running build
running build_py
running install_lib
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum/elit.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum/adipiscing.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum/consecteur.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/ipsum/__init__.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum
creating /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem/amet.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem/dolor.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem/sit.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
copying /home/bignose/Projects/debian/foo-app-1.2.3/.pybuild/pythonX.Y_3.4/build/lorem/__init__.py -> /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/elit.py to elit.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/adipiscing.py to adipiscing.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/consecteur.py to consecteur.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/ipsum/__init__.py to __init__.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/amet.py to amet.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/dolor.py to dolor.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/sit.py to sit.cpython-34.pyc
byte-compiling /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/lorem/__init__.py to __init__.cpython-34.pyc
running install_egg_info
running egg_info
creating FooApp.egg-info
writing entry points to FooApp.egg-info/entry_points.txt
writing top-level names to FooApp.egg-info/top_level.txt
writing FooApp.egg-info/PKG-INFO
writing dependency_links to FooApp.egg-info/dependency_links.txt
writing manifest file 'FooApp.egg-info/SOURCES.txt'
reading manifest file 'FooApp.egg-info/SOURCES.txt'
writing manifest file 'FooApp.egg-info/SOURCES.txt'
Copying FooApp.egg-info to /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/FooApp-1.2.3.egg-info
Skipping SOURCES.txt
running install_scripts
Installing execute-lorem script to /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/
Installing execute-ipsum script to /home/bignose/Projects/debian/foo-app-1.2.3/debian/foo-app/usr/share/foo-app/
   dh_installdocs -O--buildsystem=pybuild
   dh_installchangelogs -O--buildsystem=pybuild   
   dh_python3 -O--buildsystem=pybuild
   dh_perl -O--buildsystem=pybuild
   dh_link -O--buildsystem=pybuild
   dh_strip_nondeterminism -O--buildsystem=pybuild
   dh_compress -O--buildsystem=pybuild
   dh_fixperms -O--buildsystem=pybuild
   dh_installdeb -O--buildsystem=pybuild
   dh_gencontrol -O--buildsystem=pybuild
   dh_md5sums -O--buildsystem=pybuild
   dh_builddeb -O--buildsystem=pybuild
dpkg-deb: building package 'foo-app' in '../foo-app_1.2.3-2_all.deb'.

$ sudo dpkg -i ../foo-app_1.2.3-2_all.deb
[sudo] password for bignose:
(Reading database ... 759304 files and directories currently installed.)
Preparing to unpack ../foo-app_1.2.3-2_all.deb ...
Unpacking foo-app (1.2.3-2) over (1.2.3-2) ...
Setting up foo-app (1.2.3-2) ...

$ dpkg --listfiles foo-app
/.
/usr
/usr/bin
/usr/share
/usr/share/doc
/usr/share/doc/foo-app
/usr/share/doc/foo-app/changelog.Debian.gz
/usr/share/foo-app
/usr/share/foo-app/FooApp-1.2.3.egg-info
/usr/share/foo-app/FooApp-1.2.3.egg-info/top_level.txt
/usr/share/foo-app/FooApp-1.2.3.egg-info/dependency_links.txt
/usr/share/foo-app/FooApp-1.2.3.egg-info/entry_points.txt
/usr/share/foo-app/FooApp-1.2.3.egg-info/PKG-INFO 
/usr/share/foo-app/ipsum
/usr/share/foo-app/ipsum/elit.py
/usr/share/foo-app/ipsum/adipiscing.py
/usr/share/foo-app/ipsum/consecteur.py
/usr/share/foo-app/ipsum/__init__.py
/usr/share/foo-app/execute-ipsum
/usr/share/foo-app/execute-lorem
/usr/share/foo-app/lorem
/usr/share/foo-app/lorem/amet.py
/usr/share/foo-app/lorem/dolor.py
/usr/share/foo-app/lorem/sit.py
/usr/share/foo-app/lorem/__init__.py
/usr/share/python3
/usr/share/python3/runtime.d
/usr/share/python3/runtime.d/foo-app.rtupdate
/usr/bin/ipsum
/usr/bin/lorem
/usr/share/foo-app/FooApp

$ cat /usr/bin/lorem
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'FooApp==1.2.3','console_scripts','execute-lorem'
__requires__ = 'FooApp==1.2.3'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('FooApp==1.2.3', 'console_scripts', 'execute-lorem')()
    )

$ /usr/bin/lorem
>From FooApp.lorem.dolor.main
Hello, world!

=====

-- 
 \       “Anyone who puts a small gloss on [a] fundamental technology, |
  `\          calls it proprietary, and then tries to keep others from |
_o__)           building on it, is a thief.” —Tim O'Reilly, 2000-01-25 |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#7916 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

[Ben Finney, 2015-11-26]
> * Patch upstream's ‘setup.py’ to rename the command file to something
>   like ‘execute-lorem’, so that it can be installed alongside the
>   ‘lorem’ package directory.

instead of patching I usually let it install into /usr/bin and mv it to
/usr/share/foo/ under a different name (and then symlink) - see f.e.
gaupol's debian/rules

> * Create a symlink in the application-private directory, making an
>   application namespace from which to import the private modules.

in gaupol I use --install-lib=/usr/share/gaupol/
so that in the end I have:

 /usr/share/gaupol/gaupol - the library
 /usr/share/gaupol/run - the script
 /usr/bin/gaupol→/usr/share/gaupol/run - symlink

-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#7934 — Re: Application libraries private, Distutils metadata available for console scripts and introspection

On Nov 27, 2015, at 09:56 AM, Ben Finney wrote:

>Python's import mechanism makes it infeasible to have a module file
>named without a suffix; this makes it infeasible to name a command file
>by Unix conventions of having no suffix for the implementation language.
>
>There are many approaches to address this, which I won't detail here,
>but suffice it to say that none of them work smoothly.

I agree that it makes sense to separate the importable module from the
executable script file, but I'd argue that's good practice anyway for ease of
development, use, and testing.  And there are good tools and techniques to
take advantage of that, including setuptools entry points, or shim-scripts
with only an import and an `__name__` conditional.

I've found it good to minimize the logic in executable scripts because they
are harder to test.  If essentially all your script does is import your main()
and then call it, you can more easily incorporate a unit test for main().
Entry points make this a no-brainer; you don't even have to write the shim,
although to invoke it you need to build the package, so either use
`$python -m` or tox to create the shim in the tox-managed venv.

(I don't personally like application private library directories, but if you
do, you will have to fiddle with sys.path somehow to make your main()
importable.)

Cheers,
-Barry

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web