Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #11958 > unrolled thread

package_data in python package

Back to article view | Back to linux.debian.maint.python

Contents

  package_data in python package Félix Sipma <felix+debian@gueux.org> - 2019-10-28 15:40 +0100
    Re: package_data in python package Scott Kitterman <debian@kitterman.com> - 2019-10-28 15:50 +0100
      Re: package_data in python package Félix Sipma <felix+debian@gueux.org> - 2019-10-28 16:10 +0100
        Re: package_data in python package Scott Kitterman <debian@kitterman.com> - 2019-10-28 16:20 +0100
          Re: package_data in python package Félix Sipma <felix+debian@gueux.org> - 2019-10-28 16:30 +0100
            Re: package_data in python package Ole Streicher <olebole@debian.org> - 2019-10-28 17:20 +0100

#11958 — package_data in python package

[Multipart message — attachments visible in raw view] — view raw

Hi,

I have a problem with package_data files in a package ("khard", which is
not currently under the debian-python umbrella but it is on my todo list
to transfer it, I just didn't find the time yet) not getting installed.

I submitted an error upstream,
https://github.com/scheibler/khard/issues/231 but they can't reproduce
the problem.

I "fixed" the issue by adding a patch to add "khard/data/" to
MANIFEST.in (with "recursive-include khard/data/ *"), but I don't get
why this directory doesn't get installed by default... Is it because of
a limitation of dh-python? Have I forgot an option somewhere?

Link to the patch and the source of the package:
https://sources.debian.org/src/khard/0.15.0-2/debian/patches/0003-add-khard-data-dir-to-MANIFEST.in.patch/

Thanks for your help!

-- 
Félix

[toc] | [next] | [standalone]

#11959

On Monday, October 28, 2019 10:25:44 AM EDT Félix Sipma wrote:
> Hi,
> 
> I have a problem with package_data files in a package ("khard", which is
> not currently under the debian-python umbrella but it is on my todo list
> to transfer it, I just didn't find the time yet) not getting installed.
> 
> I submitted an error upstream,
> https://github.com/scheibler/khard/issues/231 but they can't reproduce
> the problem.
> 
> I "fixed" the issue by adding a patch to add "khard/data/" to
> MANIFEST.in (with "recursive-include khard/data/ *"), but I don't get
> why this directory doesn't get installed by default... Is it because of
> a limitation of dh-python? Have I forgot an option somewhere?
> 
> Link to the patch and the source of the package:
> https://sources.debian.org/src/khard/0.15.0-2/debian/patches/0003-add-khard-> data-dir-to-MANIFEST.in.patch/
> 
> Thanks for your help!

It's the way setuptools works.  Looking at the current upstream 
recommendations:

https://python-packaging.readthedocs.io/en/latest/non-code-files.html

your solution seems correct.

Scott K

[toc] | [prev] | [next] | [standalone]

#11960

[Multipart message — attachments visible in raw view] — view raw

On 2019-10-28 10:44-0400, Scott Kitterman wrote:
> It's the way setuptools works.  Looking at the current upstream 
> recommendations:
> 
> https://python-packaging.readthedocs.io/en/latest/non-code-files.html
> 
> your solution seems correct.

Hi Scott,

Thanks for your message. I'm not sure I get why upstream can't reproduce
the problem in this case... setuptools, as used in the setup.py seems to
install the data dir just fine, and without my patch.

That's why I thought that it was maybe a limitation of dh-python...

-- 
Félix

[toc] | [prev] | [next] | [standalone]

#11961

[Multipart message — attachments visible in raw view] — view raw

On Monday, October 28, 2019 11:02:40 AM EDT Félix Sipma wrote:
> On 2019-10-28 10:44-0400, Scott Kitterman wrote:
> > It's the way setuptools works.  Looking at the current upstream
> > recommendations:
> > 
> > https://python-packaging.readthedocs.io/en/latest/non-code-files.html
> > 
> > your solution seems correct.
> 
> Hi Scott,
> 
> Thanks for your message. I'm not sure I get why upstream can't reproduce
> the problem in this case... setuptools, as used in the setup.py seems to
> install the data dir just fine, and without my patch.
> 
> That's why I thought that it was maybe a limitation of dh-python...

It does depend considerably on how you do the install.  For example, to 
replicate what pip does (to get the data files installed) you have to do:

python setup.py install --single-version-externally-managed --record=/dev/null

I don't find I need to do that with pybuild though.  I find setuptools handling 
of data files highly mysterious and counter-intuitive, so I doubt I'll have 
more specific advice.

Scott K

[toc] | [prev] | [next] | [standalone]

#11962

[Multipart message — attachments visible in raw view] — view raw

On 2019-10-28 11:15-0400, Scott Kitterman wrote:
> On Monday, October 28, 2019 11:02:40 AM EDT Félix Sipma wrote:
>> On 2019-10-28 10:44-0400, Scott Kitterman wrote:
>>> It's the way setuptools works.  Looking at the current upstream
>>> recommendations:
>>> 
>>> https://python-packaging.readthedocs.io/en/latest/non-code-files.html
>>> 
>>> your solution seems correct.
>> 
>> Hi Scott,
>> 
>> Thanks for your message. I'm not sure I get why upstream can't reproduce
>> the problem in this case... setuptools, as used in the setup.py seems to
>> install the data dir just fine, and without my patch.
>> 
>> That's why I thought that it was maybe a limitation of dh-python...
> 
> It does depend considerably on how you do the install.  For example, to 
> replicate what pip does (to get the data files installed) you have to do:
> 
> python setup.py install --single-version-externally-managed --record=/dev/null
> 
> I don't find I need to do that with pybuild though.  I find setuptools handling 
> of data files highly mysterious and counter-intuitive, so I doubt I'll have 
> more specific advice.
> 
> Scott K

They just run "python3 setup.py install" (https://github.com/scheibler/khard/issues/231):

    $ python3 -m venv test-for-231
    $ source test-for-231/bin/activate
    $ python3 setup.py install
    $ tree test-for-231/lib/python3.7/site-packages/khard-0.15.0-py3.7.egg
    test-for-231/lib/python3.7/site-packages/khard-0.15.0-py3.7.egg
    ├── EGG-INFO
    │   ├── dependency_links.txt
    │   ├── entry_points.txt
    │   ├── not-zip-safe
    │   ├── PKG-INFO
    │   ├── requires.txt
    │   ├── SOURCES.txt
    │   └── top_level.txt
    └── khard
        ├── actions.py
        ├── address_book.py
        ├── carddav_object.py
        ├── cli.py
        ├── config.py
        ├── data
        │   ├── config.spec
        │   └── template.yaml
        ├── helpers.py
        ├── __init__.py
        ├── khard.py
        ├── __main__.py
        ├── object_type.py
        └── version.py
    
    3 directories, 20 files

For my part, I "just" build the debian package with "gbp buildpackage",
and with "dh $@ --with python3 --buildsystem=pybuild" in debian/rules.

The solutions I say are: I can keep the patch in the package (without
really understanding the need for it) or try to convince upstream to
merge it (but they can't reproduce the problem, so why would they do
that?). In both cases, probably others will encounter the same problem
in other packages... It would be nice to fix the problem once for all
(if possible :)).

Regards,

-- 
Félix

[toc] | [prev] | [next] | [standalone]

#11963

Félix Sipma <felix+debian@gueux.org> writes:
> They just run "python3 setup.py install" (https://github.com/scheibler/khard/issues/231):
> [...]
>
> For my part, I "just" build the debian package with "gbp buildpackage",
> and with "dh $@ --with python3 --buildsystem=pybuild" in debian/rules.
>
> The solutions I say are: I can keep the patch in the package (without
> really understanding the need for it) or try to convince upstream to
> merge it (but they can't reproduce the problem, so why would they do
> that?). In both cases, probably others will encounter the same problem
> in other packages... It would be nice to fix the problem once for all
> (if possible :)).

Just FYI, https://bugs.debian.org/940728 describes a similar problem,
although there the unhandled files are Python source files. pybuild
seems to handle things different from plain setup.py.

Cheers

Ole

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web