Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #12159 > unrolled thread

Understanding pybuild

Back to article view | Back to linux.debian.maint.python

Contents

  Understanding pybuild Ole Streicher <olebole@debian.org> - 2019-12-19 13:30 +0100
    Re: Understanding pybuild Dmitry Shachnev <mitya57@debian.org> - 2019-12-20 06:50 +0100
      Re: Understanding pybuild Ole Streicher <olebole@debian.org> - 2019-12-20 08:40 +0100
        Re: Understanding pybuild Dmitry Shachnev <mitya57@debian.org> - 2019-12-20 12:20 +0100

#12159 — Understanding pybuild

Hi,

I am currently updating scikit-image (skimage) to its newest version,
and I have some difficulties to understand how pybuild works (for
setup.py packages):

1. first, dh_auto_build calls "python3.X setup.py build" to create the
packages (Python files and compiled extensions) in
..pybuild/cpython3_3.X/. This however does not include the copying of any
data files.

2. dh_auto_test then tries runs the tests in .pybuild. If the test need
data files installed, they must be copied manually (right?).

3. dh_auto_install finally calls "python3.X setup.py install ..." to
install the complete packages (Python files, extensions, and data files)
from their original place to debian/tmp/. This includes a re-compilation
of the extensions.

What I do not understand now is: what is the purpose of step 1? Except
for the tests, these files seem to be unused. And for the tests, they
are not complete, since the data files are missing.

And what is the canonical way to get the data files copied to
..pybuild/...? They are already marked as such in setup.py. Doing this
manually by the Debian maintainer seems a constant source of error.

Best

Ole

[toc] | [next] | [standalone]

#12163

[Multipart message — attachments visible in raw view] — view raw

Hi Ole!

On Thu, Dec 19, 2019 at 01:09:40PM +0100, Ole Streicher wrote:
> Hi,
>
> I am currently updating scikit-image (skimage) to its newest version,
> and I have some difficulties to understand how pybuild works (for
> setup.py packages):
>
> 1. first, dh_auto_build calls "python3.X setup.py build" to create the
> packages (Python files and compiled extensions) in
> ..pybuild/cpython3_3.X/. This however does not include the copying of any
> data files.
>
> 2. dh_auto_test then tries runs the tests in .pybuild. If the test need
> data files installed, they must be copied manually (right?).

You can use debian/pybuild.testfiles to copy the test files. See pybuild(1)
for details.

--
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#12164

Hi Dmitry,

Dmitry Shachnev <mitya57@debian.org> writes:
> On Thu, Dec 19, 2019 at 01:09:40PM +0100, Ole Streicher wrote:
>> 2. dh_auto_test then tries runs the tests in .pybuild. If the test need
>> data files installed, they must be copied manually (right?).
>
> You can use debian/pybuild.testfiles to copy the test files. See pybuild(1)
> for details.

That would formalize it a bit, but still does not solve the underlying
problem: it requires that I maintain the list of these files manually,
while upstream already does this (by adding them to the list of data
files).

And also, the built time tests are with pybuild not done with the files
(anc compilations) that are finally installed, but the installed files
(especially Python extensions) are created again from scratch, doubling
the compile time and introducing another difficile source of error.

Best regards

Ole

[toc] | [prev] | [next] | [standalone]

#12165

[Multipart message — attachments visible in raw view] — view raw

On Fri, Dec 20, 2019 at 08:19:41AM +0100, Ole Streicher wrote:
> That would formalize it a bit, but still does not solve the underlying
> problem: it requires that I maintain the list of these files manually,
> while upstream already does this (by adding them to the list of data
> files).
>
> And also, the built time tests are with pybuild not done with the files
> (anc compilations) that are finally installed, but the installed files
> (especially Python extensions) are created again from scratch, doubling
> the compile time and introducing another difficile source of error.

I think pybuild just calls setup.py build and then setup.py install.

So it is a question to upstream build system, why are files recompiled during
install.

--
Dmitry Shachnev

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web