Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #14666 > unrolled thread

Different behavior of package with pyproject.toml

Back to article view | Back to linux.debian.maint.python

Contents

  Different behavior of package with pyproject.toml  Ole Streicher <olebole@debian.org> - 2022-12-03 17:50 +0100
    Re: Different behavior of package with pyproject.toml Scott Kitterman <debian@kitterman.com> - 2022-12-04 03:00 +0100
      Re: Different behavior of package with pyproject.toml Ole Streicher <olebole@debian.org> - 2022-12-04 17:10 +0100
        Re: Different behavior of package with pyproject.toml Scott Kitterman <debian@kitterman.com> - 2022-12-04 17:10 +0100
          Re: Different behavior of package with pyproject.toml Ole Streicher <olebole@debian.org> - 2022-12-04 18:10 +0100

#14666 — Different behavior of package with pyproject.toml

Hi,

I need some help with a package that switched to using pyproject.toml
only. The package is asdf-astropy, and the problem I have that it does
not package all Python files recursively: for example astropy_asdf.io
files are missing (as seen in unstable now).

https://tracker.debian.org/pkg/asdf-astropy
https://salsa.debian.org/debian-astro-team/asdf-astropy

What I do not understand is that this happens "sometimes":

* when I build the package in a clean schroot with "debuild", files in
  subpackages are missing (but no error message in the log)
  
* when I install git, then a debuild results in a complete package

* when I cut&paste the command from the build log [1], the copy process
  always happens completely, independent of whether git was installed or
  not

* when I use pbuilder, files of subpackages are never package, with our
  without git in the build dependencies

[1] python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir .pybuild/cpython3_3.11

Since the package "sometimes" is built correctly, I think that upstream
did it right; however I have no idea where the problem could be. Does
anyone have an idea?

Best

Ole

[toc] | [next] | [standalone]

#14680

[Multipart message — attachments visible in raw view] — view raw

On Saturday, December 3, 2022 11:19:06 AM EST Ole Streicher wrote:
> Hi,
> 
> I need some help with a package that switched to using pyproject.toml
> only. The package is asdf-astropy, and the problem I have that it does
> not package all Python files recursively: for example astropy_asdf.io
> files are missing (as seen in unstable now).
> 
> https://tracker.debian.org/pkg/asdf-astropy
> https://salsa.debian.org/debian-astro-team/asdf-astropy
> 
> What I do not understand is that this happens "sometimes":
> 
> * when I build the package in a clean schroot with "debuild", files in
>   subpackages are missing (but no error message in the log)
> 
> * when I install git, then a debuild results in a complete package
> 
> * when I cut&paste the command from the build log [1], the copy process
>   always happens completely, independent of whether git was installed or
>   not
> 
> * when I use pbuilder, files of subpackages are never package, with our
>   without git in the build dependencies
> 
> [1] python3.11 -m build --skip-dependency-check --no-isolation --wheel
> --outdir .pybuild/cpython3_3.11
> 
> Since the package "sometimes" is built correctly, I think that upstream
> did it right; however I have no idea where the problem could be. Does
> anyone have an idea?

My first guess is that in some circumstances setuptools is doing the installing 
and in others it's the dh-python pyproject plugin (using the installer 
module).  Looking at the build log on buildd.d.o, I can see that build is 
using the plugin/installer.

I think it would be useful to stop the build right before the install step 
starts [1], manually unpack the wheel (it's a zip file) and see if it has all 
the files in it.  If it does, then plugin/build is doing the right thing and 
it's an issue with either the plugin or the installer module.

Since the build stage uses setuptools as part of it's build process, I thing 
it's most likely an installer issue.  I recently found the installer module 
doesn't correctly handle flit data files, but I was able to work around that 
with appropriate .install files.

Scott K

[1] pybuild plugin_pyproject:118: Unpacking wheel built for python3.10 with 
"installer" module

[toc] | [prev] | [next] | [standalone]

#14684

Hi Scott,

thanks for the hint.

Scott Kitterman <debian@kitterman.com> writes:
> My first guess is that in some circumstances setuptools is doing the installing 
> and in others it's the dh-python pyproject plugin (using the installer 
> module).  Looking at the build log on buildd.d.o, I can see that build is 
> using the plugin/installer.
>
> I think it would be useful to stop the build right before the install step 
> starts [1], manually unpack the wheel (it's a zip file) and see if it has all 
> the files in it.  If it does, then plugin/build is doing the right thing and 
> it's an issue with either the plugin or the installer module.

I couldn't manage this; however when looking into the log, I see

---------------------------------8<---------------------------------------------
  dh_auto_build -O--buildsystem=pybuild
I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build" module
I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir /builds/debian-astro-team/asdf-astropy/debian/output/source_dir/.pybuild/cpython3_3.11 
/usr/lib/python3/dist-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
  warnings.warn(msg, _BetaConfiguration)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/asdf_astropy
copying asdf_astropy/_version.py -> build/lib/asdf_astropy
[... no asdf_astropy/io files here ...]
* Building wheel...
Successfully built asdf_astropy-0.3.0-py3-none-any.whl
I: pybuild plugin_pyproject:118: Unpacking wheel built for python3.11 with "installer" module
---------------------------------8<---------------------------------------------

So I would suspect that it is already the build does something wrong
here. Which package is that? pybuild-plugin-pyproject?

Best

Ole

[toc] | [prev] | [next] | [standalone]

#14685

On December 4, 2022 3:42:15 PM UTC, Ole Streicher <olebole@debian.org> wrote:
>Hi Scott,
>
>thanks for the hint.
>
>Scott Kitterman <debian@kitterman.com> writes:
>> My first guess is that in some circumstances setuptools is doing the installing 
>> and in others it's the dh-python pyproject plugin (using the installer 
>> module).  Looking at the build log on buildd.d.o, I can see that build is 
>> using the plugin/installer.
>>
>> I think it would be useful to stop the build right before the install step 
>> starts [1], manually unpack the wheel (it's a zip file) and see if it has all 
>> the files in it.  If it does, then plugin/build is doing the right thing and 
>> it's an issue with either the plugin or the installer module.
>
>I couldn't manage this; however when looking into the log, I see
>
>---------------------------------8<---------------------------------------------
>  dh_auto_build -O--buildsystem=pybuild
>I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build" module
>I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir /builds/debian-astro-team/asdf-astropy/debian/output/source_dir/.pybuild/cpython3_3.11 
>/usr/lib/python3/dist-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
>  warnings.warn(msg, _BetaConfiguration)
>running bdist_wheel
>running build
>running build_py
>creating build
>creating build/lib
>creating build/lib/asdf_astropy
>copying asdf_astropy/_version.py -> build/lib/asdf_astropy
>[... no asdf_astropy/io files here ...]
>* Building wheel...
>Successfully built asdf_astropy-0.3.0-py3-none-any.whl
>I: pybuild plugin_pyproject:118: Unpacking wheel built for python3.11 with "installer" module
>---------------------------------8<---------------------------------------------
>
>So I would suspect that it is already the build does something wrong
>here. Which package is that? pybuild-plugin-pyproject?


python3-installer

Scott K

[toc] | [prev] | [next] | [standalone]

#14686

Scott Kitterman <debian@kitterman.com> writes:
> On December 4, 2022 3:42:15 PM UTC, Ole Streicher <olebole@debian.org> wrote:
>>So I would suspect that it is already the build does something wrong
>>here. Which package is that? pybuild-plugin-pyproject?
>
>
> python3-installer


Thank you, I filed a bug.

Best

Ole

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web