Path: csiph.com!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod
From: Nicholas D Steeves <nsteeves@gmail.com>
Newsgroups: linux.debian.maint.python
Subject: Re: PEP-517/PEP-518 Support In Debian
Date: Fri, 05 Jun 2020 02:50:02 +0200
Message-ID: <Ae8Iy-4We-7@gated-at.bofh.it>
References: <zUTqG-72H-9@gated-at.bofh.it> <zV64x-6yd-3@gated-at.bofh.it> <Ae7Mt-4nS-1@gated-at.bofh.it> <Ae7W9-4r8-9@gated-at.bofh.it>
Old-Return-Path: <nsteeves@gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature"
Approved: robomod@news.nic.it
Lines: 157
Organization: linux.* mail to news gateway
Sender: robomod@news.nic.it
Xref: csiph.com linux.debian.maint.python:12628

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Hi Scott,

Thank you for the quick reply!

Scott Kitterman <debian@kitterman.com> writes:

> On Thursday, June 4, 2020 7:39:59 PM EDT Nicholas D Steeves wrote:
>> Scott Kitterman <debian@kitterman.com> writes:
>> > On Monday, April 13, 2020 5:18:53 AM EDT Dmitry Shachnev wrote:
>> >> Hi Scott!
>> >>=20
>> >> On Sun, Apr 12, 2020 at 06:31:57PM -0400, Scott Kitterman wrote:
>> >> > This being roughly the mid-point in the development cycle, I though=
t it
>> >> > might be good to see where we are in terms of future Python packagi=
ng
>> >> > developments.
>> >> >=20
>> >> > As I understand it, PEP-517 and PEP-518 are 'the future'.
>>=20
>> While importing the latest release of Fissix (a grammar parser used by
>> Bowler) I found that it had a pyproject.toml but no setup.py.
>>=20
>> >> > I took a look at the presence of pyproject.toml files in the Debian
>> >> > archive. There are currently 99 packages.  Of those, only 28 specif=
y a
>> >> > 'build-backend', which is required by 517/518 to be useful for buil=
ding
>> >> > a
>> >> > package.
>> >> >=20
>> >> > 25 specify: build-backend =3D "setuptools.build_meta" (including
>> >> > setuptools)
>> >> > 3 specify: build-backend =3D "flit_core.buildapi" (including flit)
>> >>=20
>> >> pyqt5 and pyqt5webengine specify: build-backend =3D "sipbuild.api".
>>=20
>> Is there existing documentation around if/how build-backend can be used
>> to work around the absence of setup.py?
>>=20
>> > So they do.  They didn't show up in my codesearch.d.n results, that ma=
kes
>> > me wonder what else I missed.  If anyone has an idea of a better way to
>> > track this, please speak up.
>> >=20
>> >> > If build-backend is not specified, the build system has to fall bac=
k to
>> >> > setup.py.
>> >> >=20
>> >> > I've recently package flit (just arrived in Testing) and written a =
flit
>> >> > plugin for pybuild that's pending review for merging[1].  I also
>> >> > packaged
>> >> > pep517 for our pip package, so that's available to support future
>> >> > Debian
>> >> > tool development in this area.
>> >=20
>> > P1otr merged the flit plugin a little while ago, so it'll be in the ne=
xt
>> > dh- python upload.
>>=20
>> I think this is probably what is needed to unblock work on fissix (and
>> thus Bowler), because its Makefile has:
>>=20
>>     python -m pip install -r requirements.txt
>>     python -m pip install -r requirements-dev.txt
>>     python -m flit install --symlink
>>=20
>> [snip]
>>=20
>> Any advise for existing workarounds would be appreciated.  So far the
>> only idea I've had is writing or generating a setup.py, and then adding
>> it as a quilt patch.
>
> That is probably the best approach for now.
>

Thank you for the ACK :-)

> Ultimately we have two choices:
>
> 1.  Add per-backend capability into pybuild as I have done with flit.=20=
=20
> Depending on how many of them there are, this may or may not be a reasona=
ble=20
> approach.
>
> 2.  Use the pep517 module with pip in our package build system.  I think=
=20
> Fedora has gone this direction.
>
> I prefer #1 because we want to produce a traditional bdist, not wheels, b=
ut=20
> we'll have to do the work.  I don't think we want an impenetrable morass =
of=20
> zip files in /usr/lib/python3/dist-packages.
>

I also prefer #1.  Here's an idea for reducing the work, and more
importantly reducing the Debian-specific investment in time:

Dephell (from the author of Poetry?) claims to support: egginfo, sdist,
wheel, pip, piplock, pipfile, pipfilelock, poetry, poetrylock, already
installed system-wide packages, plus setup.py, flit, conda, and
pyproject.

More interestingly it claims to be able to convert between formats, eg:

  dephell deps convert --from=3Dpyproject.toml --to=3Dsetup.py

Within a Debian packaging context this seems like it would be useful,
because packages whose upstreams change formats could continue to use
their existing rules file with the addition of a 'dephell deps convert'
step.

It seems like it might fit well with DPMT Policy, in that we could
recommend conversion to a particular format, for use with a particular
buildsystem, for uniformity of packaging.  Finally, if that conversion
capability works well then an effect is that Debian packaging is
effectively decoupled from upstream buildsystem changes, and this seems
like a good thing.

Downside: Their installation instructions (curl -L dephell.org/install |
python3) make me wonder if there is a fundamental cultural difference
that would make this a no-go.

tldr: Could integrating Dephell into dh_python solve these challenges
once and for all?


Regards,
Nicholas

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEE4qYmHjkArtfNxmcIWogwR199EGEFAl7ZlmsACgkQWogwR199
EGHzQw/6A5nU6yWBy92nxiNQMQSKEslqN0/LJFv+akslScRCNjQ2XtzVUYxVIvI6
hElg58/XAx/+1IZC8zAp4x0/peZtFcgHMpUX11LnCJh0iOGClO0lBA9uMMtao2Zb
PWmRcQ4LwanpvHY0w056KLGQqFybBOjY13u8OlgBJIINMShfuNcGtehBUcC5KO5s
Xm1Hu+FLeQXpoKGBXG2Oiyj3EbEL95er/9IHEMYf69aheke6WI5COEtUDdsW3ffM
Oi/TXO0yC3LNgmLGXxgcuegvc+w3HwpLLinEtAkeWqphkcyusb7dRRo//4ralMqZ
HaeViPfQz7e26tXsCKJ5T6r/tJVMV3nxnH3cTQdfPyYnbBhpariFjK5z0T0fMeuj
+Bnbv+Th6hY32JHSWwushqmF0t92//4kLJUqASPoUhGa0+7JGLhQYhRfLaL55mkS
P4GPw3HJo/Bh77TcSDkm07nt1QmmJd/vgWYgkbA4SjxlxMV2pLEwT7mvskAUv+P3
yaxoXLmMj2xuEVWT57KLMhSGnbkGUDvfsnv/CXzjKnUU5qq65Z4QjpDVs0+Qtwa1
Z1qYr+rKyRjopNuLdzHAKbu6xP45kj/6HW4cdtX06XkcLKAo9IZ92709pXMe9aCe
r89NI2vj6/5eTZ6BGa4OYdJJMn7B0poSzVMefG+XVSGdIN7zNA8=
=dwHG
-----END PGP SIGNATURE-----
--=-=-=--