Path: csiph.com!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod
From: Scott Kitterman <debian@kitterman.com>
Newsgroups: linux.debian.maint.python
Subject: Re: Python Wheel Related Policy Change
Date: Thu, 30 Apr 2020 22:40:02 +0200
Message-ID: <A1o8q-2w2-7@gated-at.bofh.it>
References: <A183E-11c-5@gated-at.bofh.it> <A1kev-a3-37@gated-at.bofh.it> <A1mgi-1nf-23@gated-at.bofh.it>
Old-Return-Path: <debian@kitterman.com>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="nextPart3055977.npYSNkMI7z"; micalg="pgp-sha512"; protocol="application/pgp-signature"
Approved: robomod@news.nic.it
Lines: 134
Organization: linux.* mail to news gateway
Sender: robomod@news.nic.it
Xref: csiph.com linux.debian.maint.python:12544

--nextPart3055977.npYSNkMI7z
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"

On Thursday, April 30, 2020 2:32:24 PM EDT Stefano Rivera wrote:
> Hi Scott (2020.04.30_16:23:41_+0000)
> 
> > Making dirtbike build a py3 wheel for setuptools (and pkg_resources),
> > which is technically accurate at this point since dirtbike can't build
> > wheels from a> 
> > python2 package was trivial to do:
> >  dirtbike/__init__.py | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > Three of the five insertions were comment.
> 
> That seems reasonable, although if we're going down that road, it
> probably makes no sense for any of them to be universal.

If we were talking about maintaining this for multiple release cycles with 
lots of version divergence, I would agree.  Let's not do more than we have to 
until python2 is gone (whether it is before bullseye or after).

> > If someone wants to provide a patch to dirtbike so it can also build
> > wheels
> > from python-setuptools and python-pkg-resources, then we could add then to
> > build-depends and provide both a set of py2 wheels and py3 wheels for as
> > long as they are in the archive.
> 
> The window for doing this is probably closing, too.
> 
> I hacked this in Ubuntu 20.04, just before release:
> https://launchpadlibrarian.net/475592029/python-pip_20.0.2-5_20.0.2-5ubuntu1
> .diff.gz
> 
> As you can see, python-wheel is no longer available, and some of the
> other libraries dirtbike uses are probably heading that way.
> 
> So, to make that approach work, we have to introduce more py2-only
> libraries again, or allow dirtbike to build py2 wheels while running
> under python3.

I had in mind the latter.  I looked and there's not a lot of difference in the 
metadata of a pure python wheel that's universal and one that's not.  The 
universal one has tags for both versions:

$ cat WHEEL 
Wheel-Version: 1.0
Generator: bdist_wheel (0.34.2)
Root-Is-Purelib: true
Tag: py2-none-any
Tag: py3-none-any

As a result, one could add some kind of py2 flag to dirtbike to tell it to look 
in /usr/lib/python2.7/dist-packages and make a 'universal' wheel from there.  
It could then rename the files from py2.py3-none-any.whl to py2-none-any.whl.  
The bonus Tag in the WHEEL file shouldn't hurt anything.

They key is I think we can do this without actually running python2, we just 
need to be able to install python-setuptools/pkg-resources.  That should be 
supportable ~as long as python2.7 is in the archive.

> > It still needs the Python policy change since they aren't universal, but
> > it
> > would allow setuptools to keep up for python3 in pip without dropping the
> > local wheels from python2.  We also have ipaddr back in the archive to
> > support wheeling it so pip will run with python2.
> 
> But, without the py2 setuptools wheels, I don't know much that buys us.

Technically I think it's solvable and is supportable until the python2 
interpreter goes away.

> Long-term: pypy (2.7) is going to continue to be around, until rpython
> moves to python3 (which is still officially never, but people are
> starting to poke at it).
> https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-support-python2
> 
> If we have a python 2 interpreter, it's really nice to have working
> virtualenv for it. Makes it a lot more useful for users, esp. as we
> don't have many debian libraries packaged for it.

Agreed.  As long as pip supports python2, we should try.  Worst case we can 
tell people to find their own interpreter and do a --download install with 
setuptools pinned to 44.0.0 (that approach doesn't need ipaddr because they'll 
get the upstream pip in the virtualenv too.  

> So, the options are:
> 1. pin pip dependencies at versions that support py2, forever. Obviously
>    this is a no-go.
> 2. Ship py2 and py3 wheels.
>    2.1. package the py2 weels with dirtbike. This requires bringing back
>         python-wheel, or more work on dirtbike.
>    2.2. Ship static py2 wheels (like we did pre-dirtbike). It's not like
>         upstream is continuing development...
> 3. Let virtualenv always download pip from pypi for py2. Only ship py3
>    wheels.
> 
> The easy options here are 2.2 and 3.

2.1a Where needed package 'py2' wheels with dirtbike running python3.

2.2 is using sourceless binaries, so I think it's got DFSG issues.  We need 
the preferred form of modification and a bdist_wheel is not that.

I think 2.1a would be nice if someone can manage it.  We'll need to support 3 
eventually.  I plan to implement 3 as an option when I upload pip 20.1 and 
virtualenv 2.0.18.

Scott K

--nextPart3055977.npYSNkMI7z
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part.
Content-Transfer-Encoding: 7Bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAl6rNjcACgkQeNfe+5rV
mvEddhAAtSyXbIG7E7sekRHggXqBED0SCF7W9SWucJS+iDHKc+ENcCBSzoDzh2VC
af/SUG6ma1GCnkbe2sOp6br3/pIrtu+rup+Do0aHhSO2hnkE1I3/zLPZCRNL7RYm
61YAcporEg3U9JqOKrW2RjWknpAG/15kV+eLlPauGAy0KkocNXhEEJuCIR75qOqk
3KFKh8pup1OAoJ9eND2jaCKvsLym/McP1gvfEyvZs9j1kTHDdolTmrFfTx57ZX6f
4MLL47JnAqnGVOQliqQ3NpoYQSIBds1zH3d/WUegMkYEOAj83kWDTgMntHkWzWDa
4tWUq251ZcbqypYVtO9CfUxr6x3FN9ppF1iwL8Wj7NQLr0VCfEdFRUvXSIIKPhjM
Uwa0cEyGRyzPFyMemXApSEpxSBvNZdQHMesRAUzrjlIWXuLyhgj4kj2jJfNtflA6
Bas8RsuZ3paRlkr+t1wm3wMM+02vNK/akLierlWCBcuv3wt6JRYLsRZxc7M1jsQR
TAb3bf08i9trCe+CuRgGXyTtfxWPoLu8ZIH8Kc86BcWcAiGl7DySRnLbrZROn2sG
ilGugs3ihRo0u13oD/8jqx7WRazpNA+/YT1pNHBYZdc8OaiXuWinIJd8J0aWDOx+
pc3ZDktsjYWNYswyVguqfMQtqC82x7QtX57N2bXI450+vsj8ENE=
=Qfpg
-----END PGP SIGNATURE-----

--nextPart3055977.npYSNkMI7z--