Path: csiph.com!aioe.org!bofh.it!news.nic.it!robomod
From: Scott Kitterman <debian@kitterman.com>
Newsgroups: linux.debian.maint.python
Subject: Re: Python Wheel Related Policy Change
Date: Fri, 01 May 2020 00:30:01 +0200
Message-ID: <A1pQR-3BN-1@gated-at.bofh.it>
References: <A183E-11c-5@gated-at.bofh.it> <A1o8q-2w2-7@gated-at.bofh.it> <A1pea-38B-5@gated-at.bofh.it>
Old-Return-Path: <debian@kitterman.com>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="nextPart6675062.yd42G6rWGc"; micalg="pgp-sha512"; protocol="application/pgp-signature"
Approved: robomod@news.nic.it
Lines: 127
Organization: linux.* mail to news gateway
Sender: robomod@news.nic.it
Xref: csiph.com linux.debian.maint.python:12547

--nextPart6675062.yd42G6rWGc
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"

On Thursday, April 30, 2020 5:49:20 PM EDT Stefano Rivera wrote:
> Hi Scott (2020.04.30_20:33:59_+0000)
> 
> > > That seems reasonable, although if we're going down that road, it
> > > probably makes no sense for any of them to be universal.
> > 
> > If we were talking about maintaining this for multiple release cycles with
> > lots of version divergence, I would agree.  Let's not do more than we have
> > to until python2 is gone (whether it is before bullseye or after).
> 
> I suspect pypy (2.7) will probably be around post-bullseye, unless
> somebody funds pypy to migrate rpython to python 3.
> 
> But yeah, we can change strategy later, if appropriate.

Well, we have also talked about pypy vendoring as much of the python2.7 
package as it needs to build itself so we don't have to support it in the 
archive as an active interpreter, but that's a different discussion.

> > As a result, one could add some kind of py2 flag to dirtbike to tell it to
> > look in /usr/lib/python2.7/dist-packages and make a 'universal' wheel
> > from there. It could then rename the files from py2.py3-none-any.whl to
> > py2-none-any.whl. The bonus Tag in the WHEEL file shouldn't hurt
> > anything.
> > 
> > They key is I think we can do this without actually running python2, we
> > just need to be able to install python-setuptools/pkg-resources.  That
> > should be supportable ~as long as python2.7 is in the archive.
> 
> Yep, that sounds good.
> 
> > Agreed.  As long as pip supports python2, we should try.  Worst case we
> > can
> > tell people to find their own interpreter and do a --download install with
> > setuptools pinned to 44.0.0 (that approach doesn't need ipaddr because
> > they'll get the upstream pip in the virtualenv too.
> 
> When we get to the download route: pip can parse Requires-Python, and it
> looks like virtualenv uses the system pip to download pip, so the
> pinning shouldn't be necessary, I think.

I haven't really tested that yet, so I don't doubt that.  It was an 
assumption.  My first run at virtualenv ran aground on trying to have different 
package lists depending on if the install invoked --download or not (since a 
--download install with the upstream pip doesn't need all the unvendored 
wheels).  So that's something to sort later.  We'll want to document how to do 
it even if we get a working solution that doesn't require it.

> > > So, the options are:
> > > 1. pin pip dependencies at versions that support py2, forever. Obviously
> > > 
> > >    this is a no-go.
> > > 
> > > 2. Ship py2 and py3 wheels.
> > > 
> > >    2.1. package the py2 weels with dirtbike. This requires bringing back
> > >    
> > >         python-wheel, or more work on dirtbike.
> > >    
> > >    2.2. Ship static py2 wheels (like we did pre-dirtbike). It's not like
> > >    
> > >         upstream is continuing development...
> > > 
> > > 3. Let virtualenv always download pip from pypi for py2. Only ship py3
> > > 
> > >    wheels.
> > > 
> > > The easy options here are 2.2 and 3.
> > 
> > 2.1a Where needed package 'py2' wheels with dirtbike running python3.
> 
> Ah, yeah.
> 
> Still requires keeping py2 source packages whenever a library in the pip
> dependency stack drops py2 support. But if they move slowly, that's
> fine.

So far setuptools is the only one.  It might pick up though now that they've 
done it.  We've split packages into a source for python2 and a source for 
python3 in a number of cases, so I think this scales reasonably.  We can just 
yank all the python2 sources the same time pip drops python2 or we do.

> > 2.2 is using sourceless binaries, so I think it's got DFSG issues.  We
> > need
> > the preferred form of modification and a bdist_wheel is not that.
> 
> They could be bdist_wheeled at build time, within a single source
> package.
> 
> > I think 2.1a would be nice if someone can manage it.  We'll need to
> > support 3 eventually.  I plan to implement 3 as an option when I upload
> > pip 20.1 and virtualenv 2.0.18.
> 
> Not necessarily. 2.2 is equivalent to 3, but without hitting PyPI.

Let's focus on 2.1a so we don't have to have that argument ...

I might actually have a little time tomorrow to work on it.

Scott K
--nextPart6675062.yd42G6rWGc
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part.
Content-Transfer-Encoding: 7Bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAl6rT3wACgkQeNfe+5rV
mvHu6BAAh/TTQIcolGahqbI3o+7KKwvqz7SWEm9ll+Lm/AVWiPOzt8NWnM/W3b/W
qO3XvspeQABJvWZ5Z4Hn8bHFXOlE9eSaxIFkvtjd3VkQJfwcZ9O7+lj+JvvMdZUy
hxqsYtS8Y8x3AMyoQsE2C3sNcFkWP9jblANSxElq4mkMGDWvW9ckaE++yNc/H0Qe
w1rWHYXqZtphMuZAoUp7PVHPn+JyTSAmYN3Z8ZtY4iTwrGbp/rz9zhweTXFn7yQ7
ZgWvhhYIK1WGN9YdNGdzDYygyQkBJUOWfDe89UJU+iJ2KW7JKXz5H6eTJwx0OeIN
Mi0lrGuPDlCFDd8PN+FXON1+GLYnNHxRpdyN9uHVycF+5j18vh1tmsoftuzDaoYY
3qfWj87+N4WY/9ZzmXoIjrfwNIGrPs49ABitD22/16NKUr9B3cRV5WrxymwIzv3z
cEkYNHmm754q9zN1QnccwQCDzsmNM+c+skXLv3jDwN+r6K12kXryOIpkMVs2aTv+
EgWv72xRa0QKhU1XOp5EEqL515+BdruZ5wifZAerB/2970pH+PO3zGvj72RfOkxr
ZvTu9rsU5GyLx/3ZXb+IMzRO+QHUXS7hN9GaFshCAHnqSy4UE4Pyxvhe6TwpLhMx
RxlTpxKpH1bRlIR3prEqhMEn75beCn3JaR9rX9nw3fZdOvzndS4=
=x3ph
-----END PGP SIGNATURE-----

--nextPart6675062.yd42G6rWGc--