Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #12544

Re: Python Wheel Related Policy Change

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

On Thursday, April 30, 2020 2:32:24 PM EDT Stefano Rivera wrote:
> Hi Scott (2020.04.30_16:23:41_+0000)
> 
> > Making dirtbike build a py3 wheel for setuptools (and pkg_resources),
> > which is technically accurate at this point since dirtbike can't build
> > wheels from a> 
> > python2 package was trivial to do:
> >  dirtbike/__init__.py | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > Three of the five insertions were comment.
> 
> That seems reasonable, although if we're going down that road, it
> probably makes no sense for any of them to be universal.

If we were talking about maintaining this for multiple release cycles with 
lots of version divergence, I would agree.  Let's not do more than we have to 
until python2 is gone (whether it is before bullseye or after).

> > If someone wants to provide a patch to dirtbike so it can also build
> > wheels
> > from python-setuptools and python-pkg-resources, then we could add then to
> > build-depends and provide both a set of py2 wheels and py3 wheels for as
> > long as they are in the archive.
> 
> The window for doing this is probably closing, too.
> 
> I hacked this in Ubuntu 20.04, just before release:
> https://launchpadlibrarian.net/475592029/python-pip_20.0.2-5_20.0.2-5ubuntu1
> .diff.gz
> 
> As you can see, python-wheel is no longer available, and some of the
> other libraries dirtbike uses are probably heading that way.
> 
> So, to make that approach work, we have to introduce more py2-only
> libraries again, or allow dirtbike to build py2 wheels while running
> under python3.

I had in mind the latter.  I looked and there's not a lot of difference in the 
metadata of a pure python wheel that's universal and one that's not.  The 
universal one has tags for both versions:

$ cat WHEEL 
Wheel-Version: 1.0
Generator: bdist_wheel (0.34.2)
Root-Is-Purelib: true
Tag: py2-none-any
Tag: py3-none-any

As a result, one could add some kind of py2 flag to dirtbike to tell it to look 
in /usr/lib/python2.7/dist-packages and make a 'universal' wheel from there.  
It could then rename the files from py2.py3-none-any.whl to py2-none-any.whl.  
The bonus Tag in the WHEEL file shouldn't hurt anything.

They key is I think we can do this without actually running python2, we just 
need to be able to install python-setuptools/pkg-resources.  That should be 
supportable ~as long as python2.7 is in the archive.

> > It still needs the Python policy change since they aren't universal, but
> > it
> > would allow setuptools to keep up for python3 in pip without dropping the
> > local wheels from python2.  We also have ipaddr back in the archive to
> > support wheeling it so pip will run with python2.
> 
> But, without the py2 setuptools wheels, I don't know much that buys us.

Technically I think it's solvable and is supportable until the python2 
interpreter goes away.

> Long-term: pypy (2.7) is going to continue to be around, until rpython
> moves to python3 (which is still officially never, but people are
> starting to poke at it).
> https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-support-python2
> 
> If we have a python 2 interpreter, it's really nice to have working
> virtualenv for it. Makes it a lot more useful for users, esp. as we
> don't have many debian libraries packaged for it.

Agreed.  As long as pip supports python2, we should try.  Worst case we can 
tell people to find their own interpreter and do a --download install with 
setuptools pinned to 44.0.0 (that approach doesn't need ipaddr because they'll 
get the upstream pip in the virtualenv too.  

> So, the options are:
> 1. pin pip dependencies at versions that support py2, forever. Obviously
>    this is a no-go.
> 2. Ship py2 and py3 wheels.
>    2.1. package the py2 weels with dirtbike. This requires bringing back
>         python-wheel, or more work on dirtbike.
>    2.2. Ship static py2 wheels (like we did pre-dirtbike). It's not like
>         upstream is continuing development...
> 3. Let virtualenv always download pip from pypi for py2. Only ship py3
>    wheels.
> 
> The easy options here are 2.2 and 3.

2.1a Where needed package 'py2' wheels with dirtbike running python3.

2.2 is using sourceless binaries, so I think it's got DFSG issues.  We need 
the preferred form of modification and a bdist_wheel is not that.

I think 2.1a would be nice if someone can manage it.  We'll need to support 3 
eventually.  I plan to implement 3 as an option when I upload pip 20.1 and 
virtualenv 2.0.18.

Scott K

Back to linux.debian.maint.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python Wheel Related Policy Change Scott Kitterman <debian@kitterman.com> - 2020-04-30 05:30 +0200
  Re: Python Wheel Related Policy Change Matthias Klose <doko@debian.org> - 2020-04-30 09:30 +0200
    Re: Python Wheel Related Policy Change Scott Kitterman <debian@kitterman.com> - 2020-04-30 12:00 +0200
      Re: Python Wheel Related Policy Change Scott Kitterman <debian@kitterman.com> - 2020-04-30 18:30 +0200
        Re: Python Wheel Related Policy Change Stefano Rivera <stefanor@debian.org> - 2020-04-30 20:40 +0200
          Re: Python Wheel Related Policy Change Scott Kitterman <debian@kitterman.com> - 2020-04-30 22:40 +0200
            Re: Python Wheel Related Policy Change Stefano Rivera <stefanor@debian.org> - 2020-04-30 23:50 +0200
              Re: Python Wheel Related Policy Change Scott Kitterman <debian@kitterman.com> - 2020-05-01 00:30 +0200
                Re: Python Wheel Related Policy Change Scott Kitterman <debian@kitterman.com> - 2020-05-01 22:50 +0200
                Re: Python Wheel Related Policy Change Nicholas D Steeves <nsteeves@gmail.com> - 2020-05-01 23:10 +0200

csiph-web