Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #8110

Re: Amend Debian Python Proposal to Include More Python Metadata?

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

Hey Donald!

As far as using pip to do stuff system-wide, I wrote thoughts on
http://notes.pault.ag/debian-python

As for the rest of it, distutils is actually concretely shitty, and
replacing it with setuptools
sounds sane. Sounds like a solid idea.

Perhaps we can add a Lintian warning for using distutils; it's really nasty
cruft,
and it's going to bite the archive sooner rather than later.

Cheers,
  Paul

On Thu, Jan 21, 2016 at 9:19 AM, Donald Stufft <donald@stufft.io> wrote:

> Hello!
>
> I'd like to suggest a change to the Debian Policy around Python packages
> that
> will help enable the world of Python packaging to continue to progress
> forward.
>
> First, a little bit of background:
>
> At the Python level there are three metadata formats for Python packaging:
>
> * The original, setuptools style .egg-info directories.
> * The distutils style .egg-info *file* added to distutils at some point.
> * The new and improved, wheel based .dist-info directories.
>
> The presence of any of these files will signal to Python tools that a
> particular distribution has been installed, however there are two fairly
> major
> and important differences between the distutils style, and the other two.
>
> 1. The distutils style has no provisions to record what files on the system
>    belong to the installed distribution, making it appear to Python tooling
>    that there *are* no files other than the metadata file itself.
>
> 2. The distutils style has no provisions to include additional metadata
> files
>    in the metadata, making it impossible to extend the python level
> metadata
>    with additional files.
>
> I have a series of improvements that I'd like to make to the packaging
> toolchain that will sort of build on one another, but which is not going to
> function correctly with the distutils style metadata and I'm hoping that I
> can
> convince y'all to make it policy to default to generating one of the other
> two
> kinds (with varying methods, more on that later).
>
> Concretely the thing that this is blocking right now, is that with the
> newly
> released pip 8.0 I tried to make it so that pip will refuse to uninstall a
> project that is installed with distutils style metadata. This is because
> we do
> not have any way to associate the actual .py (and others) files on disk
> with
> the installed metadata, so all we have ever done is just simply remove the
> metadata file, making it appear as if the item is uninstalled but leaving
> behind all of the actual files. However I'm going to be reverting this in a
> pip 8.0.1 release because it caused a decent amount of breakage amongst
> pip's
> users, almost all of them people who are attempting to upgrade OS provided
> packages using pip.
>
> Now, I know that upgrading OS provided packages using pip is less than
> optimal
> and I would greatly prefer that people did not do it (and I'm generally in
> agreement) however if we don't enable people to do it, they'll just
> continue to
> use an old version of pip and file bugs. It's a non starter for pip to
> make it
> impossible to do.
>
> In addition to the uninstall bit, it also means that things like pip show
> -f
> return junk information for packages installed in this way.
>
> Beyond just (eventually) enabling pip to disable uninstallations of
> distutils
> based installs this will start to allow some other future changes that I
> think
> will be more interesting to Debian. The uninstallation of distutils based
> installs comes hand in hand with pip stomping all over already existing
> files
> willy nilly because the way upgrading a project like that works is pip
> uninstalls the metadata file that says X is installed, then it just
> overwrites
> over any of the files that happen to be in it's way when it installs the
> newer
> version. If we can remove the need for pip to gleefully overwrite files to
> support these types of installed packages, then we can make it so pip will
> hard fail if it attempts to overwrite an already existing file on disk.
>
> An additional benefit here is that by switching to using the directory
> based
> options, we can add additional metadata files to the installed projects,
> much
> like the INSTALLER file from PEP376 (IIRC). This file will likely be the
> path
> to having pip refuse to touch OS owned files all together without some
> sort of
> --force flag to override the safety switch.
>
> As far as compatibility goes, pip has always forced everything to be
> installed
> using setuptools and as far as I am aware, there's no real fallout from
> doing
> so. I think in 2016 it's pretty reasonable to assume that a Python project
> is
> capable of being installed using setuptools instead of distutils.
>
> So without getting into the actual *method* of doing this (of which there
> are
> several different options with different trade offs) does this sound like
> something at all that Debian would be interested in?
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>


-- 
:wq

Back to linux.debian.maint.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-21 15:30 +0100
  Re: Amend Debian Python Proposal to Include More Python Metadata? Paul Tagliamonte <paultag@debian.org> - 2016-01-21 15:50 +0100
    Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-21 16:10 +0100
      Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-21 17:00 +0100
  Re: Amend Debian Python Proposal to Include More Python Metadata? Piotr Ożarowski <piotr@debian.org> - 2016-01-22 16:40 +0100
    Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-22 17:00 +0100
      Re: Amend Debian Python Proposal to Include More Python Metadata? Piotr Ożarowski <piotr@debian.org> - 2016-01-22 17:20 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Paul Tagliamonte <paultag@debian.org> - 2016-01-22 17:50 +0100
          Re: Amend Debian Python Proposal to Include More Python Metadata? Piotr Ożarowski <piotr@debian.org> - 2016-01-22 18:20 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-22 18:00 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Piotr Ożarowski <piotr@debian.org> - 2016-01-22 18:40 +0100
      Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-22 18:00 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-22 18:20 +0100
          Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-22 18:50 +0100
            Re: Amend Debian Python Proposal to Include More Python Metadata? Fred Drake <fred@fdrake.net> - 2016-01-22 19:40 +0100
              Re: Amend Debian Python Proposal to Include More Python Metadata? Fred Drake <fred@fdrake.net> - 2016-01-22 19:50 +0100
                Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-22 20:00 +0100
              Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-22 19:50 +0100
      Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-23 09:20 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-23 09:30 +0100
  Re: Amend Debian Python Proposal to Include More Python Metadata? Barry Warsaw <barry@debian.org> - 2016-01-22 23:40 +0100
    Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-23 00:00 +0100
      Re: Amend Debian Python Proposal to Include More Python Metadata? Barry Warsaw <barry@debian.org> - 2016-01-23 00:10 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-23 00:20 +0100
      Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-23 00:10 +0100
        Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-23 00:30 +0100
          Re: Amend Debian Python Proposal to Include More Python Metadata? Scott Kitterman <debian@kitterman.com> - 2016-01-23 01:20 +0100
            Re: Amend Debian Python Proposal to Include More Python Metadata? Donald Stufft <donald@stufft.io> - 2016-01-23 02:00 +0100
          pip as distutils/setuptools/setup.py wrapper Piotr Ożarowski <piotr@debian.org> - 2016-01-26 19:30 +0100

csiph-web