Path: csiph.com!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod
From: Simon McVittie <smcv@debian.org>
Newsgroups: linux.debian.bugs.dist,linux.debian.maint.python
Subject: Bug#1026312: Setuptools 65.5.0-1.1 breaks installing Python modules/extensions via meson
Date: Tue, 20 Dec 2022 20:10:01 +0100
Message-ID: <FEQ0h-c0AA-9@gated-at.bofh.it>
References: <FDYVX-btDj-1@gated-at.bofh.it> <FDYVX-btDj-1@gated-at.bofh.it>
X-Mailbox-Line: From debian-bugs-dist-request@lists.debian.org  Tue Dec 20 19:03:11 2022
Old-Return-Path: <debbugs@buxtehude.debian.org>
X-Spam-Flag: NO
X-Spam-Score: -3.999
Reply-To: Simon McVittie <smcv@debian.org>, 1026312@bugs.debian.org
Resent-To: debian-bugs-dist@lists.debian.org
Resent-Cc: Matthias Klose <doko@debian.org>
X-Debian-Pr-Message: followup 1026312
X-Debian-Pr-Package: src:setuptools
X-Debian-Pr-Source: setuptools
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
X-Debian-User: smcv
X-Debian-Message: from BTS
X-Mailing-List: <debian-bugs-dist@lists.debian.org> archive/latest/1749599
List-ID: <debian-bugs-dist.lists.debian.org>
List-URL: <https://lists.debian.org/debian-bugs-dist/>
Approved: robomod@news.nic.it
Lines: 73
Organization: linux.* mail to news gateway
Sender: robomod@news.nic.it
X-Original-Cc: Stefano Rivera <stefanor@debian.org>, debian-python@lists.debian.org, Lucas Nussbaum <lucas@debian.org>
X-Original-Date: Tue, 20 Dec 2022 18:59:12 +0000
X-Original-Message-ID: <Y6IGADJkq/jfA8sP@momentum.pseudorandom.co.uk>
X-Original-References: <167135830602.2874730.4603630218284602212.reportbug@ploma.enricozini.org> <167135830602.2874730.4603630218284602212.reportbug@ploma.enricozini.org>
Xref: csiph.com linux.debian.bugs.dist:1129666 linux.debian.maint.python:14762

Control: severity -1 serious
Control: block 1026526 by -1
Control: block 1026751 by -1
Control: block 1026732 by -1
Control: affects -1 + meson python3-distutils src:dbus-python src:libgit2-glib src:gi-docgen

On Sun, 18 Dec 2022 at 11:11:46 +0100, Enrico Zini wrote:
> After the 65.5.0-1.1 update, installing Python modules and extensions
> via meson makes them end up in /usr/local instead of /usr.
> 
> More details are in this debian-devel thread:
> 
> https://lists.debian.org/debian-devel/2022/12/msg00152.html
> 
> This currently breaks wreport and dballe, and xraylib when I try to
> build its Python extensions.

In Lucas Nussbaum's latest round of mass-rebuilds, multiple packages
including dbus-python, gi-docgen and libgit2-glib FTBFS because their
Python modules are installed into /usr/local instead of the expected /usr.

gi-docgen_2022.2+ds-1 is a nice simple example. When it was built on the
buildds, this happened:

> cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 meson .. --wrap-mode=nodownload --buildtype=plain --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=lib/x86_64-linux-gnu -Ddevelopment_tests=false
...
>    dh_auto_install -i -O--buildsystem=meson
>	cd obj-x86_64-linux-gnu && DESTDIR=/<<BUILDDIR>>/gi-docgen-2022.2\+ds/debian/tmp LC_ALL=C.UTF-8 ninja install
> [0/1] Installing files.
> Installing subdir /<<PKGBUILDDIR>>/gidocgen to /<<PKGBUILDDIR>>/debian/tmp/usr/lib/python3/dist-packages/gidocgen
> Installing /<<PKGBUILDDIR>>/gidocgen/utils.py to /<<PKGBUILDDIR>>/debian/tmp/usr/lib/python3/dist-packages/gidocgen
— https://buildd.debian.org/status/fetch.php?pkg=gi-docgen&arch=all&ver=2022.2%2Bds-1&stamp=1668280443&raw=0

but when it was rebuilt by Lucas' infrastructure more recently, this
happened:

> cd obj-x86_64-linux-gnu && LC_ALL=C.UTF-8 meson setup .. --wrap-mode=nodownload --buildtype=plain --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=lib/x86_64-linux-gnu -Ddevelopment_tests=false
...
>    dh_auto_install -O--buildsystem=meson
> 	cd obj-x86_64-linux-gnu && DESTDIR=/<<BUILDDIR>>/gi-docgen-2022.2\+ds/debian/tmp LC_ALL=C.UTF-8 ninja install
> [0/1] Installing files.
> Installing subdir /<<PKGBUILDDIR>>/gidocgen to /<<PKGBUILDDIR>>/debian/tmp/usr/local/lib/python3.10/dist-packages/gidocgen
> Installing /<<PKGBUILDDIR>>/gidocgen/__init__.py to /<<PKGBUILDDIR>>/debian/tmp/usr/local/lib/python3.10/dist-packages/gidocgen
— http://qa-logs.debian.net/2022/12/20/gi-docgen_2022.2+ds-1_unstable.log

and as a result, dh_install failed to find the expected files.

This appears to be a behaviour change in the build system, and I think it's
triggered by the new setuptools. Meson's python module has special cases
for distutils.command.install containing deb_system, and the "real"
distutils.command.install in python3-distutils has that scheme patched
into it; but the new python3-setuptools overrides parts of distutils:

> >>> import distutils.command.install
> >>> distutils.command.install.__file__
> '/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py'
> >>> sorted(distutils.command.install.INSTALL_SCHEMES)
> ['nt', 'nt_user', 'osx_framework_library', 'posix_home', 'posix_prefix', 'posix_user', 'pypy', 'pypy_nt']

... which notably does not contain the deb_system that Meson relies on
for the expected behaviour on Debian systems.

(See file:///usr/lib/python3/dist-packages/mesonbuild/modules/python.py
for full details)

> Fun fact: unless I missed something in sources.debian.net, there seems
> to be nobody else but me maintaining Debian packages which install
> Python modules via meson.

You must have missed something in sources.debian.net, because you're not
alone here: dbus-python, libgit2-glib and gi-docgen show similar symptoms.

    smcv