Path: csiph.com!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod
From: =?utf-8?Q?=C3=89tienne?= Mollier <etienne.mollier@mailoo.org>
Newsgroups: linux.debian.maint.python
Subject: pkg_resources modules detection issues in various packages
Date: Sat, 10 Oct 2020 14:50:01 +0200
Message-ID: <AYmtX-7FW-1@gated-at.bofh.it>
Old-Return-Path: <etienne.mollier@mailoo.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="dDRMvlgZJXvWKvBx"
Content-Disposition: inline
Approved: robomod@news.nic.it
Lines: 122
Organization: linux.* mail to news gateway
Sender: robomod@news.nic.it
Xref: csiph.com linux.debian.maint.python:12937


--dDRMvlgZJXvWKvBx
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

TL;DR:
  1. Has someone an idea of usage of the module pkg_resourses,
     specifically the iterator pkg_resources.iter_entry_points,
     for fetching local resources?
  2. Is someone aware of recent (as in the past year or so)
     breaking changes in the way the pkg_resources module works?


Greetings,

While attempting to give a hand with maintenance of various
debian-med python packages, various sources were failing their
test suites due to diverse failures to import their test modules
through the mechanism provided by the module pkg_resources.
Andreas Tille first noted this in python-cobra:

	https://lists.debian.org/debian-med/2020/09/msg00369.html

Then I hit some issues involving pkg_resources as well in Qiime,
while attempting to update it, and it turned out even the
current version does not manage to pass tests, due to failure to
load a dummy-plugin with this system:

	https://lists.debian.org/debian-med/2020/10/msg00014.html
	https://salsa.debian.org/med-team/qiime

Note the package will incorrectly report that the QIIMETEST
variable should be set, although it actually is.  Here is a
sample error from a build attempt of qiime_2019.10.0-1.dsc:

	=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
	ERROR: test_import_root (qiime2.tests.test_artifact_api.TestImports)
	----------------------------------------------------------------------
	Traceback (most recent call last):
	  File "/tmp/autopkgtest.MTf3Wq/build.GrO/src/.pybuild/cpython3_3.8_qiime/=
build/qiime2/tests/test_artifact_api.py", line 23, in setUp
	    get_dummy_plugin()
	  File "/tmp/autopkgtest.MTf3Wq/build.GrO/src/.pybuild/cpython3_3.8_qiime/=
build/qiime2/core/testing/util.py", line 19, in get_dummy_plugin
	    raise RuntimeError(
	RuntimeError: When running QIIME 2 unit tests, the QIIMETEST environment v=
ariable must be defined so that plugins required by unit tests are loaded. =
The value of the QIIMETEST environment variable can be anything. Example co=
mmand: QIIMETEST=3D1 nosetests

The get_dummy_plugin function is a wrapper around a class which
ultimately calls an iterator like this to build a for loop:

	pkg_resources.iter_entry_points(group=3D'qiime2.plugins')

where qiime2/plugins.py is available at the root of the source
code.  My main problem is: I did not manage to get this
construction to return a proper object to point to the dummy
plugin stored at that location for the test suite, even when
making sure the sys.path includes the current working directory
prior to including the pkg_resources module.


Nilesh Patra also followed up with something related to
pkg_resources on the package gubbins and the open bug #971111,
although the situation seemed different this time, as this
package shows issues in the requirements parser of pkg_resources
and not the distribution locator:

	https://lists.debian.org/debian-med/2020/10/msg00015.html
	https://bugs.debian.org/971111

I even saw something somewhat similar in the present list
archives recently, although altering the PYTHONPATH, or even the
sys.path directly, did not help in my case:

	https://lists.debian.org/debian-python/2020/10/msg00005.html

I'm afraid I'm not at ease with dealing with pkg_resources,
although I did try to make sense of how it is supposed to work,
by referring to its documentation, which I believe is here:

	https://setuptools.readthedocs.io/en/latest/pkg_resources.html

All these test failures show slightly different symptoms, I even
tested Qiime building on Buster, and the issue was already
present, while former versions of python-cobra or gubbins made
it to Buster, so it is quite possible they are unrelated after
all.  But just in case, is someone aware of recent breaking
changes in the way pkg_resources module works?

Kind Regards,
--=20
=C3=89tienne Mollier <etienne.mollier@mailoo.org>
Old rsa/3072: 5ab1 4edf 63bb ccff 8b54  2fa9 59da 56fe fff3 882d
New rsa/4096: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
Sent from /dev/pts/3, please excuse my verbosity.

--dDRMvlgZJXvWKvBx
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEj5GyJ8fW8rGUjII2eTz2fo8NEdoFAl+BrYcACgkQeTz2fo8N
EdqAWhAAkS8Xm2dNrPmtq2dTnnkIToKfxlVFDga1YqyLP610DFh03/12pW9kORIV
jxS5O/02Rg7XYVNwMA5N3+is7pMG2YwGCoOyF9nD8HN1EXxlL7zGfBZgze+6hruX
SiSdfFL1455LlqAnfHnYUfWvDE+sIPyNOMEqSsRJRl5WGBSc7QAb+6pGeGczMFqX
mm7zkjvQ1Anu/GGRXj7N3GRg3gaoSk/EKy7K+jG+cb+JmlDBVXWJTfz6sa0LfsSb
VL9oK41n7DcIGZ8yfnQcFiIq/bsilavIpaABxxPpThQN6WlRzGXEFiKmdEoNwGeJ
L9ulJLC/Sgd8xUOyyAFnZXg+6dbErohr5rREEla63/Bbc0pBDYYeqYjL4v44tiM9
DMP78XgN+SxmsM9/xPIG9DHcLz4UuRptzW9vsfhWskNKmFqh4NAY8vQUIRMN08nz
9GGxnj1HB3Ou5oLIRi8SKiMtNWbBYDTz1S3bwfiBLg7HjLCZHk8x9QrBoDTIivHP
qUXmORVwOtE+reXM8p7kauqLMT43GKI2HwpNzxiuGd93bV8sgtWZdc/jatDAacHT
hWHxPm/m2I23sspMHdt4OLkgYSnE0dQYPbQQ5N98tsoxkr66kzKBcaYLjuZwQ3I5
M5FMuFxekyu8pydEHARhY+GbWDVVuCKm7B2+g7Y5HGf7EVSmipM=
=XQmG
-----END PGP SIGNATURE-----

--dDRMvlgZJXvWKvBx--