Path: csiph.com!weretis.net!feeder8.news.weretis.net!news.samoylyk.net!gothmog.csi.it!bofh.it!news.nic.it!robomod From: Simon McVittie Newsgroups: linux.debian.maint.python Subject: Re: Policy Change Proposal: Running the upstream test suite Date: Wed, 31 Jul 2024 10:50:01 +0200 Message-ID: References: X-Mailbox-Line: From debian-python-request@lists.debian.org Wed Jul 31 08:46:44 2024 Old-Return-Path: X-Amavis-Spam-Status: No, score=-109.51 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, LDO_WHITELIST=-5, RCVD_IN_DNSWL_MED=-2.3, UNPARSEABLE_RELAY=0.001, USER_IN_DKIM_WELCOMELIST=-0.01, USER_IN_DKIM_WHITELIST=-100] autolearn=ham autolearn_force=no MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Debian-User: smcv X-Mailing-List: archive/latest/22175 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/Zqn542HOHUqVRnZF@remnant.pseudorandom.co.uk Approved: robomod@news.nic.it Lines: 37 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Date: Wed, 31 Jul 2024 09:46:27 +0100 X-Original-Message-ID: X-Original-References: <1867641997.115526135.1722243907166.JavaMail.zimbra@synchrotron-soleil.fr> <20240729134327.Horde._9QGgAmwIGxmlrG94HyAfLk@webmail.in-berlin.de> <1801597020.116294261.1722259839638.JavaMail.zimbra@synchrotron-soleil.fr> Xref: csiph.com linux.debian.maint.python:16137 On Mon, 29 Jul 2024 at 18:55:26 +0500, Andrey Rakhmatullin wrote: > On Mon, Jul 29, 2024 at 03:30:39PM +0200, PICCA Frederic-Emmanuel wrote: > > My use case, is to check that all the Dependencies computer by dh_python3 from the build tools are indeed listed in the Depends of the binary package. > > Maybe we indeed want a "minimal" autopkgtest environment For a simple smoke-test that the Python packages import succesfully, isn't this exactly autopkgtest-pkg-python? For thorough test coverage, I don't think this is possible: in general, the test suite will legitimately require installation of packages that are only needed during testing (pytest or similar), or packages that are optional for basic use of the code but mandatory for full test coverage (those might be Recommends or Suggests). > but many > upstream tests will fail in those and I don't see an automatic way to test > a random package in this way If we want to run upstream test-suites as autopkgtests, then I think ideally we want the same test coverage as autopkgtest-pkg-python, *and* the same test coverage as autopkgtest-pkg-pybuild, as separate autopkgtest test-cases with different dependencies. But I don't think we can easily have that, because the Testsuite field only takes one value. Would it be possible to make autopkgtest-pkg-pybuild absorb the functionality of autopkgtest-pkg-python, so that it generates a debian/control that does some simple smoke-tests (like autopkgtest-pkg-python does), *and* runs the upstream test suite (like autopkgtest-pkg-pybuild currently does)? If not, then the proposed policy change would result in us gaining coverage but also losing coverage. smcv