Path: csiph.com!news.mixmin.net!news2.arglkargh.de!news.karotte.org!news.szaf.org!news.samoylyk.net!gothmog.csi.it!bofh.it!news.nic.it!robomod From: Stefano Rivera Newsgroups: linux.debian.maint.python Subject: Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s" Date: Mon, 09 Sep 2024 10:40:01 +0200 Message-ID: References: X-Original-To: debian-python@lists.debian.org X-Mailbox-Line: From debian-python-request@lists.debian.org Mon Sep 9 08:30:52 2024 Old-Return-Path: X-Amavis-Spam-Status: No, score=-107.012 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, LDO_WHITELIST=-5, T_SCC_BODY_TEXT_LINE=-0.01, USER_IN_DKIM_WELCOMELIST=-0.01, USER_IN_DKIM_WHITELIST=-100] autolearn=ham autolearn_force=no X-Policyd-Weight: using cached result; rate: -4.6 Mail-Followup-To: debian-python@lists.debian.org X-Gpg-Public-Key: http://www.rivera.za.net/stefano.gpg MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: NeoMutt/20220429 X-Mailing-List: archive/latest/22321 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/20240909083036.kc6kfwj4b5ossona@satie.tumbleweed.org.za Approved: robomod@news.nic.it Lines: 38 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Date: Mon, 9 Sep 2024 08:30:36 +0000 X-Original-Message-ID: <20240909083036.kc6kfwj4b5ossona@satie.tumbleweed.org.za> X-Original-References: <1d836a0a-b926-aaa1-c04c-5a9d6910347a@debian.org> <6b7zvq5bjmmfp3nogosqrgejexhne6mjyzfsih7iyrlaixa7hv@clcppesyi4m7> <1d836a0a-b926-aaa1-c04c-5a9d6910347a@debian.org> Xref: csiph.com linux.debian.maint.python:16262 Hi Julian (2024.09.08_21:33:49_+0000) > > I built 6440 packages build-depending on dh-python in one way or > > another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs. > > My guess is that most of these 1124 have no tests at all, rather than > having a misconfigured setup. A unittest is the pybuild default test > framework, unittest is used and fails to find any tests, hence all of > these failures. Yes, almost certainly. > > 4. We could make this failure opt-in in dh-python. Maybe via an explicit > > --test-unittest option that selects the unittest runner. If you don't > > explicitly select this runner, you'd get an attempt to run tests by > > with unittest, and no failure if no tests are found. > > I like option 4 for the above reason. But implementing this would > mean that all of the packages that currently *do* use unittest > (intentionally, but without having to code it explicitly as it's the > default) would suddenly not have any tests running until they > proactively add --test-unittest or set PYBUILD_TEST_UNITTEST = 1 or > similar. This seems like an unfortunate consequence. I would leave unittest as the default runner, but without missing test detection. That's a slightly unexpected behaviour, but it makes the default case work. Downside is that you have to opt-in to missing test detection. Maybe we can have a lintian tag for that? Stefano -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272