Path: csiph.com!news.samoylyk.net!gothmog.csi.it!bofh.it!news.nic.it!robomod From: Stefano Rivera Newsgroups: linux.debian.maint.python,linux.debian.bugs.dist Subject: Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s" Date: Tue, 10 Sep 2024 17:10:01 +0200 Message-ID: References: X-Mailbox-Line: From debian-python-request@lists.debian.org Tue Sep 10 15:06:12 2024 Old-Return-Path: X-Amavis-Spam-Status: No, score=-107.012 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, LDO_WHITELIST=-5, T_SCC_BODY_TEXT_LINE=-0.01, USER_IN_DKIM_WELCOMELIST=-0.01, USER_IN_DKIM_WHITELIST=-100] autolearn=ham autolearn_force=no X-Policyd-Weight: using cached result; rate: -4.6 Mail-Followup-To: Julian Gilbey , debian-python@lists.debian.org, Louis-Philippe =?utf-8?Q?V=C3=A9ronneau?= , 1024971@bugs.debian.org X-Gpg-Public-Key: http://www.rivera.za.net/stefano.gpg MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: NeoMutt/20220429 X-Mailing-List: archive/latest/22330 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/20240910150547.gbzjkqljjydndgm7@satie.tumbleweed.org.za Approved: robomod@news.nic.it Lines: 51 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Cc: debian-python@lists.debian.org, Louis-Philippe =?utf-8?Q?V=C3=A9ronneau?= , 1024971@bugs.debian.org X-Original-Date: Tue, 10 Sep 2024 15:05:47 +0000 X-Original-Message-ID: <20240910150547.gbzjkqljjydndgm7@satie.tumbleweed.org.za> X-Original-References: <1d836a0a-b926-aaa1-c04c-5a9d6910347a@debian.org> <6b7zvq5bjmmfp3nogosqrgejexhne6mjyzfsih7iyrlaixa7hv@clcppesyi4m7> <1d836a0a-b926-aaa1-c04c-5a9d6910347a@debian.org> <20240909083152.6gthpgmyza7hxmry@satie.tumbleweed.org.za> Xref: csiph.com linux.debian.maint.python:16270 linux.debian.bugs.dist:1212199 Hi Julian (2024.09.09_15:19:51_+0000) > That seems a bit heavy to ask for. >=20 > Is there any way of identifying those packages that do genuinely use > unittest? =46rom 6438 build logs: - 651 don't call dh_auto_test - 2180 do something custom - 1989 use pytest - 25 use nose - 18 use nose2 - 23 use tox - 3 use stestr - 1561 packages use pybuild's unittest runner * 391 pass * 1170 fail + 1139 NO TESTS RAN + 33 the test suite failed (numbers don't quite add up, because this was a lot of grep | wc -l) > If there are not that many of them, then implementing a > --test-unittest option would be a good way to go. I would imagine the > following timeline: >=20 > (1) --test-unittest is introduced as an option to explicitly select > unittest as the test framework. When --test-unittest is specified, > the test will fail if no tests are found. unittest is still used as a > fallback test framework; in this case, the dh_auto_test call will > succeed if no tests are run. >=20 > (2) Add some sort of warning for pybuild-using packages that run > dh_auto_test but haven't specified a test framework and for which > autodetection of the test framework fails. If there aren't any tests > to run, an empty override_dh_auto_test target should be specified. >=20 > (3) Stop using unittest as the default test framework, and fail if no > test framework has been specified or autodetected. >=20 > But that might be overkill for something which may not actually be > much of a problem. Yeah, that can work. We can also just abort after step 2. Stefano --=20 Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272