Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #16258

Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s"

Cross-posted to 2 groups.

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

Hi Louis-Philippe (2022.11.28_01:46:58_+0200)
> Too often, a mistake or a misconfiguration leads to no tests being detected
> when trying to run the upstream testsuite.
> 
> When this happens, the result of the test command typically looks like "Ran
> 0 tests in 0.000s".

A couple of years later:

We've implemented the feature in Python 3.12, unittest's runner now
exits with return value 5 if no tests were discovered, like pytest does.
https://github.com/python/cpython/issues/113661

We initially ignored this exit status in dh-python, to allow us to
transition to Python 3.12, without too much pain.

I've just done a rebuild test without ignoring it to see how much effort
it would take to be able to use this feature:

I built 6440 packages build-depending on dh-python in one way or
another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs.

To fix these build failures, package maintainers would have these
options:

1. Get the build to run some unit tests (assuming they exist),
2. override_dh_auto_test with something noop,
3. export PYBUILD_DISABLE=test,
4. We could make this failure opt-in in dh-python. Maybe via an explicit
   --test-unittest option that selects the unittest runner. If you don't
   explicitly select this runner, you'd get an attempt to run tests by
   with unittest, and no failure if no tests are found.

The downside of options 2 and 3 is that if the package does start
gaining an upstream test suite, nothing ever will attempt to run it. I
think that's OK?

If you want to experiment yourself, there's a version of dh-python in
experimental that will treat no tests as an error.

Should we file bugs for these packages? Or implement option 4?

Attached is a current dd-list.

Stefano

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272

Back to linux.debian.maint.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-08 16:00 +0200
  Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s" Scott Kitterman <debian@kitterman.com> - 2024-09-08 19:30 +0200
    Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Louis-Philippe Véronneau <pollo@debian.org> - 2024-09-12 17:40 +0200
  Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Julian Gilbey <julian@d-and-j.net> - 2024-09-08 23:40 +0200
    Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-09 10:40 +0200
      Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Julian Gilbey <julian@d-and-j.net> - 2024-09-09 17:30 +0200
        Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-10 17:10 +0200
          Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Alexandre Detiste <alexandre.detiste@gmail.com> - 2024-09-12 16:30 +0200
            Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-12 16:30 +0200
    Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-09 10:40 +0200
  Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Timo Röhling <roehling@debian.org> - 2024-09-13 11:00 +0200

csiph-web