Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #16259

Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s"

Show all headers | View raw

On September 8, 2024 1:43:33 PM UTC, Stefano Rivera <stefanor@debian.org> wrote:
>Hi Louis-Philippe (2022.11.28_01:46:58_+0200)
>> Too often, a mistake or a misconfiguration leads to no tests being detected
>> when trying to run the upstream testsuite.
>> 
>> When this happens, the result of the test command typically looks like "Ran
>> 0 tests in 0.000s".
>
>A couple of years later:
>
>We've implemented the feature in Python 3.12, unittest's runner now
>exits with return value 5 if no tests were discovered, like pytest does.
>https://github.com/python/cpython/issues/113661
>
>We initially ignored this exit status in dh-python, to allow us to
>transition to Python 3.12, without too much pain.
>
>I've just done a rebuild test without ignoring it to see how much effort
>it would take to be able to use this feature:
>
>I built 6440 packages build-depending on dh-python in one way or
>another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs.
>
>To fix these build failures, package maintainers would have these
>options:
>
>1. Get the build to run some unit tests (assuming they exist),
>2. override_dh_auto_test with something noop,
>3. export PYBUILD_DISABLE=test,
>4. We could make this failure opt-in in dh-python. Maybe via an explicit
>   --test-unittest option that selects the unittest runner. If you don't
>   explicitly select this runner, you'd get an attempt to run tests by
>   with unittest, and no failure if no tests are found.
>
>The downside of options 2 and 3 is that if the package does start
>gaining an upstream test suite, nothing ever will attempt to run it. I
>think that's OK?
>
>If you want to experiment yourself, there's a version of dh-python in
>experimental that will treat no tests as an error.
>
>Should we file bugs for these packages? Or implement option 4?
>
>Attached is a current dd-list.
>
Thanks.  I think some variation of #4 is the right answer.  Causing a thousand packages to FTBFS isn't great.  I would find it easier to have an environment variable (similar to what you suggested for #3) instead of adding an override, but that's more of an implementation detail.

Scott K

Back to linux.debian.maint.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-08 16:00 +0200
  Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s" Scott Kitterman <debian@kitterman.com> - 2024-09-08 19:30 +0200
    Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Louis-Philippe Véronneau <pollo@debian.org> - 2024-09-12 17:40 +0200
  Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Julian Gilbey <julian@d-and-j.net> - 2024-09-08 23:40 +0200
    Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-09 10:40 +0200
      Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Julian Gilbey <julian@d-and-j.net> - 2024-09-09 17:30 +0200
        Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-10 17:10 +0200
          Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Alexandre Detiste <alexandre.detiste@gmail.com> - 2024-09-12 16:30 +0200
            Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-12 16:30 +0200
    Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Stefano Rivera <stefanor@debian.org> - 2024-09-09 10:40 +0200
  Re: Bug#1024971: pybuild: should fail when the result of running  tests is "Ran 0 tests in 0.000s" Timo Röhling <roehling@debian.org> - 2024-09-13 11:00 +0200

csiph-web