Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #16986

Re: tests defined in d/pybuild.testfiles tested twice

Show all headers | View raw

On Fri, Aug 08, 2025 at 10:51:21AM +0800, Tianyu Chen wrote:
>Yesterday I was packaging python-pywebpush[1]. To enable autopkgtest,
>I used `Testsuite: autopkgtest-pkg-pybuild' in d/control. But when
>autopkgtest stage, pytest didn't discover any tests. weepingclown and
>I dug into the problem and found tests in pywebpush/tests are not
>copied to the build directory; instead, pybuild only copies
>pyproject.toml, pytest.ini, test, and tests directories. pybuild(1)
>documented that "you can list them in debian/pybuild.testfiles file".
>
>By creating d/pybuild.testfiles and adding "pywebpush/tests" to the
>file, autopkgtest works. But reviewing the build log, we found all
>tests are running twice in dh_auto_test. The log shows:
>
>tests/test_webpush.py
>pywebpush/tests/test_webpush.py
>
>I asked on #debian-python IRC yesterday, and currently using `export
>PYBUILD_TEST_ARGS=--ignore pywebpush/tests' to avoid being tested
>twice. I also did some searching on codesearch.debian.net, and I found
>that some other packages are either tested twice or doing similar
>things to avoid being tested twice. Following are three attempts to
>ensure tested only once:
>
>src:isodate
>    export PYBUILD_BEFORE_TEST=cp -r {dir}/tests {build_dir}/
>    export PYBUILD_TEST_ARGS=tests/test_*.py
>
>src:python-cymem
>    export PYBUILD_NAME = cymem
>    export PYBUILD_TEST_ARGS = tests
>
>src:python-ewokscore
>    ifeq ($(PYBUILD_AUTOPKGTEST),1)
>    export PYBUILD_TEST_ARGS := {dir}/src/ewokscore
>    endif
>
>Are there better solutions to this problem?

These are all ultimately workarounds for https://bugs.debian.org/947800.  
I've generally found that there's unfortunately no one size that fits 
all: which option is best depends on the exact details of the package 
and how tolerant it is of its tests living in a slightly different 
location from the one it expects.  For example, some tests do relative 
imports so they'll fail if you effectively move foo/tests/ to tests/, 
while some tests have expectations about their current working directory 
relative to the test suite.

With pytest it's often (not always) easiest to tell it to run tests from 
a non-installed path, as in python-cymem and python-ewokscore above.  
Manually copying the tests into the expected subdirectory using 
PYBUILD_BEFORE_TEST and PYBUILD_AFTER_TEST works too, and that's usually 
what I end up doing with other test frameworks, but it's cumbersome 
enough that I usually only do this when I have to.

In your case I would probably drop debian/pybuild.testfiles and do 
`export PYBUILD_TEST_ARGS = {dir}/pywebpush/tests` in debian/rules, 
although it does produce mildly uglier test output, and to some extent 
this is a matter of taste.

Ideally I think we'd be able to write something like `pywebpush/tests => 
pywebpush/tests` in debian/pybuild.testfiles, in the style of dh-exec.  
Then most of these hacks would be unnecessary.

-- 
Colin Watson (he/him)                              [cjwatson@debian.org]

Back to linux.debian.maint.python | Previous | Next — Previous in thread | Find similar

Thread

tests defined in d/pybuild.testfiles tested twice Tianyu Chen <billchenchina2001@gmail.com> - 2025-08-08 05:10 +0200
  Re: tests defined in d/pybuild.testfiles tested twice Colin Watson <cjwatson@debian.org> - 2025-08-08 11:30 +0200

csiph-web