Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #13138 > unrolled thread

gotchas when running tests via pybuild?

Back to article view | Back to linux.debian.maint.python

Contents

  gotchas when running tests via pybuild? Antonio Terceiro <terceiro@debian.org> - 2021-01-22 18:20 +0100
    Re: gotchas when running tests via pybuild? Diane Trout <diane@ghic.org> - 2021-01-22 20:10 +0100
    Re: gotchas when running tests via pybuild? Thomas Goirand <zigo@debian.org> - 2021-01-24 11:30 +0100

#13138 — gotchas when running tests via pybuild?

[Multipart message — attachments visible in raw view] — view raw

Hi,

I'm working on python-eventlet, to fix a FTBFS bug and an
incompatibility with python3.9. I got both fixes ready, but I'm not
fighting with the fact that the test suite fails randomly. In my
experiments, the test suite fails ~40% of the time, but *only when run
during the build* (!!).

- I added a testsuite run to autopkgtest, and it consistently passes,
  100% of the time.
- If I run the same command as pybuild runs manually (python3 -m nose)
  on a fully patched tree, it passes 100% of the time.
- Only during the build -- when executed automatically by pybuild -- the
  test fails ~40% of the time.

The failures are the typical failures you get when testing concurrency
code: timeouts, race conditions, etc, and it happens on different tests
every time. What I don't quite understand yet is why this _only_ happens
during the Debian package build.

I already checked that the tests don't use anything else from the source
tree beyond the tests themselves and the python modules. For example the
autopkgtest copies tests/, and only it, to a temp directory and runs the
tests from there; and works every time. So in principle I wouldn't need
to have an explicit testfiles file.

Does anybody have an insight on cases like this? Are there any details
that I'm missing?

If anyone wants to try it, the git repository is up to date.

[toc] | [next] | [standalone]

#13139

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Fri, 2021-01-22 at 14:06 -0300, Antonio Terceiro wrote:
> Does anybody have an insight on cases like this? Are there any
> details
> that I'm missing?

I occasionally have tests behave differently between the buildd and
autopkgtest runner.

Things that I have encountered.

The pybuild tests are run after the package is built in
${src}/.pybuild, the autopkgtests are run on the installed package, so
if the package depends on scripts being installed, they might fail in
the builder.

There might also be different arguments to the test runner (or even
different test runners) between what pybuild decided to do and what's
done in debian/tests/control.

Maybe those ideas might help?
Diane

[toc] | [prev] | [next] | [standalone]

#13141

On 1/22/21 6:06 PM, Antonio Terceiro wrote:
> Hi,
> 
> I'm working on python-eventlet, to fix a FTBFS bug and an
> incompatibility with python3.9. I got both fixes ready, but I'm not
> fighting with the fact that the test suite fails randomly. In my
> experiments, the test suite fails ~40% of the time, but *only when run
> during the build* (!!).
> 
> - I added a testsuite run to autopkgtest, and it consistently passes,
>   100% of the time.
> - If I run the same command as pybuild runs manually (python3 -m nose)
>   on a fully patched tree, it passes 100% of the time.
> - Only during the build -- when executed automatically by pybuild -- the
>   test fails ~40% of the time.
> 
> The failures are the typical failures you get when testing concurrency
> code: timeouts, race conditions, etc, and it happens on different tests
> every time. What I don't quite understand yet is why this _only_ happens
> during the Debian package build.
> 
> I already checked that the tests don't use anything else from the source
> tree beyond the tests themselves and the python modules. For example the
> autopkgtest copies tests/, and only it, to a temp directory and runs the
> tests from there; and works every time. So in principle I wouldn't need
> to have an explicit testfiles file.
> 
> Does anybody have an insight on cases like this? Are there any details
> that I'm missing?
> 
> If anyone wants to try it, the git repository is up to date.

Hi,

Eventlet looks broken beyond just the unit tests. I did a deployment of
OpenStack on unstable, and there's lots of issues on absolutely all
daemons. Hopefully, I can fine time to investigate this next week.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web