Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.debian.maint.python > #13138

gotchas when running tests via pybuild?

Path csiph.com!aioe.org!bofh.it!news.nic.it!robomod
From Antonio Terceiro <terceiro@debian.org>
Newsgroups linux.debian.maint.python
Subject gotchas when running tests via pybuild?
Date Fri, 22 Jan 2021 18:20:01 +0100
Message-ID <BA8gh-3Cp-1@gated-at.bofh.it> (permalink)
X-Original-To debian-python@lists.debian.org
X-Mailbox-Line From debian-python-request@lists.debian.org Fri Jan 22 17:17:41 2021
Old-Return-Path <debian@terceiro.xyz>
X-Amavis-Spam-Status No, score=-8.43 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, FROM_SUSPICIOUS_NTLD=0.272, FROM_SUSPICIOUS_NTLD_FP=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, LDO_WHITELIST=-5, MURPHY_DRUGS_REL8=0.02, PGPSIGNATURE=-5, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_RP_RNBL=1.31, TO_NO_BRKTS_PCNT=2.418] autolearn=ham autolearn_force=no
X-Policyd-Weight using cached result; rate: -5.5
X-Originating-IP 198.58.116.17
Mail-Followup-To Antonio Terceiro <terceiro@debian.org>, debian-python@lists.debian.org
MIME-Version 1.0
Content-Type multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="GRMV960HhfU/cwos"
Content-Disposition inline
X-Mailing-List <debian-python@lists.debian.org> archive/latest/18914
List-ID <debian-python.lists.debian.org>
List-URL <https://lists.debian.org/debian-python/>
List-Archive https://lists.debian.org/msgid-search/YAsGDm7sRWBOgmmu@debian.org
Approved robomod@news.nic.it
Lines 57
Organization linux.* mail to news gateway
Sender robomod@news.nic.it
X-Original-Date Fri, 22 Jan 2021 14:06:22 -0300
X-Original-Message-ID <YAsGDm7sRWBOgmmu@debian.org>
Xref csiph.com linux.debian.maint.python:13138

Show key headers only | View raw


[Multipart message — attachments visible in raw view] - view raw

Hi,

I'm working on python-eventlet, to fix a FTBFS bug and an
incompatibility with python3.9. I got both fixes ready, but I'm not
fighting with the fact that the test suite fails randomly. In my
experiments, the test suite fails ~40% of the time, but *only when run
during the build* (!!).

- I added a testsuite run to autopkgtest, and it consistently passes,
  100% of the time.
- If I run the same command as pybuild runs manually (python3 -m nose)
  on a fully patched tree, it passes 100% of the time.
- Only during the build -- when executed automatically by pybuild -- the
  test fails ~40% of the time.

The failures are the typical failures you get when testing concurrency
code: timeouts, race conditions, etc, and it happens on different tests
every time. What I don't quite understand yet is why this _only_ happens
during the Debian package build.

I already checked that the tests don't use anything else from the source
tree beyond the tests themselves and the python modules. For example the
autopkgtest copies tests/, and only it, to a temp directory and runs the
tests from there; and works every time. So in principle I wouldn't need
to have an explicit testfiles file.

Does anybody have an insight on cases like this? Are there any details
that I'm missing?

If anyone wants to try it, the git repository is up to date.

Back to linux.debian.maint.python | Previous | Next — Next in thread | Find similar | Unroll thread


Thread

gotchas when running tests via pybuild? Antonio Terceiro <terceiro@debian.org> - 2021-01-22 18:20 +0100
  Re: gotchas when running tests via pybuild? Diane Trout <diane@ghic.org> - 2021-01-22 20:10 +0100
  Re: gotchas when running tests via pybuild? Thomas Goirand <zigo@debian.org> - 2021-01-24 11:30 +0100

csiph-web