Path: csiph.com!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!3.eu.feeder.erje.net!feeder.erje.net!news2.arglkargh.de!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod From: Thomas Goirand Newsgroups: linux.debian.maint.python Subject: Re: gotchas when running tests via pybuild? Date: Sun, 24 Jan 2021 11:30:02 +0100 Message-ID: References: X-Mailbox-Line: From debian-python-request@lists.debian.org Sun Jan 24 10:26:11 2021 Old-Return-Path: X-Amavis-Spam-Status: No, score=-9.729 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, LDO_WHITELIST=-5, MURPHY_DRUGS_REL8=0.02, NICE_REPLY_A=-2.749] autolearn=ham autolearn_force=no X-Policyd-Weight: using cached result; rate: -3.5 Organization: Debian User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Mailing-List: archive/latest/18918 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/8ef938f4-3b2a-cff7-5464-e36b5688fca5@debian.org Approved: robomod@news.nic.it Lines: 46 Sender: robomod@news.nic.it X-Original-Date: Sun, 24 Jan 2021 11:25:53 +0100 X-Original-Message-ID: <8ef938f4-3b2a-cff7-5464-e36b5688fca5@debian.org> X-Original-References: Xref: csiph.com linux.debian.maint.python:13141 On 1/22/21 6:06 PM, Antonio Terceiro wrote: > Hi, >=20 > I'm working on python-eventlet, to fix a FTBFS bug and an > incompatibility with python3.9. I got both fixes ready, but I'm not > fighting with the fact that the test suite fails randomly. In my > experiments, the test suite fails ~40% of the time, but *only when run > during the build* (!!). >=20 > - I added a testsuite run to autopkgtest, and it consistently passes, > 100% of the time. > - If I run the same command as pybuild runs manually (python3 -m nose) > on a fully patched tree, it passes 100% of the time. > - Only during the build -- when executed automatically by pybuild -- th= e > test fails ~40% of the time. >=20 > The failures are the typical failures you get when testing concurrency > code: timeouts, race conditions, etc, and it happens on different tests > every time. What I don't quite understand yet is why this _only_ happen= s > during the Debian package build. >=20 > I already checked that the tests don't use anything else from the sourc= e > tree beyond the tests themselves and the python modules. For example th= e > autopkgtest copies tests/, and only it, to a temp directory and runs th= e > tests from there; and works every time. So in principle I wouldn't need > to have an explicit testfiles file. >=20 > Does anybody have an insight on cases like this? Are there any details > that I'm missing? >=20 > If anyone wants to try it, the git repository is up to date. Hi, Eventlet looks broken beyond just the unit tests. I did a deployment of OpenStack on unstable, and there's lots of issues on absolutely all daemons. Hopefully, I can fine time to investigate this next week. Cheers, Thomas Goirand (zigo)