Path: csiph.com!aioe.org!bofh.it!news.nic.it!robomod From: Antonio Terceiro Newsgroups: linux.debian.maint.python Subject: gotchas when running tests via pybuild? Date: Fri, 22 Jan 2021 18:20:01 +0100 Message-ID: X-Original-To: debian-python@lists.debian.org X-Mailbox-Line: From debian-python-request@lists.debian.org Fri Jan 22 17:17:41 2021 Old-Return-Path: X-Amavis-Spam-Status: No, score=-8.43 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, FROM_SUSPICIOUS_NTLD=0.272, FROM_SUSPICIOUS_NTLD_FP=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, LDO_WHITELIST=-5, MURPHY_DRUGS_REL8=0.02, PGPSIGNATURE=-5, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_RP_RNBL=1.31, TO_NO_BRKTS_PCNT=2.418] autolearn=ham autolearn_force=no X-Policyd-Weight: using cached result; rate: -5.5 X-Originating-IP: 198.58.116.17 Mail-Followup-To: Antonio Terceiro , debian-python@lists.debian.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="GRMV960HhfU/cwos" Content-Disposition: inline X-Mailing-List: archive/latest/18914 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/YAsGDm7sRWBOgmmu@debian.org Approved: robomod@news.nic.it Lines: 57 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Date: Fri, 22 Jan 2021 14:06:22 -0300 X-Original-Message-ID: Xref: csiph.com linux.debian.maint.python:13138 --GRMV960HhfU/cwos Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I'm working on python-eventlet, to fix a FTBFS bug and an incompatibility with python3.9. I got both fixes ready, but I'm not fighting with the fact that the test suite fails randomly. In my experiments, the test suite fails ~40% of the time, but *only when run during the build* (!!). - I added a testsuite run to autopkgtest, and it consistently passes, 100% of the time. - If I run the same command as pybuild runs manually (python3 -m nose) on a fully patched tree, it passes 100% of the time. - Only during the build -- when executed automatically by pybuild -- the test fails ~40% of the time. The failures are the typical failures you get when testing concurrency code: timeouts, race conditions, etc, and it happens on different tests every time. What I don't quite understand yet is why this _only_ happens during the Debian package build. I already checked that the tests don't use anything else from the source tree beyond the tests themselves and the python modules. For example the autopkgtest copies tests/, and only it, to a temp directory and runs the tests from there; and works every time. So in principle I wouldn't need to have an explicit testfiles file. Does anybody have an insight on cases like this? Are there any details that I'm missing? If anyone wants to try it, the git repository is up to date. --GRMV960HhfU/cwos Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEst7mYDbECCn80PEM/A2xu81GC94FAmALBgwACgkQ/A2xu81G C97CyQ/+Nxy+t15Eb1/KhprPX6mCsUlhU1X+JMQiGzlO6mKbMxflkzNcVtxO0qZN aXQXCcJxiSj/I5IZuh1i7MA6SSEjY+qFJAGnGnWivbdAk6d9+vyexAIyjUnZrkB8 T9ngzvnLuGEE3tggIQu8JRBeFt18REQjbquqtu9T42lM4r1v14LDVuvofUeRripA QvaQLVyTtbPtD71RseBvqtHMo19dFw+/rVGpNCKrPL8r4C4mEInjS+BbSs2ZWvUJ kAtwupitbxpJzWJnCnE8f0Ry6fmpGhogRYqm+UN+mAyRZl2QV0N8qdg/CQ44+7ol 9XNlIEa+TMZ5enGlze/pScSRXYXipwT92OIFXxe9WinFb5bIqNlE61GlCmu2Btai MR+s0JIJHz6LjexjXktXdu4dk6aS8bHIPt2Vp5JwS6iosf+yeYBRewK4tPwS+xq8 sdcP35sylO/Sy+2N1NKujqpMQQ5hWawV1F0ex3dglUhaRS6rXeCd4cWsOCzNI7JD zfQO0xHCPEezyQtDsDpkTtyir6onPmfuqiK3FGyKnjrzROXiT53+2kYy8fr7y11v MBnhlPmPvDTA3zsW4993Qz087Nu1B9EFs32T9gQb2Ij7dZWhrFaXsF0BblsOUSez bQ4cQrtt3l7rl++gfI+CHy/7cbtm5RSdF4ozK4tlZksBXQH6ffQ= =TWLc -----END PGP SIGNATURE----- --GRMV960HhfU/cwos--