Path: csiph.com!weretis.net!feeder8.news.weretis.net!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod From: Peter Wienemann Newsgroups: linux.debian.maint.python Subject: Re: How to run unit tests? Date: Sat, 12 Dec 2020 16:40:01 +0100 Message-ID: References: X-Mailbox-Line: From debian-python-request@lists.debian.org Sat Dec 12 15:35:59 2020 Old-Return-Path: X-Amavis-Spam-Status: No, score=-9.479 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, LDO_WHITELIST=-5, MURPHY_DRUGS_REL8=0.02, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001] autolearn=ham autolearn_force=no X-Policyd-Weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 CL_IP_EQ_HELO_IP=-2 (check from: .posteo. - helo: .mout01.posteo. - helo-domain: .posteo.) FROM/MX_MATCHES_HELO(DOMAIN)=-2; rate: -5.5 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1607787342; bh=r8RoPQRMagTz9R0ItWpz4X9Tr7/Ft5pY3M4363bFQdQ=; h=Subject:To:From:Date:From; b=r7GAu8qT5k/72AbIKPCFdZ3k32K39KqiRRkJcqvaX/FkRu7PIEoNjSXqhDFw8afz1 y6WkB9EwVWVKqpFzWNoCRq1CFooGG4SHHK5WR24mqETSib6nVPVjuy1ruq+WeJBT6q eMDdsk/Rfd8m8wwP6qm31fNpP8XKkOfc7GrYbwcx1m7NzsU03+1sGgJoqotyrg5/uj 1mklhgTiEFp9PDLv3FlM7C4EDKYhlRKchFiBwk9nhyvbDhxfS6ojVsxpMDpUll+s5a 7VS5LNZqlk9P+Iml1aGYBunWBTYn6Ge0p2bGSSZwRJ1IhN9p/subEcYImUeq/CSEOx mFs2vjFNTuhLQ== User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Mailing-List: archive/latest/18830 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/4cd114ec-1ddd-cee3-8f4b-e70a7dc82dad@posteo.de Approved: robomod@news.nic.it Lines: 47 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Date: Sat, 12 Dec 2020 16:35:41 +0100 X-Original-Message-ID: <4cd114ec-1ddd-cee3-8f4b-e70a7dc82dad@posteo.de> X-Original-References: Xref: csiph.com linux.debian.maint.python:13057 Hi Andrey, thanks for sharing your thoughts. On 08.12.20 21:43, Andrey Rahmatullin wrote: > You didn't explain what actual problems do you have, but > https://github.com/lark-parser/lark/issues/792 suggests that some way you > used was skipping some of the tests? I am not familiar enough with Python unit testing for a solid judgement but I find it disturbing that the outcome of the test suite depends on the way you run it. I wonder whether this is a) due to a problem in the test suite, b) due to a problem somewhere in the tested code, c) by construction because the three different ways to run the test suite are not equivalent? >> Upstream uses option 3 in its CI testing [2] and did not see any issues with >> lark release 0.11.1. dh_auto_test seems to use option 2 by default (for the >> pybuild build system). For autopkgtest I have chosen to use option 1. The >> latter two options both fail for version 0.11.1. > Which is good? > >> After applying a patch provided by upstream [3], also option 2 works but option 1 still fails. > Which is good or bad? I'm not sure. I am asking myself the same questions. That is why I am asking for advice here. :-) >> Upstream suggested to change the way the (autopkgtest) tests are run [4]. > So are you asking only about autopkgtests, not build-time tests? Rereading it I think one should not constrain the question to autopkgtests but also consider build-time tests. >> Is there something like a general recommendation on what is the best way >> to run Python unit tests? > No (except "whatever the upstream CI runs"). This would vote for option 3: pythonX -m tests Best regards Peter