Groups > linux.debian.maint.python > #13048

Re: How to run unit tests?

Path	csiph.com!newsfeed.xs4all.nl!newsfeed9.news.xs4all.nl!news.dns-netz.com!news.freedyn.net!aioe.org!bofh.it!news.nic.it!robomod
From	Andrey Rahmatullin <wrar@debian.org>
Newsgroups	linux.debian.maint.python
Subject	Re: How to run unit tests?
Date	Tue, 08 Dec 2020 21:50:02 +0100
Message-ID	<BjS5Q-1GU-21@gated-at.bofh.it> (permalink)
References	<BjRWa-1Dl-11@gated-at.bofh.it>
X-Original-To	debian-python@lists.debian.org
X-Mailbox-Line	From debian-python-request@lists.debian.org Tue Dec 8 20:43:23 2020
Old-Return-Path	<wrar@debian.org>
X-Amavis-Spam-Status	No, score=-11.98 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, LDO_WHITELIST=-5, MURPHY_DRUGS_REL8=0.02, PGPSIGNATURE=-5] autolearn=ham autolearn_force=no
X-Policyd-Weight	using cached result; rate:hard: -3.5
Dkim-Signature	v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=wrar.name; s=20160710; h=In-Reply-To:Content-Disposition:Content-Type:MIME-Version: References:Message-ID:Subject:To:From:Date:From:Sender:Reply-To:Subject:Date: Message-ID:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Content-Disposition:In-Reply-To:References; bh=9VwIBm5RvkuS4ezbAwKxUlWm9Vjvq8IJhdzLVgKwRKs=; b=4Osx7f3xtyOOmSnXC8MbKKsSHg 1NpxQYCsJFCgZSIlAwr1VKobToOkydtIwNIFz5xZF0BIbCsmoagODdlNjl6p/tkUrjzRha0lC5thG jLWeIyl6VpFpm3wgBuNgyYEupitJOUD5+66R+CYiu1In7YYwljoGPiGMN7tub47JfXhk=;
MIME-Version	1.0
Content-Type	multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="XHx8BQ3FYXt/hSDR"
Content-Disposition	inline
X-Mailing-List	<debian-python@lists.debian.org> archive/latest/18816
List-ID	<debian-python.lists.debian.org>
List-URL	<https://lists.debian.org/debian-python/>
List-Archive	https://lists.debian.org/msgid-search/X8/lW0aLakA+U7z3@belkar.wrar.name
Approved	robomod@news.nic.it
Lines	72
Organization	linux.* mail to news gateway
Sender	robomod@news.nic.it
X-Original-Date	Wed, 9 Dec 2020 01:43:07 +0500
X-Original-Message-ID	<X8/lW0aLakA+U7z3@belkar.wrar.name>
X-Original-References	<ca4e7355-7290-16a7-805f-66e952f99d9d@posteo.de>
Xref	csiph.com linux.debian.maint.python:13048

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

On Tue, Dec 08, 2020 at 09:33:03PM +0100, Peter Wienemann wrote:
> Dear Python experts,
> 
> in trying to update the python-lark package [0] to the most recent upstream
> version, an interesting issue regarding unit tests showed up [1]. Depending
> on how one runs unit tests they either fail or not. Tried options are:
> 
> 1. PYTHONWARNINGS=d pythonX -m unittest discover -v tests
> 2. pythonX setup.py test
> 3. pythonX -m tests
Option 3 runs
https://github.com/lark-parser/lark/blob/master/tests/__main__.py which
mostly just runs unittest.main().
Option 2 seems to run the same file according to setup.py.
I'm not familiar with unittests enough know the difference between options
1 and 3.
You didn't explain what actual problems do you have, but
https://github.com/lark-parser/lark/issues/792 suggests that some way you
used was skipping some of the tests?

> Upstream uses option 3 in its CI testing [2] and did not see any issues with
> lark release 0.11.1. dh_auto_test seems to use option 2 by default (for the
> pybuild build system). For autopkgtest I have chosen to use option 1. The
> latter two options both fail for version 0.11.1. 
Which is good?

> After applying a patch provided by upstream [3], also option 2 works but option 1 still fails.
Which is good or bad? I'm not sure.

> Upstream suggested to change the way the (autopkgtest) tests are run [4].
So are you asking only about autopkgtests, not build-time tests?

> Is there something like a general recommendation on what is the best way
> to run Python unit tests?
No (except "whatever the upstream CI runs").

-- 
WBR, wRAR

Thread

How to run unit tests? Peter Wienemann <fossdev@posteo.de> - 2020-12-08 21:40 +0100
  Re: How to run unit tests? Andrey Rahmatullin <wrar@debian.org> - 2020-12-08 21:50 +0100
    Re: How to run unit tests? Peter Wienemann <fossdev@posteo.de> - 2020-12-12 16:40 +0100

csiph-web