Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #15070 > unrolled thread

How to realize different test categories?

Back to article view | Back to linux.debian.maint.python

Contents

  How to realize different test categories? c.buhtz@posteo.jp - 2023-07-28 14:20 +0200
    Re: How to realize different test categories? Timo Röhling <roehling@debian.org> - 2023-07-28 15:00 +0200
      Re: How to realize different test categories? c.buhtz@posteo.jp - 2023-07-28 15:10 +0200
    Re: How to realize different test categories? Andrey Rakhmatullin <wrar@wrar.name> - 2023-07-28 15:40 +0200
    Re: How to realize different test categories? Paul Wise <pabs@debian.org> - 2023-07-29 06:20 +0200

#15070 — How to realize different test categories?

Hello,

I'm upstream maintainer (but not founder) of "backintime".

I realized that most of the tests in my test suite are integration or 
system tests which are impossible to handle by Debians build server and 
testing system (how do you name it?).
My package do use something like "./configure && make && make test && 
sudo make install" to test and build. Works fine on a local Debian box.

I would like to have an advice from you as DPT how I should separate my 
tests into the two categories "run by debian" and "don't run by debian".

One approach in my mind is to add one new make target "make unittests" 
containing real unittests able to run on Debians build system in a 
chroot.

Another approach in my mind is asking for environment variables that 
might exists on a Debian build box? I know this approach from TravisCI 
and ReadTheDocs.

Or any other idea on your site?

Kind
Christian

[toc] | [next] | [standalone]

#15074

[Multipart message — attachments visible in raw view] — view raw

Hi Christian,

* c.buhtz@posteo.jp <c.buhtz@posteo.jp> [2023-07-28 12:14]:
>Or any other idea on your site?
I had a look at the backintime sources, and I noticed that you have
implemented a very unusual (for Python, at least) build system.
There are well-known tools and standard practices in the Python
ecosystem for packaging [1], and much of the Python packaging in
Debian is designed to work well with them.

There are also tools for test isolation such as [2]. Instead of
increasing your workload by maintaining an extra set of tests for
Debian, why not let backintime take advantage of established
practices and tools? Not only will this reduce friction with the
Debian packaging, it will generally make your package more
accessible to other Python developers, which in turn may attract
more people to help fix bugs and keep the software in good shape.


Cheers
Timo

[1] https://packaging.python.org/en/latest/tutorials/packaging-projects/
[2] https://tox.wiki/

-- 
⢀⣴⠾⠻⢶⣦⠀   ╭────────────────────────────────────────────────────╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling                                       │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄⠀⠀⠀⠀   ╰────────────────────────────────────────────────────╯

[toc] | [prev] | [next] | [standalone]

#15075

Hello Timo,

thanks for reply.

Am 28.07.2023 14:56 schrieb Timo Röhling:
> I had a look at the backintime sources, and I noticed that you

First of all: I haven't nothing. :D I'm not the founder of backintime 
(BIT) just the 3rd generation maintainer trying to modernize an over 15 
year old python application. ;)

> implemented a very unusual (for Python, at least) build system.

Yes, pyproject.toml etc is on my list.

> There are also tools for test isolation such as [2].

Tox? How can tox help me here? It is to run multiply python versions, 
virtual env and things like this.
Tox, poetry etc would just make a build system more complex then it 
need. Currently I see no advantage of it from upstream perspective but 
I'm open to new arguments.

> increasing your workload by maintaining an extra set of tests
> for Debian

No, that is not how it is. It is not an extra set of tests.

In theory you have three kinds of tests: unit-, integration- and 
system-tests.
To my knowledge only unittests are making sense to run on a Debian build 
system using chroot.

BIT's problem is having no real unittests. But if I want to make them 
accessible to Debians build system excluding the other tests.

It won't be a set of tests realted to Debian. It will just be a set of 
real unittests running everywhere.

> why not let backintime take advantage of established
> practices and tools?

I'm working on it. But breaking the whole project structure without 
knowing all details of it (because I'm not the founder) is to risky at 
the current time. Maybe when Debian 14 is stable. ;)

Kind
Christian

[toc] | [prev] | [next] | [standalone]

#15076

On Fri, Jul 28, 2023 at 12:14:33PM +0000, c.buhtz@posteo.jp wrote:
> Hello,
> 
> I'm upstream maintainer (but not founder) of "backintime".
> 
> I realized that most of the tests in my test suite are integration or system
> tests which are impossible to handle by Debians build server and testing
> system (how do you name it?).
> My package do use something like "./configure && make && make test && sudo
> make install" to test and build. Works fine on a local Debian box.
> 
> I would like to have an advice from you as DPT how I should separate my
> tests into the two categories "run by debian" and "don't run by debian".
In the context of DPT I would use pytest marks.
For poorer test systems it's possible that there is no answer except
splitting the tests into separate things and providing separate commands
for running those.

> Another approach in my mind is asking for environment variables that might
> exists on a Debian build box? I know this approach from TravisCI and
> ReadTheDocs.
You shouldn't check for Debian-specific envvars in upstream sources.

[toc] | [prev] | [next] | [standalone]

#15079

[Multipart message — attachments visible in raw view] — view raw

On Fri, 2023-07-28 at 12:14 +0000, Christian Buhtz wrote:

> I realized that most of the tests in my test suite are integration or
> system tests which are impossible to handle by Debians build server and 
> testing system (how do you name it?).

These sorts of tests are suitable for autopkgtests run on debci:

https://wiki.debian.org/ContinuousIntegration

So you can tag your tests as suitable for build time or not suitable
for build time, and then run build time tests on buildds and others on
the debci service using autopkgtests.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web