Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #13811

Bug#1000803: dh-python: add pybuild-autopkgtest, a test runner

Cross-posted to 2 groups.

Show key headers only | View raw

[Multipart message — attachments visible in raw view] - view raw

Control: reassign -1 dh-python
Control: retitle -1 dh-python: add pybuild-autopkgtest, a test runner
Control: severity -1 wishlist
Control: tag -1 + patch
Control: forwarded -1 https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27

On Mon, Dec 20, 2021 at 04:03:17PM -0300, Antonio Terceiro wrote:
> On Wed, Dec 01, 2021 at 08:19:20PM -0300, Antonio Terceiro wrote:
> > Hi,
> > 
> > Adding debian-python@l.d.o
> > 
> > The context is #1000803 where Sandro asked about reducing duplication
> > when running upstream test suites both during the build and under
> > autopkgtest.
> > 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000803
> > 
> > On Tue, Nov 30, 2021 at 12:47:42AM -0500, Sandro Tosi wrote:
> > > > This is usually solved outside of autopkgtest itself.
> > > >
> > > > For example Ruby packages have a single place where tests are specified
> > > > (debian/ruby-tests.*{rb,rake,yaml}, and the appropriate
> > > > debian/tests/control file is autogenerated by autodep8. I would say 99%
> > > > of Ruby packages require no duplication, or even explicitly specifying
> > > > commands to run tests at all.
> > > >
> > > > In general, this is most efficiently solved by package type (e.g.
> > > > programming language), because the way the tests are run at build-time
> > > > usually is tied to the build system. You then usually need some test
> > > > runner, probably extracted from, or provided by, the build system, to
> > > > also provide the same funcionality in an autopkgtest-compatible way.
> > > >
> > > > All the package types that are well supported in autodep8 have their
> > > > specialized test runner tools that avoid this type of duplication.
> > > 
> > > I'm mostly interested in the python ecosystem, and the provided
> > > autodep8 test are extremely superficial (essentially only importing
> > > the main python module packaged), which is better than nothing but not
> > > the same level as what's in d/rules.
> > > 
> > > Are you aware of any effort to expand the Python autodep8 tests to
> > > uniform them into a comprehensive, and unique place to run tests at
> > > build-time and via autopkgtest?
> > 
> > I'm not.
> > 
> > > If no such effort currently exists, what would one have to do to start
> > > developing it? Not necessarily volunteering, just gathering
> > > information.
> > 
> > In general terms, I see this being implemented like this:
> > 
> > 0) modify pybuild so one can call e.g. `pybuild --autopkgtest`. That
> > should work almost the same as `pybuild --test`, but would copy the test
> > files under ${AUTOPKGTEST_TMP} and run the tests from there, instead of
> > assuming a previously built tree and trying to chdir into
> > .pybuild/*/build.
> > 
> > 1) add a way of being able to specify pybuild parameters outside of
> > debian/rules that can be used both during the build and under
> > autopkgtest
> > 
> >    For Ruby, gem2deb-test-runner is driven by debian/ruby-tests.*, both
> >    during the build, and under autopkgtest.
> > 
> >    In pybuild, some aspects of the test run can already be done this
> >    way, e.g. debian/pybuild.testfiles. Maybe we could have
> >    debian/pybuild.env to read options in the same way as they are set in
> >    debian/rules today (with environment variables).
> > 
> > 2) update autodep8 to generate the appropriate control file to call
> >    `pybuild --autopkgtest`. This needs to be "smart" enough to not
> >    automatically add this call to packages that are not ready for it. At
> >    the moment, almost 1000 source packages have
> >    `Testsuite: autopkgtest-pkg-python`. We would probably need a new
> >    value, or (probably better) to additionally check for something else
> >    in the source tree.
> > 
> > Each item has quite some details to be figured out, but this is the
> > general idea.
> 
> I have written a prototype implementation of this, and published it as
> https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27

This is now ready to be seriously considered, please take a look.

Back to linux.debian.maint.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Bug#1000803: autopkgtest: avoid duplication with autopkgtests and how unittests are run at build-time Antonio Terceiro <terceiro@debian.org> - 2021-12-02 00:30 +0100
  Bug#1000803: autopkgtest: avoid duplication with autopkgtests and how unittests are run at build-time Antonio Terceiro <terceiro@debian.org> - 2021-12-20 20:10 +0100
    Bug#1000803: dh-python: add pybuild-autopkgtest, a test runner Antonio Terceiro <terceiro@debian.org> - 2021-12-25 14:30 +0100

csiph-web