Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.devel > #100358

automated pkg set installation and upgrade tests (was Re: Planning for libidn shared library version transition)

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

hi,

On Wed, May 26, 2021 at 11:28:24PM +0100, Simon McVittie wrote:
> piuparts only routinely tests relatively small installations - there are
> periodic QA tests done on larger systems like the union of all desktop
> tasks, but those are more expensive to run.

indeed, on jenkins.debian.net there are 421 of those package installation
tests running (called chroot-installation there for historic reasons.)

I'd be more than glad to run *more* tests there! *many* more tests!
please read this mail completly and then reply, send MR, join #debian-qa
and ask me there or some such.

those 421 tests are divided into testing these suites:

holger@jenkins:/var/lib/jenkins/jobs$ for i in stretch buster bullseye sid ; do echo -n "$i: " ; ls -1d chroot-installation_*|grep -c $i ; done
stretch: 97
buster: 208
bullseye: 162
sid: 162

so quite many of them test installation in one suite and then
upgrading to the next.

trigger_times = {
    'stretch':  '30 10 1 */2 *',
    'buster': '30 10 4,11,18,25 * *',
    'bullseye': '30 10 */3 * *',
    'sid':     '30 4 * * *',
    }

is when the jobs are triggered and that's coming from the job
definitions in https://salsa.debian.org/qa/jenkins.debian.net/-/blob/master/job-cfg/chroot-installation.yaml.py

the packages to be installed are defined in 
https://salsa.debian.org/qa/jenkins.debian.net/-/blob/master/bin/chroot-installation.sh
in lines 227 until 267.

currently the package sets being tested are mostly desktop 
environments and Debian Edu package sets:

all_targets = [
   'gnome',
   'kde',
   'kde-full',
   'cinnamon',
   'lxde',
   'lxqt',
   'xfce',
   'full_desktop',
   'qt4',
   'qt5',
   'haskell',
   'developer',
   'debconf-video',
   'education-tasks',
   'education-menus',
   'education-astronomy',
   'education-chemistry',
   'education-common',
   'education-desktop-gnome',
   'education-desktop-kde',
   'education-desktop-lxde',
   'education-desktop-lxqt',
   'education-desktop-mate',
   'education-desktop-other',
   'education-desktop-xfce',
   'education-desktop-cinnamon',
   'education-development',
   'education-electronics',
   'education-geography',
   'education-graphics',
   'education-language',
   'education-laptop',
   'education-logic-games',
   'education-ltsp-server',
   'education-main-server',
   'education-mathematics',
   'education-misc',
   'education-music',
   'education-networked',
   'education-physics',
   'education-primaryschool',
   'education-services',
   'education-standalone',
   'education-thin-client',
   'education-roaming-workstation',
   'education-video',
   'education-workstation',
   'parl-desktop-eu',
   'parl-desktop-strict',
   'parl-desktop-world',
   'design-desktop-animation',
   'design-desktop-graphics',
   'design-desktop-strict',
   'design-desktop-web',
   ]

(again from that yaml.py file linked above.)

A list of all tests is attached and a list of all tests currently flagged
being problematic is available at https://jenkins.debian.net/view/chroot-installation/view/Problems/

Let's test all the things!


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Society: Be Yourself!
Society: No, not like that.

Back to linux.debian.devel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Planning for libidn shared library version transition Simon Josefsson <simon@josefsson.org> - 2021-05-24 20:20 +0200
  Re: Planning for libidn shared library version transition Timo Röhling <timo@gaussglocke.de> - 2021-05-24 20:50 +0200
    Re: Planning for libidn shared library version transition Simon Josefsson <simon@josefsson.org> - 2021-05-24 21:00 +0200
  Re: Planning for libidn shared library version transition Andreas Metzler <ametzler@bebt.de> - 2021-05-25 19:50 +0200
    Re: Planning for libidn shared library version transition Simon Josefsson <simon@josefsson.org> - 2021-05-26 00:20 +0200
      Re: Planning for libidn shared library version transition Simon McVittie <smcv@debian.org> - 2021-05-26 11:50 +0200
    Re: Planning for libidn shared library version transition Guillem Jover <guillem@debian.org> - 2021-05-26 01:50 +0200
      Re: Planning for libidn shared library version transition Andreas Metzler <ametzler@bebt.de> - 2021-05-26 18:00 +0200
        Re: Planning for libidn shared library version transition Simon Josefsson <simon@josefsson.org> - 2021-05-26 19:20 +0200
          Re: Planning for libidn shared library version transition Simon McVittie <smcv@debian.org> - 2021-05-27 00:30 +0200
            Re: Planning for libidn shared library version transition Simon Josefsson <simon@josefsson.org> - 2021-05-27 08:40 +0200
              Re: Planning for libidn shared library version transition Simon McVittie <smcv@debian.org> - 2021-05-27 12:20 +0200
                Re: Planning for libidn shared library version transition Marvin Renich <mrvn@renich.org> - 2021-05-27 16:20 +0200
                Re: Planning for libidn shared library version transition Andrey Rahmatullin <wrar@debian.org> - 2021-05-27 16:40 +0200
                Re: Planning for libidn shared library version transition Marvin Renich <mrvn@renich.org> - 2021-05-27 19:50 +0200
                Re: Planning for libidn shared library version transition Marvin Renich <mrvn@renich.org> - 2021-05-27 19:50 +0200
                Re: Planning for libidn shared library version transition David Kalnischkies <david@kalnischkies.de> - 2021-05-27 17:00 +0200
                Re: Planning for libidn shared library version transition Simon McVittie <smcv@debian.org> - 2021-05-28 18:20 +0200
                Re: Planning for libidn shared library version transition David Kalnischkies <david@kalnischkies.de> - 2021-06-01 17:20 +0200
              Re: Planning for libidn shared library version transition Russ Allbery <rra@debian.org> - 2021-05-27 20:20 +0200
                Re: Planning for libidn shared library version transition Holger Levsen <holger@layer-acht.org> - 2021-05-27 20:30 +0200
            automated pkg set installation and upgrade tests (was Re: Planning  for libidn shared library version transition) Holger Levsen <holger@layer-acht.org> - 2021-05-27 16:50 +0200
    Re: Planning for libidn shared library version transition Simon Josefsson <simon@josefsson.org> - 2021-07-27 11:10 +0200
      Re: Planning for libidn shared library version transition Andreas Metzler <ametzler@bebt.de> - 2021-07-28 10:20 +0200

csiph-web