Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.project > #13922

Re: salsa: unconditional rebuild of all reverse dependencies on every push considered abuse (was "Salsa CI overload")

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

Hi,

On Thu, Aug 28, 2025 at 03:15:46PM +0200, Marco d'Itri wrote:
> Can you tell me more about what is wrong with varnish and how to fix it?
>
> I do not understand how it could stand out since it only has an handful of
> reverse dependencies and I rarely push to the repository before I am ready
> to upload.
> Varnish is exactly the kind of package that can break its reverse
> dependencies because they are modules which often access its internals.
> If using SALSA_CI_ENABLE_REVERSE_DEPENDENCY_BUILD is not acceptable then why
> is it available?

In the varnish case, there are only 5 reverse build dependencies, so
it's mostly fine. The point is not having those rebuilds run on user
request, and not for every single push made to the git repository. 

As I mentioned, a better version os this job will be available from
salsa-ci proper at some point, and that will allow just this: reverse
dependency rebuilds are a manually triggered job that requires human
intervention to run. The problem with the version that is currently in
use by varnish and others, which trigger the rebuilds on every single
push.

For now, I *think* this should do it:

----------------8<----------------8<----------------8<----------------- 
diff --git i/debian/rdeps-ci.yml w/debian/rdeps-ci.yml
index 57b5670fa..ab2751669 100644
--- i/debian/rdeps-ci.yml
+++ w/debian/rdeps-ci.yml
@@ -1,5 +1,7 @@
 generate-config:
   image: $SALSA_CI_IMAGES_BASE
+  rules:
+    - when: manual
   needs:
     - job: aptly
       artifacts: true
----------------8<----------------8<----------------8<-----------------

Later when this feature is available in salsa-ci proper, then it should
be a matter of dropping that file entirely (and it's inclusion from
debian/salsa-ci.yml, and just relying on the better version from
salsa-ci.

> BTW, please note that after I took over the maintenance of Varnish I already
> cut a lot the CI time by disabling the very heavy internal test suite for
> the additional rebuilds:
> 
> https://salsa.debian.org/varnish-team/varnish/-/commit/f34d2997e77afc729b162041c0c511e5c5b41594
> 
> (And I still believe that nocheck should be set by default for these tests!)

While optimizations are always welcome, having a few jobs that can take
some time is fine from the infrastructure point of view, as several
others can still run at the time time. The salsa shared CI runner can
currently run up to 64 jobs concurrently.

The problem for everyone happens when a package triggers 2k jobs, as
node-glob did, because when that batch gets to the front of the queue,
it delays everyone else's jobs for quite some time (what prompted this
thread in the first place). Having this run manually on maintainer
request, once in a while, is also OK, but not unconditionally on every
push.

Back to linux.debian.project | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Salsa CI overload Ian Jackson <ijackson@chiark.greenend.org.uk> - 2025-08-25 22:20 +0200
  Re: Salsa CI overload Louis-Philippe Véronneau <pollo@debian.org> - 2025-08-25 23:50 +0200
    Re: Salsa CI overload Simon Josefsson <simon@josefsson.org> - 2025-08-26 09:50 +0200
      Re: Salsa CI overload Louis-Philippe Véronneau <pollo@debian.org> - 2025-08-26 17:10 +0200
      Re: Salsa CI overload Otto Kekäläinen <otto@debian.org> - 2025-09-05 03:20 +0200
        Re: Salsa CI overload Lucas Nussbaum <lucas@debian.org> - 2025-09-05 15:40 +0200
          Re: Salsa CI overload Otto Kekäläinen <otto@debian.org> - 2025-09-05 19:40 +0200
            Re: Salsa CI overload Lucas Nussbaum <lucas@debian.org> - 2025-09-05 21:40 +0200
  Re: Salsa CI overload Alexander Wirt <formorer@debian.org> - 2025-08-27 00:30 +0200
    Re: Salsa CI overload Antoine Le Gonidec <vv221@debian.org> - 2025-08-27 01:30 +0200
      Re: Salsa CI overload Soren Stoutner <soren@debian.org> - 2025-08-27 23:50 +0200
      Re: Salsa CI overload Didier 'OdyX' Raboud <odyx@debian.org> - 2025-08-28 09:30 +0200
        salsa: unconditional rebuild of all reverse dependencies on every push considered abuse (was "Salsa CI overload") Antonio Terceiro <terceiro@debian.org> - 2025-08-28 15:00 +0200
          Re: salsa: unconditional rebuild of all reverse dependencies on every push considered abuse (was "Salsa CI overload") Jérémy Lal <kapouer@melix.org> - 2025-08-28 15:30 +0200
          Re: salsa: unconditional rebuild of all reverse dependencies on every push considered abuse (was "Salsa CI overload") Marco d'Itri <md@Linux.IT> - 2025-08-28 15:30 +0200
            Re: salsa: unconditional rebuild of all reverse dependencies on every push considered abuse (was "Salsa CI overload") Antonio Terceiro <terceiro@debian.org> - 2025-08-28 17:00 +0200
    Re: Salsa CI overload Ian Jackson <ijackson@chiark.greenend.org.uk> - 2025-08-27 11:40 +0200

csiph-web