Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #15141

Re: Package testing with Salsa CI for new packages

Show key headers only | View raw

Hi,

Quick answer for now...

On 19 Aug 2023 at 19:04:09, Paul Boddie wrote:
> On Friday, 18 August 2023 19:54:55 CEST Carles Pina i Estany wrote:
> > 
> > Ha! I wasn't aware of the aptly option
> > (https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built-> apt-repository and SALSA_CI_DISABLE_APTLY=0).
> > I think that I might have re-invented the wheel in a tiny part of
> > Debusine CI/CD.
> 
> It is certainly a way of propagating packages to those that might need
> them.  However, the instructions indicating how a package might access
> these dependencies appear to be deficient.

I haven't used aptly on salsa but on Monday I want to give it a try, to
see if I can replace my solution...

> It does not appear to be sufficient to merely specify the dependency
> package repositories and mark them as trusted. Doing that will just
> cause the repositories to be regarded as ignored and the GPG keys
> signalled as unrecognised.
> 
> So, the GPG keys need to be obtained. This is a hassle because something like 
> wget is needed to do that, and then apt has to be persuaded not to fail in an 
> opaque way. So the modified recipe is something like this:
> 
> before_script:
>   - apt-get update
>   - NON_INTERACTIVE=1 apt-get install -y wget
>   - echo "deb https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/
> artifacts/raw/aptly unstable main" | tee /etc/apt/sources.list.d/pkga.list
>   ...
>   - wget -q -O /etc/apt/trusted.gpg.d/emeraldtree.asc https://
> salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly/
> public-key.asc

I'm ensure that this is correct. I have:
-----
deb [arch=amd64 signed-by=/usr/share/keyrings/oracle-virtualbox-2016.gpg] https://download.virtualbox.org/virtualbox/debian bookworm contrib
-----

So I thought that the "signed-by" was needed; and that the files that I
have there are binary files (not .asc) of this type:
carles@pinux:~$ file /etc/apt/trusted.gpg.d/skype.gpg
/etc/apt/trusted.gpg.d/skype.gpg: OpenPGP Public Key Version 4, Created Wed Jun 22 09:36:35 2016, RSA (Encrypt or Sign, 2048 bits); User ID; Signature; OpenPGP Certificate
carles@pinux:~$

I don't know if a .asc files are allowed. Actually, from my
.bash_history:
---
wget -O- https://www.virtualbox.org/download/oracle_vbox_2016.asc | sudo gpg --dearmor --yes --output /usr/share/keyrings/oracle-virtualbox-2016.gpg
---

(I was following instructions, I didn't try leaving the .asc file there)

>   ...
>   - apt-get update
> 
> This seems to make the various jobs happy, but the one that I was most
> concerned with remains unhappy! I don't actually need the dependencies
> for anything other than autopkgtest, but that job employs its own
> environment where the above recipe has no effect.

You could try using:
---
variables:
    SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=ci/setup-the-repo.sh'
---
(and write and ship the ci/setup-the-repo.sh in the repo, do whatever
you need there)

Or, probably even better but less flexible:
---
variables:
    SALSA_CI_AUTOPKGTEST_ARGS: '--add-apt-source="deb http://MIRROR SUITE COMPONENT'
---
(I found add-apt-source via "man autopkgtest" in TEST BED SETUP OPTIONS.
I haven't used add-apt-source, I don't know what happens with the gpg
keys... but you could use [trusted=yes] :-| )

For the MIRROR you have salsa variables that might help if you need to
specify the pipeline.

> So, what I now need to do is to find out how I can make the new packages 
> available to autopkgtest specifically.

We will get there! (one way or another)

> > I will point out at some things that might safe some time to you
> > (great) or not (ignore! :-) ):
> > 
> > debusine's .gitlab-ci.yml:
> > https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml
> 
> This is definitely useful for examples of the CI definition syntax and how 
> hidden jobs can be used to provide additional script fragments.
> 
> [...]
> 
> > The job autopkgtest, via debci command, runs autopkgtest:
> > https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458#L65
> 
> The difficult thing appears to be the configuration of this testing 
> environment. One approach might involve overriding the existing job 
> definitions to incorporate the injection of new packages.

hopefully SALSA_CI_AUTOPKGTEST_ARGS will help you. I added this in
salsa-ci/pipeline last year because I was trying to do a similar thing
to what you are doing (I had to pin a package from backports).

(I'm just a user of salsa-ci/pipeline, not a member of the team neither
I can speak for them!)

> [...]
> 
> > It's possible for sure. Other people in this list might come up with a
> > different idea. I don't have almost any experience with Debian
> > packaging, but I have some experience on the salsa CI. So I might be
> > giving you solutions that might be sub-optimal! :-)
> 
> I think that you are giving me some good ideas at least. I do feel that I am 
> encountering problems that must have been encountered many times before, but I 
> could imagine that people do not use these tools in this way or have different 
> expectations of what they are able to accomplish.

I don't know if an FAQ, conventions, best practises or something might
help...

Cheers,

-- 
Carles Pina i Estany
https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat

Back to linux.debian.maint.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-17 17:20 +0200
  Re: Package testing with Salsa CI for new packages Carles Pina i Estany <carles@pina.cat> - 2023-08-18 10:00 +0200
    Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-18 13:20 +0200
      Re: Package testing with Salsa CI for new packages Carles Pina i Estany <carles@pina.cat> - 2023-08-18 16:20 +0200
        Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-18 19:10 +0200
          Re: Package testing with Salsa CI for new packages Carles Pina i Estany <carles@pina.cat> - 2023-08-18 20:00 +0200
            Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-19 19:10 +0200
              Re: Package testing with Salsa CI for new packages Carles Pina i Estany <carles@pina.cat> - 2023-08-19 20:40 +0200
                Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-20 00:50 +0200
                Re: Package testing with Salsa CI for new packages Carles Pina i Estany <carles@pina.cat> - 2023-08-20 14:10 +0200
                Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-20 23:20 +0200
                Re: Package testing with Salsa CI for new packages Carles Pina i Estany <carles@pina.cat> - 2023-08-21 15:10 +0200
                Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-21 19:20 +0200
  Re: Package testing with Salsa CI for new packages Andrey Rakhmatullin <wrar@wrar.name> - 2023-08-18 18:30 +0200
    Re: Package testing with Salsa CI for new packages Paul Boddie <paul@boddie.org.uk> - 2023-08-18 19:20 +0200
      Re: Package testing with Salsa CI for new packages Andrey Rakhmatullin <wrar@wrar.name> - 2023-08-18 19:50 +0200

csiph-web