Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #12749 > unrolled thread

Newcomers project: DPMT/PAPT pristine-tar verification

Back to article view | Back to linux.debian.maint.python

Contents

  Newcomers project: DPMT/PAPT pristine-tar verification Sandro Tosi <morph@debian.org> - 2020-07-10 06:40 +0200
    Re: Newcomers project: DPMT/PAPT pristine-tar verification Raphael Hertzog <hertzog@debian.org> - 2020-07-19 17:10 +0200
      Re: Newcomers project: DPMT/PAPT pristine-tar verification Sandro Tosi <morph@debian.org> - 2020-07-19 19:30 +0200
        Re: Newcomers project: DPMT/PAPT pristine-tar verification Raphael Hertzog <hertzog@debian.org> - 2020-07-20 10:50 +0200
    Re: Newcomers project: DPMT/PAPT pristine-tar verification Louis-Philippe Véronneau <pollo@debian.org> - 2020-10-06 18:10 +0200
      Re: Newcomers project: DPMT/PAPT pristine-tar verification Louis-Philippe Véronneau <pollo@debian.org> - 2020-10-06 18:20 +0200
        Re: Newcomers project: DPMT/PAPT pristine-tar verification Emmanuel Arias <eamanu@yaerobi.com> - 2020-10-06 20:40 +0200
      Re: Newcomers project: DPMT/PAPT pristine-tar verification intrigeri <intrigeri@debian.org> - 2020-10-06 20:40 +0200
    Re: Newcomers project: DPMT/PAPT pristine-tar verification Thomas Goirand <zigo@debian.org> - 2020-10-09 13:20 +0200
      Re: Newcomers project: DPMT/PAPT pristine-tar verification Stefano Rivera <stefanor@debian.org> - 2020-12-11 19:00 +0100

#12749 — Newcomers project: DPMT/PAPT pristine-tar verification

Hello,
i would like to propose a project to make sure our teams (DPMT/PAPT)
repos are using pristine-tar properly.

The checks i have in mind for now, are:

* pristine-tar branch must exist, if not -> it's a bug
* pristine-tar + upstream branch must produce the same tarball as
downloaded from the archive, if not -> it's a bug
* bonus point: fix the repo if it doesn't generate the right tarball
and or the branch is missing.
* bonus point: make this into a service that runs regularly (not
strictly necessary to be limited to us)

i guess we should have a brief discussion about additional checks
and/or procedures before "assigning" it to a volunteer. let's say up
to 2 weeks of discussion, and during the same period volunteers can
nominate themselves.

I marked this project as newcomers as it doesn't require to be a DD/DM
to work on it, you just need a salsa account and access to our teams.
a handy tool to retrieve all our repos is at

https://salsa.debian.org/python-team/tools/python-modules
https://salsa.debian.org/python-team/tools/python-apps

that contains a config file for `mr` and a `checkout` script to fetch
the repos registered in that config file.

Please feel free to discuss this project now :)

Regards,
--
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [next] | [standalone]

#12784

Hi,

On Fri, 10 Jul 2020, Sandro Tosi wrote:
> The checks i have in mind for now, are:
> 
> * pristine-tar branch must exist, if not -> it's a bug
> * pristine-tar + upstream branch must produce the same tarball as
> downloaded from the archive, if not -> it's a bug
> * bonus point: fix the repo if it doesn't generate the right tarball
> and or the branch is missing.
> * bonus point: make this into a service that runs regularly (not
> strictly necessary to be limited to us)

I would suggest that this would be a nice job for the janitor bot.
https://janitor.debian.net/

Cheers,
-- 
  ⢀⣴⠾⠻⢶⣦⠀   Raphaël Hertzog <hertzog@debian.org>
  ⣾⠁⢠⠒⠀⣿⡁
  ⢿⡄⠘⠷⠚⠋    The Debian Handbook: https://debian-handbook.info/get/
  ⠈⠳⣄⠀⠀⠀⠀   Debian Long Term Support: https://deb.li/LTS

[toc] | [prev] | [next] | [standalone]

#12786

On Sun, Jul 19, 2020 at 11:04 AM Raphael Hertzog <hertzog@debian.org> wrote:
>
> Hi,
>
> On Fri, 10 Jul 2020, Sandro Tosi wrote:
> > The checks i have in mind for now, are:
> >
> > * pristine-tar branch must exist, if not -> it's a bug
> > * pristine-tar + upstream branch must produce the same tarball as
> > downloaded from the archive, if not -> it's a bug
> > * bonus point: fix the repo if it doesn't generate the right tarball
> > and or the branch is missing.
> > * bonus point: make this into a service that runs regularly (not
> > strictly necessary to be limited to us)
>
> I would suggest that this would be a nice job for the janitor bot.
> https://janitor.debian.net/

How would you suggest implementing this?

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#12787

[ Putting Jelmer in copy ]

On Sun, 19 Jul 2020, Sandro Tosi wrote:
> On Sun, Jul 19, 2020 at 11:04 AM Raphael Hertzog <hertzog@debian.org> wrote:
> > On Fri, 10 Jul 2020, Sandro Tosi wrote:
> > > The checks i have in mind for now, are:
> > >
> > > * pristine-tar branch must exist, if not -> it's a bug
> > > * pristine-tar + upstream branch must produce the same tarball as
> > > downloaded from the archive, if not -> it's a bug
> > > * bonus point: fix the repo if it doesn't generate the right tarball
> > > and or the branch is missing.
> > > * bonus point: make this into a service that runs regularly (not
> > > strictly necessary to be limited to us)
> >
> > I would suggest that this would be a nice job for the janitor bot.
> > https://janitor.debian.net/
> 
> How would you suggest implementing this?

The janitor bot can be added to our salsa groups and he can be configured
to push changes directly instead of proposing merge requests.

All the above operations are simple automatable jobs that only result in
changes to the git repositories (mainly updates to the pristine-tar
branch).

I would thus suggest to add this functionality to the janitor and I
believe the right place for this is here:
https://jelmer.uk/code/silver-platter/tree/master/silver_platter/debian

Cheers,
-- 
  ⢀⣴⠾⠻⢶⣦⠀   Raphaël Hertzog <hertzog@debian.org>
  ⣾⠁⢠⠒⠀⣿⡁
  ⢿⡄⠘⠷⠚⠋    The Debian Handbook: https://debian-handbook.info/get/
  ⠈⠳⣄⠀⠀⠀⠀   Debian Long Term Support: https://deb.li/LTS

[toc] | [prev] | [next] | [standalone]

#12929

[Multipart message — attachments visible in raw view] — view raw

On 2020-10-03 15 h 35, Sandro Tosi wrote:
> attached the dd-list of the packages missing the pristine-tar branch (some
> may have been moved/removed, but these are actual repos in DPT)
> 
> On Fri, Jul 10, 2020 at 12:38 AM Sandro Tosi <morph@debian.org> wrote:
> 
>> Hello,
>> i would like to propose a project to make sure our teams (DPMT/PAPT)
>> repos are using pristine-tar properly.
>>
>> The checks i have in mind for now, are:
>>
>> * pristine-tar branch must exist, if not -> it's a bug
>> * pristine-tar + upstream branch must produce the same tarball as
>> downloaded from the archive, if not -> it's a bug
>> * bonus point: fix the repo if it doesn't generate the right tarball
>> and or the branch is missing.
>> * bonus point: make this into a service that runs regularly (not
>> strictly necessary to be limited to us)
>>
>> i guess we should have a brief discussion about additional checks
>> and/or procedures before "assigning" it to a volunteer. let's say up
>> to 2 weeks of discussion, and during the same period volunteers can
>> nominate themselves.
>>
>> I marked this project as newcomers as it doesn't require to be a DD/DM
>> to work on it, you just need a salsa account and access to our teams.
>> a handy tool to retrieve all our repos is at
>>
>> https://salsa.debian.org/python-team/tools/python-modules
>> https://salsa.debian.org/python-team/tools/python-apps
>>
>> that contains a config file for `mr` and a `checkout` script to fetch
>> the repos registered in that config file.
>>
>> Please feel free to discuss this project now :)

I had a chat with folks in #debian-qa last night, as I agree such checks
would be nice to have.

1. Lintian is not suited for that kind of checks, as it does not have
network access. Frankensteining lintian to do that kind of stuff would
surely be met with fierce opposition.

2. The vcswatch script [1] from the QA team already does something akin
to what we would want. It's written in Perl [2], but doesn't look
terribly complicated. When a check doesn't pass, it issues an
action-item like this one [3].

I think the first step would be modifying vcswatcher to issue warnings for:

* the absence of pristine-tar branches
* missing git tags
* repositories using 'master' instead of 'debian/master' as the main branch

Once these are flagged, we can easily script a way to fix them, maybe
even using lintian-brush?

As for the problem of pristine-tar and the upstream branch not producing
the same tarballs, I think it would be best to have that done in another
script altogether.

Cheers,

[1]: https://qa.debian.org/cgi-bin/vcswatch
[2]: https://salsa.debian.org/qa/qa/-/blob/master/cgi-bin/vcswatch
[3]: https://tracker.debian.org/action-items/1469895


-- 
  ⢀⣴⠾⠻⢶⣦⠀
  ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
  ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
  ⠈⠳⣄

[toc] | [prev] | [next] | [standalone]

#12930

[Multipart message — attachments visible in raw view] — view raw

On 2020-10-06 12 h 07, Louis-Philippe Véronneau wrote:
> On 2020-10-03 15 h 35, Sandro Tosi wrote:
>> attached the dd-list of the packages missing the pristine-tar branch (some
>> may have been moved/removed, but these are actual repos in DPT)
>>
>> On Fri, Jul 10, 2020 at 12:38 AM Sandro Tosi <morph@debian.org> wrote:
>>
>>> Hello,
>>> i would like to propose a project to make sure our teams (DPMT/PAPT)
>>> repos are using pristine-tar properly.
>>>
>>> The checks i have in mind for now, are:
>>>
>>> * pristine-tar branch must exist, if not -> it's a bug
>>> * pristine-tar + upstream branch must produce the same tarball as
>>> downloaded from the archive, if not -> it's a bug
>>> * bonus point: fix the repo if it doesn't generate the right tarball
>>> and or the branch is missing.
>>> * bonus point: make this into a service that runs regularly (not
>>> strictly necessary to be limited to us)
>>>
>>> i guess we should have a brief discussion about additional checks
>>> and/or procedures before "assigning" it to a volunteer. let's say up
>>> to 2 weeks of discussion, and during the same period volunteers can
>>> nominate themselves.
>>>
>>> I marked this project as newcomers as it doesn't require to be a DD/DM
>>> to work on it, you just need a salsa account and access to our teams.
>>> a handy tool to retrieve all our repos is at
>>>
>>> https://salsa.debian.org/python-team/tools/python-modules
>>> https://salsa.debian.org/python-team/tools/python-apps
>>>
>>> that contains a config file for `mr` and a `checkout` script to fetch
>>> the repos registered in that config file.
>>>
>>> Please feel free to discuss this project now :)
> 
> I had a chat with folks in #debian-qa last night, as I agree such checks
> would be nice to have.
> 
> 1. Lintian is not suited for that kind of checks, as it does not have
> network access. Frankensteining lintian to do that kind of stuff would
> surely be met with fierce opposition.
> 
> 2. The vcswatch script [1] from the QA team already does something akin
> to what we would want. It's written in Perl [2], but doesn't look
> terribly complicated. When a check doesn't pass, it issues an
> action-item like this one [3].
> 
> I think the first step would be modifying vcswatcher to issue warnings for:
> 
> * the absence of pristine-tar branches
> * missing git tags
> * repositories using 'master' instead of 'debian/master' as the main branch
> 
> Once these are flagged, we can easily script a way to fix them, maybe
> even using lintian-brush?

I meant Debian Janitor here.

I don't know the codebase enough, but in my mind, having the thing that
fixes problems and the thing that flags them be separate is valuable. I
don't know if Janitor follows that philosophy though.

-- 
  ⢀⣴⠾⠻⢶⣦⠀
  ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
  ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
  ⠈⠳⣄

[toc] | [prev] | [next] | [standalone]

#12932

[Multipart message — attachments visible in raw view] — view raw

Hi,

IMO this is a good job for Janitor. I don't know if I would like to add the
possibility to make the
merge without human approval, but for me a MR from Janitor could be enough.



On Tue, Oct 6, 2020 at 1:14 PM Louis-Philippe Véronneau <pollo@debian.org>
wrote:

> On 2020-10-06 12 h 07, Louis-Philippe Véronneau wrote:
> > On 2020-10-03 15 h 35, Sandro Tosi wrote:
> >> attached the dd-list of the packages missing the pristine-tar branch
> (some
> >> may have been moved/removed, but these are actual repos in DPT)
> >>
> >> On Fri, Jul 10, 2020 at 12:38 AM Sandro Tosi <morph@debian.org> wrote:
> >>
> >>> Hello,
> >>> i would like to propose a project to make sure our teams (DPMT/PAPT)
> >>> repos are using pristine-tar properly.
> >>>
> >>> The checks i have in mind for now, are:
> >>>
> >>> * pristine-tar branch must exist, if not -> it's a bug
> >>> * pristine-tar + upstream branch must produce the same tarball as
> >>> downloaded from the archive, if not -> it's a bug
> >>> * bonus point: fix the repo if it doesn't generate the right tarball
> >>> and or the branch is missing.
> >>> * bonus point: make this into a service that runs regularly (not
> >>> strictly necessary to be limited to us)
> >>>
> >>> i guess we should have a brief discussion about additional checks
> >>> and/or procedures before "assigning" it to a volunteer. let's say up
> >>> to 2 weeks of discussion, and during the same period volunteers can
> >>> nominate themselves.
> >>>
> >>> I marked this project as newcomers as it doesn't require to be a DD/DM
> >>> to work on it, you just need a salsa account and access to our teams.
> >>> a handy tool to retrieve all our repos is at
> >>>
> >>> https://salsa.debian.org/python-team/tools/python-modules
> >>> https://salsa.debian.org/python-team/tools/python-apps
> >>>
> >>> that contains a config file for `mr` and a `checkout` script to fetch
> >>> the repos registered in that config file.
> >>>
> >>> Please feel free to discuss this project now :)
> >
> > I had a chat with folks in #debian-qa last night, as I agree such checks
> > would be nice to have.
> >
> > 1. Lintian is not suited for that kind of checks, as it does not have
> > network access. Frankensteining lintian to do that kind of stuff would
> > surely be met with fierce opposition.
> >
> > 2. The vcswatch script [1] from the QA team already does something akin
> > to what we would want. It's written in Perl [2], but doesn't look
> > terribly complicated. When a check doesn't pass, it issues an
> > action-item like this one [3].
> >
> > I think the first step would be modifying vcswatcher to issue warnings
> for:
> >
> > * the absence of pristine-tar branches
> > * missing git tags
> > * repositories using 'master' instead of 'debian/master' as the main
> branch
> >
> > Once these are flagged, we can easily script a way to fix them, maybe
> > even using lintian-brush?
>
> I meant Debian Janitor here.
>
> I don't know the codebase enough, but in my mind, having the thing that
> fixes problems and the thing that flags them be separate is valuable. I
> don't know if Janitor follows that philosophy though.
>
> --
>   ⢀⣴⠾⠻⢶⣦⠀
>   ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
>   ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
>   ⠈⠳⣄
>
>
>
>

[toc] | [prev] | [next] | [standalone]

#12933

Hi,

Louis-Philippe Véronneau (2020-10-06):
> I think the first step would be modifying vcswatcher to issue warnings for:
>
> * the absence of pristine-tar branches
> * missing git tags
> * repositories using 'master' instead of 'debian/master' as the main branch

FWIW, so that nobody else spends time looking for this:
it seems that the script that identifies problems and adds them to the
DB is
https://salsa.debian.org/qa/qa/-/blob/master/data/vcswatch/vcswatch.

Here's the DB schema:
https://salsa.debian.org/qa/qa/-/blob/master/data/vcswatch/vcswatch.sql

Then
https://salsa.debian.org/qa/qa/-/blob/master/cgi-bin/vcswatch
generates the frontend HTML pages.

Cheers!

[toc] | [prev] | [next] | [standalone]

#12935

On 10/3/20 9:35 PM, Sandro Tosi wrote:
> attached the dd-list of the packages missing the pristine-tar branch
> (some may have been moved/removed, but these are actual repos in DPT)

For those:

   python-etcd3 (U)
   python-requests-mock (U)
   python-sphinxcontrib.apidoc (U)

they in fact live in the OpenStack team namespace, but I don't have
enough rights to delete the Git repositories. Can someone do it for me
please?

There's 2 more for which I'm listed, I'll see if I can fix.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#13051

Hi Thomas (2020.10.09_11:12:21_+0000)
> For those:
> 
>    python-etcd3 (U)
>    python-requests-mock (U)
>    python-sphinxcontrib.apidoc (U)
> 
> they in fact live in the OpenStack team namespace, but I don't have
> enough rights to delete the Git repositories. Can someone do it for me
> please?

Deleted.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web