Path: csiph.com!news.samoylyk.net!gothmog.csi.it!bofh.it!news.nic.it!robomod From: Scott Kitterman Newsgroups: linux.debian.maint.python Subject: Re: Policy Change Proposal: Running the upstream test suite Date: Tue, 30 Jul 2024 06:50:02 +0200 Message-ID: References: X-Mailbox-Line: From debian-python-request@lists.debian.org Tue Jul 30 04:48:11 2024 Old-Return-Path: X-Amavis-Spam-Status: No, score=-6.3 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, CR_SYSTEM8=3, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FOURLA=0.1, LDO_WHITELIST=-5, MURPHY_SCAM1=0.1, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no X-Policyd-Weight: using cached result; rate: -5.5 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailing-List: archive/latest/22170 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/11C7F1B8-E2B9-44F4-8532-2D2F0AEB493B@kitterman.com Approved: robomod@news.nic.it Lines: 112 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Date: Tue, 30 Jul 2024 04:47:18 +0000 X-Original-Message-ID: <11C7F1B8-E2B9-44F4-8532-2D2F0AEB493B@kitterman.com> X-Original-References: <02D3B06F-D4BC-48BE-A0D4-191C6FFBBFD8@kitterman.com> <653ea6ec-ccd1-4f68-9f77-125983b1326e@debian.org> <63425DF6-E03F-4F58-9983-96ACAE3B7FE8@kitterman.com> <5318a8ac-1665-49b0-a02f-797aa262d6f6@debian.org> <7F9A2174-F35A-4075-BB81-E5983BC3FC01@kitterman.com> <9947b215-e93f-4fc4-970e-5b9652e80882@debian.org> Xref: csiph.com linux.debian.maint.python:16132 On July 30, 2024 4:27:48 AM UTC, "Louis-Philippe V=C3=A9ronneau" wrote: >On 2024-07-30 12:58, Scott Kitterman wrote: >>=20 >>=20 >> On July 30, 2024 2:47:08 AM UTC, "Louis-Philippe V=C3=A9ronneau" wrote: >>> On 2024-07-30 11:09, Scott Kitterman wrote: >>>>=20 >>>>=20 >>>> On July 30, 2024 12:49:50 AM UTC, "Louis-Philippe V=C3=A9ronneau" wrote: >>>>> On 2024-07-29 21:07, Scott Kitterman wrote: >>>>>>=20 >>>>>>=20 >>>>>> On July 29, 2024 8:53:11 AM UTC, "Louis-Philippe V=C3=A9ronneau" wrote: >>>>>>> Hello, >>>>>>>=20 >>>>>>> As discussed during the DebConf24 Python BoF, I'm submitting this = change to the policy to require the use of the upstream test suite, both du= ring the build process and as an autopkgtest=2E >>>>>>>=20 >>>>>>> You can find the MR here: >>>>>>>=20 >>>>>>> https://salsa=2Edebian=2Eorg/python-team/tools/python-modules/-/me= rge_requests/24 >>>>>>>=20 >>>>>>> People present at the BoF seemed to think this was a good idea=2E = If you don't please reply to this message and make yourself heard :) >>>>>>=20 >>>>>> I understand the theory and why it's a good idea to run the test su= ite=2E I don't think it ought to be a hard requirement=2E I have several = packages where there's a test suite, but I don't run it: >>>>>>=20 >>>>>> 1=2E The largest set is packages that need test only dependencies = which are not packaged=2E When I am packaging something new which has a te= st suite, then I generally package any needed test depends=2E If those test= depends also need test depends packaged, I generally stop and don't enable= tests for things that are only in the archives to support tests=2E Noseof= yeti is an example of this=2E >>>>>=20 >>>>> That sounds like a valid technical reason not to run the tests to me= :) >>>>>=20 >>>>>> 2=2E There's at least one case where Debian has customizations tha= t cause the test suite to fail, but the failures don't seem to cause any re= al problems=2E If anyone wants to make it so the weasyprint test suite wor= ks on Debian, please knock yourself out=2E >>>>>=20 >>>>> Again, as long as you document that, I don't think it would go again= st the proposed policy change=2E >>>>>=20 >>>>>> 3=2E I also maintain multiple packages which require network acces= s to run their test suite, so they can't run tests during build, only autop= kgtests=2E >>>>>=20 >>>>> Same=2E >>>>>=20 >>>>=20 >>>> Except for #3, I don't get that from the wording in the MR=2E I don'= t think not worth the trouble is a technical reason=2E I think the real ru= le that's being proposed is that packages must run the test suit or documen= t why not=2E I don't have a problem with that, but I don't think that's wh= at it actually says now=2E I think if you were to change must to should an= d strike the word technical before reason, it would accomplish the same thi= ng and be clearer=2E I could support that=2E >>>=20 >>> Language is hard and I'm not a native English speaker :) >>>=20 >>> What I want to prevent is people not running tests because they don't = feel like it, even though it would not take them a large amount of efforts = to do so=2E >>>=20 >>> I'll strike "technical", as it seems others also interpreted this word= they way you have=2E >>>=20 >>> As for "MUST" vs "SHOULD", I believe the exception clause provides eno= ugh leeway to justify a reasonable way out in case of $VALID_REASON=2E >>>=20 >>> "SHOULD" is not particularly strong and again, I fear it would let peo= ple get away with not running tests although it wouldn't be much work to do= so=2E=2E=2E >>>=20 >>=20 >> I guess it depends on how you look at the word=2E In the context of te= chnical specifications, I tend to think of terms as defined in RFC 2119 [1]= =2E I think that's pretty close to what you are suggesting: >>=20 >> 3=2E SHOULD This word, or the adjective "RECOMMENDED", mean that ther= e >> may exist valid reasons in particular circumstances to ignore a >> particular item, but the full implications must be understood and >> carefully weighed before choosing a different course=2E >>=20 >> The way RFC 2119 defines MUST doesn't leave much room for exceptions=2E= If you want to be clear, I suggest SHOULD with a reference to RFC 2119=2E= While not universal, it is a widely used reference for definitions of the= se terms in technical specifications=2E >>=20 >> Scott K >>=20 >> [1] https://datatracker=2Eietf=2Eorg/doc/html/rfc2119#section-3 >>=20 > >Fair enough=2E I also made that change=2E Thanks, Scott K