Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #14271 > unrolled thread

Notes from the DC22 Python Team BoF

Back to article view | Back to linux.debian.maint.python

Contents

  Notes from the DC22 Python Team BoF Louis-Philippe Véronneau <pollo@debian.org> - 2022-07-23 20:00 +0200
    Re: Notes from the DC22 Python Team BoF Julian Gilbey <julian@d-and-j.net> - 2022-07-25 09:40 +0200
      Re: Notes from the DC22 Python Team BoF Thomas Goirand <zigo@debian.org> - 2022-07-27 10:00 +0200
        Re: Notes from the DC22 Python Team BoF Louis-Philippe Véronneau <pollo@debian.org> - 2022-07-27 16:20 +0200
    pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Julian Gilbey <julian@d-and-j.net> - 2022-07-25 10:10 +0200
      Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Antonio Terceiro <terceiro@debian.org> - 2022-07-26 17:00 +0200
        Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Scott Kitterman <debian@kitterman.com> - 2022-07-26 17:50 +0200
          Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Antonio Terceiro <terceiro@debian.org> - 2022-07-26 23:00 +0200
            Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Paul Wise <pabs@debian.org> - 2022-07-27 02:20 +0200
              Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Louis-Philippe Véronneau <pollo@debian.org> - 2022-07-27 05:50 +0200
        Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Julian Gilbey <julian@d-and-j.net> - 2022-07-27 10:20 +0200
          Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Louis-Philippe Véronneau <pollo@debian.org> - 2022-07-27 16:30 +0200
            Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Julian Gilbey <julian@d-and-j.net> - 2022-07-27 20:50 +0200
              Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Julian Gilbey <julian@d-and-j.net> - 2022-07-27 22:40 +0200
                Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Julian Gilbey <julian@d-and-j.net> - 2022-07-28 20:50 +0200
                Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Louis-Philippe Véronneau <pollo@debian.org> - 2022-08-18 02:00 +0200
          Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF) Simon McVittie <smcv@debian.org> - 2022-07-27 18:50 +0200

#14271 — Notes from the DC22 Python Team BoF

Hey folks,

We had a Python Team BoF at DC22 earlier today and I thought relaying 
the notes we took in gobby here would be a good idea.

----------------------------------------------------------------------
== python3.11 ==

python3.11 release has been delayed, from october 2022 to december 2022.

python3.12 is planned for october 2023.

Debian testing freeze for transitions is in January 2023, which is very 
tight.

Do we want to try python3.11 in bookworm, or do we delay it for after 
the freeze?

One problem we'll likely have, is upstream python libraries might not 
have started playing with 3.11 already. It might be hard to try to use 
beta versions right now to try and prepare the transition.

Once the default version in the archive has changed, it's hard to revert 
to an older one.

Some packages like pandas and numba (and dart?) might need an exception 
from the Release Team to allow us to upload versions more compatible 
with 3.11 after the bookworm release.

For sure, this is a lot of work for me (zigo) on the OpenStack packages. 
On each Python 3 release, this makes me fix lots of stuff. At the 
moment, upstream OpenStack isn't even on Python 1.10 on its CI...

As a datapoint, Ruby always releases on Christmas, and the Ruby team has 
never even attempted to push that as a default in the next Debian stable 
release.

3.11 beta 4 is already in unstable, people can start trying to build 
against it.

3.10 EOL is october 2026.

upstream is working on lots of speed optimizations. 3.11 has a bunch of 
these that our users would appreciate.

3.11 as an extra supported version might work out, but it will create 
subtle breakage for packages that happen not to be compatible with that 
version, so we should avoid that in a stable release.

3.11 cannot be easily tested via an archive rebuild since there are 
about 25 stages to a transition which build on top of one another.

Doko asked if it would be possible to have the Python releases 1 month 
earlier, to which people replied: "Why doesn't Debian change their 
release dates?". :(

== PEP 668 ==

https://discuss.python.org/t/pep-668-marking-python-base-environments-as-externally-managed/ 
https://peps.python.org/pep-0668/

We would like to have an directory reserved for distro-managed packages, 
where pip should not be installing anything.

upstream pip seems to be keen on that, although there are currently no 
issues in their bug report about it.

it would also be nice to have a python option to only use distro-managed 
packages on the load path.

== pybuild improvements ==

getting the autopkgtest MR in would be great

https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27

We need people to test this MR some more, although it seems fairly mature.

It might be a good idea to have a line in d/control to let us migrate 
from the existing autopkgtests running unit tests to the new automated MR.

== lintian tags requests for the team ==

pollo will write you Python-related lintian tags. Ask him to.

* find packages that are building extensions only for the current Python 
default version, instead of all the available python versions.

* warn packages still using distutils.version (removal in python 3.12)

It would be nice if lintian brush could help us migrate to pybuild-pyproject
   - pollo can ping Jelmer.

== upstream cpython patches ==

Some work has been done, but some Debian patches still need to be 
forwarded to python upstream

== where are we with python2rm? ==

pypy is still a blocker. A solution would be to bundle the python2 
source code in it.

Other blockers 
(https://release.debian.org/transitions/html/python2-rm.html):

python-defaults
python2.7
pam-python
python-stdlib-extensions
redland-bindings #938345 orphaned, key package
mozilla-devscripts (used for firefox extension building) #937085 key package
python-setuptools
python2-pip
six

== possible remote sprint? ==

a good time to have a remote sprint would be after adding python3.11 as 
the new default (and thus creating new RC bugs). Therefore, this would 
be between end of October up to early December.

== tracker.d.o dashboard ==

https://tracker.debian.org/teams/python-modules/

Would be great to have Salsa MRs on it

----------------------------------------------------------------------

Cheers,

-- 
   ⢀⣴⠾⠻⢶⣦⠀
   ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
   ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
   ⠈⠳⣄

[toc] | [next] | [standalone]

#14280

On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
> Hey folks,
> 
> We had a Python Team BoF at DC22 earlier today and I thought relaying the
> notes we took in gobby here would be a good idea.

Thanks for the notes, Louis-Philippe, and sorry I couldn't join you!

A few comments....

> ----------------------------------------------------------------------
> == python3.11 ==
> 
> python3.11 release has been delayed, from october 2022 to december 2022.
> [...]

My 2 cents' worth is as the 3.9->3.10 transition took several months,
and was quite complicated, it is not wise to attempt the 3.10->3.11
before the freeze.  We could then potentially go straight to 3.12 a
few months after the bookworm freeze rather than going to 3.11 first.
And that will probably be quite painful.

> == pybuild improvements ==
> 
> getting the autopkgtest MR in would be great
> 
> https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
> 
> We need people to test this MR some more, although it seems fairly mature.
> 
> It might be a good idea to have a line in d/control to let us migrate from
> the existing autopkgtests running unit tests to the new automated MR.

I'll take this to a separate email.

> == lintian tags requests for the team ==
> 
> pollo will write you Python-related lintian tags. Ask him to.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1004746  :-)

   Julian

[toc] | [prev] | [next] | [standalone]

#14289

On 7/25/22 09:37, Julian Gilbey wrote:
> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
>> Hey folks,
>>
>> We had a Python Team BoF at DC22 earlier today and I thought relaying the
>> notes we took in gobby here would be a good idea.
> 
> Thanks for the notes, Louis-Philippe, and sorry I couldn't join you!
> 
> A few comments....
> 
>> ----------------------------------------------------------------------
>> == python3.11 ==
>>
>> python3.11 release has been delayed, from october 2022 to december 2022.
>> [...]
> 
> My 2 cents' worth is as the 3.9->3.10 transition took several months,
> and was quite complicated, it is not wise to attempt the 3.10->3.11
> before the freeze.  We could then potentially go straight to 3.12 a
> few months after the bookworm freeze rather than going to 3.11 first.
> And that will probably be quite painful.

I agree, also because 3.10 wont be EOL before Bookworm becomes LTS.

However, it's quite tempting to upgrade to 3.11 because:
- our users will prefer having the latest stable release of Python in 
our latest release, rather than an old version.
- 3.11 has many optimization (it's said to be 25% faster)

So if we can at least TRY, it's not a so bad idea... Hopefully, we can 
take the decision to reverse if needed.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#14291

[Multipart message — attachments visible in raw view] — view raw

On 2022-07-27 03 h 52, Thomas Goirand wrote:
> On 7/25/22 09:37, Julian Gilbey wrote:
>> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
>>> Hey folks,
>>>
>>> We had a Python Team BoF at DC22 earlier today and I thought relaying 
>>> the
>>> notes we took in gobby here would be a good idea.
>>
>> Thanks for the notes, Louis-Philippe, and sorry I couldn't join you!
>>
>> A few comments....
>>
>>> ----------------------------------------------------------------------
>>> == python3.11 ==
>>>
>>> python3.11 release has been delayed, from october 2022 to december 2022.
>>> [...]
>>
>> My 2 cents' worth is as the 3.9->3.10 transition took several months,
>> and was quite complicated, it is not wise to attempt the 3.10->3.11
>> before the freeze.  We could then potentially go straight to 3.12 a
>> few months after the bookworm freeze rather than going to 3.11 first.
>> And that will probably be quite painful.
> 
> I agree, also because 3.10 wont be EOL before Bookworm becomes LTS.

As I said during the BoF, I'm also in favor of sticking with 3.10 for 
bookworm.

I think we should aim for a stable Python ecosystem. Using the time we 
have left to iron out bugs and failures in our packages for 3.10 sounds 
like a much better decision than trying to fix a ton of RC bugs for 3.11 
in time for the final freeze.

> However, it's quite tempting to upgrade to 3.11 because:
> - our users will prefer having the latest stable release of Python in 
> our latest release, rather than an old version.
> - 3.11 has many optimization (it's said to be 25% faster)
> 
> So if we can at least TRY, it's not a so bad idea... Hopefully, we can 
> take the decision to reverse if needed.

AFAIU, doko said it was pretty hard to go back, since that meant 
recompiling pretty much all the python extensions.

I'd much rather we don't waste tons of energy trying to get 3.11 and 
then deciding it's not working. We know we only have a 2-3 months window 
of opportunity to make the transition, IMO that's too short.

-- 
   ⢀⣴⠾⠻⢶⣦⠀
   ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
   ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
   ⠈⠳⣄

[toc] | [prev] | [next] | [standalone]

#14282 — pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
> == pybuild improvements ==
> 
> getting the autopkgtest MR in would be great
> 
> https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
> 
> We need people to test this MR some more, although it seems fairly mature.
> 
> It might be a good idea to have a line in d/control to let us migrate from
> the existing autopkgtests running unit tests to the new automated MR.

I've been looking at this a bit more.  I'm not sure what this last
paragraph means: the new "automated" autopkgtest will only be run if
the maintainer explicitly adds:

Testsuite: autopkgtest-pkg-pybuild

to debian/control (see the autodep8 MR at
https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
it will never automatically detect a pybuild package).
And a maintainer would presumably only add that if they are also
removing their existing debian/tests/control (or want to run the
pybuild tests in addition).

An alternative would be for the autodep8 patch to try to determine
whether to run pybuild-autopkgtest.  One approach could be:

if the package would run autopkgtest-pkg-python:
  if debian/control does not contain an override_dh_auto_test stanza:
    run pybuild-autopkgtest

Note, though, that if autodep8 is called, it will run all of the
detected tests.  (At least that is what I believe happens from reading
/usr/bin/autodep8; I haven't double-checked this.)  So, for example,
if a package specifies

Testsuite: autopkgtest-pkg-python

it will also run the autopkgtest-pkg-pybuild suite as it will be
detected as being a Python package, and vice versa.  That is a
possible reason *not* to use the above suggestion, as it would
potentially run pybuild-autopkgtest even if not desired.

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#14284 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Multipart message — attachments visible in raw view] — view raw

On Mon, Jul 25, 2022 at 09:04:27AM +0100, Julian Gilbey wrote:
> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
> > == pybuild improvements ==
> > 
> > getting the autopkgtest MR in would be great
> > 
> > https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
> > 
> > We need people to test this MR some more, although it seems fairly mature.
> > 
> > It might be a good idea to have a line in d/control to let us migrate from
> > the existing autopkgtests running unit tests to the new automated MR.
> 
> I've been looking at this a bit more.  I'm not sure what this last
> paragraph means: the new "automated" autopkgtest will only be run if
> the maintainer explicitly adds:
> 
> Testsuite: autopkgtest-pkg-pybuild
> 
> to debian/control (see the autodep8 MR at
> https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
> it will never automatically detect a pybuild package).
> And a maintainer would presumably only add that if they are also
> removing their existing debian/tests/control (or want to run the
> pybuild tests in addition).
> 
> An alternative would be for the autodep8 patch to try to determine
> whether to run pybuild-autopkgtest.  One approach could be:
> 
> if the package would run autopkgtest-pkg-python:
>   if debian/control does not contain an override_dh_auto_test stanza:
>     run pybuild-autopkgtest
> 
> Note, though, that if autodep8 is called, it will run all of the
> detected tests.  (At least that is what I believe happens from reading
> /usr/bin/autodep8; I haven't double-checked this.)  So, for example,
> if a package specifies
> 
> Testsuite: autopkgtest-pkg-python
> 
> it will also run the autopkgtest-pkg-pybuild suite as it will be
> detected as being a Python package, and vice versa.  That is a
> possible reason *not* to use the above suggestion, as it would
> potentially run pybuild-autopkgtest even if not desired.

I think the notes did not capture the consensus correctly. The point was
that it should be possible to automate updating the `Testsuite:` field
to run tests with pybuild-autopkgtest, and that we should probably do
that across team packages with the help of some scripting.

[toc] | [prev] | [next] | [standalone]

#14285 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On July 26, 2022 2:50:19 PM UTC, Antonio Terceiro <terceiro@debian.org> wrote:
>On Mon, Jul 25, 2022 at 09:04:27AM +0100, Julian Gilbey wrote:
>> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
>> > == pybuild improvements ==
>> > 
>> > getting the autopkgtest MR in would be great
>> > 
>> > https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
>> > 
>> > We need people to test this MR some more, although it seems fairly mature.
>> > 
>> > It might be a good idea to have a line in d/control to let us migrate from
>> > the existing autopkgtests running unit tests to the new automated MR.
>> 
>> I've been looking at this a bit more.  I'm not sure what this last
>> paragraph means: the new "automated" autopkgtest will only be run if
>> the maintainer explicitly adds:
>> 
>> Testsuite: autopkgtest-pkg-pybuild
>> 
>> to debian/control (see the autodep8 MR at
>> https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
>> it will never automatically detect a pybuild package).
>> And a maintainer would presumably only add that if they are also
>> removing their existing debian/tests/control (or want to run the
>> pybuild tests in addition).
>> 
>> An alternative would be for the autodep8 patch to try to determine
>> whether to run pybuild-autopkgtest.  One approach could be:
>> 
>> if the package would run autopkgtest-pkg-python:
>>   if debian/control does not contain an override_dh_auto_test stanza:
>>     run pybuild-autopkgtest
>> 
>> Note, though, that if autodep8 is called, it will run all of the
>> detected tests.  (At least that is what I believe happens from reading
>> /usr/bin/autodep8; I haven't double-checked this.)  So, for example,
>> if a package specifies
>> 
>> Testsuite: autopkgtest-pkg-python
>> 
>> it will also run the autopkgtest-pkg-pybuild suite as it will be
>> detected as being a Python package, and vice versa.  That is a
>> possible reason *not* to use the above suggestion, as it would
>> potentially run pybuild-autopkgtest even if not desired.
>
>I think the notes did not capture the consensus correctly. The point was
>that it should be possible to automate updating the `Testsuite:` field
>to run tests with pybuild-autopkgtest, and that we should probably do
>that across team packages with the help of some scripting.

I suppose this is fine as long as whoever does it fixes all the resulting RC bugs.

Scott K

[toc] | [prev] | [next] | [standalone]

#14286 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Multipart message — attachments visible in raw view] — view raw

On Tue, Jul 26, 2022 at 03:37:50PM +0000, Scott Kitterman wrote:
> 
> 
> On July 26, 2022 2:50:19 PM UTC, Antonio Terceiro <terceiro@debian.org> wrote:
> >On Mon, Jul 25, 2022 at 09:04:27AM +0100, Julian Gilbey wrote:
> >> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
> >> > == pybuild improvements ==
> >> > 
> >> > getting the autopkgtest MR in would be great
> >> > 
> >> > https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
> >> > 
> >> > We need people to test this MR some more, although it seems fairly mature.
> >> > 
> >> > It might be a good idea to have a line in d/control to let us migrate from
> >> > the existing autopkgtests running unit tests to the new automated MR.
> >> 
> >> I've been looking at this a bit more.  I'm not sure what this last
> >> paragraph means: the new "automated" autopkgtest will only be run if
> >> the maintainer explicitly adds:
> >> 
> >> Testsuite: autopkgtest-pkg-pybuild
> >> 
> >> to debian/control (see the autodep8 MR at
> >> https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
> >> it will never automatically detect a pybuild package).
> >> And a maintainer would presumably only add that if they are also
> >> removing their existing debian/tests/control (or want to run the
> >> pybuild tests in addition).
> >> 
> >> An alternative would be for the autodep8 patch to try to determine
> >> whether to run pybuild-autopkgtest.  One approach could be:
> >> 
> >> if the package would run autopkgtest-pkg-python:
> >>   if debian/control does not contain an override_dh_auto_test stanza:
> >>     run pybuild-autopkgtest
> >> 
> >> Note, though, that if autodep8 is called, it will run all of the
> >> detected tests.  (At least that is what I believe happens from reading
> >> /usr/bin/autodep8; I haven't double-checked this.)  So, for example,
> >> if a package specifies
> >> 
> >> Testsuite: autopkgtest-pkg-python
> >> 
> >> it will also run the autopkgtest-pkg-pybuild suite as it will be
> >> detected as being a Python package, and vice versa.  That is a
> >> possible reason *not* to use the above suggestion, as it would
> >> potentially run pybuild-autopkgtest even if not desired.
> >
> >I think the notes did not capture the consensus correctly. The point was
> >that it should be possible to automate updating the `Testsuite:` field
> >to run tests with pybuild-autopkgtest, and that we should probably do
> >that across team packages with the help of some scripting.
> 
> I suppose this is fine as long as whoever does it fixes all the resulting RC bugs.

Well the idea is to only actually commit the change and upload the
package with the new Testsuite value after ensuring that actually works,
i.e. that the autopkgtest passes.

We are not suggesting to do this blindly across the packages, sorry for
not making that clear.

[toc] | [prev] | [next] | [standalone]

#14287 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Multipart message — attachments visible in raw view] — view raw

On Tue, 2022-07-26 at 17:53 -0300, Antonio Terceiro wrote:

> Well the idea is to only actually commit the change and upload the
> package with the new Testsuite value after ensuring that actually works,
> i.e. that the autopkgtest passes.

This sounds like something for lintian and the Janitor. I suggest
lintian because not all Python packages are in the team and I suggest
the Janitor because it makes changes automatically, although I am not
entirely sure if it runs autopkgtests yet?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#14288 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On 2022-07-26 20 h 13, Paul Wise wrote:
> On Tue, 2022-07-26 at 17:53 -0300, Antonio Terceiro wrote:
> 
>> Well the idea is to only actually commit the change and upload the
>> package with the new Testsuite value after ensuring that actually works,
>> i.e. that the autopkgtest passes.
> 
> This sounds like something for lintian and the Janitor. I suggest
> lintian because not all Python packages are in the team and I suggest
> the Janitor because it makes changes automatically, although I am not
> entirely sure if it runs autopkgtests yet?

I agree Lintian+Janitor is probably the best tool combo for this, yes.

If Jelmer agrees to do the Janitor part, I'll be happy to do the Lintian
part once the pybuild-autopkgtest MR has been merged and we've confirmed
this feature is ready for large-scale deployment.

-- 
  ⢀⣴⠾⠻⢶⣦⠀
  ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
  ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
  ⠈⠳⣄

[toc] | [prev] | [next] | [standalone]

#14290 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On Tue, Jul 26, 2022 at 11:50:19AM -0300, Antonio Terceiro wrote:
> I think the notes did not capture the consensus correctly. The point was
> that it should be possible to automate updating the `Testsuite:` field
> to run tests with pybuild-autopkgtest, and that we should probably do
> that across team packages with the help of some scripting.

This makes more sense, thanks.

There seems to be little point running both pybuild-autopkgtest and a
manually written debian/tests/* test suite.  So would the script only
add pybuild-autopkgtest to packages which don't have a manually
written debian/tests/* suite?

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#14292 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Multipart message — attachments visible in raw view] — view raw

On 2022-07-27 04 h 18, Julian Gilbey wrote:
> 
> There seems to be little point running both pybuild-autopkgtest and a
> manually written debian/tests/* test suite.  So would the script only
> add pybuild-autopkgtest to packages which don't have a manually
> written debian/tests/* suite?

The way I see it:

1. We should have a Lintian tag for packages not using the new 
pybuild-autodep8 autopkgtest. It would be even better if this tag would 
be a pointed hint that identified 'manually' written unit test 
autopkgtests that could be replaced.

This way, you get something like:

python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests]

for python packages that have old 'manually' written unit test 
autopkgtests and:

python-foo source: not-using-pybuild-autodep8 [no-autopkgtest]

for python packages without any autopkgtest.

2. lintian-brush (or something else, but I think lintian-brush is the 
right tool) would go over these packages to:

2.1 Add the new autodep8 autopkgtests and build the package to see if 
they pass
2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds
2.3 Open a bug report if 2.1 fails

This way, we can imagine a transition that would be mostly automated. 
We'd probably have to fix the ones that failed to migrate manually, but 
it would be much less work :)

Cheers,

-- 
   ⢀⣴⠾⠻⢶⣦⠀
   ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
   ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
   ⠈⠳⣄

[toc] | [prev] | [next] | [standalone]

#14294 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On Wed, Jul 27, 2022 at 10:26:33AM -0400, Louis-Philippe Véronneau wrote:
> The way I see it:
> 
> 1. We should have a Lintian tag for packages not using the new
> pybuild-autodep8 autopkgtest. It would be even better if this tag would be a
> pointed hint that identified 'manually' written unit test autopkgtests that
> could be replaced.
> 
> This way, you get something like:
> 
> python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests]
> 
> for python packages that have old 'manually' written unit test autopkgtests
> and:
> 
> python-foo source: not-using-pybuild-autodep8 [no-autopkgtest]
> 
> for python packages without any autopkgtest.
> 
> 2. lintian-brush (or something else, but I think lintian-brush is the right
> tool) would go over these packages to:
> 
> 2.1 Add the new autodep8 autopkgtests and build the package to see if they
> pass
> 2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds
> 2.3 Open a bug report if 2.1 fails

I'd be wary about 2.2 and 2.3.  I have several packages where I know
that an automated test will fail; there are all sorts of weird cases
where I've had to write tests manually.  I would also be quite cross
if manually crafted tests were automatically removed, especially in
cases such as Simon mentioned where they do things that that
automatically generated test does not do.  Another thing I could
imagine happening is that the automated test succeeds in a trivial way
- it succeeds but doesn't actually test much because of the nature of
the package.

On the other hand, a bug report saying something like the following
seems much more reasonable: "We've tested this package using the
automated autopkgtest system and it seems to work by adding the line
'Testsuite: autopkgtest-pkg-pybuild'; please check that the automated
tests cover all of the tests of your manually written debian/tests/*
and if so, then please remove them.  The autopkgtest-pkg-pybuild logs
are attached."  This would give the maintainer the chance to decide
how best to proceed.

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#14295 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On Wed, Jul 27, 2022 at 07:45:12PM +0100, Julian Gilbey wrote:
> On Wed, Jul 27, 2022 at 10:26:33AM -0400, Louis-Philippe Véronneau wrote:
> > The way I see it:
> > 
> > 1. We should have a Lintian tag for packages not using the new
> > pybuild-autodep8 autopkgtest. It would be even better if this tag would be a
> > pointed hint that identified 'manually' written unit test autopkgtests that
> > could be replaced.
> > 
> > This way, you get something like:
> > 
> > python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests]
> > 
> > for python packages that have old 'manually' written unit test autopkgtests
> > and:
> > 
> > python-foo source: not-using-pybuild-autodep8 [no-autopkgtest]
> > 
> > for python packages without any autopkgtest.
> > 
> > 2. lintian-brush (or something else, but I think lintian-brush is the right
> > tool) would go over these packages to:
> > 
> > 2.1 Add the new autodep8 autopkgtests and build the package to see if they
> > pass
> > 2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds
> > 2.3 Open a bug report if 2.1 fails
> 
> I'd be wary about 2.2 and 2.3.  I have several packages where I know
> that an automated test will fail; there are all sorts of weird cases
> where I've had to write tests manually.  I would also be quite cross
> if manually crafted tests were automatically removed, especially in
> cases such as Simon mentioned where they do things that that
> automatically generated test does not do.  Another thing I could
> imagine happening is that the automated test succeeds in a trivial way
> - it succeeds but doesn't actually test much because of the nature of
> the package.
> 
> On the other hand, a bug report saying something like the following
> seems much more reasonable: "We've tested this package using the
> automated autopkgtest system and it seems to work by adding the line
> 'Testsuite: autopkgtest-pkg-pybuild'; please check that the automated
> tests cover all of the tests of your manually written debian/tests/*
> and if so, then please remove them.  The autopkgtest-pkg-pybuild logs
> are attached."  This would give the maintainer the chance to decide
> how best to proceed.

Here's another alternative to steps 2.1-2.3 based on this:

 For packages which currently have manually-written autopkgtests:

 2.A Try removing debian/tests and adding Testsuite:
     autopkgtest-pkg-pybuild to debian/control, then building the
     package and running autopkgtest.

 2.B If this works, then submit a bug report to the BTS as I suggested
     above.

 2.C If this does not work, don't do anything more; trust that the
     maintainer knew what they were doing when they wrote the manual
     autopkgtests.

 For packages which don't currently have manually-written
 autopkgtests:

 2.A' Try adding Testsuite: autopkgtest-pkg-pybuild to debian/control,
      then building the package and running autopkgtest.

 2.B' If this works, then either Janitor adds this line to
      debian/control or submit a bug report to the BTS to recommend
      this.  (But we would not expect Janitor to do step 2.1', so this
      might have to be a different setup, or maybe the script doing
      2.A' could leave a list of packages for Janitor, or something
      like that.)

 2.C' If this does not work, submit a wishlist bug to the BTS to
      recommend that the maintainer adds some autopkgtest tests,
      either by making the autodep8 system work or by writing some
      manual tests.

I'd be wary about adding lintian tags for this, though: with so many
packages not being able to use the autodep8 system (I vaguely recall
someone suggesting that a third of Python packages would not be able
to use the system), that's a lot of packages getting false positives.
In particular, for the suggested first version of
not-using-pybuild-autodep8 tag (which would probably be better named
manual-autopkgtest-could-be-pybuild-autodep8), how would lintian go
about identifying which packages fall into category 2.B and which into
2.C?  The second version of the tag (better named something like
no-autopkgtest-could-use-pybuild-autodep8, but that's still not very
good) is less problematic.

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#14297 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On Wed, Jul 27, 2022 at 09:32:19PM +0100, Julian Gilbey wrote:
> [...]
> > 
> > I'd be wary about 2.2 and 2.3.  I have several packages where I know
> > that an automated test will fail; there are all sorts of weird cases
> > [...]
> 
> I'd be wary about adding lintian tags for this, though: with so many
> packages not being able to use the autodep8 system (I vaguely recall
> someone suggesting that a third of Python packages would not be able
> [...]

I realise that I may have come across as quite negative.  Apologies if
that is the case - it was not my intention.  I think the
autopkgtest-pkg-pybuild/pybuild-autodep8 work is very helpful and a
positive addition to our infrastructure, and I'm very grateful to
those who've made it happen.

My only concern is with how it is introduced; because of the wide
variety of Python packages and the hugely varying nature of the
testsuites present in them (or not), I think that trying to force this
into packages is a poor idea.  (I wish that most of my packages could
simply use this pybuild-autodep8 tool.  I fear that this won't be the
case, but I will certainly adopt it for those which can.)

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#14352 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Multipart message — attachments visible in raw view] — view raw

On 2022-07-27 16 h 32, Julian Gilbey wrote:
> On Wed, Jul 27, 2022 at 07:45:12PM +0100, Julian Gilbey wrote:
>> On Wed, Jul 27, 2022 at 10:26:33AM -0400, Louis-Philippe Véronneau wrote:
>>> The way I see it:
>>>
>>> 1. We should have a Lintian tag for packages not using the new
>>> pybuild-autodep8 autopkgtest. It would be even better if this tag would be a
>>> pointed hint that identified 'manually' written unit test autopkgtests that
>>> could be replaced.
>>>
>>> This way, you get something like:
>>>
>>> python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests]
>>>
>>> for python packages that have old 'manually' written unit test autopkgtests
>>> and:
>>>
>>> python-foo source: not-using-pybuild-autodep8 [no-autopkgtest]
>>>
>>> for python packages without any autopkgtest.
>>>
>>> 2. lintian-brush (or something else, but I think lintian-brush is the right
>>> tool) would go over these packages to:
>>>
>>> 2.1 Add the new autodep8 autopkgtests and build the package to see if they
>>> pass
>>> 2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds
>>> 2.3 Open a bug report if 2.1 fails
>>
>> I'd be wary about 2.2 and 2.3.  I have several packages where I know
>> that an automated test will fail; there are all sorts of weird cases
>> where I've had to write tests manually.  I would also be quite cross
>> if manually crafted tests were automatically removed, especially in
>> cases such as Simon mentioned where they do things that that
>> automatically generated test does not do.  Another thing I could
>> imagine happening is that the automated test succeeds in a trivial way
>> - it succeeds but doesn't actually test much because of the nature of
>> the package.
>>
>> On the other hand, a bug report saying something like the following
>> seems much more reasonable: "We've tested this package using the
>> automated autopkgtest system and it seems to work by adding the line
>> 'Testsuite: autopkgtest-pkg-pybuild'; please check that the automated
>> tests cover all of the tests of your manually written debian/tests/*
>> and if so, then please remove them.  The autopkgtest-pkg-pybuild logs
>> are attached."  This would give the maintainer the chance to decide
>> how best to proceed.
> 
> Here's another alternative to steps 2.1-2.3 based on this:
> 
>   For packages which currently have manually-written autopkgtests:
> 
>   2.A Try removing debian/tests and adding Testsuite:
>       autopkgtest-pkg-pybuild to debian/control, then building the
>       package and running autopkgtest.
> 
>   2.B If this works, then submit a bug report to the BTS as I suggested
>       above.
> 
>   2.C If this does not work, don't do anything more; trust that the
>       maintainer knew what they were doing when they wrote the manual
>       autopkgtests.
> 
>   For packages which don't currently have manually-written
>   autopkgtests:
> 
>   2.A' Try adding Testsuite: autopkgtest-pkg-pybuild to debian/control,
>        then building the package and running autopkgtest.
> 
>   2.B' If this works, then either Janitor adds this line to
>        debian/control or submit a bug report to the BTS to recommend
>        this.  (But we would not expect Janitor to do step 2.1', so this
>        might have to be a different setup, or maybe the script doing
>        2.A' could leave a list of packages for Janitor, or something
>        like that.)
> 
>   2.C' If this does not work, submit a wishlist bug to the BTS to
>        recommend that the maintainer adds some autopkgtest tests,
>        either by making the autodep8 system work or by writing some
>        manual tests.
> 
> I'd be wary about adding lintian tags for this, though: with so many
> packages not being able to use the autodep8 system (I vaguely recall
> someone suggesting that a third of Python packages would not be able
> to use the system), that's a lot of packages getting false positives.
> In particular, for the suggested first version of
> not-using-pybuild-autodep8 tag (which would probably be better named
> manual-autopkgtest-could-be-pybuild-autodep8), how would lintian go
> about identifying which packages fall into category 2.B and which into
> 2.C?  The second version of the tag (better named something like
> no-autopkgtest-could-use-pybuild-autodep8, but that's still not very
> good) is less problematic.

This all makes a lot of sense to me and I think this is the way forward, 
once the MR is merged. Thanks for the discussion.

-- 
   ⢀⣴⠾⠻⢶⣦⠀
   ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
   ⢿⡄⠘⠷⠚⠋   pollo@debian.org / veronneau.org
   ⠈⠳⣄

[toc] | [prev] | [next] | [standalone]

#14293 — Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

On Wed, 27 Jul 2022 at 09:18:42 +0100, Julian Gilbey wrote:
> There seems to be little point running both pybuild-autopkgtest and a
> manually written debian/tests/* test suite.

I think it can make sense to have both. d/tests is the right place for
an integration test that checks things like "any user-facing executables
are correctly in the PATH", which an upstream-oriented test probably
cannot assert because you might be installing into ~/.local or similar.

Using src:tap.py as an example, pybuild-autopkgtest should presumably
replace d/tests/python3 and/or d/tests/python3-with-recommends (which
are tests for the library you get from "import tap"), but wouldn't test
the tappy(1) CLI entry point, which is tested by d/tests/tappy.

    smcv

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web