Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #11918 > unrolled thread

Discussing next steps for the Python2 removal

Back to article view | Back to linux.debian.maint.python

Contents

  Discussing next steps for the Python2 removal Matthias Klose <doko@debian.org> - 2019-10-22 17:10 +0200
    Re: Discussing next steps for the Python2 removal Sandro Tosi <morph@debian.org> - 2019-10-22 17:50 +0200
      Re: Discussing next steps for the Python2 removal Sandro Tosi <morph@debian.org> - 2019-10-23 04:30 +0200
      Re: Discussing next steps for the Python2 removal Sandro Tosi <morph@debian.org> - 2019-11-27 03:20 +0100
    Re: Discussing next steps for the Python2 removal Paul Gevers <elbrus@debian.org> - 2019-10-22 20:50 +0200
    Re: Discussing next steps for the Python2 removal Ondrej Novy <novy@ondrej.org> - 2019-10-24 22:10 +0200
      Re: Discussing next steps for the Python2 removal Sandro Tosi <morph@debian.org> - 2019-10-25 03:40 +0200
      Re: Discussing next steps for the Python2 removal Sandro Tosi <morph@debian.org> - 2019-10-25 03:40 +0200
        Re: Discussing next steps for the Python2 removal "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2019-10-25 09:10 +0200
          Re: Discussing next steps for the Python2 removal Ondrej Novy <novy@ondrej.org> - 2019-10-25 13:00 +0200
            Re: Discussing next steps for the Python2 removal Simon McVittie <smcv@debian.org> - 2019-10-28 11:00 +0100
              Re: Discussing next steps for the Python2 removal Ondrej Novy <novy@ondrej.org> - 2019-10-28 11:50 +0100
        Re: Discussing next steps for the Python2 removal Ondrej Novy <novy@ondrej.org> - 2019-10-25 13:00 +0200

#11918 — Discussing next steps for the Python2 removal

Paul Gevers from the release team pointed out that the Python2 removal is 
causing some uninstall-ability issues in testing because some packages 
apparently are removed too early, but never the less are migrating to testing. 
He suggested to make the removal plan more concrete and having a timeline.

So maybe people interested in joining the discussion and wanting to help could 
come to #debian-python on this Thu, Oct 24, 18:00 UTC.  If needed, with a 
follow-up on Sat, Oct 27, 18:00 UTC.

Topics could be:

  - filing removal bugs for packages which don't have a bug
    report yet.

  - when to raise bug severity for which bugs, even now
    for leaf packages?

  - bumping lintian info to warn, warn to error, ask ftp-master
    to autoreject packages for some errors?

If you cannot join, please leave some comments in the "Discussions/Proposals" 
section at https://wiki.debian.org/Python/2Removal

[toc] | [next] | [standalone]

#11919

On Tue, Oct 22, 2019 at 11:05 AM Matthias Klose <doko@debian.org> wrote:
>
> Paul Gevers from the release team pointed out that the Python2 removal is
> causing some uninstall-ability issues in testing because some packages
> apparently are removed too early, but never the less are migrating to testing.

in a hope to help preventing this from happening, i've extend the
script that generates the webpage to also send control@ mails to mark
what bugs are blocked by others, and it went live yesterday evening.

i have a semi-working script that produced ~400 package depending on
py2 packages with missing py2removal bugs, i'll try to finalize it in
the coming days and submit those bugs when done

Regards,
-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
G+: https://plus.google.com/u/0/+SandroTosi

[toc] | [prev] | [next] | [standalone]

#11923

> i have a semi-working script that produced ~400 package depending on
> py2 packages with missing py2removal bugs, i'll try to finalize it in
> the coming days and submit those bugs when done

here's the list:
https://people.debian.org/~morph/mass-bug-py2removal_take2.txt in
format

<src pkg> <version> <reason it's been flagged>

all of these sources didnt not have a py2removal filed against them
(and i'm not even checking if there are sources that still depends on
py2 packages but had their py2removal bugs closed, it may be a further
enhancement).

I'm gonna file bugs soon

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
G+: https://plus.google.com/u/0/+SandroTosi

[toc] | [prev] | [next] | [standalone]

#12048

> i have a semi-working script that produced ~400 package depending on
> py2 packages with missing py2removal bugs, i'll try to finalize it in
> the coming days and submit those bugs when done

i found approximately 140 more packages that depends on python/cython
and dont have a py2removal bug; i'll spot check that list soon and
will report the missing bugs in the coming days.

I'll also start looking into packages that closed a py2removal bug but
still have py2 dependencies (and probably i'll reopen said bug).

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#11920

[Multipart message — attachments visible in raw view] — view raw

Dear all,

On 22-10-2019 17:04, Matthias Klose wrote:
> He suggested to make the removal plan more
> concrete and having a timeline.

To be more precise on what I meant, I'm talking here about *every*
package that wants to drop a Python 2 binary package, discuss (and
ideally agree on, but I understand that that can be hard) the time line
with their reverse dependencies.

If this is done globally, fine, but please, give reverse dependencies
time to adapt. I'm already seeing quite some annoyance on the fact that
in several cases the ground is pulled away under one's package.

Yes, there's a bug in britney somewhere that allows the migration, we're
trying to find it.

Paul

For those that are curious, the britney output [1] shows at the end
binary packages in testing that are not build from source anymore but
are left in testing due to dependencies. Those that aren't in
libs/oldlibs shouldn't be there and are make their reverse dependencies
RC buggy. On top of that, Dose [2] shows packages that can't be build
from source in testing. There's quite some python2 packages listing as
causing that. Again, all those packages are RC buggy (often the bug
hasn't been filed yet).

[1] https://release.debian.org/britney/update_output.txt.gz
[2] https://qa.debian.org/dose/debcheck/src_testing_main/ (pick the
latest amd64 log, it should be empty except for cmucl)

[toc] | [prev] | [next] | [standalone]

#11935

[Multipart message — attachments visible in raw view] — view raw

Hi,

notes from this meeting (
https://gobby.debian.org/export/Teams/Python/Py2Removal):

We want to bump all bugs to RC bugs, if they are:
* leaf package
* with py2 binary
* bug is not marked as py2keep
* all modules and low popcon apps (< 300)

If package will be removed from testing (auto-rm), we are going to remove
them from unstable. We are going to run this process semi-automatically
(after more packages gets leaf). First version of list should be checked
(emailed to this ML). We need to prepare draft of RC-bumper email.

We will send email to d-d-a with current state, our next steps, etc.

We are going to bump Lintian tags severity:
* python-foo-but-no-python3-foo: warning->error
* dependency-on-python-version-marked-for-end-of-life: pedantic->warning
* new-package-should-not-package-python2-module: warning->error
* build-depends-on-python-sphinx-only: warning->error
* depends-on-python2-and-python3: info->warning
* script-uses-unversioned-python-in-shebang: info->warning

We would like to update Python policy and clearly state future of Python 2.

Thanks everybody.

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#11937

> notes from this meeting (https://gobby.debian.org/export/Teams/Python/Py2Removal):

small additional note: the py2removal tracker script now ignores
Suggests and wont mark them as rdeps

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [next] | [standalone]

#11938

On Thu, Oct 24, 2019 at 4:09 PM Ondrej Novy <novy@ondrej.org> wrote:
> We want to bump all bugs to RC bugs, if they are:

Was there anyone from the RT that ack'd this? Paul is here and i guess
he can do that now: there could be hundreds of new RC bugs in a blink.

> * leaf package
> * with py2 binary

do we really want to remove from testing (and subsequently from
debian, as mentioned below) if it both has a py2 and a py3k package?
should we limit this to py2-ONLY packages?

> * bug is not marked as py2keep
> * all modules and low popcon apps (< 300)

JFYI for now i was steering clear of removing py2 support even of
modules with high popcon (while it's true they may not have any rdeps
in the archive, there could be plenty of 3rd party code using them).

> If package will be removed from testing (auto-rm), we are going to remove them from unstable. We are going to run this process semi-automatically (after more packages gets leaf). First version of list should be checked (emailed to this ML). We need to prepare draft of RC-bumper email.

mailing an explanation can be tricky: currently the script simply
generates a mail to control@ and that's usually accepted within
seconds of it sending it. but the BTS mail server do implement
greylisting and throttling so generating a huge number of individual
emails could just result in them being queued on my machine until
accepted, and make the script generate duplicate emails (possibly
confusing the end users and definitely un-necessary).

we could go with multi-recipients emails, but first we need to
identify how many recipients is acceptable for the BTS mail severs
(there are usually rules in place to prevent spam), and that also
means there will be a rather long "control header" (for the control
commands) in the email, rendering rather ineffective the text of the
explanation, which will be at the bottom.

> We will send email to d-d-a with current state, our next steps, etc.
>
> We are going to bump Lintian tags severity:
> * python-foo-but-no-python3-foo: warning->error
> * dependency-on-python-version-marked-for-end-of-life: pedantic->warning
> * new-package-should-not-package-python2-module: warning->error
> * build-depends-on-python-sphinx-only: warning->error
> * depends-on-python2-and-python3: info->warning
> * script-uses-unversioned-python-in-shebang: info->warning

is there any plan to make some of this tags an upload-rejecting one?

thanks everyone on working to get rid of python2 from debian!

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
G+: https://plus.google.com/u/0/+SandroTosi

[toc] | [prev] | [next] | [standalone]

#11940

Sandro Tosi wrote:
> do we really want to remove from testing (and subsequently from
> debian, as mentioned below) if it both has a py2 and a py3k package?
> should we limit this to py2-ONLY packages?

Not the whole source package, but we do want to remove the py2 binary. 
Maybe there should be two versions of the email, with the one for py2/3 
packages saying 'NMU' instead of 'removal'?

[toc] | [prev] | [next] | [standalone]

#11942

[Multipart message — attachments visible in raw view] — view raw

Hi,

pá 25. 10. 2019 v 9:09 odesílatel Rebecca N. Palmer <rebecca_palmer@zoho.com>
napsal:

> Not the whole source package, but we do want to remove the py2 binary.
> Maybe there should be two versions of the email, with the one for py2/3
> packages saying 'NMU' instead of 'removal'?
>

this is not how autorm works. You can't remove from testing only one of two
binary package from same source package. You are removing  package as a
whole.

But maintainer can anytime fix that bug by removing py2 binary from source
package, which is few minutes work.

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#11956

On Fri, 25 Oct 2019 at 12:38:11 +0200, Ondrej Novy wrote:
> this is not how autorm works. You can't remove from testing only one of two
> binary package from same source package. You are removing  package as a whole.
> 
> But maintainer can anytime fix that bug by removing py2 binary from source
> package, which is few minutes work.

The maintainer of a leaf package can do this in a few minutes at any
time, but library packages with many reverse-dependencies (for example
dbus-python) don't really have that option, so I hope using autorm for
library packages is not on the table at this stage?

    smcv

[toc] | [prev] | [next] | [standalone]

#11957

[Multipart message — attachments visible in raw view] — view raw

Hi,

po 28. 10. 2019 v 10:56 odesílatel Simon McVittie <smcv@debian.org> napsal:

> The maintainer of a leaf package can do this in a few minutes at any
> time, but library packages with many reverse-dependencies (for example
> dbus-python) don't really have that option, so I hope using autorm for
> library packages is not on the table at this stage?
>

but we are still talking about __leaf__ packages so dbus-python is
irelevant, right?

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [next] | [standalone]

#11941

[Multipart message — attachments visible in raw view] — view raw

Hi,

pá 25. 10. 2019 v 3:30 odesílatel Sandro Tosi <morph@debian.org> napsal:

> do we really want to remove from testing (and subsequently from
> debian, as mentioned below) if it both has a py2 and a py3k package?
> should we limit this to py2-ONLY packages?
>

yes and no. If maintainer wants to keep package in Debian, fix is simple:
Just remove one binary package. If nobody cares (no team upload, no NMU) we
want to move forward and remove that package.

> * bug is not marked as py2keep
> > * all modules and low popcon apps (< 300)
>
> JFYI for now i was steering clear of removing py2 support even of
> modules with high popcon (while it's true they may not have any rdeps
> in the archive, there could be plenty of 3rd party code using them).
>

we want to remove Python 2 in bullseye ideally. So no 3rd party code will
be using them, because there will be no Py2 :)


> is there any plan to make some of this tags an upload-rejecting one?
>

maybe later?

-- 
Best regards
 Ondřej Nový

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web