Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1171201 > unrolled thread

Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries

Started byRuss Allbery <rra@debian.org>
First post2023-10-09 00:10 +0200
Last post2023-10-09 18:20 +0200
Articles 5 — 2 participants

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries Russ Allbery <rra@debian.org> - 2023-10-09 00:10 +0200
    Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries Russ Allbery <rra@debian.org> - 2023-10-09 04:30 +0200
      Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries Jonathan Kamens <jik@kamens.us> - 2023-10-09 05:10 +0200
    Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries Jonathan Kamens <jik@kamens.us> - 2023-10-09 17:40 +0200
      Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries Russ Allbery <rra@debian.org> - 2023-10-09 18:20 +0200

#1171201 — Bug#1053696: Upgrading Python packages showed numerous ancient changelog entries

FromRuss Allbery <rra@debian.org>
Date2023-10-09 00:10 +0200
SubjectBug#1053696: Upgrading Python packages showed numerous ancient changelog entries
Message-ID<HmK8h-etra-1@gated-at.bofh.it>
Package: apt-listchanges
Version: 4.0
Severity: normal
X-Debbugs-Cc: rra@debian.org

An apt run that included the following upgrades:

Unpacking python3.11-dev (3.11.6-3) over (3.11.6-2) ...
Unpacking libpython3.11-dev:amd64 (3.11.6-3) over (3.11.6-2) ...
Unpacking libpython3.11:amd64 (3.11.6-3) over (3.11.6-2) ...
Unpacking python3.11-venv (3.11.6-3) over (3.11.6-2) ...
Unpacking python3.11 (3.11.6-3) over (3.11.6-2) ...
Unpacking libpython3.11-stdlib:amd64 (3.11.6-3) over (3.11.6-2) ...
Unpacking python3.11-minimal (3.11.6-3) over (3.11.6-2) ...
Unpacking libpython3.11-minimal:amd64 (3.11.6-3) over (3.11.6-2) ...
Unpacking python3.11-doc (3.11.6-3) over (3.11.6-2) ...

appears to have gotten confused about what entries I would have seen before
and showed me what appeared to be the entire python3.11 changelog. I thought
it may have only been because the package name in that changelog changes
over time, but it also showed me the entries for 3.11.6-2 and 3.11.6-1,
which have the same source and binary package names and were already
installed. That makes me think something more fundamental may have gone
wrong.

I'm not completely sure which package the changelog entries were pulled from.
I spot-checked a few of them on disk and they all seemed to be truncated at
python3.8, but apt-listchanges found and displayed changelog entries going
back to python2.* versions.

-- Package-specific info:
==> /etc/apt/listchanges.conf <==
[apt]
frontend=pager
email_address=none
confirm=0
save_seen=/var/lib/apt/listchanges
which=both
no_network=false
headers=false
reverse=false


-- System Information:
Debian Release: trixie/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (500, 'unstable-debug'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 6.5.0-1-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages apt-listchanges depends on:
ii  apt                    2.7.6
ii  debconf [debconf-2.0]  1.5.82
ii  python3                3.11.4-5+b1
ii  python3-apt            2.6.0
ii  python3-debconf        1.5.82
ii  sensible-utils         0.0.20
ii  ucf                    3.0043+nmu1

apt-listchanges recommends no packages.

Versions of packages apt-listchanges suggests:
ii  chromium [www-browser]              117.0.5938.149-1
ii  firefox [www-browser]               118.0-1+b1
ii  google-chrome-stable [www-browser]  117.0.5938.149-1
ii  links [www-browser]                 2.29-1+b1
ii  lynx [www-browser]                  2.9.0dev.12-1
ii  postfix [mail-transport-agent]      3.8.2-1
ii  python3-gi                          3.46.0-1
ii  w3m [www-browser]                   0.5.3+git20230121-2
ii  xterm [x-terminal-emulator]         385-1

-- debconf information:
* apt-listchanges/headers: false
  apt-listchanges/email-format: text
* apt-listchanges/frontend: pager
* apt-listchanges/email-address:
* apt-listchanges/reverse: false
* apt-listchanges/confirm: false
* apt-listchanges/which: both
* apt-listchanges/save-seen: true
* apt-listchanges/no-network: false

[toc] | [next] | [standalone]

#1171209

FromRuss Allbery <rra@debian.org>
Date2023-10-09 04:30 +0200
Message-ID<HmObU-evNi-1@gated-at.bofh.it>
In reply to#1171201
Package: apt-listchanges
Version: 4.0
Followup-For: Bug #1053696
X-Debbugs-Cc: rra@debian.org

Same thing happened with the following upgrades:

Unpacking gcc-12 (12.3.0-10) over (12.3.0-9) ...
Unpacking libgcc-12-dev:amd64 (12.3.0-10) over (12.3.0-9) ...
Unpacking cpp-12 (12.3.0-10) over (12.3.0-9) ...
Unpacking gcc-12-base:amd64 (12.3.0-10) over (12.3.0-9) ...

apt-listchanges displayed what I think is the entire gcc changelog,
including the entries for 12.3.0-9 which were already installed.

It's not happening with every package, though. I'm not sure yet what the
common pattern is, unless it's packages that have multiple package names
in their changelog.

-- Package-specific info:
==> /etc/apt/listchanges.conf <==
[apt]
frontend=pager
email_address=none
confirm=0
save_seen=/var/lib/apt/listchanges
which=both
no_network=false
headers=false
reverse=false


-- System Information:
Debian Release: trixie/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (500, 'unstable-debug'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 6.5.0-1-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages apt-listchanges depends on:
ii  apt                    2.7.6
ii  debconf [debconf-2.0]  1.5.82
ii  python3                3.11.4-5+b1
ii  python3-apt            2.6.0
ii  python3-debconf        1.5.82
ii  sensible-utils         0.0.20
ii  ucf                    3.0043+nmu1

apt-listchanges recommends no packages.

Versions of packages apt-listchanges suggests:
ii  chromium [www-browser]              117.0.5938.149-1
ii  firefox [www-browser]               118.0-1+b1
ii  google-chrome-stable [www-browser]  117.0.5938.149-1
ii  links [www-browser]                 2.29-1+b1
ii  lynx [www-browser]                  2.9.0dev.12-1
ii  postfix [mail-transport-agent]      3.8.2-1
ii  python3-gi                          3.46.0-1
ii  w3m [www-browser]                   0.5.3+git20230121-2
ii  xterm [x-terminal-emulator]         385-1

-- debconf information:
* apt-listchanges/confirm: false
* apt-listchanges/headers: false
* apt-listchanges/which: both
* apt-listchanges/email-address:
* apt-listchanges/reverse: false
* apt-listchanges/save-seen: true
* apt-listchanges/frontend: pager
* apt-listchanges/no-network: false
  apt-listchanges/email-format: text

[toc] | [prev] | [next] | [standalone]

#1171210

FromJonathan Kamens <jik@kamens.us>
Date2023-10-09 05:10 +0200
Message-ID<HmOOB-ewgm-1@gated-at.bofh.it>
In reply to#1171209

[Multipart message — attachments visible in raw view] — view raw

You don't have to look into it anymore, I know what's happening. ;-)

Been working on it for a couple of hours. Will have an update in the 
morning, no time to write it up now.

Thanks for the report!

jik

On 10/8/23 22:23, Russ Allbery wrote:
> Package: apt-listchanges
> Version: 4.0
> Followup-For: Bug #1053696
> X-Debbugs-Cc:rra@debian.org
>
> Same thing happened with the following upgrades:
>
> Unpacking gcc-12 (12.3.0-10) over (12.3.0-9) ...
> Unpacking libgcc-12-dev:amd64 (12.3.0-10) over (12.3.0-9) ...
> Unpacking cpp-12 (12.3.0-10) over (12.3.0-9) ...
> Unpacking gcc-12-base:amd64 (12.3.0-10) over (12.3.0-9) ...
>
> apt-listchanges displayed what I think is the entire gcc changelog,
> including the entries for 12.3.0-9 which were already installed.
>
> It's not happening with every package, though. I'm not sure yet what the
> common pattern is, unless it's packages that have multiple package names
> in their changelog.
>
> -- Package-specific info:
> ==> /etc/apt/listchanges.conf <==
> [apt]
> frontend=pager
> email_address=none
> confirm=0
> save_seen=/var/lib/apt/listchanges
> which=both
> no_network=false
> headers=false
> reverse=false
>
>
> -- System Information:
> Debian Release: trixie/sid
>    APT prefers unstable
>    APT policy: (990, 'unstable'), (500, 'unstable-debug'), (500, 'testing'), (1, 'experimental')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
>
> Kernel: Linux 6.5.0-1-amd64 (SMP w/8 CPU threads; PREEMPT)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
>
> Versions of packages apt-listchanges depends on:
> ii  apt                    2.7.6
> ii  debconf [debconf-2.0]  1.5.82
> ii  python3                3.11.4-5+b1
> ii  python3-apt            2.6.0
> ii  python3-debconf        1.5.82
> ii  sensible-utils         0.0.20
> ii  ucf                    3.0043+nmu1
>
> apt-listchanges recommends no packages.
>
> Versions of packages apt-listchanges suggests:
> ii  chromium [www-browser]              117.0.5938.149-1
> ii  firefox [www-browser]               118.0-1+b1
> ii  google-chrome-stable [www-browser]  117.0.5938.149-1
> ii  links [www-browser]                 2.29-1+b1
> ii  lynx [www-browser]                  2.9.0dev.12-1
> ii  postfix [mail-transport-agent]      3.8.2-1
> ii  python3-gi                          3.46.0-1
> ii  w3m [www-browser]                   0.5.3+git20230121-2
> ii  xterm [x-terminal-emulator]         385-1
>
> -- debconf information:
> * apt-listchanges/confirm: false
> * apt-listchanges/headers: false
> * apt-listchanges/which: both
> * apt-listchanges/email-address:
> * apt-listchanges/reverse: false
> * apt-listchanges/save-seen: true
> * apt-listchanges/frontend: pager
> * apt-listchanges/no-network: false
>    apt-listchanges/email-format: text
>

[toc] | [prev] | [next] | [standalone]

#1171268

FromJonathan Kamens <jik@kamens.us>
Date2023-10-09 17:40 +0200
Message-ID<Hn0wp-eFIF-1@gated-at.bofh.it>
In reply to#1171201

[Multipart message — attachments visible in raw view] — view raw

TLDR this will be fixed in 4.1 and it's a significant enough fix that I 
should probably release 4.1 to experimental

So, this was one of the edge cases mentioned in the design documentation 
for the revamped changelog filtering logic: when the persistent database 
is not being used in a particular invocation of the program or there is 
no data for a particular package in the database (this is what you ran 
into), and the changelog data for a package is being fetched over the 
network because it is not present in the package, we can't use 
historical changelog data to determine which entries to display and 
which to filter out.

I was going to treat this as a transitional issue arising from the 
switch to the new filtering logic, but I think it's going to happen more 
often than expected and not just because of that transition (e.g., it 
will happen whenever anybody installs a new package that doesn't have 
local changelogs and then later it gets upgraded), so I went ahead and 
implemented the additional logic I had suggested in bug#1052697: in this 
one particular edge case, fall back on version-number filtering like the 
program used to do.

However, the logic I've added back is not exactly like the old logic; it 
is more robust and careful not to exclude changelog entries it 
shouldn't. As a result, there will still be cases in which it decides it 
can't reliably filter the network changelog and display all of it. 
Fortunately, however, the case you cited in this bug is not one of those 
cases. ;-)

I added a unit test for this and it failed before this change and passes 
now.

jik

On 10/8/23 18:02, Russ Allbery wrote:
> Package: apt-listchanges
> Version: 4.0
> Severity: normal
> X-Debbugs-Cc:rra@debian.org
>
> An apt run that included the following upgrades:
>
> Unpacking python3.11-dev (3.11.6-3) over (3.11.6-2) ...
> Unpacking libpython3.11-dev:amd64 (3.11.6-3) over (3.11.6-2) ...
> Unpacking libpython3.11:amd64 (3.11.6-3) over (3.11.6-2) ...
> Unpacking python3.11-venv (3.11.6-3) over (3.11.6-2) ...
> Unpacking python3.11 (3.11.6-3) over (3.11.6-2) ...
> Unpacking libpython3.11-stdlib:amd64 (3.11.6-3) over (3.11.6-2) ...
> Unpacking python3.11-minimal (3.11.6-3) over (3.11.6-2) ...
> Unpacking libpython3.11-minimal:amd64 (3.11.6-3) over (3.11.6-2) ...
> Unpacking python3.11-doc (3.11.6-3) over (3.11.6-2) ...
>
> appears to have gotten confused about what entries I would have seen before
> and showed me what appeared to be the entire python3.11 changelog. I thought
> it may have only been because the package name in that changelog changes
> over time, but it also showed me the entries for 3.11.6-2 and 3.11.6-1,
> which have the same source and binary package names and were already
> installed. That makes me think something more fundamental may have gone
> wrong.
>
> I'm not completely sure which package the changelog entries were pulled from.
> I spot-checked a few of them on disk and they all seemed to be truncated at
> python3.8, but apt-listchanges found and displayed changelog entries going
> back to python2.* versions.
>
> -- Package-specific info:
> ==> /etc/apt/listchanges.conf <==
> [apt]
> frontend=pager
> email_address=none
> confirm=0
> save_seen=/var/lib/apt/listchanges
> which=both
> no_network=false
> headers=false
> reverse=false
>
>
> -- System Information:
> Debian Release: trixie/sid
>    APT prefers unstable
>    APT policy: (990, 'unstable'), (500, 'unstable-debug'), (500, 'testing'), (1, 'experimental')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
>
> Kernel: Linux 6.5.0-1-amd64 (SMP w/8 CPU threads; PREEMPT)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
>
> Versions of packages apt-listchanges depends on:
> ii  apt                    2.7.6
> ii  debconf [debconf-2.0]  1.5.82
> ii  python3                3.11.4-5+b1
> ii  python3-apt            2.6.0
> ii  python3-debconf        1.5.82
> ii  sensible-utils         0.0.20
> ii  ucf                    3.0043+nmu1
>
> apt-listchanges recommends no packages.
>
> Versions of packages apt-listchanges suggests:
> ii  chromium [www-browser]              117.0.5938.149-1
> ii  firefox [www-browser]               118.0-1+b1
> ii  google-chrome-stable [www-browser]  117.0.5938.149-1
> ii  links [www-browser]                 2.29-1+b1
> ii  lynx [www-browser]                  2.9.0dev.12-1
> ii  postfix [mail-transport-agent]      3.8.2-1
> ii  python3-gi                          3.46.0-1
> ii  w3m [www-browser]                   0.5.3+git20230121-2
> ii  xterm [x-terminal-emulator]         385-1
>
> -- debconf information:
> * apt-listchanges/headers: false
>    apt-listchanges/email-format: text
> * apt-listchanges/frontend: pager
> * apt-listchanges/email-address:
> * apt-listchanges/reverse: false
> * apt-listchanges/confirm: false
> * apt-listchanges/which: both
> * apt-listchanges/save-seen: true
> * apt-listchanges/no-network: false
>

[toc] | [prev] | [next] | [standalone]

#1171270

FromRuss Allbery <rra@debian.org>
Date2023-10-09 18:20 +0200
Message-ID<Hn197-eGaP-1@gated-at.bofh.it>
In reply to#1171268
Jonathan Kamens <jik@kamens.us> writes:

> TLDR this will be fixed in 4.1 and it's a significant enough fix that I
> should probably release 4.1 to experimental

Great, thank you!  Let me know when that's ready to go.

> So, this was one of the edge cases mentioned in the design documentation
> for the revamped changelog filtering logic: when the persistent database
> is not being used in a particular invocation of the program or there is
> no data for a particular package in the database (this is what you ran
> into), and the changelog data for a package is being fetched over the
> network because it is not present in the package, we can't use
> historical changelog data to determine which entries to display and
> which to filter out.

I probably should know this, but I guess I don't: why did you have to
fetch changelog data for those packages from the network?  Both of them
ship truncated changelogs, but the changelogs are truncated well, well
before any entries that would be of any interest to apt-listchanges on my
system.  I probably don't understand the design model here, but I would
have expected changelogs to only be fetched over the network if
apt-listchanges exhausted the changelog shipped with the package and still
couldn't find the beginning of the entries it wanted to display.

-- 
Russ Allbery (rra@debian.org)              <https://www.eyrie.org/~eagle/>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web