Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.maint.dpkg > #11952 > unrolled thread
| Started by | Helmut Grohne <helmut@subdivi.de> |
|---|---|
| First post | 2023-04-03 14:10 +0200 |
| Last post | 2023-04-24 11:40 +0200 |
| Articles | 20 on this page of 139 — 25 participants |
Back to article view | Back to linux.debian.maint.dpkg
DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-03 14:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Guillem Jover <guillem@debian.org> - 2023-04-08 04:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-22 13:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-22 13:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Guillem Jover <guillem@debian.org> - 2023-06-21 13:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-06-21 16:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Guillem Jover <guillem@debian.org> - 2023-07-10 11:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Raphael Hertzog <hertzog@debian.org> - 2023-04-21 14:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <luca.boccassi@gmail.com> - 2023-04-21 16:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon McVittie <smcv@debian.org> - 2023-04-22 12:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <luca.boccassi@gmail.com> - 2023-04-22 14:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sam Hartman <hartmans@debian.org> - 2023-04-26 15:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-27 00:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-27 15:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-22 13:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <luca.boccassi@gmail.com> - 2023-04-22 14:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-25 21:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-04-25 22:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Raphael Hertzog <hertzog@debian.org> - 2023-04-26 15:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-04-26 11:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-04-26 11:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sven Joachim <svenjoac@gmx.de> - 2023-04-26 18:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Raphael Hertzog <hertzog@debian.org> - 2023-05-02 12:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-02 15:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Raphael Hertzog <hertzog@debian.org> - 2023-05-03 10:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-03 12:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-05-03 20:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-04 19:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Timo Röhling <roehling@debian.org> - 2023-05-05 11:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-05-05 14:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Andreas Metzler <ametzler@bebt.de> - 2023-05-05 18:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-06 00:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-05-06 07:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-06 14:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-05-06 17:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-06 18:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-06 21:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-06 21:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-06 21:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-06 23:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sean Whitton <spwhitton@spwhitton.name> - 2023-05-07 01:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-07 13:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sean Whitton <spwhitton@spwhitton.name> - 2023-05-08 20:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-09 03:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-09 06:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-09 13:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-07 08:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Ansgar <ansgar@43-1.org> - 2023-05-07 09:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Ansgar <ansgar@43-1.org> - 2023-05-07 11:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-07 13:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-05-08 05:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-08 14:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-05-08 14:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sean Whitton <spwhitton@spwhitton.name> - 2023-05-10 17:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Ansgar <ansgar@43-1.org> - 2023-05-10 21:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Russ Allbery <rra@debian.org> - 2023-05-10 23:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Ansgar <ansgar@43-1.org> - 2023-05-10 23:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Russ Allbery <rra@debian.org> - 2023-05-10 23:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Ansgar <ansgar@43-1.org> - 2023-05-11 00:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Emilio Pozuelo Monfort <pochu@debian.org> - 2023-05-11 12:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon Richter <sjr@debian.org> - 2023-05-11 14:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-11 20:00 +0200
Re: dpkg currently warning about merged-usr systems (revisited) Simon Richter <sjr@debian.org> - 2023-05-12 14:20 +0200
Re: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 17:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon Richter <sjr@debian.org> - 2023-05-12 14:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 15:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Matthew Vernon <matthew@debian.org> - 2023-05-25 14:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Helmut Grohne <helmut@subdivi.de> - 2023-05-21 16:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Ansgar <ansgar@43-1.org> - 2023-05-21 17:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-06-07 06:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-06-13 21:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Matthew Garrett <mjg59@srcf.ucam.org> - 2023-06-13 22:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-06-13 22:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Ansgar <ansgar@43-1.org> - 2023-05-26 08:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Matthew Vernon <matthew@debian.org> - 2023-05-26 09:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Ansgar <ansgar@43-1.org> - 2023-05-26 10:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-26 10:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Matthew Vernon <matthew@debian.org> - 2023-05-26 11:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-07 14:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-07 16:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-07 23:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-08 03:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-08 11:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-08 23:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-05-09 03:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sam Hartman <hartmans@debian.org> - 2023-05-08 21:10 +0200
booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Helmut Grohne <helmut@subdivi.de> - 2023-05-17 11:40 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) "G. Branden Robinson" <g.branden.robinson@gmail.com> - 2023-05-17 11:50 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Marco d'Itri <md@Linux.IT> - 2023-05-17 12:20 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Luca Boccassi <bluca@debian.org> - 2023-05-17 12:30 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Sam Hartman <hartmans@debian.org> - 2023-05-17 19:20 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Simon Richter <sjr@debian.org> - 2023-05-18 08:40 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Luca Boccassi <bluca@debian.org> - 2023-05-18 11:30 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Simon Richter <sjr@debian.org> - 2023-05-19 02:40 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Luca Boccassi <bluca@debian.org> - 2023-05-19 13:10 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Sam Hartman <hartmans@debian.org> - 2023-05-17 19:30 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Raphael Hertzog <hertzog@debian.org> - 2023-06-08 10:50 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Luca Boccassi <bluca@debian.org> - 2023-06-08 12:10 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Marco d'Itri <md@Linux.IT> - 2023-06-09 09:50 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Raphael Hertzog <hertzog@debian.org> - 2023-06-09 12:00 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Luca Boccassi <bluca@debian.org> - 2023-06-09 12:40 +0200
Re: booststrapping /usr-merged systems Bjørn Mork <bjorn@mork.no> - 2023-06-09 13:10 +0200
Re: booststrapping /usr-merged systems Marco d'Itri <md@Linux.IT> - 2023-06-09 13:40 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Johannes Schauer Marin Rodrigues <josch@debian.org> - 2023-06-09 17:40 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Helmut Grohne <helmut@subdivi.de> - 2023-06-09 15:30 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Johannes Schauer Marin Rodrigues <josch@debian.org> - 2023-06-09 17:50 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Helmut Grohne <helmut@subdivi.de> - 2023-06-09 18:30 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) HW42 <hw42@ipsumj.de> - 2023-06-09 22:30 +0200
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Helmut Grohne <helmut@subdivi.de> - 2023-06-10 07:40 +0200
Re: booststrapping /usr-merged systems Sven Joachim <svenjoac@gmx.de> - 2023-06-10 08:40 +0200
Re: booststrapping /usr-merged systems Sven Joachim <svenjoac@gmx.de> - 2023-06-10 09:00 +0200
Re: booststrapping /usr-merged systems Helmut Grohne <helmut@subdivi.de> - 2023-06-10 10:50 +0200
Re: booststrapping /usr-merged systems Sven Joachim <svenjoac@gmx.de> - 2023-06-10 19:30 +0200
Re: booststrapping /usr-merged systems Luca Boccassi <bluca@debian.org> - 2023-06-10 21:00 +0200
Re: booststrapping /usr-merged systems Timo Röhling <roehling@debian.org> - 2023-06-11 23:20 +0200
Re: booststrapping /usr-merged systems Luca Boccassi <bluca@debian.org> - 2023-06-27 21:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg David Kalnischkies <david@kalnischkies.de> - 2023-05-03 15:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2023-04-26 16:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-27 00:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-04-27 08:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-28 10:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-04-28 11:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Timo Röhling <roehling@debian.org> - 2023-04-28 12:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Luca Boccassi <bluca@debian.org> - 2023-04-29 02:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Sam Hartman <hartmans@debian.org> - 2023-05-02 17:40 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-04-28 14:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-28 15:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Marvin Renich <mrvn@renich.org> - 2023-04-29 20:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-29 22:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-28 22:20 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-02 15:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-02 15:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-05-02 16:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-22 13:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Raphael Hertzog <hertzog@debian.org> - 2023-04-21 15:10 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon Richter <sjr@debian.org> - 2023-04-21 19:30 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Helmut Grohne <helmut@subdivi.de> - 2023-04-22 13:00 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Simon McVittie <smcv@debian.org> - 2023-04-22 15:50 +0200
Re: DEP 17: Improve support for directory aliasing in dpkg Raphael Hertzog <hertzog@debian.org> - 2023-04-24 11:40 +0200
Page 5 of 7 — ← Prev page 1 2 3 4 [5] 6 7 Next page →
| From | Helmut Grohne <helmut@subdivi.de> |
|---|---|
| Date | 2023-05-07 23:20 +0200 |
| Message-ID | <GsTXr-7jIY-5@gated-at.bofh.it> |
| In reply to | #12056 |
Hi Luca, On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote: > The local/external aspect is already covered in Ansgar's reply and subthread. I hope that we can at least agree that we don't have consensus on this view. And the more I think about it, the more it becomes clear to me that this non-consensus is part of the larger disagreement we have about this whole transition. Do you see any way towards getting to common ground here? > Sure, but adding changes that are (seemingly) unnecessary for a large > percentage of affected packages also brings uncertainty. Every > software has bugs, thus it follows that injecting more software in the > way of a package being installed will likely also inject bugs. Which > doesn't mean we shouldn't consider it, however, it should be weighted > appropriately. Let me put this into perspective. In this scenario, we will have a few packages with versioned Pre-Depends on usr-is-merged. The seemingly unnecessary change here is adding more Pre-Depends of the same kind to many more packages. It seems very likely to me that one of the few Pre-Depends will cause usr-is-merged to be upgraded early and thus those possibly unnecessary Pre-Dependencies will be harmless. Do you actually have some scenario in mind that would warrant judging this as risky beyond suspicion? (Which is not to say that there is no risk as the whole affair bears quite some risk.) > Packages that need special handling will need special handling for > backporting too. This is nothing new, there was never a project-wide > guarantee that a package uploaded to testing can apply 1:1 to > backports, it is common enough to require changes/reverts/adjustments, > and if it's fine to require that in other cases, it's fine for this > case too. It seems that you missed my argument and it likely wasn't spelled out explicitly enough, so let me retry. Yes, you may need to adapt packages that are being backported. We don't disagree about that (and hope people get it right, which they won't, but so be it). The really bad thing here is that a backports upload may require changes to the package in unstable! Say we packaged foo version 1 in stable and it puts everything in /bin. Then we update foo to version 2 in unstable and foo gains a new /bin/bar. Due to the debhelper addon, this is actually shipped as /usr/bin/bar. Great. Then we backport foo version 2 to stable. Given that debhelper no longer moves, it'll be /bin/bar. Then we notice that foo is not laid out nicely and we split a bar package from it in version 3 and move /usr/bin/bar into bar. Now a user may install stable, install foo version 1, install the foo version 2 backport and then update to nextstable. In that stable upgrade, bar version 3 may be unpacked before foo version 3 and as a result /usr/bin/bar goes missing when the backported foo version 2 gets upgraded to the regular foo version 3 as this deletes /bin/bar. So when we backport a package, the unstable package may need to be modified to avoid such unpack file loss scenarios. In a simple case, we may be able to just add Conflicts, but the takeaway is that backporting a package may now break upgrades to nextstable in a way that requires fixes in nextstable to accommodate for such upgrades. > If the majority of packages are simply converted, with no manual > handling and no diversion, then it should be simple to handle: the > debhelper in stable will not perform the conversion by definition as > the logic won't be present, and any dh upload to backports will have > such logic disabled, so that other packages that get uploaded to > backports and built with either the stable or the backports debhelper > won't have any change performed on them. As much a I'd like to trust you on things actually being simple, we've seen over and over again that the simple approaches have non-trivial flaws. If you were to highlight resulting problems (and propose solutions), that would be more convincing to me than continuously labeling it simple. > Or to put it in another way: I think our defaults should prioritize > the Debian native use case. Given we ship our loader in /usr/lib/ld* > now, it makes sense to me that the default in GCC is to point to > /usr/lib/ld*. Callers can override that as needed for > third-party/external/foreign use cases. I guess you'll be having a hard time convincing the toolchain maintainers of this change, but my other point was that this is unnecessary when we can use patchelf after the fact. > > How about the long-term vision of this? Elsewhere you indicated that > > you'd like the aliasing symlinks to not be shipped by any data.tar. Does > > that imply that we'd keep patching the interpreter and using /usr/bin/sh > > forever in the essential set? If adding the links to base-files, it > > would be of temporary nature only. > > > > If adding the symlinks to base-files, how about /lib64? Would we ship it > > for all architectures or just for those that need it (e.g. amd64, > > loong64, mips64el, ppc64, ppc64el)? > > https://wiki.debian.org/ArchitectureSpecificsMemo has a list of dynamic > > loaders we also need /libx32 for x32 at least. If making this > > architecture-dependent, would base-files become Multi-Arch: same? > > ... > > I think we should leave the long term vision for another day, and > focus on your requirements for the essential set unpacking right now. Knowing the target state of a transition seems fairly fundamental to implementing it and base-files is part of the essential set. To me, it is a significant difference whether we temporarily or permanently modify the ELF interpreter in the essential set. For these reasons, I do think the answers to these questions do matter at this time. As long as we do not have answers here, we must not move ld.so nor /bin/sh regardless of whether we patch dpkg or not. Helmut
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-08 03:30 +0200 |
| Message-ID | <GsXy1-7lT5-5@gated-at.bofh.it> |
| In reply to | #12058 |
On Sun, 7 May 2023 at 22:10, Helmut Grohne <helmut@subdivi.de> wrote: > > Hi Luca, > > On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote: > > The local/external aspect is already covered in Ansgar's reply and subthread. > > I hope that we can at least agree that we don't have consensus on this > view. And the more I think about it, the more it becomes clear to me > that this non-consensus is part of the larger disagreement we have about > this whole transition. Do you see any way towards getting to common > ground here? I can see we don't agree on this matter, of course, that is clear. And I hope we can find common ground. But let me provocatively ask this first: is the same rule going to be enforced for all other changes that happen in the project that might affect external packages? If anybody points out past changes, recent or less recent, that caused issues for third party packages, will the TC ask for those changes to be reverted or otherwise modified accordingly? Will a change to Policy be proposed that spells out that third party packages cannot ever be broken, no matter what they do, and must always work? > > Sure, but adding changes that are (seemingly) unnecessary for a large > > percentage of affected packages also brings uncertainty. Every > > software has bugs, thus it follows that injecting more software in the > > way of a package being installed will likely also inject bugs. Which > > doesn't mean we shouldn't consider it, however, it should be weighted > > appropriately. > > Let me put this into perspective. In this scenario, we will have a few > packages with versioned Pre-Depends on usr-is-merged. The seemingly > unnecessary change here is adding more Pre-Depends of the same kind to > many more packages. It seems very likely to me that one of the few > Pre-Depends will cause usr-is-merged to be upgraded early and thus those > possibly unnecessary Pre-Dependencies will be harmless. Do you actually > have some scenario in mind that would warrant judging this as risky > beyond suspicion? (Which is not to say that there is no risk as the > whole affair bears quite some risk.) The more pre-depends, the more constraints we put on apt. I do not have a specific scenario in mind as we don't even have a full set of changes to look at, but it seems clear to me it will have _some_ effect, no? > > Packages that need special handling will need special handling for > > backporting too. This is nothing new, there was never a project-wide > > guarantee that a package uploaded to testing can apply 1:1 to > > backports, it is common enough to require changes/reverts/adjustments, > > and if it's fine to require that in other cases, it's fine for this > > case too. > > It seems that you missed my argument and it likely wasn't spelled out > explicitly enough, so let me retry. Yes, you may need to adapt packages > that are being backported. We don't disagree about that (and hope people > get it right, which they won't, but so be it). The really bad thing here > is that a backports upload may require changes to the package in > unstable! > > Say we packaged foo version 1 in stable and it puts everything in /bin. > Then we update foo to version 2 in unstable and foo gains a new > /bin/bar. Due to the debhelper addon, this is actually shipped as > /usr/bin/bar. Great. Then we backport foo version 2 to stable. Given > that debhelper no longer moves, it'll be /bin/bar. Then we notice that > foo is not laid out nicely and we split a bar package from it in version > 3 and move /usr/bin/bar into bar. Now a user may install stable, install > foo version 1, install the foo version 2 backport and then update to > nextstable. In that stable upgrade, bar version 3 may be unpacked before > foo version 3 and as a result /usr/bin/bar goes missing when the > backported foo version 2 gets upgraded to the regular foo version 3 as > this deletes /bin/bar. > > So when we backport a package, the unstable package may need to be > modified to avoid such unpack file loss scenarios. In a simple case, we > may be able to just add Conflicts, but the takeaway is that backporting > a package may now break upgrades to nextstable in a way that requires > fixes in nextstable to accommodate for such upgrades. Sure that's a legitimate concern, however, wouldn't it fall into the "needs special handling" bucket? It is a case where the file is moving both in location and package, so it is covered by the blank statement "either don't do that or implement the required workaround via diversion/conflict/etc". What am I missing? > > If the majority of packages are simply converted, with no manual > > handling and no diversion, then it should be simple to handle: the > > debhelper in stable will not perform the conversion by definition as > > the logic won't be present, and any dh upload to backports will have > > such logic disabled, so that other packages that get uploaded to > > backports and built with either the stable or the backports debhelper > > won't have any change performed on them. > > As much a I'd like to trust you on things actually being simple, we've > seen over and over again that the simple approaches have non-trivial > flaws. If you were to highlight resulting problems (and propose > solutions), that would be more convincing to me than continuously > labeling it simple. > > > Or to put it in another way: I think our defaults should prioritize > > the Debian native use case. Given we ship our loader in /usr/lib/ld* > > now, it makes sense to me that the default in GCC is to point to > > /usr/lib/ld*. Callers can override that as needed for > > third-party/external/foreign use cases. > > I guess you'll be having a hard time convincing the toolchain > maintainers of this change, but my other point was that this is > unnecessary when we can use patchelf after the fact. Sure that is possible, you can manually patch the essential set by hand, and that would solve your requirement. And that's certainly a viable fallback avenue to keep around. But the more I think about it, the more I am convinced that the default option working best for Debian is the one that matches the project's choice of a filesystem layout. After all, this is configurable in the toolchain for a reason. And the vast majority of the rest of the world has long since finished this transition, so I struggle to think where software built with this default wouldn't work. Bullseye will be oldoldstable at that point, and even that was default merged for new installations, and really old ones (oldoldoldoldstable at that point? I lost count) will be long EOL. I suppose they could still be around unmaintained, but who uses a toolchain from 8 years in the future to build software for an EOL distribution 8 years in the past? Normally it's the other way around, as even glibc adds new symbols and is not forward compatible. > > > How about the long-term vision of this? Elsewhere you indicated that > > > you'd like the aliasing symlinks to not be shipped by any data.tar. Does > > > that imply that we'd keep patching the interpreter and using /usr/bin/sh > > > forever in the essential set? If adding the links to base-files, it > > > would be of temporary nature only. > > > > > > If adding the symlinks to base-files, how about /lib64? Would we ship it > > > for all architectures or just for those that need it (e.g. amd64, > > > loong64, mips64el, ppc64, ppc64el)? > > > https://wiki.debian.org/ArchitectureSpecificsMemo has a list of dynamic > > > loaders we also need /libx32 for x32 at least. If making this > > > architecture-dependent, would base-files become Multi-Arch: same? > > > > ... > > > > I think we should leave the long term vision for another day, and > > focus on your requirements for the essential set unpacking right now. > > Knowing the target state of a transition seems fairly fundamental to > implementing it and base-files is part of the essential set. To me, it > is a significant difference whether we temporarily or permanently modify > the ELF interpreter in the essential set. For these reasons, I do think > the answers to these questions do matter at this time. As long as we do > not have answers here, we must not move ld.so nor /bin/sh regardless of > whether we patch dpkg or not. On the ELF interpreter, as long as we can reasonably ensure it works, I do believe we should switch it, regardless of what we do with the symlinks, how we ship/add/build/package/create/manage them, as a desired final state. Again, we should make the default in Debian work for Debian. And given the default for Debian from Bookworm onward is that the loader is in /usr/lib/, it seems perfectly reasonable to me that it software built for Debian and shipped in Debian should look there for it. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Helmut Grohne <helmut@subdivi.de> |
|---|---|
| Date | 2023-05-08 11:00 +0200 |
| Message-ID | <Gt4SR-7qzw-5@gated-at.bofh.it> |
| In reply to | #12059 |
On Mon, May 08, 2023 at 02:07:08AM +0100, Luca Boccassi wrote: > I can see we don't agree on this matter, of course, that is clear. And > I hope we can find common ground. But let me provocatively ask this > first: is the same rule going to be enforced for all other changes > that happen in the project that might affect external packages? If > anybody points out past changes, recent or less recent, that caused > issues for third party packages, will the TC ask for those changes to > be reverted or otherwise modified accordingly? Will a change to Policy > be proposed that spells out that third party packages cannot ever be > broken, no matter what they do, and must always work? I'm not sure about the TC's role in this. For the record, I am doing all of the analysis (and design work) in this thread without a TC hat. I also cannot comment on what the TC is going to rule this matter. Can we leave that aside or formally file it there if you see a need? I agree that what we support is vague at best and we can readily see from earlier conflicts that this is a recurring matter. We still disagree over how much maintainers should support sysvinit. I've also quite recently failed at properly preparing a transition (non-essential adduser) and while we could write about it in release-notes, what is going to happen is that we'll revert it for bookworm and then I can retry properly. You may also have noticed that my analysis of possible problems in this thread very much reasons about packages shipped in Debian releases. I would actually like to call external packages and local diversions unsupported, but I was rightfully criticised that this is falling short. So no, I cannot tell you where the boundary of our support is. I initially assumed it to be closer to where you paint it and am now trying to adapt to meet the expectations of others. For instance, I've also reached out to DSA and inquired on their use. While I haven't found local diversions or local statoverrides in dsa-puppet.git, it seems that a number of external packages ship files in /sbin or /lib (including udev rules and systemd units). > The more pre-depends, the more constraints we put on apt. I do not > have a specific scenario in mind as we don't even have a full set of > changes to look at, but it seems clear to me it will have _some_ > effect, no? We've been there with multiarch-support and my experience with that suggests that the primary effect is increasing the size of Packages files. Though given that you are obviously worried here, I suppose more research is warranted. > Sure that's a legitimate concern, however, wouldn't it fall into the > "needs special handling" bucket? It is a case where the file is moving > both in location and package, so it is covered by the blank statement > "either don't do that or implement the required workaround via > diversion/conflict/etc". What am I missing? You are missing the distribution of responsibility. Quite commonly, backports are performed by someone else than the package maintainer. Yet, an uncoordinated backport can now render the package in unstable rc-buggy. > But the more I think about it, the more I am convinced that the > default option working best for Debian is the one that matches the > project's choice of a filesystem layout. After all, this is > configurable in the toolchain for a reason. > And the vast majority of the rest of the world has long since finished > this transition, so I struggle to think where software built with this > default wouldn't work. Bullseye will be oldoldstable at that point, > and even that was default merged for new installations, and really old > ones (oldoldoldoldstable at that point? I lost count) will be long > EOL. I suppose they could still be around unmaintained, but who uses a > toolchain from 8 years in the future to build software for an EOL > distribution 8 years in the past? Normally it's the other way around, > as even glibc adds new symbols and is not forward compatible. This seems somewhat convincing to me. Would you reach out to toolchain maintainers to discuss this as an early change after the release of bookworm? > On the ELF interpreter, as long as we can reasonably ensure it works, > I do believe we should switch it, regardless of what we do with the > symlinks, how we ship/add/build/package/create/manage them, as a > desired final state. Again, we should make the default in Debian work > for Debian. And given the default for Debian from Bookworm onward is > that the loader is in /usr/lib/, it seems perfectly reasonable to me > that it software built for Debian and shipped in Debian should look > there for it. I suppose that we've been confusing the different approaches here. The question of what links base-files should contain mostly arises if you start from the assumption that we do not modify the ELF interpreter location. Once changing its (and /bin/sh's) location, the question of how to install those symlinks can indeed be done in base-files.postinst or at some other place where dpkg doesn't have to know much about it indeed. Would you agree to examine the approach where we don't modify the ELF interpreter location in parallel as a backup plan? Helmut
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-08 23:40 +0200 |
| Message-ID | <GtgAF-7xNV-9@gated-at.bofh.it> |
| In reply to | #12061 |
Will get to the rest later tonight, two quick points: On Mon, 8 May 2023 at 09:58, Helmut Grohne <helmut@subdivi.de> wrote: > > But the more I think about it, the more I am convinced that the > > default option working best for Debian is the one that matches the > > project's choice of a filesystem layout. After all, this is > > configurable in the toolchain for a reason. > > And the vast majority of the rest of the world has long since finished > > this transition, so I struggle to think where software built with this > > default wouldn't work. Bullseye will be oldoldstable at that point, > > and even that was default merged for new installations, and really old > > ones (oldoldoldoldstable at that point? I lost count) will be long > > EOL. I suppose they could still be around unmaintained, but who uses a > > toolchain from 8 years in the future to build software for an EOL > > distribution 8 years in the past? Normally it's the other way around, > > as even glibc adds new symbols and is not forward compatible. > > This seems somewhat convincing to me. Would you reach out to toolchain > maintainers to discuss this as an early change after the release of > bookworm? Have done so now via the gcc mailing list. > > On the ELF interpreter, as long as we can reasonably ensure it works, > > I do believe we should switch it, regardless of what we do with the > > symlinks, how we ship/add/build/package/create/manage them, as a > > desired final state. Again, we should make the default in Debian work > > for Debian. And given the default for Debian from Bookworm onward is > > that the loader is in /usr/lib/, it seems perfectly reasonable to me > > that it software built for Debian and shipped in Debian should look > > there for it. > > I suppose that we've been confusing the different approaches here. The > question of what links base-files should contain mostly arises if you > start from the assumption that we do not modify the ELF interpreter > location. Once changing its (and /bin/sh's) location, the question of > how to install those symlinks can indeed be done in base-files.postinst > or at some other place where dpkg doesn't have to know much about it > indeed. Would you agree to examine the approach where we don't modify > the ELF interpreter location in parallel as a backup plan? Yeah we definitely should do that. I think we should separate a bit long-term vs short-term on that front, as it will help reach a conclusion more quickly. I think that aspect is easy to revise, and shouldn't lock us in a particular position one way or the other. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-09 03:10 +0200 |
| Message-ID | <GtjRT-7zFA-3@gated-at.bofh.it> |
| In reply to | #12061 |
On Mon, 8 May 2023 at 09:58, Helmut Grohne <helmut@subdivi.de> wrote: > > On Mon, May 08, 2023 at 02:07:08AM +0100, Luca Boccassi wrote: > > I can see we don't agree on this matter, of course, that is clear. And > > I hope we can find common ground. But let me provocatively ask this > > first: is the same rule going to be enforced for all other changes > > that happen in the project that might affect external packages? If > > anybody points out past changes, recent or less recent, that caused > > issues for third party packages, will the TC ask for those changes to > > be reverted or otherwise modified accordingly? Will a change to Policy > > be proposed that spells out that third party packages cannot ever be > > broken, no matter what they do, and must always work? > > I'm not sure about the TC's role in this. For the record, I am doing all > of the analysis (and design work) in this thread without a TC hat. I > also cannot comment on what the TC is going to rule this matter. Can we > leave that aside or formally file it there if you see a need? Sure, it was just to keep it impersonal. Substitute TC with "interested party" or so. > I agree that what we support is vague at best and we can readily see > from earlier conflicts that this is a recurring matter. We still > disagree over how much maintainers should support sysvinit. I've also > quite recently failed at properly preparing a transition (non-essential > adduser) and while we could write about it in release-notes, what is > going to happen is that we'll revert it for bookworm and then I can > retry properly. > > You may also have noticed that my analysis of possible problems in this > thread very much reasons about packages shipped in Debian releases. I > would actually like to call external packages and local diversions > unsupported, but I was rightfully criticised that this is falling short. > > So no, I cannot tell you where the boundary of our support is. I > initially assumed it to be closer to where you paint it and am now > trying to adapt to meet the expectations of others. > > For instance, I've also reached out to DSA and inquired on their use. > While I haven't found local diversions or local statoverrides in > dsa-puppet.git, it seems that a number of external packages ship files > in /sbin or /lib (including udev rules and systemd units). I think the question of what do we want to support is an important one, and I care greatly that for this particular endeavour we do not impose a higher burden on ourselves that would otherwise be expected in different situations. If expectations are shifting considerably, we should codify it, so that everyone is held up to the same standards. > > The more pre-depends, the more constraints we put on apt. I do not > > have a specific scenario in mind as we don't even have a full set of > > changes to look at, but it seems clear to me it will have _some_ > > effect, no? > > We've been there with multiarch-support and my experience with that > suggests that the primary effect is increasing the size of Packages > files. Though given that you are obviously worried here, I suppose more > research is warranted. If that's the only thing to worry about, then I'm not worried at all! > > Sure that's a legitimate concern, however, wouldn't it fall into the > > "needs special handling" bucket? It is a case where the file is moving > > both in location and package, so it is covered by the blank statement > > "either don't do that or implement the required workaround via > > diversion/conflict/etc". What am I missing? > > You are missing the distribution of responsibility. Quite commonly, > backports are performed by someone else than the package maintainer. > Yet, an uncoordinated backport can now render the package in unstable > rc-buggy. Well, sure, but once again any backport changes can break in interesting and novel ways. The simplest of them all is getting the version wrong so that updates do not work... and yet we have very little if anything in the way to stop that, no? You can just ignore Lintian screaming at you and upload... To me that's just another case to be solved with good documentation and communication. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Sam Hartman <hartmans@debian.org> |
|---|---|
| Date | 2023-05-08 21:10 +0200 |
| Message-ID | <Gtepb-7wzk-13@gated-at.bofh.it> |
| In reply to | #12058 |
[Multipart message — attachments visible in raw view] — view raw
>>>>> "Helmut" == Helmut Grohne <helmut@subdivi.de> writes:
Helmut> Hi Luca,
Helmut> On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote:
>> The local/external aspect is already covered in Ansgar's reply
>> and subthread.
Helmut> I hope that we can at least agree that we don't have
Helmut> consensus on this view. And the more I think about it, the
Helmut> more it becomes clear to me that this non-consensus is part
Helmut> of the larger disagreement we have about this whole
Helmut> transition. Do you see any way towards getting to common
Helmut> ground here?
As someone who has been following this, I support the work Helmut and
Simon Richter have been doing.
I have more confidence in that view than the one Luca is proposing.
I also support Shawn's interpretation that being conservative here is
good.
I think even with my support we have no consensus. However hopefully we
can get a few more people who have been reading the whole thread to
chime in and a consensus will appear.
[toc] | [prev] | [next] | [standalone]
| From | Helmut Grohne <helmut@subdivi.de> |
|---|---|
| Date | 2023-05-17 11:40 +0200 |
| Subject | booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwlNv-9v2H-1@gated-at.bofh.it> |
| In reply to | #12056 |
[Multipart message — attachments visible in raw view] — view raw
Hi,
This bootstrap aspect got me and I discussed this with a number of
people and did some research.
On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote:
> I don't think this is true? At least not in the broader sense: if you
> compile something on Debian, it will obviously get linked against
> libraries and dependencies as they are in Debian.
> Perhaps what you mean is that, given an entire separate sysroot-like
> tree, passing the appropriate compiler and linker flags and
> environment variables, you can use the local compiler we ship to build
> 'foreign' programs. That is true, but again it requires to set up the
> environment appropriately, including linker flags. And the caller
> needs to ensure the environment, including linker flags, is
> appropriate for the target environment (I guess 'host' environment, in
> GNU parlance). Therefore, I don't think it would be unreasonable to
> require that if the target environment is split-usr, then the caller
> also needs to specify an appropriate
> '-Wl,--dynamic-linker=/lib/ld-whatever' option.
Given the feedback, I am convinced that changing PT_INTERP is a stupid
idea regardless of whether it is technically feasible. There must be a
better way. Let's step back a bit.
The underlying problem here is performing the initial filesystem
bootstrap. The semantics of this are a bit vague as they are not spelled
out in policy, so we will have to derive them from implementations.
I think the major players are (in descending popularity):
* debootstrap
* mmdebstrap
* cdebootstrap
* multistrap
multistrap predates mmdebstrap and when there was no mmdebstrap, I used
it a lot. When attempting to test it, I totally couldn't convince it to
bootstrap from an unsigned or locally signed repository. The patch in
#908451 didn't cut it. I also note that it creates a /lib64 -> /lib
symbolic link which feels quite incompatible with merged-/usr. For
these reasons, I am dropping multistrap from the tools under
consideration and recommend removing it from the archive. If you happen
to use multistrap, now would be a good moment to tell me. Personally,
all of my use cases of multistrap have been converted to mmdebstrap and
that made a lot of things simpler.
cdebootstrap vaguely works though unsigned operation seems dysfunctional
as it runs apt-get update during cdebootstrap-helper-apt.postinst and
that fails. I happen to not have figured out why and treat this failure
as a success.
So the most popular implementations quite evidently are debootstrap and
mmdebstrap and both "just work". I note though that they work quite
differently:
* debootstrap (depending on flags including --variant) pre-merges its
chroot while mmdebstrap relies on packages doing it.
I think that the question whether a distribution is merged is a
property of the distribution and not the bootstrap tool, so I
strongly recommend following mmdebstrap's view on this. The
debootstrap way means that we have to include patches for every
derivative, which is a process that does not scale well.
* mmdebstrap operates in two phases. It first unpacks and configures a
rather minimal set of packages and then proceeds to adding packages
passed to --include in a second phase once essential is fully
configured while debootstrap immediately unpacks everything.
I think the debootstrap approach is slightly worse here, because it
means that preinst scripts of non-essential packages cannot rely on
essential packages having been configured.
In any case, we have to deal with both behaviours.
After this little excursion into bootstrap technology, let's go back to
the /usr-merge and its effects.
I think at this point, we have quite universal consensus about the goal
of moving files to their canonical location (i.e. from / to /usr) as a
solution to the aliasing problems while we do not have consensus on
precisely how to do this (i.e. with changing dpkg or without). If you
believe that this is not consensus, please speak up.
So in a distant future our packages will not contain any files in /bin
or /lib. In particular, this affects /bin/sh and the dynamic loader,
both of which are required to run maintainer scripts, which are
currently required for creating the symbolic links. Boom.
Solutions have been proposed to this and I think they all fall into one
of the following four categories.
1. Don't move. We just keep those files that require a particular
location (such as /bin/sh or the dynamic loader) in their
non-canonical location. As such, maintainer scripts will be able to
run and perform the conversion to symbolic links afterwards.
2. Move and ship links. Since we unpack all essential data.tar before
running the first maintainer script, having one package contain the
compatibility symlinks is enough to fix the problem.
3. Move and avoid using non-canonical locations. This is the approach
where we write maintainer scripts as #!/usr/bin/sh and considered
changing PT_INTERP.
4. Change the bootstrap protocol. In essence, this has been attempted
in debootstrap by creating these symlinks prior to unpack, but no
consensus has evolved around this approach yet. The category is
wider though and generally requires changes to all bootstrapping
tools.
To see that it's just these four we can watch an imaginary bootstrap
process on an imaginary distribution. If any package contains the
symlinks, we're in category 2. Otherwise, we see whether these symlinks
exist prior to running the first maintainer script. If they do, we're in
category 4. And finally, we see whether any files are left in aliased
locations (according to the dpkg database). If there are, we're category
1 and otherwise category 3.
For instance, Luca proposed changing PT_INTERP in the toolchain, which
is category 3 here. I proposed a minor adaption where we have debhelper
change it post build using patchelf, which also amounts to category 3.
I've started with these to get started with classifying and think we can
disregard both.
For completeness sake, there is one more entry in category 3: We can run
the dynamic loader from its canonical location explicitly, so we'd
modify maintainer scripts to start with:
#!/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/bin/sh
This would only affect preinst scripts participating in bootstrap and
we'd not bother with changing PT_INTERP in the toolchain nor in
packages. Unfortunately, this completely breaks the DPKG_ROOT work and
with that, I see no viable entries in category 3 for moving forward.
Moving on to category 4 feels rather obvious, especially because work
has been done there in debootstrap. The approach in debootstrap however
is one that I see as a dead end, because it causes us to maintain this
code multiple times. It's the number of derivatives times the number of
bootstrap tools and that doesn't scale.
Category 4 is wider though and we also have other prior art at
https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular,
making architecture bootstrap become chrootless would likely be a
generic solution to this and other problems. However, any change needs
to propagate to a stable release in all bootstrapping tools. Therefore
we cannot reasonably finish the transition before forky. This makes
category 4 rather unattractive in a short term, but still worth pursuing
in a long term.
Having ruled out categories 3 and 4 maybe category 2 would be good? We
could just ship those symlinks in base-files and be done, right?
Unfortunately, we pass -k to tar in debootstrap, so when it extracts
base-files and tries to unpack the bin -> usr/bin symlink, it sees that
oh no, there already is a symlink at bin (as debootstrap placed it
there) and thus fails. So in order to make this work, we also have to
modify debootstrap (and thus are in a combination of category 3 and 4).
Conversely, if we unpack anything else that happens to ship
/bin/something before base-files, then mmdebstrap will fail to unpack
base-files. So we can only add this link to base-files after all other
(essential) packages have moved everything out of bin. That happens to
include /bin/sh. A possible solution to this is doing all of this in the
same dinstall. Then mmdebstrap works before and works afterwards.
Unfortunately, we're not yet done here. When (for instance) dash is the
last package to move /bin/sh and such to /usr and you upgrade dash, dpkg
notices that no package owns /bin anymore. Thus it helpfully deletes
/bin (the symlink). You're not happy when this happens. We remember the
silver bullet: diversions! So dash.preinst could dpkg-divert --no-rename
--divert /bin.usrmerged -add /bin and dash.postinst could revert that.
Unfortunately, such a diversion affects every package but dash and we
want it exactly the other way round. So what we could do is pass
--package dash-usrmerged (which must not exist). Then it'll actually
keep /bin safe. Unfortunately, we don't know whether dash or bash will
be the last package owning /bin, so both of them need this diversion and
this is a conflict.
And as if that wasn't enough, we also run into issues around hard links.
As dpkg unpacks a canonicalized gzip, it notices that /bin/gunzip (which
is scheduled for deletion) has the same inode as the new
/usr/bin/uncompress (because gunzip and uncompress are hard linked). As
far as I can see, all we get here is a warning and both files survive
the unpack. It is not clear to me whether this happens by chance or
reliably.
dpkg: warning: old file '/bin/uncompress' is the same as several new files! (both '/usr/bin/gunzip' and '/usr/bin/uncompress')^
dpkg: warning: old file '/bin/gunzip' is the same as several new files! (both '/usr/bin/gunzip' and '/usr/bin/uncompress')
Given what I've seen, I'm fairly convinced that I haven't reached the
bottom of it and I'm ready to conclude that this approach is fragile - a
property that is most unwelcome when we deal with the essential set.
So what's left is category 1. I looked into what the minimum set of
files to be retained could be. To do that end, I moved everything and
then reverted as much as was needed to make bootstrapping work.
* /lib64/ld-linux-x86-64.so.2 (hopefully obvious)
* /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (link target)
* /bin/sh (hopefully obvious)
* /bin/dash (link target)
* /bin/bash (usrmerge runs ldd, which is a #!/bin/bash script)
* /bin/more (update-alternatives doesn't like its absence)
* /bin/cp (unless usrmerge stops hard coding its path)
So in principle, we may be able to pull this off if we keep bash,
dash, libc6, and util-linux in their original location. In this
scenario, we're also unable to remove usrmerge from the essential set.
The major benefit of category 1 it allows us to move to any other
category at a later time and it removes aliasing effects from all but a
small set of packages.
In case you want to reproduce my analysis, I've attached a
test-usrmergebootstrap.sh script. It attempts the following strategies:
* "earlylink": add links to base-files first without converting other
packages
* "movefull": move all files to their canonical locations and add links
to base-files
* "movemost": move most files except those mentioned earlier to their
canonical locations without adding links to base-files
These strategies are tested in the following use cases:
* "cdebootstrap": bootstrap using cdebootstrap
* "debootstrap": bootstrap using debootstrap
* "debootstrap-buildd": bootstrap using debootstrap --variant=buildd
--merged-usr
* "mmdebstrap": bootstrap using mmdebstrap
* "upgrade": perform a regular bootstrap and upgrade packages to the
modified ones
Broken combinations
* *strap-earlylink: unpack error during base-files
* debootstrap*-movefull: unpack error during base-files
* upgrade-movefull: /lib64 missing unless diverted for libc6
And now you probably expect me to write some kind of conclusion, but
really I think none of our options are attractive. In a short term I
strongly recommend not moving files around that are part of the
transitively essential set, because things can (and do) go wrong in so
many surprising ways. The other major takeaway is that a significant
chunk of the problems mentioned in this mail cannot be fixed by
modifying dpkg only.
I sincerely hope that everyone will now point out why this analysis is
incomplete and misses out on covering strategies. I'd love to be wrong
about the dark picture I've drawn here.
Helmut
[toc] | [prev] | [next] | [standalone]
| From | "G. Branden Robinson" <g.branden.robinson@gmail.com> |
|---|---|
| Date | 2023-05-17 11:50 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwlXb-9v6a-13@gated-at.bofh.it> |
| In reply to | #12091 |
[Multipart message — attachments visible in raw view] — view raw
At 2023-05-17T11:30:36+0200, Helmut Grohne wrote: > This bootstrap aspect got me and I discussed this with a number of > people and did some research. I'd like to nominate you for a Russ Allbery Award for the most useful post to the thread. Your attention to concrete, empirical details, arising necessarily from practical experimentation rather than pure cogitation, made the nature of the problems here clearly comprehensible to me. And if I was befogged, I'll wager other people were too. Regards, Branden
[toc] | [prev] | [next] | [standalone]
| From | Marco d'Itri <md@Linux.IT> |
|---|---|
| Date | 2023-05-17 12:20 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <Gwmqe-9vvC-3@gated-at.bofh.it> |
| In reply to | #12091 |
[Multipart message — attachments visible in raw view] — view raw
On May 17, Helmut Grohne <helmut@subdivi.de> wrote:
> Given the feedback, I am convinced that changing PT_INTERP is a stupid
> idea regardless of whether it is technically feasible. There must be a
> better way. Let's step back a bit.
Me too, I was never persuaded.
> 4. Change the bootstrap protocol. In essence, this has been attempted
> in debootstrap by creating these symlinks prior to unpack, but no
> consensus has evolved around this approach yet. The category is
> wider though and generally requires changes to all bootstrapping
> tools.
I think that this is being dismissed too easily, mostly because the
mmdebstrap maintainer has been fighting it due to a philosophical
preference.
> Moving on to category 4 feels rather obvious, especially because work
> has been done there in debootstrap. The approach in debootstrap however
> is one that I see as a dead end, because it causes us to maintain this
> code multiple times. It's the number of derivatives times the number of
> bootstrap tools and that doesn't scale.
But the code is trivial. It is currently more complex than it is needed
in debootstrap only because initially it needed to support the biarch
libc packages, but since nowadays they create the top level symlinks
themselves then it can be made as simple as:
for dir in bin sbin lib; do
ln -s usr/"$dir" "$TARGET/$dir"
mkdir -p "$TARGET/usr/$dir"
done
Indeed, usrmerge does not have any architecture-specific knowledge
anymore.
> https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular,
> making architecture bootstrap become chrootless would likely be a
> generic solution to this and other problems. However, any change needs
> to propagate to a stable release in all bootstrapping tools. Therefore
> we cannot reasonably finish the transition before forky. This makes
Why not?
> So what's left is category 1. I looked into what the minimum set of
> files to be retained could be. To do that end, I moved everything and
> then reverted as much as was needed to make bootstrapping work.
> * /lib64/ld-linux-x86-64.so.2 (hopefully obvious)
> * /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (link target)
> * /bin/sh (hopefully obvious)
> * /bin/dash (link target)
Interesting approach. If it is so much simple, why are you not persuaded
that it could be a good solution until the bootstrapping tools are
updated?
> * /bin/bash (usrmerge runs ldd, which is a #!/bin/bash script)
> * /bin/more (update-alternatives doesn't like its absence)
> * /bin/cp (unless usrmerge stops hard coding its path)
These can be easily fixed, maybe the ldd issue too.
> The other major takeaway is that a significant
> chunk of the problems mentioned in this mail cannot be fixed by
> modifying dpkg only.
Agreed.
--
ciao,
Marco
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-17 12:30 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwmzT-9vyL-1@gated-at.bofh.it> |
| In reply to | #12091 |
On Wed, 17 May 2023 at 10:31, Helmut Grohne <helmut@subdivi.de> wrote: > > Hi, > > This bootstrap aspect got me and I discussed this with a number of > people and did some research. > > On Sun, May 07, 2023 at 12:51:21PM +0100, Luca Boccassi wrote: > > I don't think this is true? At least not in the broader sense: if you > > compile something on Debian, it will obviously get linked against > > libraries and dependencies as they are in Debian. > > Perhaps what you mean is that, given an entire separate sysroot-like > > tree, passing the appropriate compiler and linker flags and > > environment variables, you can use the local compiler we ship to build > > 'foreign' programs. That is true, but again it requires to set up the > > environment appropriately, including linker flags. And the caller > > needs to ensure the environment, including linker flags, is > > appropriate for the target environment (I guess 'host' environment, in > > GNU parlance). Therefore, I don't think it would be unreasonable to > > require that if the target environment is split-usr, then the caller > > also needs to specify an appropriate > > '-Wl,--dynamic-linker=/lib/ld-whatever' option. > > Given the feedback, I am convinced that changing PT_INTERP is a stupid > idea regardless of whether it is technically feasible. There must be a > better way. Let's step back a bit. > > The underlying problem here is performing the initial filesystem > bootstrap. The semantics of this are a bit vague as they are not spelled > out in policy, so we will have to derive them from implementations. > > I think the major players are (in descending popularity): > * debootstrap > * mmdebstrap > * cdebootstrap > * multistrap > > multistrap predates mmdebstrap and when there was no mmdebstrap, I used > it a lot. When attempting to test it, I totally couldn't convince it to > bootstrap from an unsigned or locally signed repository. The patch in > #908451 didn't cut it. I also note that it creates a /lib64 -> /lib > symbolic link which feels quite incompatible with merged-/usr. For > these reasons, I am dropping multistrap from the tools under > consideration and recommend removing it from the archive. If you happen > to use multistrap, now would be a good moment to tell me. Personally, > all of my use cases of multistrap have been converted to mmdebstrap and > that made a lot of things simpler. > > cdebootstrap vaguely works though unsigned operation seems dysfunctional > as it runs apt-get update during cdebootstrap-helper-apt.postinst and > that fails. I happen to not have figured out why and treat this failure > as a success. > > So the most popular implementations quite evidently are debootstrap and > mmdebstrap and both "just work". I note though that they work quite > differently: > * debootstrap (depending on flags including --variant) pre-merges its > chroot while mmdebstrap relies on packages doing it. > > I think that the question whether a distribution is merged is a > property of the distribution and not the bootstrap tool, so I > strongly recommend following mmdebstrap's view on this. The > debootstrap way means that we have to include patches for every > derivative, which is a process that does not scale well. > > * mmdebstrap operates in two phases. It first unpacks and configures a > rather minimal set of packages and then proceeds to adding packages > passed to --include in a second phase once essential is fully > configured while debootstrap immediately unpacks everything. > > I think the debootstrap approach is slightly worse here, because it > means that preinst scripts of non-essential packages cannot rely on > essential packages having been configured. > > In any case, we have to deal with both behaviours. > > After this little excursion into bootstrap technology, let's go back to > the /usr-merge and its effects. > > I think at this point, we have quite universal consensus about the goal > of moving files to their canonical location (i.e. from / to /usr) as a > solution to the aliasing problems while we do not have consensus on > precisely how to do this (i.e. with changing dpkg or without). If you > believe that this is not consensus, please speak up. > > So in a distant future our packages will not contain any files in /bin > or /lib. In particular, this affects /bin/sh and the dynamic loader, > both of which are required to run maintainer scripts, which are > currently required for creating the symbolic links. Boom. > > Solutions have been proposed to this and I think they all fall into one > of the following four categories. > > 1. Don't move. We just keep those files that require a particular > location (such as /bin/sh or the dynamic loader) in their > non-canonical location. As such, maintainer scripts will be able to > run and perform the conversion to symbolic links afterwards. > > 2. Move and ship links. Since we unpack all essential data.tar before > running the first maintainer script, having one package contain the > compatibility symlinks is enough to fix the problem. > > 3. Move and avoid using non-canonical locations. This is the approach > where we write maintainer scripts as #!/usr/bin/sh and considered > changing PT_INTERP. > > 4. Change the bootstrap protocol. In essence, this has been attempted > in debootstrap by creating these symlinks prior to unpack, but no > consensus has evolved around this approach yet. The category is > wider though and generally requires changes to all bootstrapping > tools. > > To see that it's just these four we can watch an imaginary bootstrap > process on an imaginary distribution. If any package contains the > symlinks, we're in category 2. Otherwise, we see whether these symlinks > exist prior to running the first maintainer script. If they do, we're in > category 4. And finally, we see whether any files are left in aliased > locations (according to the dpkg database). If there are, we're category > 1 and otherwise category 3. > > For instance, Luca proposed changing PT_INTERP in the toolchain, which > is category 3 here. I proposed a minor adaption where we have debhelper > change it post build using patchelf, which also amounts to category 3. > I've started with these to get started with classifying and think we can > disregard both. > > For completeness sake, there is one more entry in category 3: We can run > the dynamic loader from its canonical location explicitly, so we'd > modify maintainer scripts to start with: > > #!/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/bin/sh > > This would only affect preinst scripts participating in bootstrap and > we'd not bother with changing PT_INTERP in the toolchain nor in > packages. Unfortunately, this completely breaks the DPKG_ROOT work and > with that, I see no viable entries in category 3 for moving forward. Apart from DPKG_ROOT, that is also complicated because of multiarch I suppose? Ie, installing a random $arch package on !$arch. > Moving on to category 4 feels rather obvious, especially because work > has been done there in debootstrap. The approach in debootstrap however > is one that I see as a dead end, because it causes us to maintain this > code multiple times. It's the number of derivatives times the number of > bootstrap tools and that doesn't scale. > > Category 4 is wider though and we also have other prior art at > https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular, > making architecture bootstrap become chrootless would likely be a > generic solution to this and other problems. However, any change needs > to propagate to a stable release in all bootstrapping tools. Therefore > we cannot reasonably finish the transition before forky. This makes > category 4 rather unattractive in a short term, but still worth pursuing > in a long term. I'm not sure this is really a show-stopper, and it would really need to delay till forkie. We are talking about only two packages, needing updates to bootstrap a future release. We have already updated debootstrap in stable and oldstable via proposed-updates for this purpose in the past, so as long as the changes are self-contained and do not affect building older releases, it sounds to me it would be doable to do the same here. https://tracker.debian.org/news/1349768/accepted-debootstrap-10114deb10u1-source-into-oldstable-proposed-updates-oldstable-new-oldstable-proposed-updates/ > Having ruled out categories 3 and 4 maybe category 2 would be good? We > could just ship those symlinks in base-files and be done, right? > Unfortunately, we pass -k to tar in debootstrap, so when it extracts > base-files and tries to unpack the bin -> usr/bin symlink, it sees that > oh no, there already is a symlink at bin (as debootstrap placed it > there) and thus fails. So in order to make this work, we also have to > modify debootstrap (and thus are in a combination of category 3 and 4). As far as gut feelings go, it doesn't feel like to me that modifying debootstrap to deal with this would be too difficult. > Conversely, if we unpack anything else that happens to ship > /bin/something before base-files, then mmdebstrap will fail to unpack > base-files. So we can only add this link to base-files after all other > (essential) packages have moved everything out of bin. That happens to > include /bin/sh. A possible solution to this is doing all of this in the > same dinstall. Then mmdebstrap works before and works afterwards. > > Unfortunately, we're not yet done here. When (for instance) dash is the > last package to move /bin/sh and such to /usr and you upgrade dash, dpkg > notices that no package owns /bin anymore. Thus it helpfully deletes > /bin (the symlink). You're not happy when this happens. We remember the > silver bullet: diversions! So dash.preinst could dpkg-divert --no-rename > --divert /bin.usrmerged -add /bin and dash.postinst could revert that. > Unfortunately, such a diversion affects every package but dash and we > want it exactly the other way round. So what we could do is pass > --package dash-usrmerged (which must not exist). Then it'll actually > keep /bin safe. Unfortunately, we don't know whether dash or bash will > be the last package owning /bin, so both of them need this diversion and > this is a conflict. > > And as if that wasn't enough, we also run into issues around hard links. > As dpkg unpacks a canonicalized gzip, it notices that /bin/gunzip (which > is scheduled for deletion) has the same inode as the new > /usr/bin/uncompress (because gunzip and uncompress are hard linked). As > far as I can see, all we get here is a warning and both files survive > the unpack. It is not clear to me whether this happens by chance or > reliably. > > dpkg: warning: old file '/bin/uncompress' is the same as several new files! (both '/usr/bin/gunzip' and '/usr/bin/uncompress')^ > dpkg: warning: old file '/bin/gunzip' is the same as several new files! (both '/usr/bin/gunzip' and '/usr/bin/uncompress') > > Given what I've seen, I'm fairly convinced that I haven't reached the > bottom of it and I'm ready to conclude that this approach is fragile - a > property that is most unwelcome when we deal with the essential set. > > So what's left is category 1. I looked into what the minimum set of > files to be retained could be. To do that end, I moved everything and > then reverted as much as was needed to make bootstrapping work. > * /lib64/ld-linux-x86-64.so.2 (hopefully obvious) > * /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (link target) > * /bin/sh (hopefully obvious) > * /bin/dash (link target) > * /bin/bash (usrmerge runs ldd, which is a #!/bin/bash script) > * /bin/more (update-alternatives doesn't like its absence) > * /bin/cp (unless usrmerge stops hard coding its path) > > So in principle, we may be able to pull this off if we keep bash, > dash, libc6, and util-linux in their original location. In this > scenario, we're also unable to remove usrmerge from the essential set. > The major benefit of category 1 it allows us to move to any other > category at a later time and it removes aliasing effects from all but a > small set of packages. Is it necessary to avoid canonicalizing bash and util-linux? My understanding was that the reason for that was to avoid dpkg from deleting the symlink - wouldn't a single package (eg: dash for /bin, libc6 for /lib* and something else for /sbin) be enough to achieve that? Or did I misread the issue? Overall, this sounds like a fine approach to me - simple, and gets us 99.999% of the way there. It means dash and libc6 cannot move /bin/dash or /lib/ld to other packages in the Trixie cycle, but somehow I suspect that won't be an issue in reality :-) We could start with this, and then update debootstrap and mmdebstrap to also deal with this, and canonicalize dash/libc only after we are confident the bootstrapping issue is really solved. That way we can canonicalize and unblock 99.999% of the distro immediately. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Sam Hartman <hartmans@debian.org> |
|---|---|
| Date | 2023-05-17 19:20 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwsYF-9zAb-7@gated-at.bofh.it> |
| In reply to | #12091 |
[Multipart message — attachments visible in raw view] — view raw
>>>>> "Helmut" == Helmut Grohne <helmut@subdivi.de> writes:
Helmut> I think at this point, we have quite universal consensus
Helmut> about the goal of moving files to their canonical location
Helmut> (i.e. from / to /usr) as a solution to the aliasing problems
Helmut> while we do not have consensus on precisely how to do this
Helmut> (i.e. with changing dpkg or without). If you believe that
Helmut> this is not consensus, please speak up.
I agree we have strong consensus that we want to move files to their
canonical locations.
I'm not entirely sure I'd agree that we have consensus that's our
solution to the aliasing problem.
there are other competing reasons why we might want to move files to
their canonical locations.
If for example we accomplish the move to canonical locations by changing
dpkg, we might well get some form of aliasing support in dpkg.
This mostly doesn't matter.
If we're going to move files to their canonical location, it doesn't
matter much why.
There is one potential area where it might.
If the reason for preferring one bootstrap protocol over another depends
on why we're moving files to their canonical locations, I think we'd
need to dig into that very carefully and examine that part of your
consensus call a bit more.
[toc] | [prev] | [next] | [standalone]
| From | Simon Richter <sjr@debian.org> |
|---|---|
| Date | 2023-05-18 08:40 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwFsR-9HNQ-3@gated-at.bofh.it> |
| In reply to | #12095 |
Hi,
On 5/18/23 02:15, Sam Hartman wrote:
> Helmut> I think at this point, we have quite universal consensus
> Helmut> about the goal of moving files to their canonical location
> Helmut> (i.e. from / to /usr) as a solution to the aliasing problems
> Helmut> while we do not have consensus on precisely how to do this
> Helmut> (i.e. with changing dpkg or without). If you believe that
> Helmut> this is not consensus, please speak up.
> I agree we have strong consensus that we want to move files to their
> canonical locations.
> I'm not entirely sure I'd agree that we have consensus that's our
> solution to the aliasing problem.
It's the other way around: moving the files as a solution to the
aliasing problem is the strongest argument in favour of moving the files
inside the packages.
Without it, leaving them in place makes no difference for usrmerged
systems, and allows derived distributions that don't need usrmerge to
continue using our packages.
> If for example we accomplish the move to canonical locations by changing
> dpkg, we might well get some form of aliasing support in dpkg.
IMO, that is still the preferred solution:
- it is actually safe, because dpkg knows what is going on and can
reject conflicting changes
- there is no guarantee that usrmerge will be permanent or the last
transition of this kind
- it also solves the bootstrap problem
Simon
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-18 11:30 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwHO1-9Jl3-1@gated-at.bofh.it> |
| In reply to | #12097 |
On Thu, 18 May 2023 at 07:39, Simon Richter <sjr@debian.org> wrote: > > Hi, > > On 5/18/23 02:15, Sam Hartman wrote: > > > Helmut> I think at this point, we have quite universal consensus > > Helmut> about the goal of moving files to their canonical location > > Helmut> (i.e. from / to /usr) as a solution to the aliasing problems > > Helmut> while we do not have consensus on precisely how to do this > > Helmut> (i.e. with changing dpkg or without). If you believe that > > Helmut> this is not consensus, please speak up. > > > I agree we have strong consensus that we want to move files to their > > canonical locations. > > > I'm not entirely sure I'd agree that we have consensus that's our > > solution to the aliasing problem. > > It's the other way around: moving the files as a solution to the > aliasing problem is the strongest argument in favour of moving the files > inside the packages. > > Without it, leaving them in place makes no difference for usrmerged > systems, and allows derived distributions that don't need usrmerge to > continue using our packages. Not quite. Having packages only ship files under /usr (and possibly /etc) is very much a goal in itself for a lot of us. > > If for example we accomplish the move to canonical locations by changing > > dpkg, we might well get some form of aliasing support in dpkg. > > IMO, that is still the preferred solution: > > - it is actually safe, because dpkg knows what is going on and can > reject conflicting changes > - there is no guarantee that usrmerge will be permanent or the last > transition of this kind It is permanent, there are several upstream projects that will drop support for legacy layouts very soon, and it will not be re-added back. This will become more and more common, as simply most will stop caring and paying any attention to this detail. Debian is pretty much the last relevant holdout here, and that's going to end in a couple of weeks. > - it also solves the bootstrap problem It also is the least likely to succeed, and the most likely to cause significant "social" upheavals. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Simon Richter <sjr@debian.org> |
|---|---|
| Date | 2023-05-19 02:40 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GwWk1-9S1y-1@gated-at.bofh.it> |
| In reply to | #12098 |
Hi,
On 5/18/23 18:08, Luca Boccassi wrote:
>> Without it, leaving them in place makes no difference for usrmerged
>> systems, and allows derived distributions that don't need usrmerge to
>> continue using our packages.
> Not quite. Having packages only ship files under /usr (and possibly
> /etc) is very much a goal in itself for a lot of us.
My point is: that is not an end goal, because it provides no real
tangible benefit on its own.
It does make sense in the context of building immutable images, which is
*also* a bootstrapping problem, and probably worth being supported with
proper tooling.
>> - there is no guarantee that usrmerge will be permanent or the last
>> transition of this kind
> It is permanent, there are several upstream projects that will drop
> support for legacy layouts very soon, and it will not be re-added
> back.
You are currently building a "legacy" system, it will just take a bit of
time to reach that status. The less you anticipate future needs, the
faster this will be.
I understand that you are also a member of one of these upstream
projects, and that you are taking the interests of this project to
"pretty much the last relevant holdout" here. Has it occurred to you
that you are also wearing a Debian hat, and you could be taking the
interests of the Debian project to said upstream project?
>> - it also solves the bootstrap problem
> It also is the least likely to succeed, and the most likely to cause
> significant "social" upheavals.
The only social problem I see is that you are trying to create a
situation in which other people are compelled to do your work for you if
they want it to be done properly.
So far, you have been throwing out "solutions", and left the analysis of
the feasibility of those to other people, then, after three iterations,
you demanded a full write-up of all existing use cases and blanket
permission to ignore anything not brought up in this list.
The thing is: I see more enthusiasm and self-directed problem solving
skills from the interns at the company where I work, and at the same
time you are one of the top contributors of the upstream project whose
ideas of a "supported" configuration we are supposed to follow.
Simon
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-19 13:10 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <Gx5Qm-9Ysx-3@gated-at.bofh.it> |
| In reply to | #12099 |
On Fri, 19 May 2023 at 01:30, Simon Richter <sjr@debian.org> wrote: > > Hi, > > On 5/18/23 18:08, Luca Boccassi wrote: > > >> Without it, leaving them in place makes no difference for usrmerged > >> systems, and allows derived distributions that don't need usrmerge to > >> continue using our packages. > > > Not quite. Having packages only ship files under /usr (and possibly > > /etc) is very much a goal in itself for a lot of us. > > My point is: that is not an end goal, because it provides no real > tangible benefit on its own. For yourself. Again, for others, it is and it does. > It does make sense in the context of building immutable images, which is > *also* a bootstrapping problem, and probably worth being supported with > proper tooling. > > >> - there is no guarantee that usrmerge will be permanent or the last > >> transition of this kind > > > It is permanent, there are several upstream projects that will drop > > support for legacy layouts very soon, and it will not be re-added > > back. > > You are currently building a "legacy" system, it will just take a bit of > time to reach that status. The less you anticipate future needs, the > faster this will be. > > I understand that you are also a member of one of these upstream > projects, and that you are taking the interests of this project to > "pretty much the last relevant holdout" here. Has it occurred to you > that you are also wearing a Debian hat, and you could be taking the > interests of the Debian project to said upstream project? Has it occurred to you that there might be a specific reason why said upstream project has kept compatibility with legacy cruft that only benefits Debian for so many years, at non-zero cost for developers, despite everything else that's relevant having long since moved on? It is really not difficult to find out what that reason might be, and would have been better to do so before throwing around such accusations. And if you can't find it, you can always go to one of the numerous available channels and ask other maintainers. Why don't you do that, ask how come support for Debian stable for this and many other aspects is kept intact and who's behind that effort, and report back the answer? > >> - it also solves the bootstrap problem > > > It also is the least likely to succeed, and the most likely to cause > > significant "social" upheavals. > > The only social problem I see is that you are trying to create a > situation in which other people are compelled to do your work for you if > they want it to be done properly. No, the social problem is that there is one maintainer who is allowed to ignore the TC and roadblock the entire distribution, so that something that's been trivially and quickly done pretty much literally everywhere else is instead made incredibly difficult and long-winded. But despite that, we are getting there, slow and steady, and remarkably well given the difficult circumstances outwith our control. > So far, you have been throwing out "solutions", and left the analysis of > the feasibility of those to other people, then, after three iterations, > you demanded a full write-up of all existing use cases and blanket > permission to ignore anything not brought up in this list. I have literally no idea what you are talking about. I have contributed what I can in my spare time, unblocking the transition last year and helping out Helmut and others this year. This is not my day job, by the way. What have you done to help, precisely? > The thing is: I see more enthusiasm and self-directed problem solving > skills from the interns at the company where I work, and at the same > time you are one of the top contributors of the upstream project whose > ideas of a "supported" configuration we are supposed to follow. I seriously do not appreciate your tone, you are out of line. Please stop. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Sam Hartman <hartmans@debian.org> |
|---|---|
| Date | 2023-05-17 19:30 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <Gwt8l-9zDj-1@gated-at.bofh.it> |
| In reply to | #12091 |
[Multipart message — attachments visible in raw view] — view raw
>>>>> "Helmut" == Helmut Grohne <helmut@subdivi.de> writes:
Helmut> Moving on to category 4 feels rather obvious, especially
Helmut> because work has been done there in debootstrap. The
Helmut> approach in debootstrap however is one that I see as a dead
Helmut> end, because it causes us to maintain this code multiple
Helmut> times. It's the number of derivatives times the number of
Helmut> bootstrap tools and that doesn't scale.
Like others, I don't find this analysis compelling.
I'd like to better understand why the number of derivatives is in the
cross product.
I suspect most derivatives are going to move to merged /usr, and so I
suspect most if not all derivatives can be treated the same.
What am I missing?
[toc] | [prev] | [next] | [standalone]
| From | Raphael Hertzog <hertzog@debian.org> |
|---|---|
| Date | 2023-06-08 10:50 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GEjvb-etdy-3@gated-at.bofh.it> |
| In reply to | #12091 |
Hi, On Wed, 17 May 2023, Helmut Grohne wrote: > For completeness sake, there is one more entry in category 3: We can run > the dynamic loader from its canonical location explicitly, so we'd > modify maintainer scripts to start with: > > #!/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/bin/sh > > This would only affect preinst scripts participating in bootstrap and > we'd not bother with changing PT_INTERP in the toolchain nor in > packages. Unfortunately, this completely breaks the DPKG_ROOT work and > with that, I see no viable entries in category 3 for moving forward. > > Moving on to category 4 feels rather obvious, especially because work > has been done there in debootstrap. The approach in debootstrap however > is one that I see as a dead end, because it causes us to maintain this > code multiple times. It's the number of derivatives times the number of > bootstrap tools and that doesn't scale. > > Category 4 is wider though and we also have other prior art at > https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular, > making architecture bootstrap become chrootless would likely be a > generic solution to this and other problems. However, any change needs > to propagate to a stable release in all bootstrapping tools. Therefore > we cannot reasonably finish the transition before forky. This makes > category 4 rather unattractive in a short term, but still worth pursuing > in a long term. In the same spirit, I'd like to throw an idea... could we decide that base-files is the first package to be configured as part of the bootstrap protocol and change base-files maintainer's scripts into statically linked executables so that they can work even if we don't have the library loader on the ABI-compliant path? And creating the required symlinks would be done by those (standalone) maintainer scripts... I don't know if we already have some rule/invariant in the configuration order of the unpacked packages, but I doubt so. > Having ruled out categories 3 and 4 maybe category 2 would be good? We > could just ship those symlinks in base-files and be done, right? > Unfortunately, we pass -k to tar in debootstrap, so when it extracts > base-files and tries to unpack the bin -> usr/bin symlink, it sees that > oh no, there already is a symlink at bin (as debootstrap placed it > there) and thus fails. So in order to make this work, we also have to > modify debootstrap (and thus are in a combination of category 3 and 4). So when we will have fixed this, and waited for a release cycle, we can get rid of the statically compiled maintainer scripts and simply ship the symlinks in base-files. Cheers, -- ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org> ⣾⠁⢠⠒⠀⣿⡁ ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/ ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-06-08 12:10 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GEkKB-eu7e-3@gated-at.bofh.it> |
| In reply to | #12124 |
On Thu, 8 Jun 2023 at 09:46, Raphael Hertzog <hertzog@debian.org> wrote: > > Hi, > > On Wed, 17 May 2023, Helmut Grohne wrote: > > For completeness sake, there is one more entry in category 3: We can run > > the dynamic loader from its canonical location explicitly, so we'd > > modify maintainer scripts to start with: > > > > #!/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/bin/sh > > > > This would only affect preinst scripts participating in bootstrap and > > we'd not bother with changing PT_INTERP in the toolchain nor in > > packages. Unfortunately, this completely breaks the DPKG_ROOT work and > > with that, I see no viable entries in category 3 for moving forward. > > > > Moving on to category 4 feels rather obvious, especially because work > > has been done there in debootstrap. The approach in debootstrap however > > is one that I see as a dead end, because it causes us to maintain this > > code multiple times. It's the number of derivatives times the number of > > bootstrap tools and that doesn't scale. > > > > Category 4 is wider though and we also have other prior art at > > https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap. In particular, > > making architecture bootstrap become chrootless would likely be a > > generic solution to this and other problems. However, any change needs > > to propagate to a stable release in all bootstrapping tools. Therefore > > we cannot reasonably finish the transition before forky. This makes > > category 4 rather unattractive in a short term, but still worth pursuing > > in a long term. > > In the same spirit, I'd like to throw an idea... could we decide that > base-files is the first package to be configured as part of the bootstrap > protocol and change base-files maintainer's scripts into statically linked > executables so that they can work even if we don't have the library loader > on the ABI-compliant path? > > And creating the required symlinks would be done by those (standalone) > maintainer scripts... > > I don't know if we already have some rule/invariant in the configuration > order of the unpacked packages, but I doubt so. > > > Having ruled out categories 3 and 4 maybe category 2 would be good? We > > could just ship those symlinks in base-files and be done, right? > > Unfortunately, we pass -k to tar in debootstrap, so when it extracts > > base-files and tries to unpack the bin -> usr/bin symlink, it sees that > > oh no, there already is a symlink at bin (as debootstrap placed it > > there) and thus fails. So in order to make this work, we also have to > > modify debootstrap (and thus are in a combination of category 3 and 4). > > So when we will have fixed this, and waited for a release cycle, we can > get rid of the statically compiled maintainer scripts and simply ship the > symlinks in base-files. Just a note that we can always change debootstrap via bookworm-p-u, we've already done so in the past for this, so there's no requirement to wait an extra release. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Marco d'Itri <md@Linux.IT> |
|---|---|
| Date | 2023-06-09 09:50 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GEF2F-eGEC-3@gated-at.bofh.it> |
| In reply to | #12124 |
[Multipart message — attachments visible in raw view] — view raw
On Jun 08, Raphael Hertzog <hertzog@debian.org> wrote: > In the same spirit, I'd like to throw an idea... could we decide that > base-files is the first package to be configured as part of the bootstrap > protocol and change base-files maintainer's scripts into statically linked > executables so that they can work even if we don't have the library loader > on the ABI-compliant path? It could be even easier: base-files could be unpacked once without running the maintainer scripts and then "reinstalled" again later as usual. > And creating the required symlinks would be done by those (standalone) > maintainer scripts... > > I don't know if we already have some rule/invariant in the configuration > order of the unpacked packages, but I doubt so. Indeed, this would be very simple and it has already been proposed. But somebody then complained that special-casing a package would violate the design contraints he self-imposed to his own image building tool, and as we all know every Debian maintainer can veto any systemic changes that they do not like. -- ciao, Marco
[toc] | [prev] | [next] | [standalone]
| From | Raphael Hertzog <hertzog@debian.org> |
|---|---|
| Date | 2023-06-09 12:00 +0200 |
| Subject | Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg) |
| Message-ID | <GEH4t-eHQ9-1@gated-at.bofh.it> |
| In reply to | #12133 |
[Multipart message — attachments visible in raw view] — view raw
On Fri, 09 Jun 2023, Marco d'Itri wrote: > On Jun 08, Raphael Hertzog <hertzog@debian.org> wrote: > > > In the same spirit, I'd like to throw an idea... could we decide that > > base-files is the first package to be configured as part of the bootstrap > > protocol and change base-files maintainer's scripts into statically linked > > executables so that they can work even if we don't have the library loader > > on the ABI-compliant path? > It could be even easier: base-files could be unpacked once without > running the maintainer scripts and then "reinstalled" again later as > usual. I think you are missing the point here, that only works if the package is shipping the symlinks. And the idea is to not do this immediately because it breaks debootstrap: if I understood correctly unpacking base-files with the symlinks would fail if debootstrap had already pre-created those symlinks (due to a -k option that we should get rid of in /usr/share/debootstrap/scripts/debian-common). Hence the special maintainer script to create the required symlinks without relying on /bin/sh or any dynamically linked executable. > > And creating the required symlinks would be done by those (standalone) > > maintainer scripts... > > > > I don't know if we already have some rule/invariant in the configuration > > order of the unpacked packages, but I doubt so. > > Indeed, this would be very simple and it has already been proposed. > But somebody then complained that special-casing a package would violate > the design contraints he self-imposed to his own image building tool, > and as we all know every Debian maintainer can veto any systemic changes > that they do not like. That's not very helpful. Nobody has vetoed anything here. But I agree that it would be cleaner if we could reach a situation where we can just unpack all packages and have a working system where we can just "dpkg --configure -a" and be done. You don't care about this goal, it's fine, but it's not a reason to paint this as a black/white picture. We can have both, we just need an intermediate step. I understand some would rather just be done with this transition (so am I...), but going the extra mile here doesn't seem unreasonable. --- Coming back to my initial suggestion, I realize however that while the maintainer script can run, dpkg itself will not run in the chroot so if debootstrap is relying on dpkg to do the initial base-files configuration, this will not work. So this looks like that we will have to continue to rely on debootstrap to create the symlinks and we will have to fix it so that it can properly unpack a base-files containing /bin and /lib as symlinks on top of existing symlinks. And the actual switch to include /bin and /lib symlinks in base-files can be done once we have fixed debootstrap in all relevant releases. And after we can stop pre-creating those symlinks in debootstrap for all future releases. Cheers, -- ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org> ⣾⠁⢠⠒⠀⣿⡁ ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/ ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
[toc] | [prev] | [next] | [standalone]
Page 5 of 7 — ← Prev page 1 2 3 4 [5] 6 7 Next page →
Back to top | Article view | linux.debian.maint.dpkg
csiph-web