Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.bugs.dist > #1146390 > unrolled thread
| Started by | Ansgar <ansgar@43-1.org> |
|---|---|
| First post | 2023-05-11 00:00 +0200 |
| Last post | 2023-06-13 22:30 +0200 |
| Articles | 20 on this page of 63 — 20 participants |
Back to article view | Back to linux.debian.bugs.dist
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Ansgar <ansgar@43-1.org> - 2023-05-11 00:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Ansgar <ansgar@43-1.org> - 2023-05-11 00:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Ansgar <ansgar@43-1.org> - 2023-05-18 19:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Emilio Pozuelo Monfort <pochu@debian.org> - 2023-05-11 12:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon Richter <sjr@debian.org> - 2023-05-11 14:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-11 20:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon Richter <sjr@debian.org> - 2023-05-12 14:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 15:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Matthew Vernon <matthew@debian.org> - 2023-05-25 14:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-05-12 07:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-12 10:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 12:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-12 12:50 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 13:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-12 13:20 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 14:20 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-12 16:40 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-12 17:40 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-15 01:30 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Peter Pentchev <roam@ringlet.net> - 2023-05-15 02:10 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-15 02:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Russ Allbery <rra@debian.org> - 2023-05-15 02:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-15 02:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-15 03:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Russ Allbery <rra@debian.org> - 2023-05-15 03:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-15 04:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Russ Allbery <rra@debian.org> - 2023-05-15 17:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-16 03:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-15 03:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Johannes Schauer Marin Rodrigues <josch@debian.org> - 2023-05-15 07:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Bdale Garbee <bdale@gag.com> - 2023-05-15 15:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-15 15:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Bdale Garbee <bdale@gag.com> - 2023-05-15 18:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Matthew Vernon <matthew@debian.org> - 2023-05-15 18:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Bdale Garbee <bdale@gag.com> - 2023-05-15 18:30 +0200
Bug#1035904: What does merged /usr bring us Sam Hartman <hartmans@debian.org> - 2023-05-15 20:40 +0200
Bug#1035904: What does merged /usr bring us Sam Hartman <hartmans@debian.org> - 2023-05-15 20:40 +0200
Bug#1035904: What does merged /usr bring us Jonathan Carter <jcc@debian.org> - 2023-05-15 20:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Sam Hartman <hartmans@debian.org> - 2023-05-15 19:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Steve McIntyre <steve@einval.com> - 2023-05-15 15:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-15 16:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon McVittie <smcv@debian.org> - 2023-05-15 20:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon McVittie <smcv@debian.org> - 2023-05-15 20:00 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-16 04:00 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Simon McVittie <smcv@debian.org> - 2023-05-16 10:30 +0200
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-17 01:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Roger Lynn <usenet@rilynn.me.uk> - 2023-05-17 22:50 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-05-18 20:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Gunnar Wolf <gwolf@debian.org> - 2023-05-18 20:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-05-18 20:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Luca Boccassi <bluca@debian.org> - 2023-05-18 20:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Matthias Klumpp <matthias@tenstral.net> - 2023-05-18 21:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Arnaud Rebillout <arnaudr@debian.org> - 2023-05-19 05:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Bastian Blank <waldi@debian.org> - 2023-05-18 21:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Gunnar Wolf <gwolf@debian.org> - 2023-05-18 21:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-05-19 16:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Helmut Grohne <helmut@subdivi.de> - 2023-05-21 16:30 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Ansgar <ansgar@43-1.org> - 2023-05-21 17:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg) Luca Boccassi <bluca@debian.org> - 2023-05-21 17:00 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-06-07 06:40 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-06-13 21:20 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Matthew Garrett <mjg59@srcf.ucam.org> - 2023-06-13 22:10 +0200
Bug#1035904: dpkg currently warning about merged-usr systems (revisited) Ansgar <ansgar@43-1.org> - 2023-06-13 22:30 +0200
Page 2 of 4 — ← Prev page 1 [2] 3 4 Next page →
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-15 02:30 +0200 |
| Subject | Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <Gvug9-8Xjm-1@gated-at.bofh.it> |
| In reply to | #1146752 |
On Mon, 15 May 2023 at 01:07, Peter Pentchev <roam@ringlet.net> wrote: > > On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote: > > On Sun, 14 May 2023 at 22:37, Josh Triplett <josh@joshtriplett.org> wrote: > > > > > > On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote: > > > > The loader is still available via the old path, so external/third > > > > party/local/other software works unchanged. This should negatively > > > > only affect our 1st party packages, when running on a non-merged > > > > distro. > > > > And are _all_ our packages really 100% compatible with other distros > > > > at all? Are they even supposed to be? > > > > > > People build things on Debian that are not Debian packages. People > > > compile binaries on Debian, and expect them to work on any system that > > > has sufficiently new libraries. > > > > > > This is *not* about Debian packages failing to work on other > > > distributions; this is about *software compiled on Debian* faliing to > > > work in other environments. > > > > Why would "software compiled on Debian" fail to work in other > > environments? Well, there are many reasons actually, people invented > > containers/flatpaks/snaps exactly for that reason. But nothing do with > > anything discussed here though, as far as I can tell? > > If an ELF executable, compiled on Debian, records its interpreter as > /usr/lib/ld-linux.so.2, what happens when one tries to run it on > a non-usr-merged system? Even one with a recent enough glibc version? This is not about locally built ELF executables, no difference in those. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Russ Allbery <rra@debian.org> |
|---|---|
| Date | 2023-05-15 02:20 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <Gvu6t-8Xg0-3@gated-at.bofh.it> |
| In reply to | #1146748 |
Luca Boccassi <bluca@debian.org> writes: > Why would "software compiled on Debian" fail to work in other > environments? Well, there are many reasons actually, people invented > containers/flatpaks/snaps exactly for that reason. But nothing do with > anything discussed here though, as far as I can tell? My understanding is that this specific thread is about a mention that we may want to change PT_INTERP to /usr/lib64/ld-linux-x86-64.so.2 or some similar path. If PT_INTERP points to a file that doesn't exist, the program is obviously not going to run. The Linux x86_64 ABI says it must point to /lib64/ld-linux-x86-64.so.2. If we build binaries that use some other value, then we are not building ABI-compliant binaries and they may not run on other systems. This is the whole point of an ABI. An obvious specific example of such a system would be one that didn't merge /usr and thus only had /lib64/ld-linux-x86-64.so.2 and not any other path, but that's just one obvious example. There may be others; the whole point of an ABI is that you do not change things like this, not even if you can't personally imagine why your change wouldn't be harmful. There's a whole process for changing an ABI that involves everyone else agreeing as well, and unless one goes through that process, the ABI is what it is. Debian not building ABI-compliant binaries would be highly surprising. Incidentally, that remains true even if we only do that in distribution packages. I certainly have copied binaries from a Debian package to other Linux systems before for various reasons and expected them to run. Sure, this might not work for other reasons outside of our control, but that's no reason to be gratuitously incompatible by breaking the ABI, particularly for what seem to be annoyances of our own creation with known workarounds. -- Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-15 02:50 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <Gvuzv-8Xpy-1@gated-at.bofh.it> |
| In reply to | #1146753 |
On Mon, 15 May 2023 at 01:14, Russ Allbery <rra@debian.org> wrote: > > Luca Boccassi <bluca@debian.org> writes: > > > Why would "software compiled on Debian" fail to work in other > > environments? Well, there are many reasons actually, people invented > > containers/flatpaks/snaps exactly for that reason. But nothing do with > > anything discussed here though, as far as I can tell? > > My understanding is that this specific thread is about a mention that we > may want to change PT_INTERP to /usr/lib64/ld-linux-x86-64.so.2 or some > similar path. > > If PT_INTERP points to a file that doesn't exist, the program is obviously > not going to run. The Linux x86_64 ABI says it must point to > /lib64/ld-linux-x86-64.so.2. If we build binaries that use some other > value, then we are not building ABI-compliant binaries and they may not > run on other systems. This is the whole point of an ABI. This is not about locally compiled software or such, only packages (and maybe even just a subset of them). > An obvious specific example of such a system would be one that didn't > merge /usr and thus only had /lib64/ld-linux-x86-64.so.2 and not any other > path, but that's just one obvious example. There may be others; the whole > point of an ABI is that you do not change things like this, not even if > you can't personally imagine why your change wouldn't be harmful. There's > a whole process for changing an ABI that involves everyone else agreeing > as well, and unless one goes through that process, the ABI is what it is. > Debian not building ABI-compliant binaries would be highly surprising. That's self-evidently not true, as there are other distributions where that already happens, it's been already mentioned. Besides, we are not talking about sacred religious texts - the point is making things work. If they do, is it _really_ non-compliant/incompatible? > Incidentally, that remains true even if we only do that in distribution > packages. I certainly have copied binaries from a Debian package to other > Linux systems before for various reasons and expected them to run. Sure, > this might not work for other reasons outside of our control, but that's > no reason to be gratuitously incompatible by breaking the ABI, > particularly for what seem to be annoyances of our own creation with known > workarounds. Thanks, that's the first actual real example mentioned so far. And it's an interesting one: taking a $random Debian package and using it on a completely different, non-Debian system. Is that a supported use case? If so, does that mean that I can go ahead and raise a Severity: serious bug on any package that doesn't work in such a way? Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Steve McIntyre <steve@einval.com> |
|---|---|
| Date | 2023-05-15 03:10 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvuSR-8XLC-5@gated-at.bofh.it> |
| In reply to | #1146756 |
I'm *trying* to assume good faith here, but I'm running out of energy to do so. On Mon, May 15, 2023 at 01:42:27AM +0100, Luca Boccassi wrote: >On Mon, 15 May 2023 at 01:14, Russ Allbery <rra@debian.org> wrote: > >> Incidentally, that remains true even if we only do that in distribution >> packages. I certainly have copied binaries from a Debian package to other >> Linux systems before for various reasons and expected them to run. Sure, >> this might not work for other reasons outside of our control, but that's >> no reason to be gratuitously incompatible by breaking the ABI, >> particularly for what seem to be annoyances of our own creation with known >> workarounds. > >Thanks, that's the first actual real example mentioned so far. And >it's an interesting one: taking a $random Debian package and using it >on a completely different, non-Debian system. Is that a supported use >case? If so, does that mean that I can go ahead and raise a Severity: >serious bug on any package that doesn't work in such a way? Russ has described copying *binaries* out of packages and running them elsewhere. I've done that too, from time to time. This is one of the things made possible by the ABI contract being followed. You are the one proposing to break that contract, thereby *guaranteeing* this will fail on systems where otherwise it could work. I think the onus is on *you* to justify why this is a valid and useful thing to do. Your apparent lack of care for agreed standards here is horrifying. -- Steve McIntyre, Cambridge, UK. steve@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
[toc] | [prev] | [next] | [standalone]
| From | Russ Allbery <rra@debian.org> |
|---|---|
| Date | 2023-05-15 03:40 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <Gvvcd-8XSo-3@gated-at.bofh.it> |
| In reply to | #1146756 |
Luca Boccassi <bluca@debian.org> writes: > That's self-evidently not true, as there are other distributions where > that already happens, it's been already mentioned. You've mentioned this a couple of times but I don't think I've seen the message where the details were explained. Maybe this was only in your message posted to debian-gcc, which wasn't part of this thread? (It's also possible that I just missed it somewhere.) That message only mentions GUIX, which I don't know very much about, but my recollection (maybe wrong?) is that it's a NIX variant that is doing special tricks to support immutable package trees and roll-forward/roll-back upgrades. I can see why that might be motivation to build incompatible binaries in order to preserve some other invariant they're trying for as the point of their distribution (in particular, I suspect they're pinning binaries to a specific version of the dynamic loader as part of the whole immutable tree strategy). That's a perfectly fine decision in a distribution that's trying to do something very different and is a bit of a science experiment, but I don't think that describes Debian. (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh, major player in Linux distributions, and I'm not sure how much they care about compatibility with anyone else.) > Besides, we are not talking about sacred religious texts - the point is > making things work. If they do, is it _really_ > non-compliant/incompatible? I understand your point in making this argument, but please understand that this sort of willingness to change things if one didn't think they would cause problems didn't work very well, and was part of what led to the development of standardized ABIs in the first place. Those of us who have been around longer than Linux have ABIs have a bit of a strong reaction here (I think this is also what you're seeing from Steve), because we remember the bad old days. I still have compatibility code around to handle the fact that gcc on IRIX miscompiled calls to inet_ntoa because gcc didn't correctly implement the IRIX ABI. People are very bad at judging whether their new idea would be *really* incompatible. This is why these days everyone tries to follow the ABI pretty closely. And in any case, changing PT_INTERP is trivially and obviously incompatible; the binary will simply not run on a system that doesn't have that path. So it's not like we have to carefully judge nuance here. Your argument, so far as I can tell, is basically "but no one will ever want to run those binaries on a non-/usr-merged system anyway," which is basically conceding the incompatibility point since the ABI doesn't require merged /usr. There's also some other history here: Debian is not super-happy with the PT_INTERP because ideally we'd prefer it use a path compatible with our multiarch approach. I believe we raised that and no one had any interest in trying to change anything, so we lived with the limitations that creates. (And I think that was the right decision.) > Thanks, that's the first actual real example mentioned so far. And it's > an interesting one: taking a $random Debian package and using it on a > completely different, non-Debian system. Is that a supported use case? > If so, does that mean that I can go ahead and raise a Severity: serious > bug on any package that doesn't work in such a way? I feel like you're distorting my argument here to try to make some sort of slippery slope argument, and it's coming across as possibly more aggressive than you had intended. The world does not divide neatly into supported and unsupported use cases. There are a lot of things I do to computers that I expect to work in some situations but not in others. That includes, say, having a Debian chroot on some other OS and running binaries from that chroot without going into the chroot. Often that will absolutely not work. Sometimes it will work, and it's convenient that it will work for some obscure recovery situations or other weird one-off use cases. I've also copied files from working systems to broken systems running a different distribution before, and there's a list of caveats as long as my arm, but sometimes it's a quick fix for something. But mostly my reaction is because breaking the ABI is a Really Big Deal. Constructing the Linux ABI and getting the details actually published was a hard-fought, arduous endeavor. I doubt anyone enjoyed it; it's the sort of annoying compatibility work that provides tons of small, subtle benefits and takes a great deal of truly thankless work, and people often don't realize all the tiny ways that it has made the world a better place, or the range of weird compatibility problems that can arise from messing with it. Diverging from it is not something to do lightly, precisely *because* it's often extremely difficult to understand what the effects could be or what might break. While I appreciate how it would make bootstrapping Debian somewhat more convenient in this case, I am unconvinced that this is a good enough reason to undermine one of the foundations of what makes Linux a collective and fairly mutually compatible ecosystem. I realize it's not necessarily obvious that changing PT_INTERP for some binaries is a big deal, in part because it's not even obvious that it's part of the ABI. That's why people who are familiar with the ABI process are jumping in to say "please don't touch that, this is a big deal to us." -- Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-15 04:40 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <Gvw8h-8Ywq-5@gated-at.bofh.it> |
| In reply to | #1146762 |
On Mon, 15 May 2023 at 02:26, Russ Allbery <rra@debian.org> wrote: > > Luca Boccassi <bluca@debian.org> writes: > > > That's self-evidently not true, as there are other distributions where > > that already happens, it's been already mentioned. > > You've mentioned this a couple of times but I don't think I've seen the > message where the details were explained. Maybe this was only in your > message posted to debian-gcc, which wasn't part of this thread? (It's > also possible that I just missed it somewhere.) > > That message only mentions GUIX, which I don't know very much about, but > my recollection (maybe wrong?) is that it's a NIX variant that is doing > special tricks to support immutable package trees and > roll-forward/roll-back upgrades. I can see why that might be motivation > to build incompatible binaries in order to preserve some other invariant > they're trying for as the point of their distribution (in particular, I > suspect they're pinning binaries to a specific version of the dynamic > loader as part of the whole immutable tree strategy). That's a perfectly > fine decision in a distribution that's trying to do something very > different and is a bit of a science experiment, but I don't think that > describes Debian. > > (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh, major > player in Linux distributions, and I'm not sure how much they care about > compatibility with anyone else.) This is a counter-example to confute the assertion that *everybody* does the same thing, which has been made multiple times. I'm not sure whether it's an experiment or not, I mean if you ask their users/developers I think they'd tell you it's very much production ready, but it is largely irrelevant: it exists, and that's the only reason why it was mentioned, as it shows that it is _possible_ to do that and be a working distribution. Doesn't imply it's automatically desirable, but that was not the intention. > > Besides, we are not talking about sacred religious texts - the point is > > making things work. If they do, is it _really_ > > non-compliant/incompatible? > > I understand your point in making this argument, but please understand > that this sort of willingness to change things if one didn't think they > would cause problems didn't work very well, and was part of what led to > the development of standardized ABIs in the first place. Those of us who > have been around longer than Linux have ABIs have a bit of a strong > reaction here (I think this is also what you're seeing from Steve), > because we remember the bad old days. I still have compatibility code > around to handle the fact that gcc on IRIX miscompiled calls to inet_ntoa > because gcc didn't correctly implement the IRIX ABI. > > People are very bad at judging whether their new idea would be *really* > incompatible. This is why these days everyone tries to follow the ABI > pretty closely. > > And in any case, changing PT_INTERP is trivially and obviously > incompatible; the binary will simply not run on a system that doesn't have > that path. So it's not like we have to carefully judge nuance here. Your > argument, so far as I can tell, is basically "but no one will ever want to > run those binaries on a non-/usr-merged system anyway," which is basically > conceding the incompatibility point since the ABI doesn't require merged > /usr. Not quite: my argument is that binaries from these packages are not intended and not required to be ran on non-Debian systems, so there's no incompatibility introduced in the first place - everything still works where it is supposed to, exactly as it was before. > There's also some other history here: Debian is not super-happy with the > PT_INTERP because ideally we'd prefer it use a path compatible with our > multiarch approach. I believe we raised that and no one had any interest > in trying to change anything, so we lived with the limitations that > creates. (And I think that was the right decision.) > > > Thanks, that's the first actual real example mentioned so far. And it's > > an interesting one: taking a $random Debian package and using it on a > > completely different, non-Debian system. Is that a supported use case? > > If so, does that mean that I can go ahead and raise a Severity: serious > > bug on any package that doesn't work in such a way? > > I feel like you're distorting my argument here to try to make some sort of > slippery slope argument, and it's coming across as possibly more > aggressive than you had intended. No aggression intended whatsoever, sorry if it appeared that way. I am trying to understand what the rules are. > The world does not divide neatly into supported and unsupported use cases. > There are a lot of things I do to computers that I expect to work in some > situations but not in others. That includes, say, having a Debian chroot > on some other OS and running binaries from that chroot without going into > the chroot. Often that will absolutely not work. Sometimes it will work, > and it's convenient that it will work for some obscure recovery situations > or other weird one-off use cases. I've also copied files from working > systems to broken systems running a different distribution before, and > there's a list of caveats as long as my arm, but sometimes it's a quick > fix for something. Vast, vast majority of binaries from existing packages will already not work out of the box in that use case though. Why are the existing incompatibilities allowed, and this isn't? > But mostly my reaction is because breaking the ABI is a Really Big Deal. > Constructing the Linux ABI and getting the details actually published was > a hard-fought, arduous endeavor. I doubt anyone enjoyed it; it's the sort > of annoying compatibility work that provides tons of small, subtle > benefits and takes a great deal of truly thankless work, and people often > don't realize all the tiny ways that it has made the world a better place, > or the range of weird compatibility problems that can arise from messing > with it. Diverging from it is not something to do lightly, precisely > *because* it's often extremely difficult to understand what the effects > could be or what might break. I am afraid I am a bit more "pragmatic" than that. I am very interested in what works and what doesn't. Whether it conforms to the letter of the Sacred Ancient Texts... interesting for sure, but secondary. > While I appreciate how it would make bootstrapping Debian somewhat more > convenient in this case, I am unconvinced that this is a good enough > reason to undermine one of the foundations of what makes Linux a > collective and fairly mutually compatible ecosystem. > > I realize it's not necessarily obvious that changing PT_INTERP for some > binaries is a big deal, in part because it's not even obvious that it's > part of the ABI. That's why people who are familiar with the ABI process > are jumping in to say "please don't touch that, this is a big deal to us." Let me ask a simple policy question: why do I have to abide to your uncodified, unwritten use case of running a Debian program from a Debian package from outside a Debian chroot on a foreign, unmodified non-Debian system, but you don't have to abide by my uncodified, unwritten use case of running a Debian program on a Debian system with only a Debian /usr? And also, why it is only this change that has to abide to such use case, and the myriad of cases that cannot possibly work in such a setup are given a free pass (I mean I'm pretty sure the only executables that are guaranteed to work as you mention are golang and rustlang, where everything is statically compiled, everything else is already rc-buggy by that standard)? Isn't this the very reason we have a Policy for, so that we don't get to cherry-pick arbitrary use cases to block things we don't like? Are you really sure we want to be in a place where anybody can bring up an out-of-policy use case as a valid reason to block something they don't like? Again, I am fine if we want to say that Debian-specific changes and Debianisms are bad and we cannot do anything that jeopardises interoperability, cross-distribution harmony and mutual compatibility. But let's do that then, and start by writing it down in Policy so that it applies fairly and equally to all cases. Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Russ Allbery <rra@debian.org> |
|---|---|
| Date | 2023-05-15 17:30 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvI9r-96cX-9@gated-at.bofh.it> |
| In reply to | #1146765 |
Luca Boccassi <bluca@debian.org> writes: > On Mon, 15 May 2023 at 02:26, Russ Allbery <rra@debian.org> wrote: >> (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh, >> major player in Linux distributions, and I'm not sure how much they >> care about compatibility with anyone else.) > This is a counter-example to confute the assertion that *everybody* does > the same thing, which has been made multiple times. I'm not sure whether > it's an experiment or not, I mean if you ask their users/developers I > think they'd tell you it's very much production ready, but it is largely > irrelevant: it exists, and that's the only reason why it was mentioned, > as it shows that it is _possible_ to do that and be a working > distribution. Doesn't imply it's automatically desirable, but that was > not the intention. Ah, okay, I'm happy to agree with that point: you can violate the ABI and continue to be a working distribution. (There are a lot of parts of the ABI that if you violated them you would not have a working distribution, but this is not one of them so far as I can tell.) > Not quite: my argument is that binaries from these packages are not > intended and not required to be ran on non-Debian systems, so there's no > incompatibility introduced in the first place - everything still works > where it is supposed to, exactly as it was before. I think we're saying the same thing but quibbling over phrasing. I'd put that as saying that it's fine for the binaries of certain core Debian packages to be incompatible with the ABI because they're not intended to be used outside of Debian. (In other words, I'm talking about incompatibility as a concrete, testable property of a binary, and I think you're talking about incompatibility as a more abstract concept of a distribution.) > No aggression intended whatsoever, sorry if it appeared that way. I am > trying to understand what the rules are. Well, the rule that I'd ideally set is don't break the ABI, even if it's not obvious why breaking the ABI is a bad idea or you can't see any bad consequences that could come from it, unless the reason for breaking the ABI is absolutely central to the mission and purpose of Debian. That said, it's not like we've never shipped a binary in Debian with a different PT_INTERP. (I vaguely remember that some programming language uses PT_INTERP tricks for some sort of private binary scheme? Objective CAML or something? I ran across it once years ago and can't remember the details. Also, IIRC klibc does some sort of PT_INTERP trick in some situations that I don't remember the details of, although I don't think it does that with general binaries.) So I do see your point that you would prefer the rule to be more pragmatic than that. My counterargument is that this proposal seems to mostly be about avoiding having to create a symlink at a critical point in the bootstrap process, and while it's tricky to get the timing right (and therefore kind of annoying), the resulting usable system has to have that symlink anyway (I think there's no disagreement about that). Not following the ABI for core binaries seems like a scary change with unknown consquences to a bunch of core packages to solve what looks like a relatively minor (if admittedly annoying!) problem. Note that the target of PT_INTERP on Debian is *already* a symlink, to /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 which was the multiarch path that we actually want to use. We're already ensuring compatibility with a symlink and I think we should just keep doing that and not venture into these waters. >> The world does not divide neatly into supported and unsupported use >> cases. There are a lot of things I do to computers that I expect to >> work in some situations but not in others. That includes, say, having >> a Debian chroot on some other OS and running binaries from that chroot >> without going into the chroot. Often that will absolutely not work. >> Sometimes it will work, and it's convenient that it will work for some >> obscure recovery situations or other weird one-off use cases. I've >> also copied files from working systems to broken systems running a >> different distribution before, and there's a list of caveats as long as >> my arm, but sometimes it's a quick fix for something. > Vast, vast majority of binaries from existing packages will already > not work out of the box in that use case though. I'm not sure why you think this is true and it makes me wonder if maybe my intuition about cross-distribution compatibility is wrong. I would expect to be able to copy, say, most (all?) binaries from coreutils from a Debian system to some other distribution and run it (and that's exactly the sort of binary that is useful in this kind of cross-distribution rescue case). Is this not true today? What breaks? Note that we're not talking about complicated packages with lots of runtime like, say, Emacs. As I understand it your proposal wouldn't change PT_INTERP for that binary anyway. We're presumably talking about the kind of binaries that you need to bootstrap a minimal system, so packages like coreutils or bash. And I would indeed expect those binaries to be generally portable, as long as the same critical shared libraries are available on other systems (in this case, PCRE2 and ncurses). > Why are the existing incompatibilities allowed, and this isn't? Because it breaks the ABI. I know you don't find that answer satisfying, but that really is my answer. > I am afraid I am a bit more "pragmatic" than that. I am very interested > in what works and what doesn't. Whether it conforms to the letter of the > Sacred Ancient Texts... interesting for sure, but secondary. Right, this is our point of fundamental disagreement. > Let me ask a simple policy question: why do I have to abide to your > uncodified, unwritten use case of running a Debian program from a Debian > package from outside a Debian chroot on a foreign, unmodified non-Debian > system, but you don't have to abide by my uncodified, unwritten use case > of running a Debian program on a Debian system with only a Debian /usr? Well, the concrete answer here is that in order to make this change, you're going to have to convince a bunch of Debian core package maintainers to go along with this change and build their binaries in ways that break the ABI. I believe at least some of them are going to have the same reaction that Steve and I had, and will require a lot of convincing. > And also, why it is only this change that has to abide to such use case, > and the myriad of cases that cannot possibly work in such a setup are > given a free pass (I mean I'm pretty sure the only executables that are > guaranteed to work as you mention are golang and rustlang, where > everything is statically compiled, everything else is already rc-buggy > by that standard)? I really would be surprised if coreutils didn't work, but maybe I'm wrong? It does appear to generally have a PCRE2 requirement, but I would expect a compatible PCRE2 in other distributions of a similar vintage. > Isn't this the very reason we have a Policy for, so that we don't get to > cherry-pick arbitrary use cases to block things we don't like? Policy indeed probably doesn't say that you have to follow the Linux ABI for normal binaries that aren't doing weird language-ecosystem-specific things, but that's partly because it's so foundational and so automatic in normal uses of compilers and linkers that I don't think we ever had any reason to put it into Policy. It certainly didn't occur to me that anyone would want to change the ABI for a subset of Debian packages. Also, I thought you didn't care about Sacred Ancient Texts like Policy. :) > Are you really sure we want to be in a place where anybody can bring up > an out-of-policy use case as a valid reason to block something they > don't like? Yes, I absolutely want to be in a place where anyone can raise any reason that matters to them in public discussion as a reason not to do something they think would be harmful! This is the whole point of working collaboratively together on a shared endeavor. Everyone gets a say! That doesn't mean they get a *veto*, but they get a *say*, and that say is weighted by how much perceived expertise they have and how directly the plan affects their work in Debian. I personally am certainly not expecting to have a veto or even that strong of a say here. This doesn't affect any of my packages so far as I can tell, and I'm far from an expert on the ABI. I've just been around for long enough to pick up something by osmosis. I mostly jumped in because it felt like you and Steve were just yelling at each other and I thought I might be able to explain some of where he was coming from in a way that may make more sense. > Again, I am fine if we want to say that Debian-specific changes and > Debianisms are bad and we cannot do anything that jeopardises > interoperability, cross-distribution harmony and mutual compatibility. > But let's do that then, and start by writing it down in Policy so that > it applies fairly and equally to all cases. I would love to get into a situation where Policy is a comprehensive guide to everything people would like to have rules about in Debian, but I am also pretty sure that if I quit my day job and made Policy my full-time job, that would still not be done in ten years. Distributions are complicated with a lot of moving parts and a lot of those aren't written down. This is why we have people with substantial personal expertise around who can think through novel situations and try to figure out what the possible options and consequences are. It's a good thing! My starting point is that "follow the ABI" is like a safety interlock. Whenever you park a car on a hill, you set the parking brake. You don't try to figure out whether this hill is steep enough to make the car roll, or do complex geometry calculations to try to figure out where the car might roll too; you just set the parking brake always. That makes it a habit, which has its own valuable properties. It means that you set the parking brake in a bunch of places where it's unnecessary to do so in some objective sense, but who cares, it's a habit, it's automatic. You're saying "we don't need to set the parking brake here." You might be right! I'm thinking through the negative consequences of that, but I'm not saying that the specific examples that have come to mind so far are all that compelling. My primary argument is that the logic of always setting the parking brake and not thinking about it is what's compelling, and you're asking us to give up a point of consistency that acts like a safety interlock, and I'm saying that makes me very uncomfortable because it requires we go through this complex process of figuring out what's might go wrong and we also create an inconsistency at a core level of the distribution that may have unforseen consequences. It's a lot easier mentally and conceptually to just always set the parking brake, and complexity is the enemy of any large software project. -- Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-16 03:50 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvRZ7-9c12-1@gated-at.bofh.it> |
| In reply to | #1146828 |
On Mon, 15 May 2023 at 16:18, Russ Allbery <rra@debian.org> wrote: > > Luca Boccassi <bluca@debian.org> writes: > > On Mon, 15 May 2023 at 02:26, Russ Allbery <rra@debian.org> wrote: > > >> (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh, > >> major player in Linux distributions, and I'm not sure how much they > >> care about compatibility with anyone else.) > > > This is a counter-example to confute the assertion that *everybody* does > > the same thing, which has been made multiple times. I'm not sure whether > > it's an experiment or not, I mean if you ask their users/developers I > > think they'd tell you it's very much production ready, but it is largely > > irrelevant: it exists, and that's the only reason why it was mentioned, > > as it shows that it is _possible_ to do that and be a working > > distribution. Doesn't imply it's automatically desirable, but that was > > not the intention. > > Ah, okay, I'm happy to agree with that point: you can violate the ABI and > continue to be a working distribution. (There are a lot of parts of the > ABI that if you violated them you would not have a working distribution, > but this is not one of them so far as I can tell.) > > > Not quite: my argument is that binaries from these packages are not > > intended and not required to be ran on non-Debian systems, so there's no > > incompatibility introduced in the first place - everything still works > > where it is supposed to, exactly as it was before. > > I think we're saying the same thing but quibbling over phrasing. I'd put > that as saying that it's fine for the binaries of certain core Debian > packages to be incompatible with the ABI because they're not intended to > be used outside of Debian. (In other words, I'm talking about > incompatibility as a concrete, testable property of a binary, and I think > you're talking about incompatibility as a more abstract concept of a > distribution.) > > > No aggression intended whatsoever, sorry if it appeared that way. I am > > trying to understand what the rules are. > > Well, the rule that I'd ideally set is don't break the ABI, even if it's > not obvious why breaking the ABI is a bad idea or you can't see any bad > consequences that could come from it, unless the reason for breaking the > ABI is absolutely central to the mission and purpose of Debian. > > That said, it's not like we've never shipped a binary in Debian with a > different PT_INTERP. (I vaguely remember that some programming language > uses PT_INTERP tricks for some sort of private binary scheme? Objective > CAML or something? I ran across it once years ago and can't remember the > details. Also, IIRC klibc does some sort of PT_INTERP trick in some > situations that I don't remember the details of, although I don't think it > does that with general binaries.) So I do see your point that you would > prefer the rule to be more pragmatic than that. > > My counterargument is that this proposal seems to mostly be about avoiding > having to create a symlink at a critical point in the bootstrap process, > and while it's tricky to get the timing right (and therefore kind of > annoying), the resulting usable system has to have that symlink anyway (I > think there's no disagreement about that). Not following the ABI for core > binaries seems like a scary change with unknown consquences to a bunch of > core packages to solve what looks like a relatively minor (if admittedly > annoying!) problem. > > Note that the target of PT_INTERP on Debian is *already* a symlink, to > /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 which was the multiarch path > that we actually want to use. We're already ensuring compatibility with a > symlink and I think we should just keep doing that and not venture into > these waters. It's not even a proposal, it's a discussion to see "what would actually break if X happened". That's why I'm asking for concrete use cases, rather than theoretical points. Also, I am interested in what the rules are. For example, glibc compatibility is just as important if not more, why does that follow a different standard? If anything, adding a missing symlink is trivial and a single command. Adding a missing symbol to a shared object... not quite. There is an endless list of downstream-only debianisms that break cross-compatibility in just the same way (if not worse), and yet nobody cares about those. Why? > >> The world does not divide neatly into supported and unsupported use > >> cases. There are a lot of things I do to computers that I expect to > >> work in some situations but not in others. That includes, say, having > >> a Debian chroot on some other OS and running binaries from that chroot > >> without going into the chroot. Often that will absolutely not work. > >> Sometimes it will work, and it's convenient that it will work for some > >> obscure recovery situations or other weird one-off use cases. I've > >> also copied files from working systems to broken systems running a > >> different distribution before, and there's a list of caveats as long as > >> my arm, but sometimes it's a quick fix for something. > > > Vast, vast majority of binaries from existing packages will already > > not work out of the box in that use case though. > > I'm not sure why you think this is true and it makes me wonder if maybe my > intuition about cross-distribution compatibility is wrong. I would expect > to be able to copy, say, most (all?) binaries from coreutils from a Debian > system to some other distribution and run it (and that's exactly the sort > of binary that is useful in this kind of cross-distribution rescue case). > Is this not true today? What breaks? > > Note that we're not talking about complicated packages with lots of > runtime like, say, Emacs. As I understand it your proposal wouldn't > change PT_INTERP for that binary anyway. We're presumably talking about > the kind of binaries that you need to bootstrap a minimal system, so > packages like coreutils or bash. And I would indeed expect those binaries > to be generally portable, as long as the same critical shared libraries > are available on other systems (in this case, PCRE2 and ncurses). Is that really the case? Let's test that hypothesis: root@focal:/tmp# grep VERSION /etc/os-release VERSION="20.04 LTS (Focal Fossa)" VERSION_ID="20.04" VERSION_CODENAME=focal root@focal:/tmp# wget http://ftp.uk.debian.org/debian/pool/main/c/coreutils/coreutils_9.1-1_amd64.deb --2023-05-16 02:07:48-- http://ftp.uk.debian.org/debian/pool/main/c/coreutils/coreutils_9.1-1_amd64.deb Resolving ftp.uk.debian.org (ftp.uk.debian.org)... 2001:1b40:5600:ff80:f8ee::1, 78.129.164.123 Connecting to ftp.uk.debian.org (ftp.uk.debian.org)|2001:1b40:5600:ff80:f8ee::1|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2896560 (2.8M) [application/octet-stream] Saving to: 'coreutils_9.1-1_amd64.deb' coreutils_9.1-1_amd64.deb 100%[===============================================>] 2.76M 9.66MB/s in 0.3s 2023-05-16 02:07:48 (9.66 MB/s) - 'coreutils_9.1-1_amd64.deb' saved [2896560/2896560] root@focal:/tmp# dpkg -x coreutils_9.1-1_amd64.deb cu root@focal:/tmp# ./cu/usr/bin/stat --help ./cu/usr/bin/stat: /lib/x86_64-linux-gnu/libselinux.so.1: no version information available (required by ./cu/usr/bin/stat) ./cu/usr/bin/stat: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./cu/usr/bin/stat) ./cu/usr/bin/stat: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./cu/usr/bin/stat) Whoops. Does this make coreutils rc-buggy now? ;-) > > Why are the existing incompatibilities allowed, and this isn't? > > Because it breaks the ABI. I know you don't find that answer satisfying, > but that really is my answer. Its purpose is compatibility. If there's no compatibility in the first place, what's its purpose? Or to put it in another way, wouldn't it be more useful to talk about compatibility requirements instead? > > I am afraid I am a bit more "pragmatic" than that. I am very interested > > in what works and what doesn't. Whether it conforms to the letter of the > > Sacred Ancient Texts... interesting for sure, but secondary. > > Right, this is our point of fundamental disagreement. > > > Let me ask a simple policy question: why do I have to abide to your > > uncodified, unwritten use case of running a Debian program from a Debian > > package from outside a Debian chroot on a foreign, unmodified non-Debian > > system, but you don't have to abide by my uncodified, unwritten use case > > of running a Debian program on a Debian system with only a Debian /usr? > > Well, the concrete answer here is that in order to make this change, > you're going to have to convince a bunch of Debian core package > maintainers to go along with this change and build their binaries in ways > that break the ABI. I believe at least some of them are going to have the > same reaction that Steve and I had, and will require a lot of convincing. > > > And also, why it is only this change that has to abide to such use case, > > and the myriad of cases that cannot possibly work in such a setup are > > given a free pass (I mean I'm pretty sure the only executables that are > > guaranteed to work as you mention are golang and rustlang, where > > everything is statically compiled, everything else is already rc-buggy > > by that standard)? > > I really would be surprised if coreutils didn't work, but maybe I'm wrong? > It does appear to generally have a PCRE2 requirement, but I would expect a > compatible PCRE2 in other distributions of a similar vintage. > > > Isn't this the very reason we have a Policy for, so that we don't get to > > cherry-pick arbitrary use cases to block things we don't like? > > Policy indeed probably doesn't say that you have to follow the Linux ABI > for normal binaries that aren't doing weird language-ecosystem-specific > things, but that's partly because it's so foundational and so automatic in > normal uses of compilers and linkers that I don't think we ever had any > reason to put it into Policy. It certainly didn't occur to me that anyone > would want to change the ABI for a subset of Debian packages. And it probably should _not_ say anything about it. However, if cross-distribution compatibility is a core requirement, if not for the whole archive for a subset of it, shouldn't that be defined and written down in the policy? > Also, I thought you didn't care about Sacred Ancient Texts like Policy. :) Policy continuously changes, so it's not really Sacred, is it now :-) > > Are you really sure we want to be in a place where anybody can bring up > > an out-of-policy use case as a valid reason to block something they > > don't like? > > Yes, I absolutely want to be in a place where anyone can raise any reason > that matters to them in public discussion as a reason not to do something > they think would be harmful! This is the whole point of working > collaboratively together on a shared endeavor. Everyone gets a say! That > doesn't mean they get a *veto*, but they get a *say*, and that say is > weighted by how much perceived expertise they have and how directly the > plan affects their work in Debian. It did look like a veto to me. More importantly, isn't relying on passersby to spot alleged harmful changes dangerous, especially for undocumented, uncodified and untested use cases, like unspecified and vague cross-compatibility requirements? > I personally am certainly not expecting to have a veto or even that strong > of a say here. This doesn't affect any of my packages so far as I can > tell, and I'm far from an expert on the ABI. I've just been around for > long enough to pick up something by osmosis. I mostly jumped in because > it felt like you and Steve were just yelling at each other and I thought I > might be able to explain some of where he was coming from in a way that > may make more sense. I don't believe I've done any yelling here. > > Again, I am fine if we want to say that Debian-specific changes and > > Debianisms are bad and we cannot do anything that jeopardises > > interoperability, cross-distribution harmony and mutual compatibility. > > But let's do that then, and start by writing it down in Policy so that > > it applies fairly and equally to all cases. > > I would love to get into a situation where Policy is a comprehensive guide > to everything people would like to have rules about in Debian, but I am > also pretty sure that if I quit my day job and made Policy my full-time > job, that would still not be done in ten years. > > Distributions are complicated with a lot of moving parts and a lot of > those aren't written down. This is why we have people with substantial > personal expertise around who can think through novel situations and try > to figure out what the possible options and consequences are. It's a good > thing! > > My starting point is that "follow the ABI" is like a safety interlock. > Whenever you park a car on a hill, you set the parking brake. You don't > try to figure out whether this hill is steep enough to make the car roll, > or do complex geometry calculations to try to figure out where the car > might roll too; you just set the parking brake always. That makes it a > habit, which has its own valuable properties. It means that you set the > parking brake in a bunch of places where it's unnecessary to do so in some > objective sense, but who cares, it's a habit, it's automatic. > > You're saying "we don't need to set the parking brake here." You might be > right! I'm thinking through the negative consequences of that, but I'm > not saying that the specific examples that have come to mind so far are > all that compelling. My primary argument is that the logic of always > setting the parking brake and not thinking about it is what's compelling, > and you're asking us to give up a point of consistency that acts like a > safety interlock, and I'm saying that makes me very uncomfortable because > it requires we go through this complex process of figuring out what's > might go wrong and we also create an inconsistency at a core level of the > distribution that may have unforseen consequences. It's a lot easier > mentally and conceptually to just always set the parking brake, and > complexity is the enemy of any large software project. What if "setting the parking brake" is not enough, as the wheels are already off and rolling downhill, as shown above, because while everybody was looking at the parking brakes lever somebody ran off with the bolts that kept the wheels attached? Why is it worth worrying about compatibility with something that is already not compatible, and it's not expected to be compatible for almost all other aspects? Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Steve McIntyre <steve@einval.com> |
|---|---|
| Date | 2023-05-15 03:00 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvuJb-8XsO-3@gated-at.bofh.it> |
| In reply to | #1146748 |
On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote: >On Sun, 14 May 2023 at 22:37, Josh Triplett <josh@joshtriplett.org> wrote: > >> The x86-64 ABI is set. Feel free to make the case to the next >> architecture designer that their new ABI should have the dynamic linker >> in `/usr/lib`. That would *not* have the same downsides, as long as >> everyone agrees on a path. > >In practice it is not, though. There are other distributions that >change PT_INTERP for their own purposes, they've already been listed >in this thread. And I am still not hearing any concrete, factual use >case that would be impaired by such a change. I'm beginning to >seriously think there aren't any? Is that really the case? The ABI has been agreed and set down in documentation that *just about* everybody has been following since its inception. This includes the most basic set of definitions of what an x86-64 program must look like, including the interpreter path. If this path is changed, then *at the most basic level* we'd be making programs that are not valid by the ABI we've agreed to. This is an *external interface contract*, not something we should ever consider changing without significant cross- and inter-project discussion. Pointing at gentoo or nixos as examples of projects that have decided to break compatibility doesn't cut it, I'm afraid. They're well known for changing fundamental things around Linux and (basically) not caring about interoperability. That attitude is *not* Debian's. -- Steve McIntyre, Cambridge, UK. steve@einval.com "The problem with defending the purity of the English language is that English is about as pure as a cribhouse whore. We don't just borrow words; on occasion, English has pursued other languages down alleyways to beat them unconscious and rifle their pockets for new vocabulary." -- James D. Nicoll
[toc] | [prev] | [next] | [standalone]
| From | Johannes Schauer Marin Rodrigues <josch@debian.org> |
|---|---|
| Date | 2023-05-15 07:00 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvyjL-905x-7@gated-at.bofh.it> |
| In reply to | #1146758 |
[Multipart message — attachments visible in raw view] — view raw
Hi, Quoting Steve McIntyre (2023-05-15 02:54:02) > On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote: > >On Sun, 14 May 2023 at 22:37, Josh Triplett <josh@joshtriplett.org> wrote: > > > >> The x86-64 ABI is set. Feel free to make the case to the next > >> architecture designer that their new ABI should have the dynamic linker > >> in `/usr/lib`. That would *not* have the same downsides, as long as > >> everyone agrees on a path. > > > >In practice it is not, though. There are other distributions that > >change PT_INTERP for their own purposes, they've already been listed > >in this thread. And I am still not hearing any concrete, factual use > >case that would be impaired by such a change. I'm beginning to > >seriously think there aren't any? Is that really the case? > > The ABI has been agreed and set down in documentation that *just > about* everybody has been following since its inception. This includes > the most basic set of definitions of what an x86-64 program must look > like, including the interpreter path. If this path is changed, then > *at the most basic level* we'd be making programs that are not valid > by the ABI we've agreed to. This is an *external interface contract*, > not something we should ever consider changing without significant > cross- and inter-project discussion. > > Pointing at gentoo or nixos as examples of projects that have decided > to break compatibility doesn't cut it, I'm afraid. They're well known > for changing fundamental things around Linux and (basically) not caring about > interoperability. That attitude is *not* Debian's. me and Luca have different ideas about how bootstrapping a Debian chroot should look like and I don't want to make an argument *for* changing PT_INTERP here as I think that keeping compatibility with others by following ABI is a good thing and because I think (and hope -- but Helmut is doing that analysis right now) that the debootstrap problem can be solved in a way I envision without changing PT_INTERP. But what I do not understand about the argument against Luca's proposal is: Obviously, with Luca's proposal, binaries from packages built with a different dynamic linker path in them would not work on distributions without merged-/usr symlinks. But if the property of stuff from Debian being able to run on non-Debian non-merged-/usr systems is an important one, then why was it okay to have merged-/usr as the default? Because with merged-/usr we already changed the interface contract for a lot of things because now binaries and libraries can also be found at other locations than on non-merged-/usr systems. A script with a /usr/bin/bash shebang built on and for Debian will not work on a system without the symlinks. So did we not years ago decide, that the result of the "cross- and inter-project discussion" is, that everybody is going merged-/usr and that's why we need it too and that's why it is okay to build a system where binaries and scripts built for it just may not run on those other systems that do not do it? With merged-/usr we already *did* "change fundamental things around" for reasons that are really not clear to me (but which i do not want to discuss here) and as a result did not "care about interoperability" (with those who do not also adopt it). In my own Debian work I so far only got extra work because of merged-/usr and I do not see the benefits (yet) and I was hoping that "changing fundamental things around Linux and (basically) not caring about interoperability" was *not* Debian's attitude but alas here we are. So have we not already burned the bridges to the non-merged-/usr world? Why was it okay back then to say "we can make this change because all other important players are doing merged-/usr so we can/have to as well". And now in the PT_INTERP discussion somehow we care again about those systems? I thought we already had the "cross- and inter-project discussion" about merged-/usr and because the result was "yes, go for it" we did it too. But if that is the case, why do we now care for a subset of the interoperability problems caused by merged-/usr for systems that don't have it? As I said, I don't care much about the PT_INTERP value but I don't understand yet, why this argument about interoperability with non-merged-/usr systems is working now but it didn't wasn't enough to stop another very fundamental change in how we build a Linux distro. Thanks! cheers, josch
[toc] | [prev] | [next] | [standalone]
| From | Bdale Garbee <bdale@gag.com> |
|---|---|
| Date | 2023-05-15 15:00 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvFXX-94J5-1@gated-at.bofh.it> |
| In reply to | #1146771 |
[Multipart message — attachments visible in raw view] — view raw
Merged-/usr seems to me to have brought great pain with no discernable benefit to Debian so far, and I at least have completely lost the thread on what the point of doing it was supposed to be. So, using it as a justification for further harm to user and system expectations isn't compelling. Bdale On May 14, 2023 10:48:04 PM MDT, Johannes Schauer Marin Rodrigues <josch@debian.org> wrote: >Hi, > >Quoting Steve McIntyre (2023-05-15 02:54:02) >> On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote: >> >On Sun, 14 May 2023 at 22:37, Josh Triplett <josh@joshtriplett.org> wrote: >> > >> >> The x86-64 ABI is set. Feel free to make the case to the next >> >> architecture designer that their new ABI should have the dynamic linker >> >> in `/usr/lib`. That would *not* have the same downsides, as long as >> >> everyone agrees on a path. >> > >> >In practice it is not, though. There are other distributions that >> >change PT_INTERP for their own purposes, they've already been listed >> >in this thread. And I am still not hearing any concrete, factual use >> >case that would be impaired by such a change. I'm beginning to >> >seriously think there aren't any? Is that really the case? >> >> The ABI has been agreed and set down in documentation that *just >> about* everybody has been following since its inception. This includes >> the most basic set of definitions of what an x86-64 program must look >> like, including the interpreter path. If this path is changed, then >> *at the most basic level* we'd be making programs that are not valid >> by the ABI we've agreed to. This is an *external interface contract*, >> not something we should ever consider changing without significant >> cross- and inter-project discussion. >> >> Pointing at gentoo or nixos as examples of projects that have decided >> to break compatibility doesn't cut it, I'm afraid. They're well known >> for changing fundamental things around Linux and (basically) not caring about >> interoperability. That attitude is *not* Debian's. > >me and Luca have different ideas about how bootstrapping a Debian chroot should >look like and I don't want to make an argument *for* changing PT_INTERP here as >I think that keeping compatibility with others by following ABI is a good thing >and because I think (and hope -- but Helmut is doing that analysis right now) >that the debootstrap problem can be solved in a way I envision without changing >PT_INTERP. But what I do not understand about the argument against Luca's >proposal is: > >Obviously, with Luca's proposal, binaries from packages built with a different >dynamic linker path in them would not work on distributions without merged-/usr >symlinks. But if the property of stuff from Debian being able to run on >non-Debian non-merged-/usr systems is an important one, then why was it okay to >have merged-/usr as the default? Because with merged-/usr we already changed >the interface contract for a lot of things because now binaries and libraries >can also be found at other locations than on non-merged-/usr systems. A script >with a /usr/bin/bash shebang built on and for Debian will not work on a system >without the symlinks. > >So did we not years ago decide, that the result of the "cross- and >inter-project discussion" is, that everybody is going merged-/usr and that's >why we need it too and that's why it is okay to build a system where binaries >and scripts built for it just may not run on those other systems that do not do >it? With merged-/usr we already *did* "change fundamental things around" for >reasons that are really not clear to me (but which i do not want to discuss >here) and as a result did not "care about interoperability" (with those who do >not also adopt it). In my own Debian work I so far only got extra work because >of merged-/usr and I do not see the benefits (yet) and I was hoping that >"changing fundamental things around Linux and (basically) not caring about >interoperability" was *not* Debian's attitude but alas here we are. > >So have we not already burned the bridges to the non-merged-/usr world? Why was >it okay back then to say "we can make this change because all other important >players are doing merged-/usr so we can/have to as well". And now in the >PT_INTERP discussion somehow we care again about those systems? I thought we >already had the "cross- and inter-project discussion" about merged-/usr and >because the result was "yes, go for it" we did it too. But if that is the case, >why do we now care for a subset of the interoperability problems caused by >merged-/usr for systems that don't have it? > >As I said, I don't care much about the PT_INTERP value but I don't understand >yet, why this argument about interoperability with non-merged-/usr systems is >working now but it didn't wasn't enough to stop another very fundamental change >in how we build a Linux distro. > >Thanks! > >cheers, josch
[toc] | [prev] | [next] | [standalone]
| From | Luca Boccassi <bluca@debian.org> |
|---|---|
| Date | 2023-05-15 15:20 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvGhj-955j-1@gated-at.bofh.it> |
| In reply to | #1146814 |
On Mon, 15 May 2023 at 13:51, Bdale Garbee <bdale@gag.com> wrote: > > Merged-/usr seems to me to have brought great pain with no discernable benefit to Debian so far, and I at least have completely lost the thread on what the point of doing it was supposed to be. So, using it as a justification for further harm to user and system expectations isn't compelling. Are you able to provide an example of such "harm"? Kind regards, Luca Boccassi
[toc] | [prev] | [next] | [standalone]
| From | Bdale Garbee <bdale@gag.com> |
|---|---|
| Date | 2023-05-15 18:00 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvIM9-96q1-5@gated-at.bofh.it> |
| In reply to | #1146819 |
[Multipart message — attachments visible in raw view] — view raw
I could. Can you provide an example of actual value delivered to Debian from merged-/usr? Bdale On May 15, 2023 7:15:53 AM MDT, Luca Boccassi <bluca@debian.org> wrote: >On Mon, 15 May 2023 at 13:51, Bdale Garbee <bdale@gag.com> wrote: >> >> Merged-/usr seems to me to have brought great pain with no discernable benefit to Debian so far, and I at least have completely lost the thread on what the point of doing it was supposed to be. So, using it as a justification for further harm to user and system expectations isn't compelling. > >Are you able to provide an example of such "harm"? > >Kind regards, >Luca Boccassi >
[toc] | [prev] | [next] | [standalone]
| From | Matthew Vernon <matthew@debian.org> |
|---|---|
| Date | 2023-05-15 18:20 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvJ5v-96M6-15@gated-at.bofh.it> |
| In reply to | #1146831 |
On 15/05/2023 16:54, Bdale Garbee wrote: > I could. > > Can you provide an example of actual value delivered to Debian from > merged-/usr? With respect, I don't think this line of argument is going to get us very far - this bug isn't about whether we should undo usr-merge, so I don't think a debate on the merits or otherwise of usr-merge is germane. [I think this applies to the contrary side of the argument also] Regards, Matthew
[toc] | [prev] | [next] | [standalone]
| From | Bdale Garbee <bdale@gag.com> |
|---|---|
| Date | 2023-05-15 18:30 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvJfb-96Py-3@gated-at.bofh.it> |
| In reply to | #1146834 |
[Multipart message — attachments visible in raw view] — view raw
You are of course correct. I remain unconvinced that anything related to the work on merged-/usr to date should be considered as a positive justification for actions discussed in this thread, but we can just let the rest drop. Bdale On May 15, 2023 10:08:00 AM MDT, Matthew Vernon <matthew@debian.org> wrote: >On 15/05/2023 16:54, Bdale Garbee wrote: >> I could. >> >> Can you provide an example of actual value delivered to Debian from merged-/usr? > >With respect, I don't think this line of argument is going to get us very far - this bug isn't about whether we should undo usr-merge, so I don't think a debate on the merits or otherwise of usr-merge is germane. > >[I think this applies to the contrary side of the argument also] > >Regards, > >Matthew >
[toc] | [prev] | [next] | [standalone]
| From | Sam Hartman <hartmans@debian.org> |
|---|---|
| Date | 2023-05-15 20:40 +0200 |
| Subject | Bug#1035904: What does merged /usr bring us |
| Message-ID | <GvLgZ-981u-7@gated-at.bofh.it> |
| In reply to | #1146836 |
Hi. Off list, I wanted to try to explain what I think merged /usr has brought us that is positive. I want to stress that I'm not a huge fan of merged /usr, and I know you've encouraged me not to argue from a devil's advocate position in the past. All the things I cite here are things I actually think are a positive value, but I fully agree they do not justify the change to me. * Normalization of paths. Whether things ended up in /usr/bin or /bin tended to vary between unixes and especially between linux distros. That has produced a lot of complexity over the years in build systems. But it's also produced a lot of non-portable software--stuff written either for Redhat or Ubuntu that manages to get the path wrong for the other, and doesn't have the complexity of something like autoconf to detect the change. It's kind of nice to ignore all that. * There's work, from people related to the systemd crowd, to effectively get to a place where the initial state of a system is very close to empty with a few symlinks and a read-only /usr. And then possibly things get filled in a bit on first boot if necessary. So, for example the systemd style split of /usr/lib/systemd/system vs /etc/systemd/system where the /etc path generally gets to be empty at least initially is part of this. In some ways it reminds me of how AIX dealt with diskless workstations. (I realize that's based on how Sun did it, but I never dove into SunOS or Solaris as deep as AIX). The goal is more for containers or for deploying updates to EOT devices than for diskless. The ideais kind of cool. The updater/installer/container creater knows very little and can start with an initial state. It's easy to figure out what has been customized because anything in /etc is a customization. doing a factory reset is fairly easy. It works well with signed OS images and ostree for deploying updates/the sort of thing Endless does. And yet there are probably other ways to get all the same benefits. --Sam
[toc] | [prev] | [next] | [standalone]
| From | Sam Hartman <hartmans@debian.org> |
|---|---|
| Date | 2023-05-15 20:40 +0200 |
| Subject | Bug#1035904: What does merged /usr bring us |
| Message-ID | <GvLh0-981u-21@gated-at.bofh.it> |
| In reply to | #1146853 |
>>>>> "Sam" == Sam Hartman <hartmans@debian.org> writes:
Sam> Hi. Off list, I wanted to try to explain what I think merged
My apology for sending a mail intended to be private to the bug. It was
not my intent to clutter an already cluttered discussion. I was really
just trying to help provide what understanding I had to a friend.
--Sam
[toc] | [prev] | [next] | [standalone]
| From | Jonathan Carter <jcc@debian.org> |
|---|---|
| Date | 2023-05-15 20:50 +0200 |
| Subject | Bug#1035904: What does merged /usr bring us |
| Message-ID | <GvLqF-984M-3@gated-at.bofh.it> |
| In reply to | #1146853 |
On 2023/05/15 20:26, Sam Hartman wrote: > I want to stress that I'm not a huge fan of merged /usr, and I know > you've encouraged me not to argue from a devil's advocate position in > the past. And this is where I stop reading any further. To merge or not to merge is no longer an interesting or more importantly, a relevant discussion. The project has made the choice to implement it, after we've been one of the last major distributions to hold out on implementing it. Sure, it's been bumpy, and there's potential downsides, but none of the technical problems are as harmful as still trying to make this a debate. So, please, file bugs or fix bugs and support the people who need to complete the remaining bits of merged-/usr, or otherwise please move on. -Jonathan
[toc] | [prev] | [next] | [standalone]
| From | Sam Hartman <hartmans@debian.org> |
|---|---|
| Date | 2023-05-15 19:00 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvJId-96ZO-1@gated-at.bofh.it> |
| In reply to | #1146834 |
>>>>> "Matthew" == Matthew Vernon <matthew@debian.org> writes:
Matthew> On 15/05/2023 16:54, Bdale Garbee wrote:
>> I could.
>>
>> Can you provide an example of actual value delivered to Debian
>> from merged-/usr?
Matthew> With respect, I don't think this line of argument is going
Matthew> to get us very far - this bug isn't about whether we should
Matthew> undo usr-merge, so I don't think a debate on the merits or
Matthew> otherwise of usr-merge is germane.
I actually think the whole ABI issue is basically irrelevant for this
bug.
I think that in this bug at least we are hoping to discuss the
appropriateness of the dpkg warning for sources downstreams grab from
the Debian archivje.
That issue seems quite sufficient for one bug:-)
[toc] | [prev] | [next] | [standalone]
| From | Steve McIntyre <steve@einval.com> |
|---|---|
| Date | 2023-05-15 15:40 +0200 |
| Subject | Bug#1035904: dpkg currently warning about merged-usr systems (revisited) |
| Message-ID | <GvGAF-95bP-7@gated-at.bofh.it> |
| In reply to | #1146771 |
Hey Johannes, On Mon, May 15, 2023 at 06:48:04AM +0200, Johannes Schauer Marin Rodrigues wrote: >Quoting Steve McIntyre (2023-05-15 02:54:02) >> >> Pointing at gentoo or nixos as examples of projects that have decided >> to break compatibility doesn't cut it, I'm afraid. They're well known >> for changing fundamental things around Linux and (basically) not caring about >> interoperability. That attitude is *not* Debian's. > >me and Luca have different ideas about how bootstrapping a Debian chroot should >look like and I don't want to make an argument *for* changing PT_INTERP here as >I think that keeping compatibility with others by following ABI is a good thing >and because I think (and hope -- but Helmut is doing that analysis right now) >that the debootstrap problem can be solved in a way I envision without changing >PT_INTERP. But what I do not understand about the argument against Luca's >proposal is: > >Obviously, with Luca's proposal, binaries from packages built with a different >dynamic linker path in them would not work on distributions without merged-/usr >symlinks. But if the property of stuff from Debian being able to run on >non-Debian non-merged-/usr systems is an important one, then why was it okay to >have merged-/usr as the default? Because with merged-/usr we already changed >the interface contract for a lot of things because now binaries and libraries >can also be found at other locations than on non-merged-/usr systems. A script >with a /usr/bin/bash shebang built on and for Debian will not work on a system >without the symlinks. Despite the massive upheavals of merged-/usr in *other* ways, it's actually a *minor* change as far as compatibility is concerned here. The runtime linker (aka interpreter) is responsible for resolving symbols and finding needed libraries. So long as *that* bit works OK, then ~everything else should follow OK. This is how it's possible to have things work across distros: binaries don't actually care where the libraries live, or whether they came from rpm or deb packaging, etc. The issue at hand here is that the interpreter path is the most basic thing that matters for this compatibility. Break this and *nothing* can work. >So did we not years ago decide, that the result of the "cross- and >inter-project discussion" is, that everybody is going merged-/usr and that's >why we need it too and that's why it is okay to build a system where binaries >and scripts built for it just may not run on those other systems that do not do >it? With merged-/usr we already *did* "change fundamental things around" for >reasons that are really not clear to me (but which i do not want to discuss >here) and as a result did not "care about interoperability" (with those who do >not also adopt it). In my own Debian work I so far only got extra work because >of merged-/usr and I do not see the benefits (yet) and I was hoping that >"changing fundamental things around Linux and (basically) not caring about >interoperability" was *not* Debian's attitude but alas here we are. > >So have we not already burned the bridges to the non-merged-/usr world? Why was >it okay back then to say "we can make this change because all other important >players are doing merged-/usr so we can/have to as well". And now in the >PT_INTERP discussion somehow we care again about those systems? I thought we >already had the "cross- and inter-project discussion" about merged-/usr and >because the result was "yes, go for it" we did it too. But if that is the case, >why do we now care for a subset of the interoperability problems caused by >merged-/usr for systems that don't have it? This change is absolutely *not* needed to make merged-/usr work; if anybody is claiming that it is, then they are not being 100% honest with us. All the other distros doing merged-/usr have done it without making this change, and it's also been working OK for us so far without this change. Breaking an agreed interface contract like this is axiomatically *wrong* and will hurt us and the rest of the Linux ecosystem. -- Steve McIntyre, Cambridge, UK. steve@einval.com "You can't barbecue lettuce!" -- Ellie Crane
[toc] | [prev] | [next] | [standalone]
Page 2 of 4 — ← Prev page 1 [2] 3 4 Next page →
Back to top | Article view | linux.debian.bugs.dist
csiph-web