Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1569031 > unrolled thread

[PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h Ingo Molnar <mingo@kernel.org> - 2017-01-28 23:20 +0100
    Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Sam Ravnborg <sam@ravnborg.org> - 2017-01-29 18:20 +0100
      Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Ingo Molnar <mingo@kernel.org> - 2017-01-30 09:00 +0100
        Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Sam Ravnborg <sam@ravnborg.org> - 2017-01-31 06:50 +0100
          Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Ingo Molnar <mingo@kernel.org> - 2017-01-31 17:50 +0100
            Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Sam Ravnborg <sam@ravnborg.org> - 2017-01-31 18:30 +0100
              Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Joe Perches <joe@perches.com> - 2017-01-31 19:10 +0100
              Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Ingo Molnar <mingo@kernel.org> - 2017-01-31 19:10 +0100
                Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Sam Ravnborg <sam@ravnborg.org> - 2017-01-31 20:30 +0100
                  Re: [PATCH 01/50] x86/boot/e820: Introduce  arch/x86/include/asm/e820/types.h Ingo Molnar <mingo@kernel.org> - 2017-02-01 10:00 +0100

#1569031 — [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

First baby steps towards saner e820 headers: create an exact copy of
arch/x86/include/uapi/asm/e820.h and use it from the asm/e820.h file.

No other changes - this is done to decouple the code from UAPI headers,
plus to make sure that subsequent modifications to the file can be more
clearly seen.

The plan is to keep the old UAPI header in place but the kernel won't
use it anymore - and after some time we'll try to remove it. (User-space
tools better have local copies of headers anyway, instead of relying
on kernel headers.)

This gives the kernel the freedom to reorganize the e820 code.

Cc: Alex Thorlton <athorlton@sgi.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Huang, Ying <ying.huang@intel.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Jackson <pj@sgi.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rafael J. Wysocki <rjw@sisk.pl>
Cc: Tejun Heo <tj@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wei Yang <richard.weiyang@gmail.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/include/asm/e820.h       |  4 +--
 arch/x86/include/asm/e820/types.h | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 83 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/e820.h b/arch/x86/include/asm/e820.h
index 67313f3a9874..55c1d76c169f 100644
--- a/arch/x86/include/asm/e820.h
+++ b/arch/x86/include/asm/e820.h
@@ -5,12 +5,12 @@
  * E820_X_MAX is the maximum size of the extended E820 table.  The extended
  * table may contain up to 3 extra E820 entries per possible NUMA node, so we
  * make room for 3 * MAX_NUMNODES possible entries, beyond the standard 128.
- * Also note that E820_X_MAX *must* be defined before we include uapi/asm/e820.h.
+ * Also note that E820_X_MAX *must* be defined before we include asm/e820/types.h.
  */
 #include <linux/numa.h>
 #define E820_X_MAX (E820MAX + 3 * MAX_NUMNODES)
 
-#include <uapi/asm/e820.h>
+#include <asm/e820/types.h>
 
 #ifndef __ASSEMBLY__
 /* see comment in arch/x86/kernel/e820.c */
diff --git a/arch/x86/include/asm/e820/types.h b/arch/x86/include/asm/e820/types.h
new file mode 100644
index 000000000000..9dafe59cf6e2
--- /dev/null
+++ b/arch/x86/include/asm/e820/types.h
@@ -0,0 +1,81 @@
+#ifndef _UAPI_ASM_X86_E820_H
+#define _UAPI_ASM_X86_E820_H
+#define E820MAP	0x2d0		/* our map */
+#define E820MAX	128		/* number of entries in E820MAP */
+
+/*
+ * Legacy E820 BIOS limits us to 128 (E820MAX) nodes due to the
+ * constrained space in the zeropage.  If we have more nodes than
+ * that, and if we've booted off EFI firmware, then the EFI tables
+ * passed us from the EFI firmware can list more nodes.  Size our
+ * internal memory map tables to have room for these additional
+ * nodes, based on up to three entries per node for which the
+ * kernel was built: MAX_NUMNODES == (1 << CONFIG_NODES_SHIFT),
+ * plus E820MAX, allowing space for the possible duplicate E820
+ * entries that might need room in the same arrays, prior to the
+ * call to sanitize_e820_map() to remove duplicates.  The allowance
+ * of three memory map entries per node is "enough" entries for
+ * the initial hardware platform motivating this mechanism to make
+ * use of additional EFI map entries.  Future platforms may want
+ * to allow more than three entries per node or otherwise refine
+ * this size.
+ */
+
+#ifndef __KERNEL__
+#define E820_X_MAX E820MAX
+#endif
+
+#define E820NR	0x1e8		/* # entries in E820MAP */
+
+#define E820_RAM	1
+#define E820_RESERVED	2
+#define E820_ACPI	3
+#define E820_NVS	4
+#define E820_UNUSABLE	5
+#define E820_PMEM	7
+
+/*
+ * This is a non-standardized way to represent ADR or NVDIMM regions that
+ * persist over a reboot.  The kernel will ignore their special capabilities
+ * unless the CONFIG_X86_PMEM_LEGACY option is set.
+ *
+ * ( Note that older platforms also used 6 for the same type of memory,
+ *   but newer versions switched to 12 as 6 was assigned differently.  Some
+ *   time they will learn... )
+ */
+#define E820_PRAM	12
+
+/*
+ * reserved RAM used by kernel itself
+ * if CONFIG_INTEL_TXT is enabled, memory of this type will be
+ * included in the S3 integrity calculation and so should not include
+ * any memory that BIOS might alter over the S3 transition
+ */
+#define E820_RESERVED_KERN        128
+
+#ifndef __ASSEMBLY__
+#include <linux/types.h>
+struct e820entry {
+	__u64 addr;	/* start of memory segment */
+	__u64 size;	/* size of memory segment */
+	__u32 type;	/* type of memory segment */
+} __attribute__((packed));
+
+struct e820map {
+	__u32 nr_map;
+	struct e820entry map[E820_X_MAX];
+};
+
+#define ISA_START_ADDRESS	0xa0000
+#define ISA_END_ADDRESS		0x100000
+
+#define BIOS_BEGIN		0x000a0000
+#define BIOS_END		0x00100000
+
+#define BIOS_ROM_BASE		0xffe00000
+#define BIOS_ROM_END		0xffffffff
+
+#endif /* __ASSEMBLY__ */
+
+
+#endif /* _UAPI_ASM_X86_E820_H */
-- 
2.7.4

[toc] | [next] | [standalone]

#1569234 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

On Sat, Jan 28, 2017 at 11:11:22PM +0100, Ingo Molnar wrote:
> 
> The plan is to keep the old UAPI header in place but the kernel won't
> use it anymore - and after some time we'll try to remove it. (User-space
> tools better have local copies of headers anyway, instead of relying
> on kernel headers.)

The idea with uapi is the the kernel provides a sane set of headers
to be used by user space.
So we avoid random copies that is maintained by random people in random
ways resulting in random bugs.

The step(s) outlined here can only result in inconsistency and
cannot benefit neither user space nor the kernel in the long run.

The uapi shall be lean and clean headers, and shall include
no info whatsoever that is not relevant for user space.

But requiring all user space programs (diverse libc variants,
other programs) to maintain their own copy can only result in
inconsistencies that is the benefit for no one.

	Sam

[toc] | [prev] | [next] | [standalone]

#1569449 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

* Sam Ravnborg <sam@ravnborg.org> wrote:

> On Sat, Jan 28, 2017 at 11:11:22PM +0100, Ingo Molnar wrote:
> > 
> > The plan is to keep the old UAPI header in place but the kernel won't
> > use it anymore - and after some time we'll try to remove it. (User-space
> > tools better have local copies of headers anyway, instead of relying
> > on kernel headers.)
> 
> The idea with uapi is the the kernel provides a sane set of headers
> to be used by user space.
> So we avoid random copies that is maintained by random people in random
> ways resulting in random bugs.

Your argument is simplistic which presents a false dichotomy: maintaining a copy 
or fully sharing the header are not the only two options available to share the 
information in the headers between the kernel and tooling: for example perf uses a 
half-automated method where headers are copied from the kernel, but also checked 
automatically against the upstream kernel, and a (non-fatal) warning is emitted 
during the build if the upstream header has changed.

For example today the perf build shows these UAPI header warnings:

 Warning: arch/powerpc/include/uapi/asm/kvm.h differs from kernel
 Warning: arch/arm/include/uapi/asm/kvm.h differs from kernel

... because new bits were added to those two UAPI headers. For example the new ARM 
bits were:

triton:~/tip/tools/perf> diff -up ../arch/arm/include/uapi/asm/kvm.h ../../arch/arm/include/uapi/asm/kvm.h
--- ../arch/arm/include/uapi/asm/kvm.h  2017-01-23 10:10:18.846003002 +0100
+++ ../../arch/arm/include/uapi/asm/kvm.h       2017-01-28 09:35:12.383587930 +0100
@@ -84,6 +84,15 @@ struct kvm_regs {
 #define KVM_VGIC_V2_DIST_SIZE          0x1000
 #define KVM_VGIC_V2_CPU_SIZE           0x2000
 
+/* Supported VGICv3 address types  */
+#define KVM_VGIC_V3_ADDR_TYPE_DIST     2
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST   3
+#define KVM_VGIC_ITS_ADDR_TYPE         4
+
+#define KVM_VGIC_V3_DIST_SIZE          SZ_64K
+#define KVM_VGIC_V3_REDIST_SIZE                (2 * SZ_64K)
+#define KVM_VGIC_V3_ITS_SIZE           (2 * SZ_64K)
+
 #define KVM_ARM_VCPU_POWER_OFF         0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_PSCI_0_2          1 /* CPU uses PSCI v0.2 */
 
... so for these changes the perf side copy can be updated safely. Had the changes 
been more intricate, the changes can be copied too - while adopting the tooling 
source code as well.

See the tools/perf/check-headers.sh script.

This IMHO is a far more intelligent and far more robust approach than blind 
sharing or detached copies, because it:

 - forces new changes from upstream to be considered and adapted by tooling

 - header (and thus ABI) synchronization is guaranteed (eventually)

 - it does not actually couple the two source code bases in a rigid fashion:

 - the upstream kernel is free to change those headers (at least from perf's POV)
   in any sane way, those changes can be adapted.

 - every step is conscious and there's no way to accidentally break tooling via
   header changes - nor does tooling hinder the kernel from progressing its source 
   code base.

It's basically a script based COW filesystem with guaranteed propagation and 
guaranteed synchronization.

> The step(s) outlined here can only result in inconsistency and
> cannot benefit neither user space nor the kernel in the long run.

That's simply not true, see above.

> The uapi shall be lean and clean headers, and shall include
> no info whatsoever that is not relevant for user space.

I agree with that characterization, and that will be even more so with my changes: 
my series makes uapi/asm/bootparam.h more self-contained, more lean - while still 
defining the full ABI.

> But requiring all user space programs (diverse libc variants,
> other programs) to maintain their own copy can only result in
> inconsistencies that is the benefit for no one.

That's simply not true, see above.

Note that my changes try to keep the 'UAPI promise' (in that the old e820.h header 
is still around), while still modifying the kernel side.

What _IS_ insane is to somehow construe the UAPI headers as a rigid construct that 
forces the kernel source to keep using poorly chosen names like 'struct e820entry' 
forever...

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1570354 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

On Mon, Jan 30, 2017 at 08:58:33AM +0100, Ingo Molnar wrote:
> 
> * Sam Ravnborg <sam@ravnborg.org> wrote:
> 
> > On Sat, Jan 28, 2017 at 11:11:22PM +0100, Ingo Molnar wrote:
> > > 
> > > The plan is to keep the old UAPI header in place but the kernel won't
> > > use it anymore - and after some time we'll try to remove it. (User-space
> > > tools better have local copies of headers anyway, instead of relying
> > > on kernel headers.)
> > 
> > The idea with uapi is the the kernel provides a sane set of headers
> > to be used by user space.
> > So we avoid random copies that is maintained by random people in random
> > ways resulting in random bugs.
> 
> Your argument is simplistic which presents a false dichotomy: maintaining a copy 
> or fully sharing the header are not the only two options available to share the 
> information in the headers between the kernel and tooling: for example perf uses a 
> half-automated method where headers are copied from the kernel, but also checked 
> automatically against the upstream kernel, and a (non-fatal) warning is emitted 
> during the build if the upstream header has changed.

Obvious when there are two vastly different solutions there are some
middle ground solutions too.
And perf implements a tool based method to check that the manually
copied/created/maintained headers are in sync with the kernel.
which is just another way to maintain a set of copied headers.

But this does not at all address the point which is that it is
an unessesary burden to put on all the _users_ of the kernel uapi
headers to amintain their own copy of the kernel headers
(tool assisted or not).

The rationale behind requesting the users of the kernel uapi headers
seems to be that the kernel people would like to have the freedom
to fix wrong doings in the past without violatign the uapi.
So the benefit here is simpler headers on the kernel side compared
to all users of kernel uapi heders to manually (tool assisted or not)
maintaining their own copies of the uapi kernel headers.

Heck - if this is how it should be done then the the kernel
should provide the copy of the headers that describe the
uapi and the kernel should have the tool infrastructure to check
that the headers are OK.
Not all the users outside the kernelspace.

So just to repeat - it is an error prone design to let users
of the kernel uapi maintain their own copies of the kernel
uapi header. It is the job of the kernel.

	Sam

[toc] | [prev] | [next] | [standalone]

#1570891 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

* Sam Ravnborg <sam@ravnborg.org> wrote:

> Obvious when there are two vastly different solutions there are some
> middle ground solutions too.
>
> And perf implements a tool based method to check that the manually 
> copied/created/maintained headers are in sync with the kernel. which is just 
> another way to maintain a set of copied headers.
> 
> But this does not at all address the point which is that it is an unessesary 
> burden to put on all the _users_ of the kernel uapi headers to amintain their 
> own copy of the kernel headers (tool assisted or not).

In 99% of the cases it's trivial to keep them updated on the tooling side. 
Basically a copy only has to be made if a change is done - and changes are not 
that common in the ABI space.

> The rationale behind requesting the users of the kernel uapi headers seems to be 
> that the kernel people would like to have the freedom to fix wrong doings in the 
> past without violatign the uapi.
>
> So the benefit here is simpler headers on the kernel side compared
> to all users of kernel uapi heders to manually (tool assisted or not)
> maintaining their own copies of the uapi kernel headers.

I was reacting to this original claim of yours:

> > > The idea with uapi is the the kernel provides a sane set of headers to be 
> > > used by user space.
> > >
> > > So we avoid random copies that is maintained by random people in random ways 
> > > resulting in random bugs.

There's no 'random copies' - they are copies of past, valid versions of the kernel 
headers, and there's a mechanism to update them to the latest.

> Heck - if this is how it should be done then the the kernel
> should provide the copy of the headers that describe the
> uapi and the kernel should have the tool infrastructure to check
> that the headers are OK.
> Not all the users outside the kernelspace.
> 
> So just to repeat - it is an error prone design to let users
> of the kernel uapi maintain their own copies of the kernel
> uapi header. It is the job of the kernel.

But "random copies" is not what perf does. Tell me, how is the perf mechanism of 
using the headers "error-prone"? It's a delayed COW mechanism - COW is not an 
error-prone concept in any way ...

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1570921 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

> > 
> > So just to repeat - it is an error prone design to let users
> > of the kernel uapi maintain their own copies of the kernel
> > uapi header. It is the job of the kernel.
> 
> But "random copies" is not what perf does. Tell me, how is the perf mechanism of 
> using the headers "error-prone"? It's a delayed COW mechanism - COW is not an 
> error-prone concept in any way ...

The whole concept that user space have the burden to maintain
a set of headers describing the uapi provided by the kernel is the point
of discussion.

The randomness come into play when a user space developer are
faced with the challenge that the programm require access to something
described by the kernel uapi and then have to hunt for a header
that describes said uapi.

In this thread we have covered one rational reason to push thus
burden to user space - to give the kernel the freedom to repair
past stupidity (being that in naming or some other sort).

So lets turn around the arguments - and from a user space
perspective what is the benefit of maintaining a set of headers
describing the kernel uapi?

Obviously this allows user space to name thing exactly the
way they like, and allows user space to put all sorts of strange
things in the header files describing the kernel uapi.

This is just not enough good reasons why the user space
developer shall create headers files describing
the kerneluapi and maintain tooling to maintain the header
files describing the kernel uapi.

Are there other benefits that is missed which makes the
concept of letting user space maintain header files describing
the kernel uapi a good idea that is missed?

	Sam

[toc] | [prev] | [next] | [standalone]

#1570955 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

On Tue, 2017-01-31 at 19:00 +0100, Ingo Molnar wrote:
> * Sam Ravnborg <sam@ravnborg.org> wrote:
> > In this thread we have covered one rational reason to push thus
> > burden to user space - to give the kernel the freedom to repair
> > past stupidity (being that in naming or some other sort).
> 
> There's no real "burden" for heaven's sake: it's having to execute a 'cp' every 
> now and then and check whether the result still builds (it will build just fine in 
> the overwhelming majority of cases).

Defining what is it that encompasses "userspace", the
thing that's nominally guaranteed to "not be broken",
here is the real question.

Is it just the ABI or is it more than that?

[toc] | [prev] | [next] | [standalone]

#1570959 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

* Sam Ravnborg <sam@ravnborg.org> wrote:

> > > 
> > > So just to repeat - it is an error prone design to let users
> > > of the kernel uapi maintain their own copies of the kernel
> > > uapi header. It is the job of the kernel.
> > 
> > But "random copies" is not what perf does. Tell me, how is the perf mechanism of 
> > using the headers "error-prone"? It's a delayed COW mechanism - COW is not an 
> > error-prone concept in any way ...
> 
> The whole concept that user space have the burden to maintain a set of headers 
> describing the uapi provided by the kernel is the point of discussion.

That's your characterisation and I think it's misleading: they don't have to 
"maintain a set of headers", they can, if they'd like to:

   "maintain a _copy_ of a small number of headers that the typical user-space 
    tooling project is interested in"

which is different from how you framed it.

> The randomness come into play when a user space developer are faced with the 
> challenge that the programm require access to something described by the kernel 
> uapi and then have to hunt for a header that describes said uapi.
>
> In this thread we have covered one rational reason to push thus
> burden to user space - to give the kernel the freedom to repair
> past stupidity (being that in naming or some other sort).

There's no real "burden" for heaven's sake: it's having to execute a 'cp' every 
now and then and check whether the result still builds (it will build just fine in 
the overwhelming majority of cases).

> So lets turn around the arguments - and from a user space
> perspective what is the benefit of maintaining a set of headers
> describing the kernel uapi?

Firsty, the headers are not maintained by the user-space project, 99.999% of the 
maintenance is done by the kernel developers.

> Obviously this allows user space to name thing exactly the way they like, and 
> allows user space to put all sorts of strange things in the header files 
> describing the kernel uapi.

Huh? Again, my suggestion is to to _copy_ the kernel header the tooling project is 
interested in as-is, and this is exactly what perf does. We don't change the UAPI 
headers, we copy them verbatim and check during the build that the copy is exact.

> Are there other benefits that is missed which makes the
> concept of letting user space maintain header files describing
> the kernel uapi a good idea that is missed?

Yes, you missed a lot of the benefits.

Firstly, the user-space tooling project that relies on some UAPI header with Linux 
kernel ABI details in it, if it so wishes, maintains a _copy_ of the affected 
headers, which is vastly less work and 'burden' than 'maintaining headers'.

Secondly, doing this copy has numerous advantages over using the UAPI headers from 
the distro installation:

 - Additions to the ABI are monitored by the tooling project. Say there's 
   external tooling that relies on the perf system call, and uses a copy of 
   perf_events.h. It can see new ABI additions in new versions of the file, and 
   can add support for them. If on the other hand the file is just shared with 
   whatever distro kernel headers there are on the system, there's no information 
   about ABI progress.

   Advantage: changes to the ABI can be seen in the tool's Git repository,
              the "git log" of the header shows how the versions got updated and 
              how the tool adopted to new ABI additions.

 - The tooling project is more self-contained: it can build just fine in more 
   limited environments that don't have the kernel headers installed.

   Advantage: more robust tool build process, fewer dependencies.

 - So the tooling project can copy the kernel header _before_ the updated kernel
   header trickles through to distros. This has the advantage of being able to add
   tooling support for new ABI additions before it all goes into the 'official'
   UAPI headers a distro includes, and speeds up ABI propagation.

   Advantage: faster ABI propagation to tooling.

 - In fact this model allows the tooling project to host new kernel ABI additions 
   in its development process, so sometimes it has the new UAPI header _before_ 
   the upstream kernel has it. (Obviously the upstream kernel has to pull it for 
   it to become the official UAPI header, so this only works for subsystems where
   there's good communication between the tooling developers and the kernel 
   developers.)

   This is what tools/perf does quite often.

   Advantage: better development model where tooling can develop new kernel 
              features hand in hand with kernel developers.

I.e. most of the advantages of the copied headers are for the _tooling project_, 
not for the kernel.

Your one sided characterization of the copying method as a 'burden' ignores all 
that.

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1571024 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

Hi Ingo.

> >
> > In this thread we have covered one rational reason to push thus
> > burden to user space - to give the kernel the freedom to repair
> > past stupidity (being that in naming or some other sort).
> 
> There's no real "burden" for heaven's sake: it's having to execute a 'cp' every 
> now and then and check whether the result still builds (it will build just fine in 
> the overwhelming majority of cases).

We are obvious so far away in our perception of what is easy for
a user-space developers that is is not even funny.

> > So lets turn around the arguments - and from a user space
> > perspective what is the benefit of maintaining a set of headers
> > describing the kernel uapi?
> 
> Firsty, the headers are not maintained by the user-space project, 99.999% of the 
> maintenance is done by the kernel developers.

In the inital mail triggering this plan was that the kernel
is moving away from having uapi headers what-so-ever.

Quoting the original mail:
"
The plan is to keep the old UAPI header in place but the kernel won't
use it anymore - and after some time we'll try to remove it. 
"

Translated:
The plan is that the kernel will stop using headers from uapi/*
The headers will be left for a while and then they will be deleted.

The mail was centered about e820 - so maybe the outlined plan
was only for e820.
But I read it as a general pan - hence this mail thread.

So this is the plan that this argument is about.
No tooling will magically make files appear again.
And there is no benefit from user space that the kernel remove the files.

And the removing of files from uapi/ makes is hard for user space.
Because then a user-space developers have to find a definition
of the uapi somewhere else.

> Huh? Again, my suggestion is to to _copy_ the kernel header the tooling project is 
> interested in as-is, and this is exactly what perf does.
Here the proposal was stop using the header in uapi/ and then later delete it.
So there i no file to copy - making the copying and tooling part irrelevant.

> Yes, you missed a lot of the benefits.
> 
> Firstly, the user-space tooling project that relies on some UAPI header with Linux 
> kernel ABI details in it, if it so wishes, maintains a _copy_ of the affected 
> headers, which is vastly less work and 'burden' than 'maintaining headers'.

It is by no means a benefit to have a copy, rather than using the source.
There is a reason why we do not have multiple copies of the headers in the kernel
when we can avoid it. asm-generic is one way the kernel avoid the burden
of maintaining copies of headers.

All your remaining argumens zapped - I see the point of view.
But there are many other solutions for the same set of problems.

Perf being intimidate with the kernel is not the best example to come up with.
Think about to 100's of program that uses a few ioclt to talk with drivers etc.

	Sam

[toc] | [prev] | [next] | [standalone]

#1571334 — Re: [PATCH 01/50] x86/boot/e820: Introduce arch/x86/include/asm/e820/types.h

* Sam Ravnborg <sam@ravnborg.org> wrote:

> > Firsty, the headers are not maintained by the user-space project, 99.999% of 
> > the maintenance is done by the kernel developers.
> 
> In the inital mail triggering this plan was that the kernel is moving away from 
> having uapi headers what-so-ever.

No, that is a misunderstanding:

> Quoting the original mail:
> "
> The plan is to keep the old UAPI header in place but the kernel won't
> use it anymore - and after some time we'll try to remove it. 
> "

You misunderstood my mail and you misunderstood the patch: we transition from the 
old UAPI header to a new one, but the exported data structures are still kept!

If you check the patches you'll see that bootparam.h still exports the e820_entry 
data structure. The 'old' header is simply one that is being phased out (if we 
can) - but the information is still exported.

> Translated:
> The plan is that the kernel will stop using headers from uapi/*
> The headers will be left for a while and then they will be deleted.

No, not at all.

> Perf being intimidate with the kernel is not the best example to come up with. 

No, that's wrong too, most larger tooling projects that care about feature 
propagation latency in fact already do something quite similar to what perf does.

For example the tooling side of GPU drivers (libdrm) has a copy of all the 
relevant UAPI headers:

triton:~/libdrm/include/drm> ls -l
total 316
-rw-rw-r-- 1 mingo mingo 19119 Feb  1 09:47 amdgpu_drm.h
-rw-rw-r-- 1 mingo mingo 11850 Feb  1 09:47 drm_fourcc.h
-rw-rw-r-- 1 mingo mingo 27613 Feb  1 09:47 drm.h
-rw-rw-r-- 1 mingo mingo 18313 Feb  1 09:47 drm_mode.h
-rw-rw-r-- 1 mingo mingo  2701 Feb  1 09:47 drm_sarea.h
-rw-rw-r-- 1 mingo mingo 46684 Feb  1 09:47 i915_drm.h
-rw-rw-r-- 1 mingo mingo  7895 Feb  1 09:47 mach64_drm.h
-rw-rw-r-- 1 mingo mingo 12923 Feb  1 09:47 mga_drm.h
-rw-rw-r-- 1 mingo mingo  5662 Feb  1 09:47 nouveau_drm.h
-rw-rw-r-- 1 mingo mingo  4217 Feb  1 09:47 qxl_drm.h
-rw-rw-r-- 1 mingo mingo  9901 Feb  1 09:47 r128_drm.h
-rw-rw-r-- 1 mingo mingo 38509 Feb  1 09:47 radeon_drm.h
-rw-rw-r-- 1 mingo mingo  5201 Feb  1 09:47 README
-rw-rw-r-- 1 mingo mingo  7054 Feb  1 09:47 savage_drm.h
-rw-rw-r-- 1 mingo mingo  2534 Feb  1 09:47 sis_drm.h
-rw-rw-r-- 1 mingo mingo  5526 Feb  1 09:47 tegra_drm.h
-rw-rw-r-- 1 mingo mingo  9534 Feb  1 09:47 vc4_drm.h
-rw-rw-r-- 1 mingo mingo  8291 Feb  1 09:47 via_drm.h
-rw-rw-r-- 1 mingo mingo  4704 Feb  1 09:47 virtgpu_drm.h
-rw-rw-r-- 1 mingo mingo 31225 Feb  1 09:47 vmwgfx_drm.h

For example i915_drm.h is a copy of include/uapi/drm/i915_drm.h, which is being 
synched between the two projects regularly.

> Think about to 100's of program that uses a few ioclt to talk with drivers etc.

Those can use distro UAPI headers just fine, if they don't care about the 6-12 
months delay it takes to get updated kernel headers. I.e. what you propose works 
for well-established ABIs that have been around for years.

To actually _progress_ with a tooling project, in close cooperation with the 
kernel side, the UAPI method of sharing via distro headers as-is hinders 
development agility big time...

Distro UAPI headers work fine in a world where the kernel is a static entity and 
does not update its ABIs. I.e. it only works if there's no actual kernel side 
extensions to the ABI. The whole UAPI distro headers approach is designed for the 
case where the style of sharing the headers matters the least: for a stagnant 
kernel or a stagnant tooling project ...

Btw., his kind of rigid, suboptimal, latency laden method of sharing information 
between the kernel and tooling might be one of the reasons why in general the 
Linux tooling landscape sucks, compared to other OSs...

Thanks,

	Ingo

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web