Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1484696 > unrolled thread

[tip regression] efi: Allow drivers to reserve boot services forever == toxic

Started byMike Galbraith <umgwanakikbuti@gmail.com>
First post2016-09-16 08:10 +0200
Last post2016-09-19 13:20 +0200
Articles 10 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-09-16 08:10 +0200
    Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Matt Fleming <matt@codeblueprint.co.uk> - 2016-09-16 11:40 +0200
      Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Matt Fleming <matt@codeblueprint.co.uk> - 2016-09-16 16:40 +0200
        Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Mark Rutland <mark.rutland@arm.com> - 2016-09-16 16:50 +0200
          Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-09-16 20:30 +0200
            Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Mark Rutland <mark.rutland@arm.com> - 2016-09-19 11:00 +0200
        Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-09-16 18:40 +0200
          Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Matt Fleming <matt@codeblueprint.co.uk> - 2016-09-17 22:00 +0200
            Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-09-18 08:10 +0200
              Re: [tip regression]  efi: Allow drivers to reserve boot services  forever == toxic Matt Fleming <matt@codeblueprint.co.uk> - 2016-09-19 13:20 +0200

#1484696 — [tip regression] efi: Allow drivers to reserve boot services forever == toxic

FromMike Galbraith <umgwanakikbuti@gmail.com>
Date2016-09-16 08:10 +0200
Subject[tip regression] efi: Allow drivers to reserve boot services forever == toxic
Message-ID<shUvv-O3-3@gated-at.bofh.it>
Hi Matt,

My workstation started instant rebooting with tip.  I bisected it to..

   efi/esrt: Use efi_mem_reserve() and avoid a kmalloc()

..but seems it's really $subject, as box works fine with the below.

---
 drivers/firmware/efi/efi.c |    1 +
 1 file changed, 1 insertion(+)

--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -415,6 +415,7 @@ void __init efi_mem_reserve(phys_addr_t
 {
 	if (!memblock_is_region_reserved(addr, size))
 		memblock_reserve(addr, size);
+	else
 
 	/*
 	 * Some architectures (x86) reserve all boot services ranges


No idea what's going boom down in efi_arch_mem_reserve().

[toc] | [next] | [standalone]

#1484841

FromMatt Fleming <matt@codeblueprint.co.uk>
Date2016-09-16 11:40 +0200
Message-ID<shXMJ-2PA-9@gated-at.bofh.it>
In reply to#1484696
On Fri, 16 Sep, at 08:05:12AM, Mike Galbraith wrote:
> Hi Matt,
> 
> My workstation started instant rebooting with tip.  I bisected it to..
> 
>    efi/esrt: Use efi_mem_reserve() and avoid a kmalloc()
> 
> ..but seems it's really $subject, as box works fine with the below.
> 
> ---
>  drivers/firmware/efi/efi.c |    1 +
>  1 file changed, 1 insertion(+)
> 
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -415,6 +415,7 @@ void __init efi_mem_reserve(phys_addr_t
>  {
>  	if (!memblock_is_region_reserved(addr, size))
>  		memblock_reserve(addr, size);
> +	else
>  
>  	/*
>  	 * Some architectures (x86) reserve all boot services ranges
> 
> 
> No idea what's going boom down in efi_arch_mem_reserve().

Urgh, that's bad. Which machine is this? Can you paste the dmesg after
booting with the efi=debug kernel parameter, either with your above
patch or with the commit reverted?

[toc] | [prev] | [next] | [standalone]

#1485058

FromMatt Fleming <matt@codeblueprint.co.uk>
Date2016-09-16 16:40 +0200
Message-ID<si2t3-5We-7@gated-at.bofh.it>
In reply to#1484841
On Fri, 16 Sep, at 12:00:59PM, Mike Galbraith wrote:
> 
> Ok, here's the whole thing just in case.  Hope it's not too big.
 
[...]

> [    0.000000] esrt: Reserving ESRT space from 0x00000000def87998 to 0x00000000def879d0.

OK, that's 56 bytes and yet I realise that at no point in the
efi_mem_reserve() call path do we round up to the nearest page size
even though the EFI memory map only deals with EFI_PAGE_SIZE regions.

Could you try this patch?

-------->8--------

From a24b38999965808355c461f8490670266162eabf Mon Sep 17 00:00:00 2001
From: Matt Fleming <matt@codeblueprint.co.uk>
Date: Fri, 16 Sep 2016 15:12:47 +0100
Subject: [PATCH] x86/efi: Round EFI memmap reservations to EFI_PAGE_SIZE

Mike reported that his machine started rebooting during boot after,

  commit 8e80632fb23f ("efi/esrt: Use efi_mem_reserve() and avoid a kmalloc()")

The ESRT table on his machine is 56 bytes and at no point in the
efi_arch_mem_reserve() call path is that size rounded up to
EFI_PAGE_SIZE.

Since the EFI memory map only deals with whole pages, inserting an EFI
memory region with 56 bytes results in a new entry covering zero
pages, and completely screws up the calculations for the old regions
that were trimmed.

Round all sizes upwards to the nearest EFI_PAGE_SIZE boundary.

Additionally, efi_memmap_insert() expects the mem::range::end value to
be one less than the end address for the region.

Reported-by: Mike Galbraith <umgwanakikbuti@gmail.com>
Cc: Peter Jones <pjones@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>
---
 arch/x86/platform/efi/quirks.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c
index f14b7a9da24b..e881b4b2ffd6 100644
--- a/arch/x86/platform/efi/quirks.c
+++ b/arch/x86/platform/efi/quirks.c
@@ -201,8 +201,10 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)
 		return;
 	}
 
+	size = round_up(size, EFI_PAGE_SIZE);
+
 	mr.range.start = addr;
-	mr.range.end = addr + size;
+	mr.range.end = addr + size - 1;
 	mr.attribute = md.attribute | EFI_MEMORY_RUNTIME;
 
 	num_entries = efi_memmap_split_count(&md, &mr.range);
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1485065

FromMark Rutland <mark.rutland@arm.com>
Date2016-09-16 16:50 +0200
Message-ID<si2CK-608-13@gated-at.bofh.it>
In reply to#1485058
On Fri, Sep 16, 2016 at 03:30:07PM +0100, Matt Fleming wrote:
> On Fri, 16 Sep, at 12:00:59PM, Mike Galbraith wrote:
> > 
> > Ok, here's the whole thing just in case.  Hope it's not too big.
>  
> [...]
> 
> > [    0.000000] esrt: Reserving ESRT space from 0x00000000def87998 to 0x00000000def879d0.
> 
> OK, that's 56 bytes and yet I realise that at no point in the
> efi_mem_reserve() call path do we round up to the nearest page size
> even though the EFI memory map only deals with EFI_PAGE_SIZE regions.

I note that the base is also not aligned to EFI_PAGE_SIZE. Shouldn't we
round that down, too?

[...]

> diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c
> index f14b7a9da24b..e881b4b2ffd6 100644
> --- a/arch/x86/platform/efi/quirks.c
> +++ b/arch/x86/platform/efi/quirks.c
> @@ -201,8 +201,10 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)
>  		return;
>  	}
>  
> +	size = round_up(size, EFI_PAGE_SIZE);

i.e. have:

	size += addr % EFI_PAGE_SIZE;
	size = round_up(size, EFI_PAGE_SIZE);
	addr = round_down(base, EFI_PAGE_SIZE);

Thanks,
Mark.

[toc] | [prev] | [next] | [standalone]

#1485230

FromMike Galbraith <umgwanakikbuti@gmail.com>
Date2016-09-16 20:30 +0200
Message-ID<si63E-8pm-15@gated-at.bofh.it>
In reply to#1485065
On Fri, 2016-09-16 at 15:45 +0100, Mark Rutland wrote:

> > diff --git a/arch/x86/platform/efi/quirks.c
> > b/arch/x86/platform/efi/quirks.c
> > index f14b7a9da24b..e881b4b2ffd6 100644
> > --- a/arch/x86/platform/efi/quirks.c
> > +++ b/arch/x86/platform/efi/quirks.c
> > @@ -201,8 +201,10 @@ void __init efi_arch_mem_reserve(phys_addr_t
> > addr, u64 size)
> >  		return;
> >  	}
> >  
> > +	size = round_up(size, EFI_PAGE_SIZE);
> 
> i.e. have:
> 
> 	size += addr % EFI_PAGE_SIZE;
> 	size = round_up(size, EFI_PAGE_SIZE);
> 	addr = round_down(base, EFI_PAGE_SIZE);
                          ^^^^

s/base/addr did work, was that what you meant to write, or am I lucky? 
--- a/arch/x86/platform/efi/quirks.c
+++ b/arch/x86/platform/efi/quirks.c
@@ -201,8 +201,12 @@ void __init efi_arch_mem_reserve(phys_ad
 		return;
 	}
 
+	size += addr % EFI_PAGE_SIZE;
+	size = round_up(size, EFI_PAGE_SIZE);
+	addr = round_down(addr, EFI_PAGE_SIZE);
+
 	mr.range.start = addr;
-	mr.range.end = addr + size;
+	mr.range.end = addr + size - 1;
 	mr.attribute = md.attribute | EFI_MEMORY_RUNTIME;
 
 	num_entries = efi_memmap_split_count(&md, &mr.range);

[toc] | [prev] | [next] | [standalone]

#1486274

FromMark Rutland <mark.rutland@arm.com>
Date2016-09-19 11:00 +0200
Message-ID<sj2AF-3cw-19@gated-at.bofh.it>
In reply to#1485230
On Fri, Sep 16, 2016 at 08:26:16PM +0200, Mike Galbraith wrote:
> On Fri, 2016-09-16 at 15:45 +0100, Mark Rutland wrote:
> 
> > > diff --git a/arch/x86/platform/efi/quirks.c
> > > b/arch/x86/platform/efi/quirks.c
> > > index f14b7a9da24b..e881b4b2ffd6 100644
> > > --- a/arch/x86/platform/efi/quirks.c
> > > +++ b/arch/x86/platform/efi/quirks.c
> > > @@ -201,8 +201,10 @@ void __init efi_arch_mem_reserve(phys_addr_t
> > > addr, u64 size)
> > >  		return;
> > >  	}
> > >  
> > > +	size = round_up(size, EFI_PAGE_SIZE);
> > 
> > i.e. have:
> > 
> > 	size += addr % EFI_PAGE_SIZE;
> > 	size = round_up(size, EFI_PAGE_SIZE);
> > 	addr = round_down(base, EFI_PAGE_SIZE);
>                           ^^^^
> 
> s/base/addr did work, was that what you meant to write, or am I lucky? 

That was what I meant to write, though the two statements aren't
necessarily mutually exclusive. ;)

I'll have to leave it to you and Matt to figure out what the real
solution is -- I'm not familiar with this code and was just
pattern-matching.

Thanks,
Mark.

[toc] | [prev] | [next] | [standalone]

#1485169

FromMike Galbraith <umgwanakikbuti@gmail.com>
Date2016-09-16 18:40 +0200
Message-ID<si4lc-7e8-31@gated-at.bofh.it>
In reply to#1485058
On Fri, 2016-09-16 at 15:30 +0100, Matt Fleming wrote:
> On Fri, 16 Sep, at 12:00:59PM, Mike Galbraith wrote:
> > 
> > Ok, here's the whole thing just in case.  Hope it's not too big.
>  
> [...]
> 
> > [    0.000000] esrt: Reserving ESRT space from 0x00000000def87998 to 0x00000000def879d0.
> 
> OK, that's 56 bytes and yet I realise that at no point in the
> efi_mem_reserve() call path do we round up to the nearest page size
> even though the EFI memory map only deals with EFI_PAGE_SIZE regions.
> 
> Could you try this patch?

No dice.  The numbers changed, but box still goes *poof*.

---
 arch/x86/platform/efi/quirks.c |    4 ++++
 1 file changed, 4 insertions(+)

--- a/arch/x86/platform/efi/quirks.c
+++ b/arch/x86/platform/efi/quirks.c
@@ -227,7 +227,11 @@ void __init efi_arch_mem_reserve(phys_ad
 	efi_memmap_insert(&efi.memmap, new, &mr);
 	early_memunmap(new, new_size);
 
+#if 0
 	efi_memmap_install(new_phys, num_entries);
+#else
+	printk(KERN_DEBUG "MIKE efi_memmap_install(0x%Lx, %d);\n", new_phys, num_entries);
+#endif
 }
 
 /*

homer:..debug/tracing # journalctl|grep MIKE
virgin+above
Sep 16 15:51:15 homer kernel: MIKE efi_memmap_install(0x9d640, 51);
Sep 16 15:55:07 homer kernel: MIKE efi_memmap_install(0x9d640, 51);
+patch+above
Sep 16 18:13:46 homer kernel: MIKE efi_memmap_install(0x9d600, 53);

[toc] | [prev] | [next] | [standalone]

#1485602

FromMatt Fleming <matt@codeblueprint.co.uk>
Date2016-09-17 22:00 +0200
Message-ID<sitWi-6G2-9@gated-at.bofh.it>
In reply to#1485169
On Fri, 16 Sep, at 06:34:55PM, Mike Galbraith wrote:
> On Fri, 2016-09-16 at 15:30 +0100, Matt Fleming wrote:
> > On Fri, 16 Sep, at 12:00:59PM, Mike Galbraith wrote:
> > > 
> > > Ok, here's the whole thing just in case.  Hope it's not too big.
> >  
> > [...]
> > 
> > > [    0.000000] esrt: Reserving ESRT space from 0x00000000def87998 to 0x00000000def879d0.
> > 
> > OK, that's 56 bytes and yet I realise that at no point in the
> > efi_mem_reserve() call path do we round up to the nearest page size
> > even though the EFI memory map only deals with EFI_PAGE_SIZE regions.
> > 
> > Could you try this patch?
> 
> No dice.  The numbers changed, but box still goes *poof*.
> 
> ---
>  arch/x86/platform/efi/quirks.c |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> --- a/arch/x86/platform/efi/quirks.c
> +++ b/arch/x86/platform/efi/quirks.c
> @@ -227,7 +227,11 @@ void __init efi_arch_mem_reserve(phys_ad
>  	efi_memmap_insert(&efi.memmap, new, &mr);
>  	early_memunmap(new, new_size);
>  
> +#if 0
>  	efi_memmap_install(new_phys, num_entries);
> +#else
> +	printk(KERN_DEBUG "MIKE efi_memmap_install(0x%Lx, %d);\n", new_phys, num_entries);
> +#endif
>  }
>  
>  /*
> 
> homer:..debug/tracing # journalctl|grep MIKE
> virgin+above
> Sep 16 15:51:15 homer kernel: MIKE efi_memmap_install(0x9d640, 51);
> Sep 16 15:55:07 homer kernel: MIKE efi_memmap_install(0x9d640, 51);
> +patch+above
> Sep 16 18:13:46 homer kernel: MIKE efi_memmap_install(0x9d600, 53);

These addresses are pretty low. Can you try the hacky patch plus
Ricardo's change in commit 3dad6f7f6975 ("x86/efi: Defer efi_esrt_init
until after memblock_x86_fill"). He fixed a bug where it's possible to
run out of memblock regions.

[toc] | [prev] | [next] | [standalone]

#1485692

FromMike Galbraith <umgwanakikbuti@gmail.com>
Date2016-09-18 08:10 +0200
Message-ID<siDsB-4oB-1@gated-at.bofh.it>
In reply to#1485602
On Sat, 2016-09-17 at 20:58 +0100, Matt Fleming wrote:

> These addresses are pretty low. Can you try the hacky patch plus
> Ricardo's change in commit 3dad6f7f6975 ("x86/efi: Defer efi_esrt_init
> until after memblock_x86_fill"). He fixed a bug where it's possible to
> run out of memblock regions.

---
 arch/x86/platform/efi/quirks.c |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

--- a/arch/x86/platform/efi/quirks.c
+++ b/arch/x86/platform/efi/quirks.c
@@ -201,8 +201,12 @@ void __init efi_arch_mem_reserve(phys_ad
 		return;
 	}
 
+//XXX	size += addr % EFI_PAGE_SIZE;
+//MATT	size = round_up(size, EFI_PAGE_SIZE);
+//XXX	addr = round_down(addr, EFI_PAGE_SIZE);
+
 	mr.range.start = addr;
-	mr.range.end = addr + size;
+	mr.range.end = addr + size;//MATT - 1;
 	mr.attribute = md.attribute | EFI_MEMORY_RUNTIME;
 
 	num_entries = efi_memmap_split_count(&md, &mr.range);
@@ -225,7 +229,12 @@ void __init efi_arch_mem_reserve(phys_ad
 	efi_memmap_insert(&efi.memmap, new, &mr);
 	early_memunmap(new, new_size);
 
+#if 0
 	efi_memmap_install(new_phys, num_entries);
+#endif
+	printk(KERN_DEBUG "MIKE md.phys_addr:0x%Lx  md.virt_addr:0x%Lx  md.num_pages:%Ld\n", md.phys_addr, md.virt_addr, md.num_pages);
+	printk(KERN_DEBUG "MIKE mr.range.start:0x%Lx  mr.range.end:0x%Lx\n", mr.range.start, mr.range.end);
+	printk(KERN_DEBUG "MIKE efi_memmap_install(0x%Lx, %d);\n", new_phys, num_entries);
 }
 
 /*

tip v4.8-rc6-654-g0d941b5745e4

virgin
[    0.000000] MIKE md.phys_addr:0xded81000  md.virt_addr:0x0  md.num_pages:520
[    0.000000] MIKE mr.range.start:0xdef87998  mr.range.end:0xdef879d0
[    0.000000] MIKE efi_memmap_install(0x9d640, 51);

+MATT changes
[    0.000000] MIKE md.phys_addr:0xded81000  md.virt_addr:0x0  md.num_pages:520
[    0.000000] MIKE mr.range.start:0xdef87998  mr.range.end:0xdef88997
[    0.000000] MIKE efi_memmap_install(0x9d600, 53);

+MATT changes + XXX changes (boots/works without #if 0)
[    0.000000] MIKE md.phys_addr:0xded81000  md.virt_addr:0x0  md.num_pages:520
[    0.000000] MIKE mr.range.start:0xdef87000  mr.range.end:0xdef87fff
[    0.000000] MIKE efi_memmap_install(0x9d640, 51);

[toc] | [prev] | [next] | [standalone]

#1486382

FromMatt Fleming <matt@codeblueprint.co.uk>
Date2016-09-19 13:20 +0200
Message-ID<sj4Ma-4KI-55@gated-at.bofh.it>
In reply to#1485692
On Sun, 18 Sep, at 08:09:32AM, Mike Galbraith wrote:
> 
> +MATT changes + XXX changes (boots/works without #if 0)
> [    0.000000] MIKE md.phys_addr:0xded81000  md.virt_addr:0x0  md.num_pages:520
> [    0.000000] MIKE mr.range.start:0xdef87000  mr.range.end:0xdef87fff
> [    0.000000] MIKE efi_memmap_install(0x9d640, 51);

Brilliant. This is what I've got queued up. Thanks everyone.

----8<----

From 7e750e3289a44fe3ad693bde45aea1ad8577dd2a Mon Sep 17 00:00:00 2001
From: Matt Fleming <matt@codeblueprint.co.uk>
Date: Fri, 16 Sep 2016 15:12:47 +0100
Subject: [PATCH] x86/efi: Round EFI memmap reservations to EFI_PAGE_SIZE

Mike Galbraith reported that his machine started rebooting during boot
after,

  commit 8e80632fb23f ("efi/esrt: Use efi_mem_reserve() and avoid a kmalloc()")

The ESRT table on his machine is 56 bytes and at no point in the
efi_arch_mem_reserve() call path is that size rounded up to
EFI_PAGE_SIZE, nor is the start address on an EFI_PAGE_SIZE boundary.

Since the EFI memory map only deals with whole pages, inserting an EFI
memory region with 56 bytes results in a new entry covering zero
pages, and completely screws up the calculations for the old regions
that were trimmed.

Round all sizes upwards, and start addresses downwards, to the nearest
EFI_PAGE_SIZE boundary.

Additionally, efi_memmap_insert() expects the mem::range::end value to
be one less than the end address for the region.

Reported-by: Mike Galbraith <umgwanakikbuti@gmail.com>
Reported-by: Mike Krinkin <krinkin.m.u@gmail.com>
Cc: Peter Jones <pjones@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Taku Izumi <izumi.taku@jp.fujitsu.com>
Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>
---
 arch/x86/platform/efi/quirks.c |  6 +++++-
 drivers/firmware/efi/memmap.c  | 11 +++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c
index f14b7a9da24b..10aca63a50d7 100644
--- a/arch/x86/platform/efi/quirks.c
+++ b/arch/x86/platform/efi/quirks.c
@@ -201,8 +201,12 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)
 		return;
 	}
 
+	size += addr % EFI_PAGE_SIZE;
+	size = round_up(size, EFI_PAGE_SIZE);
+	addr = round_down(addr, EFI_PAGE_SIZE);
+
 	mr.range.start = addr;
-	mr.range.end = addr + size;
+	mr.range.end = addr + size - 1;
 	mr.attribute = md.attribute | EFI_MEMORY_RUNTIME;
 
 	num_entries = efi_memmap_split_count(&md, &mr.range);
diff --git a/drivers/firmware/efi/memmap.c b/drivers/firmware/efi/memmap.c
index cd96086fd851..f03ddecd232b 100644
--- a/drivers/firmware/efi/memmap.c
+++ b/drivers/firmware/efi/memmap.c
@@ -225,6 +225,17 @@ void __init efi_memmap_insert(struct efi_memory_map *old_memmap, void *buf,
 	m_end = mem->range.end;
 	m_attr = mem->attribute;
 
+	/*
+	 * The EFI memory map deals with regions in EFI_PAGE_SIZE
+	 * units. Ensure that the region described by 'mem' is aligned
+	 * correctly.
+	 */
+	if (!IS_ALIGNED(m_start, EFI_PAGE_SIZE) ||
+	    !IS_ALIGNED(m_end + 1, EFI_PAGE_SIZE)) {
+		WARN_ON(1);
+		return;
+	}
+
 	for (old = old_memmap->map, new = buf;
 	     old < old_memmap->map_end;
 	     old += old_memmap->desc_size, new += old_memmap->desc_size) {
-- 
2.9.3

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web