Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1646104 > unrolled thread

[PATCH v2 0/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

Back to article view | Back to linux.kernel

Contents

  [PATCH v2 0/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system Baoquan He <bhe@redhat.com> - 2017-05-20 14:10 +0200
    [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system Baoquan He <bhe@redhat.com> - 2017-05-20 14:10 +0200
      Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct  mapping section for SGI UV system Thomas Garnier <thgarnie@google.com> - 2017-05-21 22:40 +0200
        Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct  mapping section for SGI UV system Baoquan He <bhe@redhat.com> - 2017-05-22 01:20 +0200
          Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct  mapping section for SGI UV system Baoquan He <bhe@redhat.com> - 2017-05-22 01:20 +0200
            Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct  mapping section for SGI UV system Thomas Garnier <thgarnie@google.com> - 2017-05-22 19:10 +0200

#1646104 — [PATCH v2 0/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

This is v2 post.

This patchset is trying to fix a bug that SGI UV system casually hang
during boot with KASLR enabled. The root cause is that mm KASLR adapts
size of the direct mapping section only based on the system RAM size.
Then later when map SGI UV MMIOH region into the direct mapping during
rest_init() invocation, it might go beyond of the directing mapping
section and step into VMALLOC or VMEMMAP area, then BUG_ON triggered.

The fix is adding a helper function is_early_uv_system to check UV system
earlier, then call the helper function in kernel_randomize_memory() to
check if it's a SGI UV system, if yes, we keep the size of direct mapping
section to be 64TB just as nokslr.

With this fix, SGI UV system can have 64TB direct mapping size always,
and the starting address of direct mapping/vmalloc/vmemmap and the padding
between them can still be randomized to enhance the system security.

v1->v2:
    1. Mike suggested making is_early_uv_system() an inline function and be
    put in include/asm/uv/uv.h so that they can adjust them easier in the
    future.

    2. Split the v1 code into uv part and mm KASLR part as Mike suggested.

Baoquan He (2):
  x86/UV: Introduce a helper function to check UV system at earlier
    stage
  x86/mm/KASLR: Do not adapt the size of the direct mapping section for
    SGI UV system

 arch/x86/include/asm/uv/uv.h | 6 ++++++
 arch/x86/mm/kaslr.c          | 3 ++-
 2 files changed, 8 insertions(+), 1 deletion(-)

-- 
2.5.5

[toc] | [next] | [standalone]

#1646105 — [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

On SGI UV system, kernel casually hang with kaslr enabled.

The back trace is:

kernel BUG at arch/x86/mm/init_64.c:311!
invalid opcode: 0000 [#1] SMP
[...]
RIP: 0010:__init_extra_mapping+0x188/0x196
[...]
Call Trace:
 init_extra_mapping_uc+0x13/0x15
 map_high+0x67/0x75
 map_mmioh_high_uv3+0x20a/0x219
 uv_system_init_hub+0x12d9/0x1496
 uv_system_init+0x27/0x29
 native_smp_prepare_cpus+0x28d/0x2d8
 kernel_init_freeable+0xdd/0x253
 ? rest_init+0x80/0x80
 kernel_init+0xe/0x110
 ret_from_fork+0x2c/0x40

The root cause is that SGI UV system needs map its MMIOH region to direct
mapping section and the mapping happens in rest_init(). However mm KASLR
is done in kernel_randomize_memory() which is much earlier than MMIOH
mapping of SGI UV and doesn't count in the MMIOH regions. When kaslr
disabled, there are 64TB space for system RAM to do direct mapping. Both
system RAM and SGI UV MMIOH region share this 64TB space. With kaslr
enabled, mm KASLR only reserves the actual size of system RAM plus 10TB
for direct mapping usage. Then later MMIOH mapping of SGI UV could go
beyond the upper bound of direct mapping section to step into VMALLOC or
VMEMMAP area. Then the BUG_ON() in __init_extra_mapping() will be
triggered.

E.g on the SGI UV3 machine where this bug is reported , there are two MMIOH
regions:

[    1.519001] UV: Map MMIOH0_HI 0xffc00000000 - 0x100000000000
[    1.523001] UV: Map MMIOH1_HI 0x100000000000 - 0x200000000000

They are [16TB-16G, 16TB) and [16TB, 32TB). On this machine, 512G ram are
spread out to 1TB regions. Then above two SGI MMIOH regions also will be
mapped into the direct mapping section.

To fix it, we need check if it's SGI UV system by calling
is_early_uv_system() in kernel_randomize_memory(). If yes, do not adapt the
size of the direct mapping section. Do it now.

Signed-off-by: Baoquan He <bhe@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org 
Cc: Thomas Garnier <thgarnie@google.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
---
 arch/x86/mm/kaslr.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c
index aed2064..20b0456 100644
--- a/arch/x86/mm/kaslr.c
+++ b/arch/x86/mm/kaslr.c
@@ -27,6 +27,7 @@
 #include <asm/pgtable.h>
 #include <asm/setup.h>
 #include <asm/kaslr.h>
+#include <asm/uv/uv.h>
 
 #include "mm_internal.h"
 
@@ -123,7 +124,7 @@ void __init kernel_randomize_memory(void)
 		CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING;
 
 	/* Adapt phyiscal memory region size based on available memory */
-	if (memory_tb < kaslr_regions[0].size_tb)
+	if (memory_tb < kaslr_regions[0].size_tb && !is_early_uv_system())
 		kaslr_regions[0].size_tb = memory_tb;
 
 	/* Calculate entropy available between regions */
-- 
2.5.5

[toc] | [prev] | [next] | [standalone]

#1646425 — Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

On Sat, May 20, 2017 at 5:02 AM, Baoquan He <bhe@redhat.com> wrote:
> On SGI UV system, kernel casually hang with kaslr enabled.
>
> The back trace is:
>
> kernel BUG at arch/x86/mm/init_64.c:311!
> invalid opcode: 0000 [#1] SMP
> [...]
> RIP: 0010:__init_extra_mapping+0x188/0x196
> [...]
> Call Trace:
>  init_extra_mapping_uc+0x13/0x15
>  map_high+0x67/0x75
>  map_mmioh_high_uv3+0x20a/0x219
>  uv_system_init_hub+0x12d9/0x1496
>  uv_system_init+0x27/0x29
>  native_smp_prepare_cpus+0x28d/0x2d8
>  kernel_init_freeable+0xdd/0x253
>  ? rest_init+0x80/0x80
>  kernel_init+0xe/0x110
>  ret_from_fork+0x2c/0x40
>
> The root cause is that SGI UV system needs map its MMIOH region to direct
> mapping section and the mapping happens in rest_init(). However mm KASLR
> is done in kernel_randomize_memory() which is much earlier than MMIOH
> mapping of SGI UV and doesn't count in the MMIOH regions. When kaslr
> disabled, there are 64TB space for system RAM to do direct mapping. Both
> system RAM and SGI UV MMIOH region share this 64TB space. With kaslr
> enabled, mm KASLR only reserves the actual size of system RAM plus 10TB
> for direct mapping usage. Then later MMIOH mapping of SGI UV could go
> beyond the upper bound of direct mapping section to step into VMALLOC or
> VMEMMAP area. Then the BUG_ON() in __init_extra_mapping() will be
> triggered.
>
> E.g on the SGI UV3 machine where this bug is reported , there are two MMIOH
> regions:
>
> [    1.519001] UV: Map MMIOH0_HI 0xffc00000000 - 0x100000000000
> [    1.523001] UV: Map MMIOH1_HI 0x100000000000 - 0x200000000000
>
> They are [16TB-16G, 16TB) and [16TB, 32TB). On this machine, 512G ram are
> spread out to 1TB regions. Then above two SGI MMIOH regions also will be
> mapped into the direct mapping section.
>
> To fix it, we need check if it's SGI UV system by calling
> is_early_uv_system() in kernel_randomize_memory(). If yes, do not adapt the
> size of the direct mapping section. Do it now.
>
> Signed-off-by: Baoquan He <bhe@redhat.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: x86@kernel.org
> Cc: Thomas Garnier <thgarnie@google.com>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
>  arch/x86/mm/kaslr.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c
> index aed2064..20b0456 100644
> --- a/arch/x86/mm/kaslr.c
> +++ b/arch/x86/mm/kaslr.c
> @@ -27,6 +27,7 @@
>  #include <asm/pgtable.h>
>  #include <asm/setup.h>
>  #include <asm/kaslr.h>
> +#include <asm/uv/uv.h>
>
>  #include "mm_internal.h"
>
> @@ -123,7 +124,7 @@ void __init kernel_randomize_memory(void)
>                 CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING;
>
>         /* Adapt phyiscal memory region size based on available memory */
> -       if (memory_tb < kaslr_regions[0].size_tb)
> +       if (memory_tb < kaslr_regions[0].size_tb && !is_early_uv_system())

Given your example, any way we could just restrict memory_tb to be
32TB? Or different configurations will result in different mappings?

>                 kaslr_regions[0].size_tb = memory_tb;
>
>         /* Calculate entropy available between regions */
> --
> 2.5.5
>



-- 
Thomas

[toc] | [prev] | [next] | [standalone]

#1646470 — Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

On 05/21/17 at 01:38pm, Thomas Garnier wrote:
> On Sat, May 20, 2017 at 5:02 AM, Baoquan He <bhe@redhat.com> wrote:
> >  arch/x86/mm/kaslr.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c
> > index aed2064..20b0456 100644
> > --- a/arch/x86/mm/kaslr.c
> > +++ b/arch/x86/mm/kaslr.c
> > @@ -27,6 +27,7 @@
> >  #include <asm/pgtable.h>
> >  #include <asm/setup.h>
> >  #include <asm/kaslr.h>
> > +#include <asm/uv/uv.h>
> >
> >  #include "mm_internal.h"
> >
> > @@ -123,7 +124,7 @@ void __init kernel_randomize_memory(void)
> >                 CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING;
> >
> >         /* Adapt phyiscal memory region size based on available memory */
> > -       if (memory_tb < kaslr_regions[0].size_tb)
> > +       if (memory_tb < kaslr_regions[0].size_tb && !is_early_uv_system())
> 
> Given your example, any way we could just restrict memory_tb to be
> 32TB? Or different configurations will result in different mappings?

Thanks for looking into this, Thomas!

For that machine where I used to reproduce the bug and test, 32TB memory
need be mapped to the direct mapping region. I am not sure if SGI UV
system has larger MMIOH region now or in the future in different machine.
If they have machine owning MMIOH region bigger than 64TB, then it's a
problem SGI UV need fix because that will break system whether kaslr
enabled or not.

Hi Mike, Russ and Frank,

About Thomas's question, could you help answer it? Could other SGI UV
system has MMIOH region bigger than 32TB?

Thanks
Baoquan

> 
> >                 kaslr_regions[0].size_tb = memory_tb;
> >
> >         /* Calculate entropy available between regions */

[toc] | [prev] | [next] | [standalone]

#1646471 — Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

Sorry, forget 'To' Mike, Russ and Frank

On 05/22/17 at 07:14am, Baoquan He wrote:
> On 05/21/17 at 01:38pm, Thomas Garnier wrote:
> > On Sat, May 20, 2017 at 5:02 AM, Baoquan He <bhe@redhat.com> wrote:
> > >  arch/x86/mm/kaslr.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c
> > > index aed2064..20b0456 100644
> > > --- a/arch/x86/mm/kaslr.c
> > > +++ b/arch/x86/mm/kaslr.c
> > > @@ -27,6 +27,7 @@
> > >  #include <asm/pgtable.h>
> > >  #include <asm/setup.h>
> > >  #include <asm/kaslr.h>
> > > +#include <asm/uv/uv.h>
> > >
> > >  #include "mm_internal.h"
> > >
> > > @@ -123,7 +124,7 @@ void __init kernel_randomize_memory(void)
> > >                 CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING;
> > >
> > >         /* Adapt phyiscal memory region size based on available memory */
> > > -       if (memory_tb < kaslr_regions[0].size_tb)
> > > +       if (memory_tb < kaslr_regions[0].size_tb && !is_early_uv_system())
> > 
> > Given your example, any way we could just restrict memory_tb to be
> > 32TB? Or different configurations will result in different mappings?
> 
> Thanks for looking into this, Thomas!
> 
> For that machine where I used to reproduce the bug and test, 32TB memory
> need be mapped to the direct mapping region. I am not sure if SGI UV
> system has larger MMIOH region now or in the future in different machine.
> If they have machine owning MMIOH region bigger than 64TB, then it's a
> problem SGI UV need fix because that will break system whether kaslr
> enabled or not.
> 
> Hi Mike, Russ and Frank,
> 
> About Thomas's question, could you help answer it? Could other SGI UV
> system has MMIOH region bigger than 32TB?
> 
> Thanks
> Baoquan
> 
> > 
> > >                 kaslr_regions[0].size_tb = memory_tb;
> > >
> > >         /* Calculate entropy available between regions */

[toc] | [prev] | [next] | [standalone]

#1647189 — Re: [PATCH v2 2/2] x86/mm/KASLR: Do not adapt size of the direct mapping section for SGI UV system

On Mon, May 22, 2017 at 9:30 AM, Mike Travis <mike.travis@hpe.com> wrote:
>
>
> On 5/21/2017 4:17 PM, Baoquan He wrote:
>
> Sorry, forget 'To' Mike, Russ and Frank
>
> On 05/22/17 at 07:14am, Baoquan He wrote:
>
> On 05/21/17 at 01:38pm, Thomas Garnier wrote:
>
> On Sat, May 20, 2017 at 5:02 AM, Baoquan He <bhe@redhat.com> wrote:
>
>  arch/x86/mm/kaslr.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c
> index aed2064..20b0456 100644
> --- a/arch/x86/mm/kaslr.c
> +++ b/arch/x86/mm/kaslr.c
> @@ -27,6 +27,7 @@
>  #include <asm/pgtable.h>
>  #include <asm/setup.h>
>  #include <asm/kaslr.h>
> +#include <asm/uv/uv.h>
>
>  #include "mm_internal.h"
>
> @@ -123,7 +124,7 @@ void __init kernel_randomize_memory(void)
>                 CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING;
>
>         /* Adapt phyiscal memory region size based on available memory */
> -       if (memory_tb < kaslr_regions[0].size_tb)
> +       if (memory_tb < kaslr_regions[0].size_tb && !is_early_uv_system())
>
> Given your example, any way we could just restrict memory_tb to be
> 32TB? Or different configurations will result in different mappings?
>
> Thanks for looking into this, Thomas!
>
> For that machine where I used to reproduce the bug and test, 32TB memory
> need be mapped to the direct mapping region. I am not sure if SGI UV
> system has larger MMIOH region now or in the future in different machine.
> If they have machine owning MMIOH region bigger than 64TB, then it's a
> problem SGI UV need fix because that will break system whether kaslr
> enabled or not.
>
> Hi Mike, Russ and Frank,
>
> About Thomas's question, could you help answer it? Could other SGI UV
> system has MMIOH region bigger than 32TB?
>
>
> While the region is much smaller it can occupy address space > 32TB, up to
> 64TB - <MMIOH size>.
> On a system with 64TB, part of the address space is taken from RAM to
> accommodate this region.
> This has been true since UV1.

I see, it would be better to know the different places to tailor the
memory_tb accordingly. I understand that might be difficult to do and
I rather have KASLR memory randomization working for now.

Reviewed-by: thgarnie@google.com

>
> Thanks
> Baoquan
>
>                 kaslr_regions[0].size_tb = memory_tb;
>
>         /* Calculate entropy available between regions */
>
>



-- 
Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web