Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1620134 > unrolled thread

Re: KASLR causes intermittent boot failures on some systems

Started byJeff Moyer <jmoyer@redhat.com>
First post2017-04-10 18:00 +0200
Last post2017-04-10 21:20 +0200
Articles 5 — 2 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: KASLR causes intermittent boot failures on some systems Jeff Moyer <jmoyer@redhat.com> - 2017-04-10 18:00 +0200
    Re: KASLR causes intermittent boot failures on some systems Kees Cook <keescook@chromium.org> - 2017-04-10 20:20 +0200
      Re: KASLR causes intermittent boot failures on some systems Jeff Moyer <jmoyer@redhat.com> - 2017-04-10 20:30 +0200
        Re: KASLR causes intermittent boot failures on some systems Kees Cook <keescook@chromium.org> - 2017-04-10 21:10 +0200
          Re: KASLR causes intermittent boot failures on some systems Jeff Moyer <jmoyer@redhat.com> - 2017-04-10 21:20 +0200

#1620134 — Re: KASLR causes intermittent boot failures on some systems

FromJeff Moyer <jmoyer@redhat.com>
Date2017-04-10 18:00 +0200
SubjectRe: KASLR causes intermittent boot failures on some systems
Message-ID<tuJTu-2D4-73@gated-at.bofh.it>
Kees Cook <keescook@chromium.org> writes:

> On Fri, Apr 7, 2017 at 7:41 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>> Hi,
>>
>> commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory
>> regions") causes some of my systems with persistent memory (whether real
>> or emulated) to fail to boot with a couple of different crash
>> signatures.  The first signature is a NMI watchdog lockup of all but 1
>> cpu, which causes much difficulty in extracting useful information from
>> the console.  The second variant is an invalid paging request, listed
>> below.
>
> Just to rule out some of the stuff in the boot path, does booting with
> "nokaslr" solve this? (i.e. I want to figure out if this is from some
> of the rearrangements done that are exposed under that commit, or if
> it is genuinely the randomization that is killing the systems...)

Adding "nokaslr" to the boot line does indeed make the problem go away.

Cheers,
Jeff

[toc] | [next] | [standalone]

#1620582

FromKees Cook <keescook@chromium.org>
Date2017-04-10 20:20 +0200
Message-ID<tuM4W-4ll-15@gated-at.bofh.it>
In reply to#1620134
On Mon, Apr 10, 2017 at 8:49 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
> Kees Cook <keescook@chromium.org> writes:
>
>> On Fri, Apr 7, 2017 at 7:41 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>> Hi,
>>>
>>> commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory
>>> regions") causes some of my systems with persistent memory (whether real
>>> or emulated) to fail to boot with a couple of different crash
>>> signatures.  The first signature is a NMI watchdog lockup of all but 1
>>> cpu, which causes much difficulty in extracting useful information from
>>> the console.  The second variant is an invalid paging request, listed
>>> below.
>>
>> Just to rule out some of the stuff in the boot path, does booting with
>> "nokaslr" solve this? (i.e. I want to figure out if this is from some
>> of the rearrangements done that are exposed under that commit, or if
>> it is genuinely the randomization that is killing the systems...)
>
> Adding "nokaslr" to the boot line does indeed make the problem go away.

Are you booting with a memmap= flag?

-Kees

-- 
Kees Cook
Pixel Security

[toc] | [prev] | [next] | [standalone]

#1620601

FromJeff Moyer <jmoyer@redhat.com>
Date2017-04-10 20:30 +0200
Message-ID<tuMeB-4oO-5@gated-at.bofh.it>
In reply to#1620582
Kees Cook <keescook@chromium.org> writes:

> On Mon, Apr 10, 2017 at 8:49 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>> Kees Cook <keescook@chromium.org> writes:
>>
>>> On Fri, Apr 7, 2017 at 7:41 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>>> Hi,
>>>>
>>>> commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory
>>>> regions") causes some of my systems with persistent memory (whether real
>>>> or emulated) to fail to boot with a couple of different crash
>>>> signatures.  The first signature is a NMI watchdog lockup of all but 1
>>>> cpu, which causes much difficulty in extracting useful information from
>>>> the console.  The second variant is an invalid paging request, listed
>>>> below.
>>>
>>> Just to rule out some of the stuff in the boot path, does booting with
>>> "nokaslr" solve this? (i.e. I want to figure out if this is from some
>>> of the rearrangements done that are exposed under that commit, or if
>>> it is genuinely the randomization that is killing the systems...)
>>
>> Adding "nokaslr" to the boot line does indeed make the problem go away.
>
> Are you booting with a memmap= flag?

From my first email:

[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.11.0-rc5+
root=/dev/mapper/rhel_intel--lizardhead--04-root ro memmap=192G!1024G
crashkernel=auto rd.lvm.lv=rhel_intel-lizardhead-04/root
rd.lvm.lv=rhel_intel-lizardhead-04/swap console=ttyS0,115200n81
LANG=en_US.UTF-8

Did you not receive the attachments?

Cheers,
Jeff

[toc] | [prev] | [next] | [standalone]

#1620633

FromKees Cook <keescook@chromium.org>
Date2017-04-10 21:10 +0200
Message-ID<tuMRj-4S4-11@gated-at.bofh.it>
In reply to#1620601
On Mon, Apr 10, 2017 at 11:22 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
> Kees Cook <keescook@chromium.org> writes:
>
>> On Mon, Apr 10, 2017 at 8:49 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>> Kees Cook <keescook@chromium.org> writes:
>>>
>>>> On Fri, Apr 7, 2017 at 7:41 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>>>> Hi,
>>>>>
>>>>> commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory
>>>>> regions") causes some of my systems with persistent memory (whether real
>>>>> or emulated) to fail to boot with a couple of different crash
>>>>> signatures.  The first signature is a NMI watchdog lockup of all but 1
>>>>> cpu, which causes much difficulty in extracting useful information from
>>>>> the console.  The second variant is an invalid paging request, listed
>>>>> below.
>>>>
>>>> Just to rule out some of the stuff in the boot path, does booting with
>>>> "nokaslr" solve this? (i.e. I want to figure out if this is from some
>>>> of the rearrangements done that are exposed under that commit, or if
>>>> it is genuinely the randomization that is killing the systems...)
>>>
>>> Adding "nokaslr" to the boot line does indeed make the problem go away.
>>
>> Are you booting with a memmap= flag?
>
> From my first email:
>
> [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.11.0-rc5+
> root=/dev/mapper/rhel_intel--lizardhead--04-root ro memmap=192G!1024G
> crashkernel=auto rd.lvm.lv=rhel_intel-lizardhead-04/root
> rd.lvm.lv=rhel_intel-lizardhead-04/swap console=ttyS0,115200n81
> LANG=en_US.UTF-8
>
> Did you not receive the attachments?

I see it now, thanks!

The memmap parsing was added in -rc1 (f28442497b5ca), so I'd expect
that to be handled. Hmmm.

-Kees

-- 
Kees Cook
Pixel Security

[toc] | [prev] | [next] | [standalone]

#1620636

FromJeff Moyer <jmoyer@redhat.com>
Date2017-04-10 21:20 +0200
Message-ID<tuN0Z-4VF-1@gated-at.bofh.it>
In reply to#1620633
Kees Cook <keescook@chromium.org> writes:

> On Mon, Apr 10, 2017 at 11:22 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>> Kees Cook <keescook@chromium.org> writes:
>>
>>> On Mon, Apr 10, 2017 at 8:49 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>>> Kees Cook <keescook@chromium.org> writes:
>>>>
>>>>> On Fri, Apr 7, 2017 at 7:41 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> commit 021182e52fe01 ("x86/mm: Enable KASLR for physical mapping memory
>>>>>> regions") causes some of my systems with persistent memory (whether real
>>>>>> or emulated) to fail to boot with a couple of different crash
>>>>>> signatures.  The first signature is a NMI watchdog lockup of all but 1
>>>>>> cpu, which causes much difficulty in extracting useful information from
>>>>>> the console.  The second variant is an invalid paging request, listed
>>>>>> below.
>>>>>
>>>>> Just to rule out some of the stuff in the boot path, does booting with
>>>>> "nokaslr" solve this? (i.e. I want to figure out if this is from some
>>>>> of the rearrangements done that are exposed under that commit, or if
>>>>> it is genuinely the randomization that is killing the systems...)
>>>>
>>>> Adding "nokaslr" to the boot line does indeed make the problem go away.
>>>
>>> Are you booting with a memmap= flag?
>>
>> From my first email:
>>
>> [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.11.0-rc5+
>> root=/dev/mapper/rhel_intel--lizardhead--04-root ro memmap=192G!1024G
>> crashkernel=auto rd.lvm.lv=rhel_intel-lizardhead-04/root
>> rd.lvm.lv=rhel_intel-lizardhead-04/swap console=ttyS0,115200n81
>> LANG=en_US.UTF-8
>>
>> Did you not receive the attachments?
>
> I see it now, thanks!
>
> The memmap parsing was added in -rc1 (f28442497b5ca), so I'd expect
> that to be handled. Hmmm.

I can also reproduce this on a system with real persistent memory, which
does not require the memmap parameter.

Cheers,
Jeff

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web