Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1230589 > unrolled thread

Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late()

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late() Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 20:00 +0200
    Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late() Andy Lutomirski <luto@amacapital.net> - 2015-09-22 20:10 +0200
      Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late() Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 20:30 +0200
        Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late() Andy Lutomirski <luto@amacapital.net> - 2015-09-22 20:40 +0200
          Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late() Linus Torvalds <torvalds@linux-foundation.org> - 2015-09-22 20:50 +0200
            Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late() Andy Lutomirski <luto@amacapital.net> - 2015-09-22 21:00 +0200

#1230589 — Re: [PATCH 05/11] mm: Introduce arch_pgd_init_late()

On Mon, Sep 21, 2015 at 11:23 PM, Ingo Molnar <mingo@kernel.org> wrote:
> Add a late PGD init callback to places that allocate a new MM
> with a new PGD: copy_process() and exec().
>
> The purpose of this callback is to allow architectures to implement
> lockless initialization of task PGDs, to remove the scalability
> limit of pgd_list/pgd_lock.

Do we really need this?

Can't we just initialize the pgd when we allocate it, knowing that
it's not in sync, but just depend on the vmalloc fault to add in any
kernel entries that we might have missed?

I liked the other patches in the series because they remove code and
simplify things. This patch I don't like.

There may be some reason we need it that I missed, and which makes me
go "Duh!" when you tell me. But please do tell me.

               Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1230597

On Tue, Sep 22, 2015 at 10:55 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Sep 21, 2015 at 11:23 PM, Ingo Molnar <mingo@kernel.org> wrote:
>> Add a late PGD init callback to places that allocate a new MM
>> with a new PGD: copy_process() and exec().
>>
>> The purpose of this callback is to allow architectures to implement
>> lockless initialization of task PGDs, to remove the scalability
>> limit of pgd_list/pgd_lock.
>
> Do we really need this?
>
> Can't we just initialize the pgd when we allocate it, knowing that
> it's not in sync, but just depend on the vmalloc fault to add in any
> kernel entries that we might have missed?

I really really hate the vmalloc fault thing.  It seems to work,
rather to my surprise.  It doesn't *deserve* to work, because of
things like the percpu TSS accesses in the entry code that happen
without a valid stack.

For all I know, there's a long history of this hitting on monster
non-SMAP systems that are all buggy and rootable but no one notices
because it's rare.  On SMAP with non-malicious userspace, it's an
instant double fault.  With malicious userspace, it's rootable
regardless of SMAP, but it's much harder with SMAP.

If we start every mm with a fully zeroed pgd (which is what I think
you're suggesting), then this starts affecting small systems as in
addition to monster systems.

I'd really rather go in the other directoin and completely eliminate
vmalloc faults.  We could do that by eagerly initializing all pgd, or
we could do it by tracking, per-pgd, how up-to-date it is and fixing
it up in switch_mm.  The latter is a bit nasty on SMP.

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1230672

On Tue, Sep 22, 2015 at 11:00 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>
> I really really hate the vmalloc fault thing.  It seems to work,
> rather to my surprise.  It doesn't *deserve* to work, because of
> things like the percpu TSS accesses in the entry code that happen
> without a valid stack.

The thing is, I think you're misguided in your hatred.

The reason I say that is because I think we should just embrace the
fact that faults can and do happen in the kernel in very inconvenient
places, and not just in code we "control".

Even if you get rid of the vmalloc fault, you'll still have debug
faults, and you'll still have NMI's and horrible crazy machine check
faults.

I actually think teh vmalloc fault is a good way to just let people
know "pretty much anything can trap, deal with it".

And I think trying to eliminate them is the wrong thing, because it
forces us to be so damn synchronized. This whole patch-series is a
prime example of why that is a bad bad things. We want to have _less_
synchronization.

                Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1230702

On Tue, Sep 22, 2015 at 11:26 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Sep 22, 2015 at 11:00 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>
>> I really really hate the vmalloc fault thing.  It seems to work,
>> rather to my surprise.  It doesn't *deserve* to work, because of
>> things like the percpu TSS accesses in the entry code that happen
>> without a valid stack.
>
> The thing is, I think you're misguided in your hatred.
>
> The reason I say that is because I think we should just embrace the
> fact that faults can and do happen in the kernel in very inconvenient
> places, and not just in code we "control".
>
> Even if you get rid of the vmalloc fault, you'll still have debug
> faults, and you'll still have NMI's and horrible crazy machine check
> faults.
>
> I actually think teh vmalloc fault is a good way to just let people
> know "pretty much anything can trap, deal with it".
>
> And I think trying to eliminate them is the wrong thing, because it
> forces us to be so damn synchronized. This whole patch-series is a
> prime example of why that is a bad bad things. We want to have _less_
> synchronization.

Sure, pretty much anything can trap, but we need to do *something* to
deal with it.

Debug faults can't happen with bad stacks any more (now that we honor
the kprobe blacklist), which means that debug faults could, in theory,
move off the IST stack.  The SYSENTER + debug mess doesn't have any
stack problem.

NMIs and MCEs are special, and we deal with that using IST and all
kinds of mess.

I don't think that anyone really wants to move #PF to IST, which means
that we simply cannot handle vmalloc faults that happen when switching
stacks after SYSCALL, no matter what fanciness we shove into the
page_fault asm.  If we move #PF to IST, then we have to worry about
page_fault -> nmi -> page_fault, which would be a clusterf*ck.

AMD gave us a pile of misguided architectural turds, and we have to
deal with it.  My preference is to simplify dealing with it by getting
rid of vmalloc faults so that we can at least reliably touch percpu
memory without faulting.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1230755

On Tue, Sep 22, 2015 at 11:37 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> kinds of mess.
>
> I don't think that anyone really wants to move #PF to IST, which means
> that we simply cannot handle vmalloc faults that happen when switching
> stacks after SYSCALL, no matter what fanciness we shove into the
> page_fault asm.

But that's fine. The kernel stack is special.  So yes, we want to make
sure that the kernel stack is always mapped in the thread whose stack
it is.

But that's not a big and onerous guarantee to make. Not when the
*real* problem is "random vmalloc allocations made by other processes
that we are not in the least interested in, and we don't want to add
synchronization for".

                         Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1230762

On Tue, Sep 22, 2015 at 11:44 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Sep 22, 2015 at 11:37 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> kinds of mess.
>>
>> I don't think that anyone really wants to move #PF to IST, which means
>> that we simply cannot handle vmalloc faults that happen when switching
>> stacks after SYSCALL, no matter what fanciness we shove into the
>> page_fault asm.
>
> But that's fine. The kernel stack is special.  So yes, we want to make
> sure that the kernel stack is always mapped in the thread whose stack
> it is.
>
> But that's not a big and onerous guarantee to make. Not when the
> *real* problem is "random vmalloc allocations made by other processes
> that we are not in the least interested in, and we don't want to add
> synchronization for".
>

It's the kernel stack, the TSS (for sp0) and rsp_scratch at least.
But yes, that's not that onerous, and it's never lazily initialized
elsewhere.

How about this (long-term, not right now): Never free pgd entries.
For each pgd, track the number of populated kernel entries.  Also
track the global (init_mm) number of existing kernel entries.  At
context switch time, if new_pgd has fewer entries that the total, sync
it.

This hits *at most* 256 times per thread, and otherwise it's just a
single unlikely branch.  It guarantees that we only ever take a
vmalloc fault when accessing maps that didn't exist when we last
context switched, which gets us all of the important percpu stuff and
the kernel stack, even if we schedule onto a cpu that didn't exist
when the mm was created.

--Andy

>                          Linus



-- 
Andy Lutomirski
AMA Capital Management, LLC
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web