Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1668765 > unrolled thread

[PATCH] x86/mm: Don't reenter flush_tlb_func_common()

Back to article view | Back to linux.kernel

Contents

  [PATCH] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-19 06:50 +0200
    Re: [PATCH] x86/mm: Don't reenter flush_tlb_func_common() zhong jiang <zhongjiang@huawei.com> - 2017-06-19 15:40 +0200
      Re: [PATCH] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-19 17:10 +0200

#1668765 — [PATCH] x86/mm: Don't reenter flush_tlb_func_common()

It was historically possible to have two concurrent TLB flushes
targeting the same CPU: one initiated locally and one initiated
remotely.  This can now cause an OOPS in leave_mm() at
arch/x86/mm/tlb.c:47:

        if (this_cpu_read(cpu_tlbstate.state) == TLBSTATE_OK)
                BUG();

with this call trace:
 flush_tlb_func_local arch/x86/mm/tlb.c:239 [inline]
 flush_tlb_mm_range+0x26d/0x370 arch/x86/mm/tlb.c:317

Without reentrancy, this OOPS is impossible: leave_mm() is only
called if we're not in TLBSTATE_OK, but then we're unexpectedly
in TLBSTATE_OK in leave_mm().

This can be caused by flush_tlb_func_remote() happening between
the two checks and calling leave_mm(), resulting in two consecutive
leave_mm() calls on the same CPU with no intervening switch_mm()
calls.

We never saw this OOPS before because the old leave_mm()
implementation didn't put us back in TLBSTATE_OK, so the assertion
didn't fire.

Nadav noticed the reentrancy issue in a different context, but
neither of us realized that it caused a problem yet.

Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Reported-by: "Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>
Fixes: 3d28ebceaffa ("x86/mm: Rework lazy TLB to track the actual loaded mm")
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/mm/tlb.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 2a5e851f2035..f06239c6919f 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -208,6 +208,9 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
 static void flush_tlb_func_common(const struct flush_tlb_info *f,
 				  bool local, enum tlb_flush_reason reason)
 {
+	/* This code cannot presently handle being reentered. */
+	VM_WARN_ON(!irqs_disabled());
+
 	if (this_cpu_read(cpu_tlbstate.state) != TLBSTATE_OK) {
 		leave_mm(smp_processor_id());
 		return;
@@ -313,8 +316,12 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
 		info.end = TLB_FLUSH_ALL;
 	}
 
-	if (mm == this_cpu_read(cpu_tlbstate.loaded_mm))
+	if (mm == this_cpu_read(cpu_tlbstate.loaded_mm)) {
+		local_irq_disable();
 		flush_tlb_func_local(&info, TLB_LOCAL_MM_SHOOTDOWN);
+		local_irq_enable();
+	}
+
 	if (cpumask_any_but(mm_cpumask(mm), cpu) < nr_cpu_ids)
 		flush_tlb_others(mm_cpumask(mm), &info);
 	put_cpu();
@@ -370,8 +377,12 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
 
 	int cpu = get_cpu();
 
-	if (cpumask_test_cpu(cpu, &batch->cpumask))
+	if (cpumask_test_cpu(cpu, &batch->cpumask)) {
+		local_irq_disable();
 		flush_tlb_func_local(&info, TLB_LOCAL_SHOOTDOWN);
+		local_irq_enable();
+	}
+
 	if (cpumask_any_but(&batch->cpumask, cpu) < nr_cpu_ids)
 		flush_tlb_others(&batch->cpumask, &info);
 	cpumask_clear(&batch->cpumask);
-- 
2.9.4

[toc] | [next] | [standalone]

#1669072

On 2017/6/19 12:48, Andy Lutomirski wrote:
> It was historically possible to have two concurrent TLB flushes
> targeting the same CPU: one initiated locally and one initiated
> remotely.  This can now cause an OOPS in leave_mm() at
> arch/x86/mm/tlb.c:47:
>
>         if (this_cpu_read(cpu_tlbstate.state) == TLBSTATE_OK)
>                 BUG();
>
> with this call trace:
>  flush_tlb_func_local arch/x86/mm/tlb.c:239 [inline]
>  flush_tlb_mm_range+0x26d/0x370 arch/x86/mm/tlb.c:317
>
> Without reentrancy, this OOPS is impossible: leave_mm() is only
> called if we're not in TLBSTATE_OK, but then we're unexpectedly
> in TLBSTATE_OK in leave_mm().
>
> This can be caused by flush_tlb_func_remote() happening between
> the two checks and calling leave_mm(), resulting in two consecutive
> leave_mm() calls on the same CPU with no intervening switch_mm()
> calls.
>
> We never saw this OOPS before because the old leave_mm()
> implementation didn't put us back in TLBSTATE_OK, so the assertion
> didn't fire.
  HI, Andy

  Today, I see same OOPS in linux 3.4 stable. It prove that it indeed has fired.
   but It is rarely to appear.  I review the code. I found the a  issue.
  when current->mm is NULL,  leave_mm will be called. but  it maybe in
  TLBSTATE_OK,  eg: unuse_mm call after task->mm = NULL , but before enter_lazy_tlb.

   therefore,  it will fire. is it right?

  Thanks
  zhongjiang
> Nadav noticed the reentrancy issue in a different context, but
> neither of us realized that it caused a problem yet.
>
> Cc: Nadav Amit <nadav.amit@gmail.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Reported-by: "Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>
> Fixes: 3d28ebceaffa ("x86/mm: Rework lazy TLB to track the actual loaded mm")
> Signed-off-by: Andy Lutomirski <luto@kernel.org>
> ---
>  arch/x86/mm/tlb.c | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
> index 2a5e851f2035..f06239c6919f 100644
> --- a/arch/x86/mm/tlb.c
> +++ b/arch/x86/mm/tlb.c
> @@ -208,6 +208,9 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
>  static void flush_tlb_func_common(const struct flush_tlb_info *f,
>  				  bool local, enum tlb_flush_reason reason)
>  {
> +	/* This code cannot presently handle being reentered. */
> +	VM_WARN_ON(!irqs_disabled());
> +
>  	if (this_cpu_read(cpu_tlbstate.state) != TLBSTATE_OK) {
>  		leave_mm(smp_processor_id());
>  		return;
> @@ -313,8 +316,12 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
>  		info.end = TLB_FLUSH_ALL;
>  	}
>  
> -	if (mm == this_cpu_read(cpu_tlbstate.loaded_mm))
> +	if (mm == this_cpu_read(cpu_tlbstate.loaded_mm)) {
> +		local_irq_disable();
>  		flush_tlb_func_local(&info, TLB_LOCAL_MM_SHOOTDOWN);
> +		local_irq_enable();
> +	}
> +
>  	if (cpumask_any_but(mm_cpumask(mm), cpu) < nr_cpu_ids)
>  		flush_tlb_others(mm_cpumask(mm), &info);
>  	put_cpu();
> @@ -370,8 +377,12 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
>  
>  	int cpu = get_cpu();
>  
> -	if (cpumask_test_cpu(cpu, &batch->cpumask))
> +	if (cpumask_test_cpu(cpu, &batch->cpumask)) {
> +		local_irq_disable();
>  		flush_tlb_func_local(&info, TLB_LOCAL_SHOOTDOWN);
> +		local_irq_enable();
> +	}
> +
>  	if (cpumask_any_but(&batch->cpumask, cpu) < nr_cpu_ids)
>  		flush_tlb_others(&batch->cpumask, &info);
>  	cpumask_clear(&batch->cpumask);

[toc] | [prev] | [next] | [standalone]

#1669190

On Mon, Jun 19, 2017 at 6:33 AM, zhong jiang <zhongjiang@huawei.com> wrote:
> On 2017/6/19 12:48, Andy Lutomirski wrote:
>> It was historically possible to have two concurrent TLB flushes
>> targeting the same CPU: one initiated locally and one initiated
>> remotely.  This can now cause an OOPS in leave_mm() at
>> arch/x86/mm/tlb.c:47:
>>
>>         if (this_cpu_read(cpu_tlbstate.state) == TLBSTATE_OK)
>>                 BUG();
>>
>> with this call trace:
>>  flush_tlb_func_local arch/x86/mm/tlb.c:239 [inline]
>>  flush_tlb_mm_range+0x26d/0x370 arch/x86/mm/tlb.c:317
>>
>> Without reentrancy, this OOPS is impossible: leave_mm() is only
>> called if we're not in TLBSTATE_OK, but then we're unexpectedly
>> in TLBSTATE_OK in leave_mm().
>>
>> This can be caused by flush_tlb_func_remote() happening between
>> the two checks and calling leave_mm(), resulting in two consecutive
>> leave_mm() calls on the same CPU with no intervening switch_mm()
>> calls.
>>
>> We never saw this OOPS before because the old leave_mm()
>> implementation didn't put us back in TLBSTATE_OK, so the assertion
>> didn't fire.
>   HI, Andy
>
>   Today, I see same OOPS in linux 3.4 stable. It prove that it indeed has fired.
>    but It is rarely to appear.  I review the code. I found the a  issue.
>   when current->mm is NULL,  leave_mm will be called. but  it maybe in
>   TLBSTATE_OK,  eg: unuse_mm call after task->mm = NULL , but before enter_lazy_tlb.
>
>    therefore,  it will fire. is it right?

Is there a code path that does this?

Also, the IPI handler on 3.4 looks like this:

        if (f->flush_mm == percpu_read(cpu_tlbstate.active_mm)) {
                if (percpu_read(cpu_tlbstate.state) == TLBSTATE_OK) {
                        if (f->flush_va == TLB_FLUSH_ALL)
                                local_flush_tlb();
                        else
                                __flush_tlb_one(f->flush_va);
                } else
                        leave_mm(cpu);
        }

but leave_mm() checks the same condition (cpu_tlbstate.state, not
current->mm).  How is the BUG triggering?

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web