Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1701796 > unrolled thread

[PATCH v6 0/7] fixes of TLB batching races

Started byNadav Amit <namit@vmware.com>
First post2017-08-02 09:40 +0200
Last post2017-08-03 01:30 +0200
Articles 10 — 6 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v6 0/7] fixes of TLB batching races Nadav Amit <namit@vmware.com> - 2017-08-02 09:40 +0200
    [PATCH v6 5/7] mm: make tlb_flush_pending global Nadav Amit <namit@vmware.com> - 2017-08-02 09:40 +0200
      Re: [PATCH v6 5/7] mm: make tlb_flush_pending global Minchan Kim <minchan@kernel.org> - 2017-08-03 01:30 +0200
      Re: [PATCH v6 5/7] mm: make tlb_flush_pending global Andrew Morton <akpm@linux-foundation.org> - 2017-08-03 01:30 +0200
        Re: [PATCH v6 5/7] mm: make tlb_flush_pending global Minchan Kim <minchan@kernel.org> - 2017-08-03 01:40 +0200
      Re: [PATCH v6 5/7] mm: make tlb_flush_pending global kbuild test robot <lkp@intel.com> - 2017-08-03 18:50 +0200
    [PATCH v6 4/7] mm: refactoring TLB gathering API Nadav Amit <namit@vmware.com> - 2017-08-02 09:40 +0200
      Re: [PATCH v6 4/7] mm: refactoring TLB gathering API Peter Zijlstra <peterz@infradead.org> - 2017-08-11 11:30 +0200
        Re: [PATCH v6 4/7] mm: refactoring TLB gathering API Nadav Amit <nadav.amit@gmail.com> - 2017-08-11 19:20 +0200
    Re: [PATCH v6 0/7] fixes of TLB batching races Minchan Kim <minchan@kernel.org> - 2017-08-03 01:30 +0200

#1701796 — [PATCH v6 0/7] fixes of TLB batching races

FromNadav Amit <namit@vmware.com>
Date2017-08-02 09:40 +0200
Subject[PATCH v6 0/7] fixes of TLB batching races
Message-ID<u9Wq6-6U6-5@gated-at.bofh.it>
It turns out that Linux TLB batching mechanism suffers from various races.
Races that are caused due to batching during reclamation were recently
handled by Mel and this patch-set deals with others. The more fundamental
issue is that concurrent updates of the page-tables allow for TLB flushes
to be batched on one core, while another core changes the page-tables.
This other core may assume a PTE change does not require a flush based on
the updated PTE value, while it is unaware that TLB flushes are still
pending.

This behavior affects KSM (which may result in memory corruption) and
MADV_FREE and MADV_DONTNEED (which may result in incorrect behavior). A
proof-of-concept can easily produce the wrong behavior of MADV_DONTNEED.
Memory corruption in KSM is harder to produce in practice, but was observed
by hacking the kernel and adding a delay before flushing and replacing the
KSM page.

Finally, there is also one memory barrier missing, which may affect
architectures with weak memory model.

v5 -> v6:
* Combining with Minchan Kim's patch set, adding ack's (Andrew)
* Minor: missing header, typos (Nadav)
* Renaming arch_generic_tlb_finish_mmu (Mel)

Michnan's v1 -> v2 (combined):
* TLB batching API separation core part from arch specific one (Mel)
* introduce mm_tlb_flush_nested (Mel)

v4 -> v5:
* Fixing embarrassing build mistake (0day)

v3 -> v4:
* Change function names to indicate they inc/dec and not set/clear
  (Sergey)
* Avoid additional barriers, and instead revert the patch that accessed
  mm_tlb_flush_pending without a lock (Mel)

v2 -> v3:
* Do not init tlb_flush_pending if it is not defined without (Sergey)
* Internalize memory barriers to mm_tlb_flush_pending (Minchan) 

v1 -> v2:
* Explain the implications of the implications of the race (Andrew)
* Mark the patch that address the race as stable (Andrew)
* Add another patch to clean the use of barriers (Andrew)

Minchan Kim (4):
  mm: refactoring TLB gathering API
  mm: make tlb_flush_pending global
  mm: fix MADV_[FREE|DONTNEED] TLB flush miss problem
  mm: fix KSM data corruption

Nadav Amit (3):
  mm: migrate: prevent racy access to tlb_flush_pending
  mm: migrate: fix barriers around tlb_flush_pending
  Revert "mm: numa: defer TLB flush for THP migration as long as
    possible"

 arch/arm/include/asm/tlb.h  | 11 ++++++--
 arch/ia64/include/asm/tlb.h |  8 ++++--
 arch/s390/include/asm/tlb.h | 17 +++++++-----
 arch/sh/include/asm/tlb.h   |  8 +++---
 arch/um/include/asm/tlb.h   | 13 ++++++---
 fs/proc/task_mmu.c          |  7 +++--
 include/asm-generic/tlb.h   |  7 ++---
 include/linux/mm_types.h    | 64 +++++++++++++++++++++++++++------------------
 kernel/fork.c               |  2 +-
 mm/debug.c                  |  4 +--
 mm/huge_memory.c            |  7 +++++
 mm/ksm.c                    |  3 ++-
 mm/memory.c                 | 41 ++++++++++++++++++++++++-----
 mm/migrate.c                |  6 -----
 mm/mprotect.c               |  4 +--
 15 files changed, 135 insertions(+), 67 deletions(-)

-- 
2.11.0

[toc] | [next] | [standalone]

#1701797 — [PATCH v6 5/7] mm: make tlb_flush_pending global

FromNadav Amit <namit@vmware.com>
Date2017-08-02 09:40 +0200
Subject[PATCH v6 5/7] mm: make tlb_flush_pending global
Message-ID<u9Wq7-6U6-23@gated-at.bofh.it>
In reply to#1701796
From: Minchan Kim <minchan@kernel.org>

Currently, tlb_flush_pending is used only for CONFIG_[NUMA_BALANCING|
COMPACTION] but upcoming patches to solve subtle TLB flush batching
problem will use it regardless of compaction/NUMA so this patch
doesn't remove the dependency.

Signed-off-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
---
 include/linux/mm_types.h | 21 ---------------------
 mm/debug.c               |  2 --
 2 files changed, 23 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 248f4ed1f3e1..fc44315df47a 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -487,14 +487,12 @@ struct mm_struct {
 	/* numa_scan_seq prevents two threads setting pte_numa */
 	int numa_scan_seq;
 #endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 	/*
 	 * An operation with batched TLB flushing is going on. Anything that
 	 * can move process memory needs to flush the TLB when moving a
 	 * PROT_NONE or PROT_NUMA mapped page.
 	 */
 	atomic_t tlb_flush_pending;
-#endif
 	struct uprobes_state uprobes_state;
 #ifdef CONFIG_HUGETLB_PAGE
 	atomic_long_t hugetlb_usage;
@@ -524,7 +522,6 @@ extern void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
 extern void tlb_finish_mmu(struct mmu_gather *tlb,
 				unsigned long start, unsigned long end);
 
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 /*
  * Memory barriers to keep this state in sync are graciously provided by
  * the page table locks, outside of which no page table modifications happen.
@@ -565,24 +562,6 @@ static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 	smp_mb__before_atomic();
 	atomic_dec(&mm->tlb_flush_pending);
 }
-#else
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
-	return false;
-}
-
-static inline void init_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-
-static inline void inc_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-
-static inline void dec_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-#endif
 
 struct vm_fault;
 
diff --git a/mm/debug.c b/mm/debug.c
index d70103bb4731..18a9b15b1e37 100644
--- a/mm/debug.c
+++ b/mm/debug.c
@@ -158,9 +158,7 @@ void dump_mm(const struct mm_struct *mm)
 #ifdef CONFIG_NUMA_BALANCING
 		mm->numa_next_scan, mm->numa_scan_offset, mm->numa_scan_seq,
 #endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 		atomic_read(&mm->tlb_flush_pending),
-#endif
 		mm->def_flags, &mm->def_flags
 	);
 }
-- 
2.11.0

[toc] | [prev] | [next] | [standalone]

#1702564 — Re: [PATCH v6 5/7] mm: make tlb_flush_pending global

FromMinchan Kim <minchan@kernel.org>
Date2017-08-03 01:30 +0200
SubjectRe: [PATCH v6 5/7] mm: make tlb_flush_pending global
Message-ID<uabfs-8iN-33@gated-at.bofh.it>
In reply to#1701797
On Wed, Aug 02, 2017 at 10:28:47PM +0800, kbuild test robot wrote:
> Hi Minchan,
> 
> [auto build test WARNING on linus/master]
> [also build test WARNING on v4.13-rc3]
> [cannot apply to next-20170802]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> 
> url:    https://github.com/0day-ci/linux/commits/Nadav-Amit/mm-migrate-prevent-racy-access-to-tlb_flush_pending/20170802-205715
> config: sh-allyesconfig (attached as .config)
> compiler: sh4-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
> reproduce:
>         wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         make.cross ARCH=sh 
> 
> All warnings (new ones prefixed by >>):
> 
>    In file included from include/linux/printk.h:6:0,
>                     from include/linux/kernel.h:13,
>                     from mm/debug.c:8:
>    mm/debug.c: In function 'dump_mm':
> >> include/linux/kern_levels.h:4:18: warning: format '%lx' expects argument of type 'long unsigned int', but argument 40 has type 'int' [-Wformat=]

Thanks. lkp.

This patch should fix it.

From 5be44c3cbe0e4149cc8b438f2e3fcad046091a29 Mon Sep 17 00:00:00 2001
From: Minchan Kim <minchan@kernel.org>
Date: Tue, 1 Aug 2017 14:04:58 +0900
Subject: [PATCH v6 5/7] mm: make tlb_flush_pending global

Currently, tlb_flush_pending is used only for CONFIG_[NUMA_BALANCING|
COMPACTION] but upcoming patches to solve subtle TLB flush bacting
problem will use it regardless of compaction/numa so this patch
doesn't remove the dependency.

Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
 include/linux/mm_types.h | 21 ---------------------
 mm/debug.c               |  4 ----
 2 files changed, 25 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index c605f2a3a68e..892a7b0196fd 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -487,14 +487,12 @@ struct mm_struct {
 	/* numa_scan_seq prevents two threads setting pte_numa */
 	int numa_scan_seq;
 #endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 	/*
 	 * An operation with batched TLB flushing is going on. Anything that
 	 * can move process memory needs to flush the TLB when moving a
 	 * PROT_NONE or PROT_NUMA mapped page.
 	 */
 	atomic_t tlb_flush_pending;
-#endif
 #ifdef CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
 	/* See flush_tlb_batched_pending() */
 	bool tlb_flush_batched;
@@ -528,7 +526,6 @@ extern void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
 extern void tlb_finish_mmu(struct mmu_gather *tlb,
 				unsigned long start, unsigned long end);
 
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 /*
  * Memory barriers to keep this state in sync are graciously provided by
  * the page table locks, outside of which no page table modifications happen.
@@ -569,24 +566,6 @@ static inline void dec_tlb_flush_pending(struct mm_struct *mm)
 	smp_mb__before_atomic();
 	atomic_dec(&mm->tlb_flush_pending);
 }
-#else
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
-	return false;
-}
-
-static inline void init_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-
-static inline void inc_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-
-static inline void dec_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-#endif
 
 struct vm_fault;
 
diff --git a/mm/debug.c b/mm/debug.c
index d70103bb4731..5715448ab0b5 100644
--- a/mm/debug.c
+++ b/mm/debug.c
@@ -124,9 +124,7 @@ void dump_mm(const struct mm_struct *mm)
 #ifdef CONFIG_NUMA_BALANCING
 		"numa_next_scan %lu numa_scan_offset %lu numa_scan_seq %d\n"
 #endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 		"tlb_flush_pending %d\n"
-#endif
 		"def_flags: %#lx(%pGv)\n",
 
 		mm, mm->mmap, mm->vmacache_seqnum, mm->task_size,
@@ -158,9 +156,7 @@ void dump_mm(const struct mm_struct *mm)
 #ifdef CONFIG_NUMA_BALANCING
 		mm->numa_next_scan, mm->numa_scan_offset, mm->numa_scan_seq,
 #endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 		atomic_read(&mm->tlb_flush_pending),
-#endif
 		mm->def_flags, &mm->def_flags
 	);
 }
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1702566 — Re: [PATCH v6 5/7] mm: make tlb_flush_pending global

FromAndrew Morton <akpm@linux-foundation.org>
Date2017-08-03 01:30 +0200
SubjectRe: [PATCH v6 5/7] mm: make tlb_flush_pending global
Message-ID<uabfs-8iN-27@gated-at.bofh.it>
In reply to#1701797
On Wed, 2 Aug 2017 22:28:47 +0800 kbuild test robot <lkp@intel.com> wrote:

> Hi Minchan,
> 
> [auto build test WARNING on linus/master]
> [also build test WARNING on v4.13-rc3]
> [cannot apply to next-20170802]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> 
> url:    https://github.com/0day-ci/linux/commits/Nadav-Amit/mm-migrate-prevent-racy-access-to-tlb_flush_pending/20170802-205715
> config: sh-allyesconfig (attached as .config)
> compiler: sh4-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
> reproduce:
>         wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         make.cross ARCH=sh 
> 
> All warnings (new ones prefixed by >>):
> 
>    In file included from include/linux/printk.h:6:0,
>                     from include/linux/kernel.h:13,
>                     from mm/debug.c:8:
>    mm/debug.c: In function 'dump_mm':
> >> include/linux/kern_levels.h:4:18: warning: format '%lx' expects argument of type 'long unsigned int', but argument 40 has type 'int' [-Wformat=]
>
> ...
>

This?

From: Andrew Morton <akpm@linux-foundation.org>
Subject: mm-make-tlb_flush_pending-global-fix

remove more ifdefs from world's ugliest printk statement

Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Nadav Amit <namit@vmware.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/debug.c |    2 --
 1 file changed, 2 deletions(-)

diff -puN include/linux/mm_types.h~mm-make-tlb_flush_pending-global-fix include/linux/mm_types.h
diff -puN mm/debug.c~mm-make-tlb_flush_pending-global-fix mm/debug.c
--- a/mm/debug.c~mm-make-tlb_flush_pending-global-fix
+++ a/mm/debug.c
@@ -124,9 +124,7 @@ void dump_mm(const struct mm_struct *mm)
 #ifdef CONFIG_NUMA_BALANCING
 		"numa_next_scan %lu numa_scan_offset %lu numa_scan_seq %d\n"
 #endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 		"tlb_flush_pending %d\n"
-#endif
 		"def_flags: %#lx(%pGv)\n",
 
 		mm, mm->mmap, mm->vmacache_seqnum, mm->task_size,
_

[toc] | [prev] | [next] | [standalone]

#1702569 — Re: [PATCH v6 5/7] mm: make tlb_flush_pending global

FromMinchan Kim <minchan@kernel.org>
Date2017-08-03 01:40 +0200
SubjectRe: [PATCH v6 5/7] mm: make tlb_flush_pending global
Message-ID<uabp7-8mA-3@gated-at.bofh.it>
In reply to#1702566
On Wed, Aug 02, 2017 at 04:27:58PM -0700, Andrew Morton wrote:
> On Wed, 2 Aug 2017 22:28:47 +0800 kbuild test robot <lkp@intel.com> wrote:
> 
> > Hi Minchan,
> > 
> > [auto build test WARNING on linus/master]
> > [also build test WARNING on v4.13-rc3]
> > [cannot apply to next-20170802]
> > [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> > 
> > url:    https://github.com/0day-ci/linux/commits/Nadav-Amit/mm-migrate-prevent-racy-access-to-tlb_flush_pending/20170802-205715
> > config: sh-allyesconfig (attached as .config)
> > compiler: sh4-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
> > reproduce:
> >         wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> >         chmod +x ~/bin/make.cross
> >         # save the attached .config to linux build tree
> >         make.cross ARCH=sh 
> > 
> > All warnings (new ones prefixed by >>):
> > 
> >    In file included from include/linux/printk.h:6:0,
> >                     from include/linux/kernel.h:13,
> >                     from mm/debug.c:8:
> >    mm/debug.c: In function 'dump_mm':
> > >> include/linux/kern_levels.h:4:18: warning: format '%lx' expects argument of type 'long unsigned int', but argument 40 has type 'int' [-Wformat=]
> >
> > ...
> >
> 
> This?
> 
> From: Andrew Morton <akpm@linux-foundation.org>
> Subject: mm-make-tlb_flush_pending-global-fix
> 
> remove more ifdefs from world's ugliest printk statement
> 
> Cc: Mel Gorman <mgorman@techsingularity.net>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Nadav Amit <namit@vmware.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

I'm a bit late.
Thanks for the fix, Andrew!

[toc] | [prev] | [next] | [standalone]

#1703270 — Re: [PATCH v6 5/7] mm: make tlb_flush_pending global

Fromkbuild test robot <lkp@intel.com>
Date2017-08-03 18:50 +0200
SubjectRe: [PATCH v6 5/7] mm: make tlb_flush_pending global
Message-ID<uartT-2D5-3@gated-at.bofh.it>
In reply to#1701797

[Multipart message — attachments visible in raw view] — view raw

Hi Minchan,

[auto build test WARNING on linus/master]
[also build test WARNING on v4.13-rc3]
[cannot apply to next-20170803]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Nadav-Amit/mm-migrate-prevent-racy-access-to-tlb_flush_pending/20170802-205715
config: x86_64-randconfig-a0-08032207 (attached as .config)
compiler: gcc-4.4 (Debian 4.4.7-8) 4.4.7
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All warnings (new ones prefixed by >>):

   mm/debug.c: In function 'dump_mm':
>> mm/debug.c:102: warning: format '%#lx' expects type 'long unsigned int', but argument 40 has type 'int'
>> mm/debug.c:102: warning: format '%p' expects type 'void *', but argument 41 has type 'long unsigned int'
   mm/debug.c:102: warning: too many arguments for format

vim +102 mm/debug.c

82742a3a5 Sasha Levin           2014-10-09   99  
31c9afa6d Sasha Levin           2014-10-09  100  void dump_mm(const struct mm_struct *mm)
31c9afa6d Sasha Levin           2014-10-09  101  {
7a82ca0d6 Andrew Morton         2014-10-09 @102  	pr_emerg("mm %p mmap %p seqnum %d task_size %lu\n"
31c9afa6d Sasha Levin           2014-10-09  103  #ifdef CONFIG_MMU
31c9afa6d Sasha Levin           2014-10-09  104  		"get_unmapped_area %p\n"
31c9afa6d Sasha Levin           2014-10-09  105  #endif
31c9afa6d Sasha Levin           2014-10-09  106  		"mmap_base %lu mmap_legacy_base %lu highest_vm_end %lu\n"
dc6c9a35b Kirill A. Shutemov    2015-02-11  107  		"pgd %p mm_users %d mm_count %d nr_ptes %lu nr_pmds %lu map_count %d\n"
31c9afa6d Sasha Levin           2014-10-09  108  		"hiwater_rss %lx hiwater_vm %lx total_vm %lx locked_vm %lx\n"
846383359 Konstantin Khlebnikov 2016-01-14  109  		"pinned_vm %lx data_vm %lx exec_vm %lx stack_vm %lx\n"
31c9afa6d Sasha Levin           2014-10-09  110  		"start_code %lx end_code %lx start_data %lx end_data %lx\n"
31c9afa6d Sasha Levin           2014-10-09  111  		"start_brk %lx brk %lx start_stack %lx\n"
31c9afa6d Sasha Levin           2014-10-09  112  		"arg_start %lx arg_end %lx env_start %lx env_end %lx\n"
31c9afa6d Sasha Levin           2014-10-09  113  		"binfmt %p flags %lx core_state %p\n"
31c9afa6d Sasha Levin           2014-10-09  114  #ifdef CONFIG_AIO
31c9afa6d Sasha Levin           2014-10-09  115  		"ioctx_table %p\n"
31c9afa6d Sasha Levin           2014-10-09  116  #endif
31c9afa6d Sasha Levin           2014-10-09  117  #ifdef CONFIG_MEMCG
31c9afa6d Sasha Levin           2014-10-09  118  		"owner %p "
31c9afa6d Sasha Levin           2014-10-09  119  #endif
31c9afa6d Sasha Levin           2014-10-09  120  		"exe_file %p\n"
31c9afa6d Sasha Levin           2014-10-09  121  #ifdef CONFIG_MMU_NOTIFIER
31c9afa6d Sasha Levin           2014-10-09  122  		"mmu_notifier_mm %p\n"
31c9afa6d Sasha Levin           2014-10-09  123  #endif
31c9afa6d Sasha Levin           2014-10-09  124  #ifdef CONFIG_NUMA_BALANCING
31c9afa6d Sasha Levin           2014-10-09  125  		"numa_next_scan %lu numa_scan_offset %lu numa_scan_seq %d\n"
31c9afa6d Sasha Levin           2014-10-09  126  #endif
31c9afa6d Sasha Levin           2014-10-09  127  #if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
31c9afa6d Sasha Levin           2014-10-09  128  		"tlb_flush_pending %d\n"
31c9afa6d Sasha Levin           2014-10-09  129  #endif
b8eceeb99 Vlastimil Babka       2016-03-15  130  		"def_flags: %#lx(%pGv)\n",
31c9afa6d Sasha Levin           2014-10-09  131  
31c9afa6d Sasha Levin           2014-10-09  132  		mm, mm->mmap, mm->vmacache_seqnum, mm->task_size,
31c9afa6d Sasha Levin           2014-10-09  133  #ifdef CONFIG_MMU
31c9afa6d Sasha Levin           2014-10-09  134  		mm->get_unmapped_area,
31c9afa6d Sasha Levin           2014-10-09  135  #endif
31c9afa6d Sasha Levin           2014-10-09  136  		mm->mmap_base, mm->mmap_legacy_base, mm->highest_vm_end,
31c9afa6d Sasha Levin           2014-10-09  137  		mm->pgd, atomic_read(&mm->mm_users),
31c9afa6d Sasha Levin           2014-10-09  138  		atomic_read(&mm->mm_count),
31c9afa6d Sasha Levin           2014-10-09  139  		atomic_long_read((atomic_long_t *)&mm->nr_ptes),
dc6c9a35b Kirill A. Shutemov    2015-02-11  140  		mm_nr_pmds((struct mm_struct *)mm),
31c9afa6d Sasha Levin           2014-10-09  141  		mm->map_count,
31c9afa6d Sasha Levin           2014-10-09  142  		mm->hiwater_rss, mm->hiwater_vm, mm->total_vm, mm->locked_vm,
846383359 Konstantin Khlebnikov 2016-01-14  143  		mm->pinned_vm, mm->data_vm, mm->exec_vm, mm->stack_vm,
31c9afa6d Sasha Levin           2014-10-09  144  		mm->start_code, mm->end_code, mm->start_data, mm->end_data,
31c9afa6d Sasha Levin           2014-10-09  145  		mm->start_brk, mm->brk, mm->start_stack,
31c9afa6d Sasha Levin           2014-10-09  146  		mm->arg_start, mm->arg_end, mm->env_start, mm->env_end,
31c9afa6d Sasha Levin           2014-10-09  147  		mm->binfmt, mm->flags, mm->core_state,
31c9afa6d Sasha Levin           2014-10-09  148  #ifdef CONFIG_AIO
31c9afa6d Sasha Levin           2014-10-09  149  		mm->ioctx_table,
31c9afa6d Sasha Levin           2014-10-09  150  #endif
31c9afa6d Sasha Levin           2014-10-09  151  #ifdef CONFIG_MEMCG
31c9afa6d Sasha Levin           2014-10-09  152  		mm->owner,
31c9afa6d Sasha Levin           2014-10-09  153  #endif
31c9afa6d Sasha Levin           2014-10-09  154  		mm->exe_file,
31c9afa6d Sasha Levin           2014-10-09  155  #ifdef CONFIG_MMU_NOTIFIER
31c9afa6d Sasha Levin           2014-10-09  156  		mm->mmu_notifier_mm,
31c9afa6d Sasha Levin           2014-10-09  157  #endif
31c9afa6d Sasha Levin           2014-10-09  158  #ifdef CONFIG_NUMA_BALANCING
31c9afa6d Sasha Levin           2014-10-09  159  		mm->numa_next_scan, mm->numa_scan_offset, mm->numa_scan_seq,
31c9afa6d Sasha Levin           2014-10-09  160  #endif
fd2fc6e1f Nadav Amit            2017-08-01  161  		atomic_read(&mm->tlb_flush_pending),
b8eceeb99 Vlastimil Babka       2016-03-15  162  		mm->def_flags, &mm->def_flags
31c9afa6d Sasha Levin           2014-10-09  163  	);
31c9afa6d Sasha Levin           2014-10-09  164  }
31c9afa6d Sasha Levin           2014-10-09  165  

:::::: The code at line 102 was first introduced by commit
:::::: 7a82ca0d6437261d0727ce472ae4f3a05a9ce5f7 mm/debug.c: use pr_emerg()

:::::: TO: Andrew Morton <akpm@linux-foundation.org>
:::::: CC: Linus Torvalds <torvalds@linux-foundation.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[toc] | [prev] | [next] | [standalone]

#1701799 — [PATCH v6 4/7] mm: refactoring TLB gathering API

FromNadav Amit <namit@vmware.com>
Date2017-08-02 09:40 +0200
Subject[PATCH v6 4/7] mm: refactoring TLB gathering API
Message-ID<u9Wq7-6U6-27@gated-at.bofh.it>
In reply to#1701796
From: Minchan Kim <minchan@kernel.org>

This patch is a preparatory patch for solving race problems caused by
TLB batch.  For that, we will increase/decrease TLB flush pending count
of mm_struct whenever tlb_[gather|finish]_mmu is called.

Before making it simple, this patch separates architecture specific
part and rename it to arch_tlb_[gather|finish]_mmu and generic part
just calls it.

It shouldn't change any behavior.

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: linux-arch@vger.kernel.org
Cc: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
---
 arch/arm/include/asm/tlb.h  |  6 ++++--
 arch/ia64/include/asm/tlb.h |  6 ++++--
 arch/s390/include/asm/tlb.h | 12 ++++++------
 arch/sh/include/asm/tlb.h   |  6 ++++--
 arch/um/include/asm/tlb.h   |  8 +++++---
 include/asm-generic/tlb.h   |  7 ++++---
 include/linux/mm_types.h    |  6 ++++++
 mm/memory.c                 | 28 +++++++++++++++++++++-------
 8 files changed, 54 insertions(+), 25 deletions(-)

diff --git a/arch/arm/include/asm/tlb.h b/arch/arm/include/asm/tlb.h
index 3f2eb76243e3..7f5b2a2d3861 100644
--- a/arch/arm/include/asm/tlb.h
+++ b/arch/arm/include/asm/tlb.h
@@ -148,7 +148,8 @@ static inline void tlb_flush_mmu(struct mmu_gather *tlb)
 }
 
 static inline void
-tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end)
+arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+			unsigned long start, unsigned long end)
 {
 	tlb->mm = mm;
 	tlb->fullmm = !(start | (end+1));
@@ -166,7 +167,8 @@ tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start
 }
 
 static inline void
-tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
+arch_tlb_finish_mmu(struct mmu_gather *tlb,
+			unsigned long start, unsigned long end)
 {
 	tlb_flush_mmu(tlb);
 
diff --git a/arch/ia64/include/asm/tlb.h b/arch/ia64/include/asm/tlb.h
index fced197b9626..93cadc04ac62 100644
--- a/arch/ia64/include/asm/tlb.h
+++ b/arch/ia64/include/asm/tlb.h
@@ -168,7 +168,8 @@ static inline void __tlb_alloc_page(struct mmu_gather *tlb)
 
 
 static inline void
-tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end)
+arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+			unsigned long start, unsigned long end)
 {
 	tlb->mm = mm;
 	tlb->max = ARRAY_SIZE(tlb->local);
@@ -185,7 +186,8 @@ tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start
  * collected.
  */
 static inline void
-tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
+arch_tlb_finish_mmu(struct mmu_gather *tlb,
+			unsigned long start, unsigned long end)
 {
 	/*
 	 * Note: tlb->nr may be 0 at this point, so we can't rely on tlb->start_addr and
diff --git a/arch/s390/include/asm/tlb.h b/arch/s390/include/asm/tlb.h
index 853b2a3d8dee..0e59ef57e234 100644
--- a/arch/s390/include/asm/tlb.h
+++ b/arch/s390/include/asm/tlb.h
@@ -47,10 +47,9 @@ struct mmu_table_batch {
 extern void tlb_table_flush(struct mmu_gather *tlb);
 extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
 
-static inline void tlb_gather_mmu(struct mmu_gather *tlb,
-				  struct mm_struct *mm,
-				  unsigned long start,
-				  unsigned long end)
+static inline void
+arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+			unsigned long start, unsigned long end)
 {
 	tlb->mm = mm;
 	tlb->start = start;
@@ -76,8 +75,9 @@ static inline void tlb_flush_mmu(struct mmu_gather *tlb)
 	tlb_flush_mmu_free(tlb);
 }
 
-static inline void tlb_finish_mmu(struct mmu_gather *tlb,
-				  unsigned long start, unsigned long end)
+static inline void
+arch_tlb_finish_mmu(struct mmu_gather *tlb,
+		unsigned long start, unsigned long end)
 {
 	tlb_flush_mmu(tlb);
 }
diff --git a/arch/sh/include/asm/tlb.h b/arch/sh/include/asm/tlb.h
index 46e0d635e36f..89786560dbd4 100644
--- a/arch/sh/include/asm/tlb.h
+++ b/arch/sh/include/asm/tlb.h
@@ -36,7 +36,8 @@ static inline void init_tlb_gather(struct mmu_gather *tlb)
 }
 
 static inline void
-tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end)
+arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+		unsigned long start, unsigned long end)
 {
 	tlb->mm = mm;
 	tlb->start = start;
@@ -47,7 +48,8 @@ tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start
 }
 
 static inline void
-tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
+arch_tlb_finish_mmu(struct mmu_gather *tlb,
+		unsigned long start, unsigned long end)
 {
 	if (tlb->fullmm)
 		flush_tlb_mm(tlb->mm);
diff --git a/arch/um/include/asm/tlb.h b/arch/um/include/asm/tlb.h
index 600a2e9bfee2..2a901eca7145 100644
--- a/arch/um/include/asm/tlb.h
+++ b/arch/um/include/asm/tlb.h
@@ -45,7 +45,8 @@ static inline void init_tlb_gather(struct mmu_gather *tlb)
 }
 
 static inline void
-tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end)
+arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+		unsigned long start, unsigned long end)
 {
 	tlb->mm = mm;
 	tlb->start = start;
@@ -80,12 +81,13 @@ tlb_flush_mmu(struct mmu_gather *tlb)
 	tlb_flush_mmu_free(tlb);
 }
 
-/* tlb_finish_mmu
+/* arch_tlb_finish_mmu
  *	Called at the end of the shootdown operation to free up any resources
  *	that were required.
  */
 static inline void
-tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
+arch_tlb_finish_mmu(struct mmu_gather *tlb,
+		unsigned long start, unsigned long end)
 {
 	tlb_flush_mmu(tlb);
 
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 8afa4335e5b2..8f71521e7a44 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -112,10 +112,11 @@ struct mmu_gather {
 
 #define HAVE_GENERIC_MMU_GATHER
 
-void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end);
+void arch_tlb_gather_mmu(struct mmu_gather *tlb,
+	struct mm_struct *mm, unsigned long start, unsigned long end);
 void tlb_flush_mmu(struct mmu_gather *tlb);
-void tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start,
-							unsigned long end);
+void arch_tlb_finish_mmu(struct mmu_gather *tlb,
+			 unsigned long start, unsigned long end);
 extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page,
 				   int page_size);
 
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 2956513619a7..248f4ed1f3e1 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -518,6 +518,12 @@ static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
 	return mm->cpu_vm_mask_var;
 }
 
+struct mmu_gather;
+extern void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+				unsigned long start, unsigned long end);
+extern void tlb_finish_mmu(struct mmu_gather *tlb,
+				unsigned long start, unsigned long end);
+
 #if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
 /*
  * Memory barriers to keep this state in sync are graciously provided by
diff --git a/mm/memory.c b/mm/memory.c
index bb11c474857e..7848b5030be0 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -215,12 +215,8 @@ static bool tlb_next_batch(struct mmu_gather *tlb)
 	return true;
 }
 
-/* tlb_gather_mmu
- *	Called to initialize an (on-stack) mmu_gather structure for page-table
- *	tear-down from @mm. The @fullmm argument is used when @mm is without
- *	users and we're going to destroy the full address space (exit/execve).
- */
-void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end)
+void arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+				unsigned long start, unsigned long end)
 {
 	tlb->mm = mm;
 
@@ -275,7 +271,8 @@ void tlb_flush_mmu(struct mmu_gather *tlb)
  *	Called at the end of the shootdown operation to free up any resources
  *	that were required.
  */
-void tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
+void arch_tlb_finish_mmu(struct mmu_gather *tlb,
+		unsigned long start, unsigned long end)
 {
 	struct mmu_gather_batch *batch, *next;
 
@@ -398,6 +395,23 @@ void tlb_remove_table(struct mmu_gather *tlb, void *table)
 
 #endif /* CONFIG_HAVE_RCU_TABLE_FREE */
 
+/* tlb_gather_mmu
+ *	Called to initialize an (on-stack) mmu_gather structure for page-table
+ *	tear-down from @mm. The @fullmm argument is used when @mm is without
+ *	users and we're going to destroy the full address space (exit/execve).
+ */
+void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+			unsigned long start, unsigned long end)
+{
+	arch_tlb_gather_mmu(tlb, mm, start, end);
+}
+
+void tlb_finish_mmu(struct mmu_gather *tlb,
+		unsigned long start, unsigned long end)
+{
+	arch_tlb_finish_mmu(tlb, start, end);
+}
+
 /*
  * Note: this doesn't free the actual pages themselves. That
  * has been handled earlier when unmapping all the memory regions.
-- 
2.11.0

[toc] | [prev] | [next] | [standalone]

#1709402 — Re: [PATCH v6 4/7] mm: refactoring TLB gathering API

FromPeter Zijlstra <peterz@infradead.org>
Date2017-08-11 11:30 +0200
SubjectRe: [PATCH v6 4/7] mm: refactoring TLB gathering API
Message-ID<udeqt-1IZ-9@gated-at.bofh.it>
In reply to#1701799
On Tue, Aug 01, 2017 at 05:08:15PM -0700, Nadav Amit wrote:
> From: Minchan Kim <minchan@kernel.org>
> 
> This patch is a preparatory patch for solving race problems caused by
> TLB batch.  For that, we will increase/decrease TLB flush pending count
> of mm_struct whenever tlb_[gather|finish]_mmu is called.
> 
> Before making it simple, this patch separates architecture specific
> part and rename it to arch_tlb_[gather|finish]_mmu and generic part
> just calls it.

I absolutely hate this. We should unify this stuff, not diverge it
further.

[toc] | [prev] | [next] | [standalone]

#1709869 — Re: [PATCH v6 4/7] mm: refactoring TLB gathering API

FromNadav Amit <nadav.amit@gmail.com>
Date2017-08-11 19:20 +0200
SubjectRe: [PATCH v6 4/7] mm: refactoring TLB gathering API
Message-ID<udlLj-6j5-3@gated-at.bofh.it>
In reply to#1709402
Peter Zijlstra <peterz@infradead.org> wrote:

> On Tue, Aug 01, 2017 at 05:08:15PM -0700, Nadav Amit wrote:
>> From: Minchan Kim <minchan@kernel.org>
>> 
>> This patch is a preparatory patch for solving race problems caused by
>> TLB batch.  For that, we will increase/decrease TLB flush pending count
>> of mm_struct whenever tlb_[gather|finish]_mmu is called.
>> 
>> Before making it simple, this patch separates architecture specific
>> part and rename it to arch_tlb_[gather|finish]_mmu and generic part
>> just calls it.
> 
> I absolutely hate this. We should unify this stuff, not diverge it
> further.

Agreed, but I don’t see how this patch makes the situation any worse.

I’ll review your other comments by tomorrow due to some personal
constraints.

[toc] | [prev] | [next] | [standalone]

#1702568

FromMinchan Kim <minchan@kernel.org>
Date2017-08-03 01:30 +0200
Message-ID<uabfs-8iN-31@gated-at.bofh.it>
In reply to#1701796
On Tue, Aug 01, 2017 at 05:08:11PM -0700, Nadav Amit wrote:
> It turns out that Linux TLB batching mechanism suffers from various races.
> Races that are caused due to batching during reclamation were recently
> handled by Mel and this patch-set deals with others. The more fundamental
> issue is that concurrent updates of the page-tables allow for TLB flushes
> to be batched on one core, while another core changes the page-tables.
> This other core may assume a PTE change does not require a flush based on
> the updated PTE value, while it is unaware that TLB flushes are still
> pending.
> 
> This behavior affects KSM (which may result in memory corruption) and
> MADV_FREE and MADV_DONTNEED (which may result in incorrect behavior). A
> proof-of-concept can easily produce the wrong behavior of MADV_DONTNEED.
> Memory corruption in KSM is harder to produce in practice, but was observed
> by hacking the kernel and adding a delay before flushing and replacing the
> KSM page.
> 
> Finally, there is also one memory barrier missing, which may affect
> architectures with weak memory model.
> 
> v5 -> v6:
> * Combining with Minchan Kim's patch set, adding ack's (Andrew)
> * Minor: missing header, typos (Nadav)
> * Renaming arch_generic_tlb_finish_mmu (Mel)

Thanks for intergrating/correction, Nadav.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web