Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1157878 > unrolled thread

[PATCHv6 00/36] THP refcounting redesign

Back to article view | Back to linux.kernel

Contents

  [PATCHv6 00/36] THP refcounting redesign "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:10 +0200
    [PATCHv6 12/36] thp: drop all split_huge_page()-related code "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:10 +0200
    [PATCHv6 18/36] arm, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:10 +0200
    [PATCHv6 01/36] mm, proc: adjust PSS calculation "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:10 +0200
    [PATCHv6 28/36] mm, numa: skip PTE-mapped THP on numa fault "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:10 +0200
    [PATCHv6 02/36] rmap: add argument to charge compound page "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:10 +0200
    [PATCHv6 24/36] x86, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 08/36] khugepaged: ignore pmd tables with THP mapped with ptes "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 19/36] mips, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 33/36] migrate_pages: try to split pages on qeueuing "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 13/36] mm: drop tail page refcounting "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 27/36] mm: differentiate page_mapped() from page_mapcount() for compound pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 32/36] thp: reintroduce split_huge_page() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 21/36] s390, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 29/36] thp: implement split_huge_pmd() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 20/36] powerpc, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 17/36] arm64, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 34/36] thp: introduce deferred_split_huge_page() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 30/36] thp: add option to setup migration entiries during PMD split "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 10/36] mm, vmstats: new THP splitting event "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 23/36] tile, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 06/36] mm: handle PTE-mapped tail pages in gerneric fast gup implementaiton "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 31/36] thp, mm: split_huge_page(): caller need to lock page "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 35/36] mm: re-enable THP "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200
    [PATCHv6 16/36] mm, thp: remove compound_lock "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-06-03 19:20 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1157919 — [PATCHv6 23/36] tile, thp: remove infrastructure for handling splitting PMDs

With new refcounting we don't need to mark PMDs splitting. Let's drop
code to handle this.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/tile/include/asm/pgtable.h | 10 ----------
 1 file changed, 10 deletions(-)

diff --git a/arch/tile/include/asm/pgtable.h b/arch/tile/include/asm/pgtable.h
index 2b05ccbebed9..96cecf55522e 100644
--- a/arch/tile/include/asm/pgtable.h
+++ b/arch/tile/include/asm/pgtable.h
@@ -489,16 +489,6 @@ static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
 #define has_transparent_hugepage() 1
 #define pmd_trans_huge pmd_huge_page
-
-static inline pmd_t pmd_mksplitting(pmd_t pmd)
-{
-	return pte_pmd(hv_pte_set_client2(pmd_pte(pmd)));
-}
-
-static inline int pmd_trans_splitting(pmd_t pmd)
-{
-	return hv_pte_get_client2(pmd_pte(pmd));
-}
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 
 /*
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1157920 — [PATCHv6 06/36] mm: handle PTE-mapped tail pages in gerneric fast gup implementaiton

With new refcounting we are going to see THP tail pages mapped with PTE.
Generic fast GUP rely on page_cache_get_speculative() to obtain
reference on page. page_cache_get_speculative() always fails on tail
pages, because ->_count on tail pages is always zero.

Let's handle tail pages in gup_pte_range().

New split_huge_page() will rely on migration entries to freeze page's
counts. Recheck PTE value after page_cache_get_speculative() on head
page should be enough to serialize against split.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Sasha Levin <sasha.levin@oracle.com>
Acked-by: Jerome Marchand <jmarchan@redhat.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 mm/gup.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index 2dd6706ea29f..d975bdc2465f 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1051,7 +1051,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		 * for an example see gup_get_pte in arch/x86/mm/gup.c
 		 */
 		pte_t pte = READ_ONCE(*ptep);
-		struct page *page;
+		struct page *head, *page;
 
 		/*
 		 * Similar to the PMD case below, NUMA hinting must take slow
@@ -1063,15 +1063,17 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 
 		VM_BUG_ON(!pfn_valid(pte_pfn(pte)));
 		page = pte_page(pte);
+		head = compound_head(page);
 
-		if (!page_cache_get_speculative(page))
+		if (!page_cache_get_speculative(head))
 			goto pte_unmap;
 
 		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
-			put_page(page);
+			put_page(head);
 			goto pte_unmap;
 		}
 
+		VM_BUG_ON_PAGE(compound_head(page) != head, page);
 		pages[*nr] = page;
 		(*nr)++;
 
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1157922 — [PATCHv6 31/36] thp, mm: split_huge_page(): caller need to lock page

We're going to use migration entries instead of compound_lock() to
stabilize page refcounts. Setup and remove migration entries require
page to be locked.

Some of split_huge_page() callers already have the page locked. Let's
require everybody to lock the page before calling split_huge_page().

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Sasha Levin <sasha.levin@oracle.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 mm/memory-failure.c | 10 ++++++++--
 mm/migrate.c        |  8 ++++++--
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 1cf7f2988422..0d9989a36d32 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1143,15 +1143,18 @@ int memory_failure(unsigned long pfn, int trapno, int flags)
 				put_page(hpage);
 			return -EBUSY;
 		}
+		lock_page(hpage);
 		if (unlikely(split_huge_page(hpage))) {
 			pr_err("MCE: %#lx: thp split failed\n", pfn);
 			if (TestClearPageHWPoison(p))
 				atomic_long_sub(nr_pages, &num_poisoned_pages);
+			unlock_page(hpage);
 			put_page(p);
 			if (p != hpage)
 				put_page(hpage);
 			return -EBUSY;
 		}
+		unlock_page(hpage);
 		VM_BUG_ON_PAGE(!page_count(p), p);
 		hpage = compound_head(p);
 	}
@@ -1714,10 +1717,13 @@ int soft_offline_page(struct page *page, int flags)
 		return -EBUSY;
 	}
 	if (!PageHuge(page) && PageTransHuge(hpage)) {
-		if (PageAnon(hpage) && unlikely(split_huge_page(hpage))) {
+		lock_page(page);
+		ret = split_huge_page(hpage);
+		unlock_page(page);
+		if (unlikely(ret)) {
 			pr_info("soft offline: %#lx: failed to split THP\n",
 				pfn);
-			return -EBUSY;
+			return ret;
 		}
 	}
 
diff --git a/mm/migrate.c b/mm/migrate.c
index dfd24cb7afc6..8bb2107b8751 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -933,9 +933,13 @@ static ICE_noinline int unmap_and_move(new_page_t get_new_page,
 		goto out;
 	}
 
-	if (unlikely(PageTransHuge(page)))
-		if (unlikely(split_huge_page(page)))
+	if (unlikely(PageTransHuge(page))) {
+		lock_page(page);
+		rc = split_huge_page(page);
+		unlock_page(page);
+		if (rc)
 			goto out;
+	}
 
 	rc = __unmap_and_move(page, newpage, force, mode);
 
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1157923 — [PATCHv6 35/36] mm: re-enable THP

All parts of THP with new refcounting are now in place. We can now allow
to enable THP.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Sasha Levin <sasha.levin@oracle.com>
---
 mm/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/Kconfig b/mm/Kconfig
index c973f416cbe5..e79de2bd12cd 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -410,7 +410,7 @@ config NOMMU_INITIAL_TRIM_EXCESS
 
 config TRANSPARENT_HUGEPAGE
 	bool "Transparent Hugepage Support"
-	depends on HAVE_ARCH_TRANSPARENT_HUGEPAGE && BROKEN
+	depends on HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	select COMPACTION
 	help
 	  Transparent Hugepages allows the kernel to use huge pages and
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1157924 — [PATCHv6 16/36] mm, thp: remove compound_lock

We are going to use migration entries to stabilize page counts. It means
we don't need compound_lock() for that.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Sasha Levin <sasha.levin@oracle.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/linux/mm.h         | 35 -----------------------------------
 include/linux/page-flags.h | 12 +-----------
 mm/debug.c                 |  3 ---
 mm/memcontrol.c            | 11 +++--------
 4 files changed, 4 insertions(+), 57 deletions(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index f079d9ffc797..31cd5be081cf 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -403,41 +403,6 @@ static inline int is_vmalloc_or_module_addr(const void *x)
 
 extern void kvfree(const void *addr);
 
-static inline void compound_lock(struct page *page)
-{
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	VM_BUG_ON_PAGE(PageSlab(page), page);
-	bit_spin_lock(PG_compound_lock, &page->flags);
-#endif
-}
-
-static inline void compound_unlock(struct page *page)
-{
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	VM_BUG_ON_PAGE(PageSlab(page), page);
-	bit_spin_unlock(PG_compound_lock, &page->flags);
-#endif
-}
-
-static inline unsigned long compound_lock_irqsave(struct page *page)
-{
-	unsigned long uninitialized_var(flags);
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	local_irq_save(flags);
-	compound_lock(page);
-#endif
-	return flags;
-}
-
-static inline void compound_unlock_irqrestore(struct page *page,
-					      unsigned long flags)
-{
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	compound_unlock(page);
-	local_irq_restore(flags);
-#endif
-}
-
 /*
  * The atomic page->_mapcount, starts from -1: so that transitions
  * both from it and to it can be tracked, using atomic_inc_and_test
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 91b7f9b2b774..74b7cece1dfa 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -106,9 +106,6 @@ enum pageflags {
 #ifdef CONFIG_MEMORY_FAILURE
 	PG_hwpoison,		/* hardware poisoned page. Don't touch */
 #endif
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	PG_compound_lock,
-#endif
 	__NR_PAGEFLAGS,
 
 	/* Filesystems */
@@ -683,12 +680,6 @@ static inline void ClearPageSlabPfmemalloc(struct page *page)
 #define __PG_MLOCKED		0
 #endif
 
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-#define __PG_COMPOUND_LOCK		(1 << PG_compound_lock)
-#else
-#define __PG_COMPOUND_LOCK		0
-#endif
-
 /*
  * Flags checked when a page is freed.  Pages being freed should not have
  * these flags set.  It they are, there is a problem.
@@ -698,8 +689,7 @@ static inline void ClearPageSlabPfmemalloc(struct page *page)
 	 1 << PG_private | 1 << PG_private_2 | \
 	 1 << PG_writeback | 1 << PG_reserved | \
 	 1 << PG_slab	 | 1 << PG_swapcache | 1 << PG_active | \
-	 1 << PG_unevictable | __PG_MLOCKED | __PG_HWPOISON | \
-	 __PG_COMPOUND_LOCK)
+	 1 << PG_unevictable | __PG_MLOCKED | __PG_HWPOISON )
 
 /*
  * Flags checked when a page is prepped for return by the page allocator.
diff --git a/mm/debug.c b/mm/debug.c
index 3eb3ac2fcee7..9dfcd77e7354 100644
--- a/mm/debug.c
+++ b/mm/debug.c
@@ -45,9 +45,6 @@ static const struct trace_print_flags pageflag_names[] = {
 #ifdef CONFIG_MEMORY_FAILURE
 	{1UL << PG_hwpoison,		"hwpoison"	},
 #endif
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	{1UL << PG_compound_lock,	"compound_lock"	},
-#endif
 };
 
 static void dump_flags(unsigned long flags,
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 1c7c6bb69d3c..d0d862e4ca74 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2761,9 +2761,7 @@ struct mem_cgroup *__mem_cgroup_from_kmem(void *ptr)
 
 /*
  * Because tail pages are not marked as "used", set it. We're under
- * zone->lru_lock, 'splitting on pmd' and compound_lock.
- * charge/uncharge will be never happen and move_account() is done under
- * compound_lock(), so we don't have to take care of races.
+ * zone->lru_lock and migration entries setup in all page mappings.
  */
 void mem_cgroup_split_huge_fixup(struct page *head)
 {
@@ -4738,9 +4736,7 @@ static struct page *mc_handle_file_pte(struct vm_area_struct *vma,
  * @from: mem_cgroup which the page is moved from.
  * @to:	mem_cgroup which the page is moved to. @from != @to.
  *
- * The caller must confirm following.
- * - page is not on LRU (isolate_page() is useful.)
- * - compound_lock is held when nr_pages > 1
+ * The caller must make sure the page is not on LRU (isolate_page() is useful.)
  *
  * This function doesn't do "charge" to new cgroup and doesn't do "uncharge"
  * from old cgroup.
@@ -5061,8 +5057,7 @@ static int mem_cgroup_move_charge_pte_range(pmd_t *pmd,
 	struct page *page;
 
 	/*
-	 * We don't take compound_lock() here but no race with splitting thp
-	 * happens because:
+	 * No race with splitting thp happens because:
 	 *  - if pmd_trans_huge_lock() returns 1, the relevant thp is not
 	 *    under splitting, which means there's no concurrent thp split,
 	 *  - if another thread runs into split_huge_page() just after we
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web