Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1583347 > unrolled thread

[PATCH 0/6] Enable parallel page migration

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/6] Enable parallel page migration Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-02-17 12:30 +0100
    [PATCH 6/6] sysctl: Add global tunable mt_page_copy Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-02-17 12:30 +0100
      Re: [PATCH 6/6] sysctl: Add global tunable mt_page_copy kbuild test robot <lkp@intel.com> - 2017-02-17 16:40 +0100
    [PATCH 2/6] mm/migrate: Make migrate_mode types non-exclusive Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-02-17 12:30 +0100
    [PATCH 1/6] mm/migrate: Add new mode parameter to migrate_page_copy() function Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-02-17 12:30 +0100
    [PATCH 4/6] mm/migrate: Add new migrate mode MIGRATE_MT Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-02-17 12:30 +0100
    Re: [PATCH 0/6] Enable parallel page migration Balbir Singh <bsingharora@gmail.com> - 2017-02-22 06:10 +0100
      Re: [PATCH 0/6] Enable parallel page migration Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-02-22 07:00 +0100
        Re: [PATCH 0/6] Enable parallel page migration Balbir Singh <bsingharora@gmail.com> - 2017-02-22 12:00 +0100

#1583347 — [PATCH 0/6] Enable parallel page migration

	This patch series is base on the work posted by Zi Yan back in
November 2016 (https://lkml.org/lkml/2016/11/22/457) but includes some
amount clean up and re-organization. This series depends on THP migration
optimization patch series posted by Naoya Horiguchi on 8th November 2016
(https://lwn.net/Articles/705879/). Though Zi Yan has recently reposted
V3 of the THP migration patch series (https://lwn.net/Articles/713667/),
this series is yet to be rebased.

	Primary motivation behind this patch series is to achieve higher
bandwidth of memory migration when ever possible using multi threaded
instead of a single threaded copy. Did all the experiments using a two
socket X86 sytsem (Intel(R) Xeon(R) CPU E5-2650). All the experiments
here have same allocation size 4K * 100000 (which did not split evenly
for the 2MB huge pages). Here are the results.

Vanilla:

Moved 100000 normal pages in 247.000000 msecs 1.544412 GBs
Moved 100000 normal pages in 238.000000 msecs 1.602814 GBs
Moved 195 huge pages in 252.000000 msecs 1.513769 GBs
Moved 195 huge pages in 257.000000 msecs 1.484318 GBs

THP migration improvements:

Moved 100000 normal pages in 302.000000 msecs 1.263145 GBs
Moved 100000 normal pages in 262.000000 msecs 1.455991 GBs
Moved 195 huge pages in 120.000000 msecs 3.178914 GBs
Moved 195 huge pages in 129.000000 msecs 2.957130 GBs

THP migration improvements + Multi threaded page copy:

Moved 100000 normal pages in 1589.000000 msecs 0.240069 GBs **
Moved 100000 normal pages in 1932.000000 msecs 0.197448 GBs **
Moved 195 huge pages in 54.000000 msecs 7.064254 GBs ***
Moved 195 huge pages in 86.000000 msecs 4.435694 GBs ***


**      Using multi threaded copy can be detrimental to performance if
	used for regular pages which are way too small. But then the
	framework provides the means to use it if some kernel/driver
	caller or user application wants to use it.

***     These applications have used the new MPOL_MF_MOVE_MT flag while
	calling the system calls like mbind() and move_pages().

On POWER8 the improvements are similar when tested with a draft patch
which enables migration at PMD level. Not putting out the results here
as the kernel is not stable with the that draft patch and crashes some
times. We are working on enabling PMD level migration on POWER8 and will
test this series out thoroughly when its ready.

Patch Series Description::

Patch 1: Add new parameter to migrate_page_copy and copy_huge_page so
	 that it can differentiate between when to use single threaded
	 version (MIGRATE_ST) or multi threaded version (MIGRATE_MT).

Patch 2: Make migrate_mode types non-exclusive.

Patch 3: Add copy_pages_mthread function which does the actual multi
	 threaded copy. This involves splitting the copy work into
	 chunks, selecting threads and submitting copy jobs in the
	 work queues.

Patch 4: Add new migrate mode MIGRATE_MT to be used by higher level
	 migration functions.

Patch 5: Add new migration flag MPOL_MF_MOVE_MT for migration system
	 calls to be used in the user space.

Patch 6: Define global mt_page_copy tunable which turns on the multi
	 threaded page copy no matter what for all migrations on the
	 system.

Outstanding Issues::

Issue 1: The usefulness of the global multi threaded copy tunable i.e
	 vm.mt_page_copy. It makes sense and helps in validating the
	 framework. Should this be moved to debugfs instead ?

Issue 2: We choose nr_copythreads = 8 as maximum number of threads on
	 a node can be 8 on any architecture (Which is on POWER8 if
	 I am not missing any other arch which might have equal or
	 more number of threads per node). It just denotes max number
	 of threads and we will be adjusted based on cpumask_weight
	 value on destination node. Can we do better, suggestions ?

Issue 3: Multi threaded page migration works best with threads allocated
	 at different physical cores, not all in the same hyper-threaded
	 core. Work queues submitted jobs consume scheduler slots from
	 the given thread to execute the copy. This can interfere with
	 scheduling and affect some already running tasks on the system.
	 Should we be looking into arch topology information, scheduler
	 cpu idle details to decide on which threads to use before going
	 for multi threaded copy ? Abort multi threaded copy and fallback
	 to regular copy at times when the parameters are not good ?

Any comments, suggestions are welcome.

Zi Yan (6):
  mm/migrate: Add new mode parameter to migrate_page_copy() function
  mm/migrate: Make migrate_mode types non-exclussive
  mm/migrate: Add copy_pages_mthread function
  mm/migrate: Add new migrate mode MIGRATE_MT
  mm/migrate: Add new migration flag MPOL_MF_MOVE_MT for syscalls
  sysctl: Add global tunable mt_page_copy

 fs/aio.c                       |  2 +-
 fs/f2fs/data.c                 |  2 +-
 fs/hugetlbfs/inode.c           |  2 +-
 fs/ubifs/file.c                |  2 +-
 include/linux/highmem.h        |  2 +
 include/linux/migrate.h        |  6 ++-
 include/linux/migrate_mode.h   |  8 ++--
 include/uapi/linux/mempolicy.h |  4 +-
 kernel/sysctl.c                | 10 +++++
 mm/Makefile                    |  2 +
 mm/compaction.c                | 20 +++++-----
 mm/copy_pages_mthread.c        | 87 ++++++++++++++++++++++++++++++++++++++++++
 mm/mempolicy.c                 |  7 +++-
 mm/migrate.c                   | 81 +++++++++++++++++++++++++++------------
 14 files changed, 190 insertions(+), 45 deletions(-)
 create mode 100644 mm/copy_pages_mthread.c

-- 
2.9.3

[toc] | [next] | [standalone]

#1583348 — [PATCH 6/6] sysctl: Add global tunable mt_page_copy

From: Zi Yan <ziy@nvidia.com>

A new global sysctl tunable 'mt_page_copy' is added which will override
syscall specific requests and enable multi threaded page copy during
all migrations on the system. This tunable is disabled by default.

Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
---
 kernel/sysctl.c | 10 ++++++++++
 mm/migrate.c    | 14 ++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 1aea594..e5f7ca9 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -98,6 +98,7 @@
 #if defined(CONFIG_SYSCTL)
 
 /* External variables not in a header file. */
+extern int mt_page_copy;
 extern int suid_dumpable;
 #ifdef CONFIG_COREDUMP
 extern int core_uses_pid;
@@ -1346,6 +1347,15 @@ static struct ctl_table vm_table[] = {
 		.proc_handler   = &hugetlb_mempolicy_sysctl_handler,
 	},
 #endif
+	{
+		.procname	= "mt_page_copy",
+		.data		= &mt_page_copy,
+		.maxlen		= sizeof(mt_page_copy),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec,
+		.extra1		= &zero,
+		.extra2		= &one,
+	},
 	 {
 		.procname	= "hugetlb_shm_group",
 		.data		= &sysctl_hugetlb_shm_group,
diff --git a/mm/migrate.c b/mm/migrate.c
index 660c4b2..75b6d7a 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -48,6 +48,8 @@
 
 #include "internal.h"
 
+int mt_page_copy;
+
 /*
  * migrate_prep() needs to be called before we start compiling a list of pages
  * to be migrated using isolate_lru_page(). If scheduling work on other CPUs is
@@ -611,6 +613,9 @@ static void copy_huge_page(struct page *dst, struct page *src,
 		nr_pages = hpage_nr_pages(src);
 	}
 
+	if (mt_page_copy)
+		mode |= MIGRATE_MT;
+
 	if (mode & MIGRATE_MT)
 		rc = copy_pages_mthread(dst, src, nr_pages);
 
@@ -629,6 +634,9 @@ void migrate_page_copy(struct page *newpage, struct page *page,
 {
 	int cpupid;
 
+	if (mt_page_copy)
+		mode |= MIGRATE_MT;
+
 	if (PageHuge(page) || PageTransHuge(page)) {
 		copy_huge_page(newpage, page, mode);
 	} else {
@@ -695,6 +703,12 @@ void migrate_page_copy(struct page *newpage, struct page *page,
 }
 EXPORT_SYMBOL(migrate_page_copy);
 
+static int __init mt_page_copy_init(void)
+{
+	mt_page_copy = 0;
+	return 0;
+}
+subsys_initcall(mt_page_copy_init);
 /************************************************************
  *                    Migration functions
  ***********************************************************/
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1583536 — Re: [PATCH 6/6] sysctl: Add global tunable mt_page_copy

[Multipart message — attachments visible in raw view] — view raw

Hi Zi,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.10-rc8 next-20170217]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/Enable-parallel-page-migration/20170217-200523
config: i386-randconfig-a0-02131010 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

>> kernel/built-in.o:(.data+0x1ec8): undefined reference to `mt_page_copy'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[toc] | [prev] | [next] | [standalone]

#1583350 — [PATCH 2/6] mm/migrate: Make migrate_mode types non-exclusive

From: Zi Yan <ziy@nvidia.com>

It basically changes the enum declaration from numbers to bit positions
so that they can be used in combination which was not the case earlier.
No functionality has been changed.

Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
---
 include/linux/migrate_mode.h |  8 ++++----
 mm/compaction.c              | 20 ++++++++++----------
 mm/migrate.c                 | 14 +++++++-------
 3 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h
index b3b9acb..89c1700 100644
--- a/include/linux/migrate_mode.h
+++ b/include/linux/migrate_mode.h
@@ -8,10 +8,10 @@
  * MIGRATE_SYNC will block when migrating pages
  */
 enum migrate_mode {
-	MIGRATE_ASYNC,
-	MIGRATE_SYNC_LIGHT,
-	MIGRATE_SYNC,
-	MIGRATE_ST
+	MIGRATE_ASYNC		= 1<<0,
+	MIGRATE_SYNC_LIGHT	= 1<<1,
+	MIGRATE_SYNC		= 1<<2,
+	MIGRATE_ST		= 1<<3,
 };
 
 #endif		/* MIGRATE_MODE_H_INCLUDED */
diff --git a/mm/compaction.c b/mm/compaction.c
index 949198d..1a481af 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -296,7 +296,7 @@ static void update_pageblock_skip(struct compact_control *cc,
 	if (migrate_scanner) {
 		if (pfn > zone->compact_cached_migrate_pfn[0])
 			zone->compact_cached_migrate_pfn[0] = pfn;
-		if (cc->mode != MIGRATE_ASYNC &&
+		if (!(cc->mode & MIGRATE_ASYNC) &&
 		    pfn > zone->compact_cached_migrate_pfn[1])
 			zone->compact_cached_migrate_pfn[1] = pfn;
 	} else {
@@ -329,7 +329,7 @@ static void update_pageblock_skip(struct compact_control *cc,
 static bool compact_trylock_irqsave(spinlock_t *lock, unsigned long *flags,
 						struct compact_control *cc)
 {
-	if (cc->mode == MIGRATE_ASYNC) {
+	if (cc->mode & MIGRATE_ASYNC) {
 		if (!spin_trylock_irqsave(lock, *flags)) {
 			cc->contended = true;
 			return false;
@@ -370,7 +370,7 @@ static bool compact_unlock_should_abort(spinlock_t *lock,
 	}
 
 	if (need_resched()) {
-		if (cc->mode == MIGRATE_ASYNC) {
+		if (cc->mode & MIGRATE_ASYNC) {
 			cc->contended = true;
 			return true;
 		}
@@ -393,7 +393,7 @@ static inline bool compact_should_abort(struct compact_control *cc)
 {
 	/* async compaction aborts if contended */
 	if (need_resched()) {
-		if (cc->mode == MIGRATE_ASYNC) {
+		if (cc->mode & MIGRATE_ASYNC) {
 			cc->contended = true;
 			return true;
 		}
@@ -688,7 +688,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 	 */
 	while (unlikely(too_many_isolated(zone))) {
 		/* async migration should just abort */
-		if (cc->mode == MIGRATE_ASYNC)
+		if (cc->mode & MIGRATE_ASYNC)
 			return 0;
 
 		congestion_wait(BLK_RW_ASYNC, HZ/10);
@@ -700,7 +700,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 	if (compact_should_abort(cc))
 		return 0;
 
-	if (cc->direct_compaction && (cc->mode == MIGRATE_ASYNC)) {
+	if (cc->direct_compaction && (cc->mode & MIGRATE_ASYNC)) {
 		skip_on_failure = true;
 		next_skip_pfn = block_end_pfn(low_pfn, cc->order);
 	}
@@ -1195,7 +1195,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
 	struct page *page;
 	const isolate_mode_t isolate_mode =
 		(sysctl_compact_unevictable_allowed ? ISOLATE_UNEVICTABLE : 0) |
-		(cc->mode != MIGRATE_SYNC ? ISOLATE_ASYNC_MIGRATE : 0);
+		(!(cc->mode & MIGRATE_SYNC) ? ISOLATE_ASYNC_MIGRATE : 0);
 
 	/*
 	 * Start at where we last stopped, or beginning of the zone as
@@ -1241,7 +1241,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
 		 * Async compaction is optimistic to see if the minimum amount
 		 * of work satisfies the allocation.
 		 */
-		if (cc->mode == MIGRATE_ASYNC &&
+		if ((cc->mode & MIGRATE_ASYNC) &&
 		    !migrate_async_suitable(get_pageblock_migratetype(page)))
 			continue;
 
@@ -1481,7 +1481,7 @@ static enum compact_result compact_zone(struct zone *zone, struct compact_contro
 	unsigned long start_pfn = zone->zone_start_pfn;
 	unsigned long end_pfn = zone_end_pfn(zone);
 	const int migratetype = gfpflags_to_migratetype(cc->gfp_mask);
-	const bool sync = cc->mode != MIGRATE_ASYNC;
+	const bool sync = !(cc->mode & MIGRATE_ASYNC);
 
 	ret = compaction_suitable(zone, cc->order, cc->alloc_flags,
 							cc->classzone_idx);
@@ -1577,7 +1577,7 @@ static enum compact_result compact_zone(struct zone *zone, struct compact_contro
 			 * order-aligned block, so skip the rest of it.
 			 */
 			if (cc->direct_compaction &&
-						(cc->mode == MIGRATE_ASYNC)) {
+						(cc->mode & MIGRATE_ASYNC)) {
 				cc->migrate_pfn = block_end_pfn(
 						cc->migrate_pfn - 1, cc->order);
 				/* Draining pcplists is useless in this case */
diff --git a/mm/migrate.c b/mm/migrate.c
index 13fa938..63c3682 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -352,7 +352,7 @@ static bool buffer_migrate_lock_buffers(struct buffer_head *head,
 	struct buffer_head *bh = head;
 
 	/* Simple case, sync compaction */
-	if (mode != MIGRATE_ASYNC) {
+	if (!(mode & MIGRATE_ASYNC)) {
 		do {
 			get_bh(bh);
 			lock_buffer(bh);
@@ -453,7 +453,7 @@ int migrate_page_move_mapping(struct address_space *mapping,
 	 * the mapping back due to an elevated page count, we would have to
 	 * block waiting on other references to be dropped.
 	 */
-	if (mode == MIGRATE_ASYNC && head &&
+	if ((mode & MIGRATE_ASYNC) && head &&
 			!buffer_migrate_lock_buffers(head, mode)) {
 		page_ref_unfreeze(page, expected_count);
 		spin_unlock_irq(&mapping->tree_lock);
@@ -739,7 +739,7 @@ int buffer_migrate_page(struct address_space *mapping,
 	 * with an IRQ-safe spinlock held. In the sync case, the buffers
 	 * need to be locked now
 	 */
-	if (mode != MIGRATE_ASYNC)
+	if (!(mode & MIGRATE_ASYNC))
 		BUG_ON(!buffer_migrate_lock_buffers(head, mode));
 
 	ClearPagePrivate(page);
@@ -821,7 +821,7 @@ static int fallback_migrate_page(struct address_space *mapping,
 {
 	if (PageDirty(page)) {
 		/* Only writeback pages in full synchronous migration */
-		if (mode != MIGRATE_SYNC)
+		if (!(mode & MIGRATE_SYNC))
 			return -EBUSY;
 		return writeout(mapping, page);
 	}
@@ -930,7 +930,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
 	bool is_lru = !__PageMovable(page);
 
 	if (!trylock_page(page)) {
-		if (!force || mode == MIGRATE_ASYNC)
+		if (!force || (mode & MIGRATE_ASYNC))
 			goto out;
 
 		/*
@@ -959,7 +959,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
 		 * the retry loop is too short and in the sync-light case,
 		 * the overhead of stalling is too much
 		 */
-		if (mode != MIGRATE_SYNC) {
+		if (!(mode & MIGRATE_SYNC)) {
 			rc = -EBUSY;
 			goto out_unlock;
 		}
@@ -1229,7 +1229,7 @@ static int unmap_and_move_huge_page(new_page_t get_new_page,
 		return -ENOMEM;
 
 	if (!trylock_page(hpage)) {
-		if (!force || mode != MIGRATE_SYNC)
+		if (!force || !(mode & MIGRATE_SYNC))
 			goto out;
 		lock_page(hpage);
 	}
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1583351 — [PATCH 1/6] mm/migrate: Add new mode parameter to migrate_page_copy() function

From: Zi Yan <ziy@nvidia.com>

This is a prerequisite change required to make page migration framewok
copy in different modes like the default single threaded or the new
multi threaded one yet to be introduced in follow up patches. This
does not change any existing functionality. Only migrate_page_copy()
and copy_huge_page() function's signatures are affected.

Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
---
 fs/aio.c                     |  2 +-
 fs/f2fs/data.c               |  2 +-
 fs/hugetlbfs/inode.c         |  2 +-
 fs/ubifs/file.c              |  2 +-
 include/linux/migrate.h      |  6 ++++--
 include/linux/migrate_mode.h |  1 +
 mm/migrate.c                 | 14 ++++++++------
 7 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/fs/aio.c b/fs/aio.c
index 873b4ca..ba3f6eb 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -418,7 +418,7 @@ static int aio_migratepage(struct address_space *mapping, struct page *new,
 	 * events from being lost.
 	 */
 	spin_lock_irqsave(&ctx->completion_lock, flags);
-	migrate_page_copy(new, old);
+	migrate_page_copy(new, old, MIGRATE_ST);
 	BUG_ON(ctx->ring_pages[idx] != old);
 	ctx->ring_pages[idx] = new;
 	spin_unlock_irqrestore(&ctx->completion_lock, flags);
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 9ac2625..ad41356 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1997,7 +1997,7 @@ int f2fs_migrate_page(struct address_space *mapping,
 		SetPagePrivate(newpage);
 	set_page_private(newpage, page_private(page));
 
-	migrate_page_copy(newpage, page);
+	migrate_page_copy(newpage, page, MIGRATE_ST);
 
 	return MIGRATEPAGE_SUCCESS;
 }
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 54de77e..0e16512f 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -850,7 +850,7 @@ static int hugetlbfs_migrate_page(struct address_space *mapping,
 	rc = migrate_huge_page_move_mapping(mapping, newpage, page);
 	if (rc != MIGRATEPAGE_SUCCESS)
 		return rc;
-	migrate_page_copy(newpage, page);
+	migrate_page_copy(newpage, page, MIGRATE_ST);
 
 	return MIGRATEPAGE_SUCCESS;
 }
diff --git a/fs/ubifs/file.c b/fs/ubifs/file.c
index b0d7837..293616f 100644
--- a/fs/ubifs/file.c
+++ b/fs/ubifs/file.c
@@ -1482,7 +1482,7 @@ static int ubifs_migrate_page(struct address_space *mapping,
 		SetPagePrivate(newpage);
 	}
 
-	migrate_page_copy(newpage, page);
+	migrate_page_copy(newpage, page, MIGRATE_ST);
 	return MIGRATEPAGE_SUCCESS;
 }
 #endif
diff --git a/include/linux/migrate.h b/include/linux/migrate.h
index ae8d475..d843b8f 100644
--- a/include/linux/migrate.h
+++ b/include/linux/migrate.h
@@ -42,7 +42,8 @@ extern void putback_movable_page(struct page *page);
 
 extern int migrate_prep(void);
 extern int migrate_prep_local(void);
-extern void migrate_page_copy(struct page *newpage, struct page *page);
+extern void migrate_page_copy(struct page *newpage, struct page *page,
+			enum migrate_mode mode);
 extern int migrate_huge_page_move_mapping(struct address_space *mapping,
 				  struct page *newpage, struct page *page);
 extern int migrate_page_move_mapping(struct address_space *mapping,
@@ -61,7 +62,8 @@ static inline int migrate_prep(void) { return -ENOSYS; }
 static inline int migrate_prep_local(void) { return -ENOSYS; }
 
 static inline void migrate_page_copy(struct page *newpage,
-				     struct page *page) {}
+				     struct page *page,
+				     enum migrate_mode mode) {}
 
 static inline int migrate_huge_page_move_mapping(struct address_space *mapping,
 				  struct page *newpage, struct page *page)
diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h
index ebf3d89..b3b9acb 100644
--- a/include/linux/migrate_mode.h
+++ b/include/linux/migrate_mode.h
@@ -11,6 +11,7 @@ enum migrate_mode {
 	MIGRATE_ASYNC,
 	MIGRATE_SYNC_LIGHT,
 	MIGRATE_SYNC,
+	MIGRATE_ST
 };
 
 #endif		/* MIGRATE_MODE_H_INCLUDED */
diff --git a/mm/migrate.c b/mm/migrate.c
index 87f4d0f..13fa938 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -589,7 +589,8 @@ static void __copy_gigantic_page(struct page *dst, struct page *src,
 	}
 }
 
-static void copy_huge_page(struct page *dst, struct page *src)
+static void copy_huge_page(struct page *dst, struct page *src,
+				enum migrate_mode mode)
 {
 	int i;
 	int nr_pages;
@@ -618,12 +619,13 @@ static void copy_huge_page(struct page *dst, struct page *src)
 /*
  * Copy the page to its new location
  */
-void migrate_page_copy(struct page *newpage, struct page *page)
+void migrate_page_copy(struct page *newpage, struct page *page,
+					   enum migrate_mode mode)
 {
 	int cpupid;
 
 	if (PageHuge(page) || PageTransHuge(page))
-		copy_huge_page(newpage, page);
+		copy_huge_page(newpage, page, mode);
 	else
 		copy_highpage(newpage, page);
 
@@ -705,7 +707,7 @@ int migrate_page(struct address_space *mapping,
 	if (rc != MIGRATEPAGE_SUCCESS)
 		return rc;
 
-	migrate_page_copy(newpage, page);
+	migrate_page_copy(newpage, page, mode);
 	return MIGRATEPAGE_SUCCESS;
 }
 EXPORT_SYMBOL(migrate_page);
@@ -755,7 +757,7 @@ int buffer_migrate_page(struct address_space *mapping,
 
 	SetPagePrivate(newpage);
 
-	migrate_page_copy(newpage, page);
+	migrate_page_copy(newpage, page, MIGRATE_ST);
 
 	bh = head;
 	do {
@@ -1968,7 +1970,7 @@ int migrate_misplaced_transhuge_page(struct mm_struct *mm,
 	/* anon mapping, we can simply copy page->mapping to the new page: */
 	new_page->mapping = page->mapping;
 	new_page->index = page->index;
-	migrate_page_copy(new_page, page);
+	migrate_page_copy(new_page, page, MIGRATE_ST);
 	WARN_ON(PageLRU(new_page));
 
 	/* Recheck the target PMD */
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1583352 — [PATCH 4/6] mm/migrate: Add new migrate mode MIGRATE_MT

From: Zi Yan <ziy@nvidia.com>

This change adds a new migration mode called MIGRATE_MT to enable multi
threaded page copy implementation inside copy_huge_page() function by
selectively calling copy_pages_mthread() when requested. But it still
falls back using the regular page copy mechanism instead the previous
multi threaded attempt fails. It also attempts multi threaded copy for
regular pages.

Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
---
 include/linux/migrate_mode.h |  1 +
 mm/migrate.c                 | 25 ++++++++++++++++++-------
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h
index 89c1700..d344ad6 100644
--- a/include/linux/migrate_mode.h
+++ b/include/linux/migrate_mode.h
@@ -12,6 +12,7 @@ enum migrate_mode {
 	MIGRATE_SYNC_LIGHT	= 1<<1,
 	MIGRATE_SYNC		= 1<<2,
 	MIGRATE_ST		= 1<<3,
+	MIGRATE_MT		= 1<<4,
 };
 
 #endif		/* MIGRATE_MODE_H_INCLUDED */
diff --git a/mm/migrate.c b/mm/migrate.c
index 63c3682..6ac3572 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -594,6 +594,7 @@ static void copy_huge_page(struct page *dst, struct page *src,
 {
 	int i;
 	int nr_pages;
+	int rc = -EFAULT;
 
 	if (PageHuge(src)) {
 		/* hugetlbfs page */
@@ -610,10 +611,14 @@ static void copy_huge_page(struct page *dst, struct page *src,
 		nr_pages = hpage_nr_pages(src);
 	}
 
-	for (i = 0; i < nr_pages; i++) {
-		cond_resched();
-		copy_highpage(dst + i, src + i);
-	}
+	if (mode & MIGRATE_MT)
+		rc = copy_pages_mthread(dst, src, nr_pages);
+
+	if (rc)
+		for (i = 0; i < nr_pages; i++) {
+			cond_resched();
+			copy_highpage(dst + i, src + i);
+		}
 }
 
 /*
@@ -624,10 +629,16 @@ void migrate_page_copy(struct page *newpage, struct page *page,
 {
 	int cpupid;
 
-	if (PageHuge(page) || PageTransHuge(page))
+	if (PageHuge(page) || PageTransHuge(page)) {
 		copy_huge_page(newpage, page, mode);
-	else
-		copy_highpage(newpage, page);
+	} else {
+		if (mode & MIGRATE_MT) {
+			if (copy_pages_mthread(newpage, page, 1))
+				copy_highpage(newpage, page);
+		} else {
+			copy_highpage(newpage, page);
+		}
+	}
 
 	if (PageError(page))
 		SetPageError(newpage);
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1585919

On Fri, Feb 17, 2017 at 04:54:47PM +0530, Anshuman Khandual wrote:
> 	This patch series is base on the work posted by Zi Yan back in
> November 2016 (https://lkml.org/lkml/2016/11/22/457) but includes some
> amount clean up and re-organization. This series depends on THP migration
> optimization patch series posted by Naoya Horiguchi on 8th November 2016
> (https://lwn.net/Articles/705879/). Though Zi Yan has recently reposted
> V3 of the THP migration patch series (https://lwn.net/Articles/713667/),
> this series is yet to be rebased.
> 
> 	Primary motivation behind this patch series is to achieve higher
> bandwidth of memory migration when ever possible using multi threaded
> instead of a single threaded copy. Did all the experiments using a two
> socket X86 sytsem (Intel(R) Xeon(R) CPU E5-2650). All the experiments
> here have same allocation size 4K * 100000 (which did not split evenly
> for the 2MB huge pages). Here are the results.
> 
> Vanilla:
> 
> Moved 100000 normal pages in 247.000000 msecs 1.544412 GBs
> Moved 100000 normal pages in 238.000000 msecs 1.602814 GBs
> Moved 195 huge pages in 252.000000 msecs 1.513769 GBs
> Moved 195 huge pages in 257.000000 msecs 1.484318 GBs
> 
> THP migration improvements:
> 
> Moved 100000 normal pages in 302.000000 msecs 1.263145 GBs

Is there a decrease here for normal pages?

> Moved 100000 normal pages in 262.000000 msecs 1.455991 GBs
> Moved 195 huge pages in 120.000000 msecs 3.178914 GBs
> Moved 195 huge pages in 129.000000 msecs 2.957130 GBs
> 
> THP migration improvements + Multi threaded page copy:
> 
> Moved 100000 normal pages in 1589.000000 msecs 0.240069 GBs **

Ditto?

> Moved 100000 normal pages in 1932.000000 msecs 0.197448 GBs **
> Moved 195 huge pages in 54.000000 msecs 7.064254 GBs ***
> Moved 195 huge pages in 86.000000 msecs 4.435694 GBs ***
>

Could you also comment on the CPU utilization impact of these
patches. 

Balbir Singh.

[toc] | [prev] | [next] | [standalone]

#1585941

On 02/22/2017 10:34 AM, Balbir Singh wrote:
> On Fri, Feb 17, 2017 at 04:54:47PM +0530, Anshuman Khandual wrote:
>> 	This patch series is base on the work posted by Zi Yan back in
>> November 2016 (https://lkml.org/lkml/2016/11/22/457) but includes some
>> amount clean up and re-organization. This series depends on THP migration
>> optimization patch series posted by Naoya Horiguchi on 8th November 2016
>> (https://lwn.net/Articles/705879/). Though Zi Yan has recently reposted
>> V3 of the THP migration patch series (https://lwn.net/Articles/713667/),
>> this series is yet to be rebased.
>>
>> 	Primary motivation behind this patch series is to achieve higher
>> bandwidth of memory migration when ever possible using multi threaded
>> instead of a single threaded copy. Did all the experiments using a two
>> socket X86 sytsem (Intel(R) Xeon(R) CPU E5-2650). All the experiments
>> here have same allocation size 4K * 100000 (which did not split evenly
>> for the 2MB huge pages). Here are the results.
>>
>> Vanilla:
>>
>> Moved 100000 normal pages in 247.000000 msecs 1.544412 GBs
>> Moved 100000 normal pages in 238.000000 msecs 1.602814 GBs
>> Moved 195 huge pages in 252.000000 msecs 1.513769 GBs
>> Moved 195 huge pages in 257.000000 msecs 1.484318 GBs
>>
>> THP migration improvements:
>>
>> Moved 100000 normal pages in 302.000000 msecs 1.263145 GBs
> 
> Is there a decrease here for normal pages?

Yeah.

> 
>> Moved 100000 normal pages in 262.000000 msecs 1.455991 GBs
>> Moved 195 huge pages in 120.000000 msecs 3.178914 GBs
>> Moved 195 huge pages in 129.000000 msecs 2.957130 GBs
>>
>> THP migration improvements + Multi threaded page copy:
>>
>> Moved 100000 normal pages in 1589.000000 msecs 0.240069 GBs **
> 
> Ditto?

Yeah, I have already mentioned about this after these data in
the cover letter. This new flag is controlled from user space
while invoking the system calls. Users should be careful in
using it for scenarios where its useful and avoid it for cases
where it hurts.

> 
>> Moved 100000 normal pages in 1932.000000 msecs 0.197448 GBs **
>> Moved 195 huge pages in 54.000000 msecs 7.064254 GBs ***
>> Moved 195 huge pages in 86.000000 msecs 4.435694 GBs ***
>>
> 
> Could you also comment on the CPU utilization impact of these
> patches. 

Yeah, it really makes sense to analyze this impact. I have mentioned
about this in the outstanding issues section of the series. But what
exactly we need to analyze from CPU utilization impact point of view
? Like whats the probability that the work queue requested jobs will
throw some tasks from the run queue and make them starve for some
more time ? Could you please give some details on this ?

[toc] | [prev] | [next] | [standalone]

#1586062

On 22/02/17 16:55, Anshuman Khandual wrote:
> On 02/22/2017 10:34 AM, Balbir Singh wrote:
>> On Fri, Feb 17, 2017 at 04:54:47PM +0530, Anshuman Khandual wrote:
>>> 	This patch series is base on the work posted by Zi Yan back in
>>> November 2016 (https://lkml.org/lkml/2016/11/22/457) but includes some
>>> amount clean up and re-organization. This series depends on THP migration
>>> optimization patch series posted by Naoya Horiguchi on 8th November 2016
>>> (https://lwn.net/Articles/705879/). Though Zi Yan has recently reposted
>>> V3 of the THP migration patch series (https://lwn.net/Articles/713667/),
>>> this series is yet to be rebased.
>>>
>>> 	Primary motivation behind this patch series is to achieve higher
>>> bandwidth of memory migration when ever possible using multi threaded
>>> instead of a single threaded copy. Did all the experiments using a two
>>> socket X86 sytsem (Intel(R) Xeon(R) CPU E5-2650). All the experiments
>>> here have same allocation size 4K * 100000 (which did not split evenly
>>> for the 2MB huge pages). Here are the results.
>>>
>>> Vanilla:
>>>
>>> Moved 100000 normal pages in 247.000000 msecs 1.544412 GBs
>>> Moved 100000 normal pages in 238.000000 msecs 1.602814 GBs
>>> Moved 195 huge pages in 252.000000 msecs 1.513769 GBs
>>> Moved 195 huge pages in 257.000000 msecs 1.484318 GBs
>>>
>>> THP migration improvements:
>>>
>>> Moved 100000 normal pages in 302.000000 msecs 1.263145 GBs
>>
>> Is there a decrease here for normal pages?
> 
> Yeah.
> 
>>
>>> Moved 100000 normal pages in 262.000000 msecs 1.455991 GBs
>>> Moved 195 huge pages in 120.000000 msecs 3.178914 GBs
>>> Moved 195 huge pages in 129.000000 msecs 2.957130 GBs
>>>
>>> THP migration improvements + Multi threaded page copy:
>>>
>>> Moved 100000 normal pages in 1589.000000 msecs 0.240069 GBs **
>>
>> Ditto?
> 
> Yeah, I have already mentioned about this after these data in
> the cover letter. This new flag is controlled from user space
> while invoking the system calls. Users should be careful in
> using it for scenarios where its useful and avoid it for cases
> where it hurts.

Fair enough, I wonder if _MT should be disabled for normal pages
and allow only THP migration. I think it might be worth evaluating
the overheads

> 
>>
>>> Moved 100000 normal pages in 1932.000000 msecs 0.197448 GBs **
>>> Moved 195 huge pages in 54.000000 msecs 7.064254 GBs ***
>>> Moved 195 huge pages in 86.000000 msecs 4.435694 GBs ***
>>>
>>
>> Could you also comment on the CPU utilization impact of these
>> patches. 
> 
> Yeah, it really makes sense to analyze this impact. I have mentioned
> about this in the outstanding issues section of the series. But what
> exactly we need to analyze from CPU utilization impact point of view
> ? Like whats the probability that the work queue requested jobs will
> throw some tasks from the run queue and make them starve for some
> more time ? Could you please give some details on this ?
> 

I wonder if the CPU utilization is so high that its hurting the CPU
(system time) at the cost of increased migration speeds. We may need
a trade-off (see my comment above)

Balbir Singh.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web