Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1187758
| From | Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH 4.0 33/58] mm, thp: respect MPOL_PREFERRED policy with non-local node |
| Date | 2015-07-19 21:20 +0200 |
| Message-ID | <pO2hY-3q1-33@gated-at.bofh.it> (permalink) |
| References | <pO2hX-3q1-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
4.0-stable review patch. If anyone has any objections, please let me know.
------------------
From: Vlastimil Babka <vbabka@suse.cz>
commit 0867a57c4f80a566dda1bac975b42fcd857cb489 upstream.
Since commit 077fcf116c8c ("mm/thp: allocate transparent hugepages on
local node"), we handle THP allocations on page fault in a special way -
for non-interleave memory policies, the allocation is only attempted on
the node local to the current CPU, if the policy's nodemask allows the
node.
This is motivated by the assumption that THP benefits cannot offset the
cost of remote accesses, so it's better to fallback to base pages on the
local node (which might still be available, while huge pages are not due
to fragmentation) than to allocate huge pages on a remote node.
The nodemask check prevents us from violating e.g. MPOL_BIND policies
where the local node is not among the allowed nodes. However, the
current implementation can still give surprising results for the
MPOL_PREFERRED policy when the preferred node is different than the
current CPU's local node.
In such case we should honor the preferred node and not use the local
node, which is what this patch does. If hugepage allocation on the
preferred node fails, we fall back to base pages and don't try other
nodes, with the same motivation as is done for the local node hugepage
allocations. The patch also moves the MPOL_INTERLEAVE check around to
simplify the hugepage specific test.
The difference can be demonstrated using in-tree transhuge-stress test
on the following 2-node machine where half memory on one node was
occupied to show the difference.
> numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 24 25 26 27 28 29 30 31 32 33 34 35
node 0 size: 7878 MB
node 0 free: 3623 MB
node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23 36 37 38 39 40 41 42 43 44 45 46 47
node 1 size: 8045 MB
node 1 free: 7818 MB
node distances:
node 0 1
0: 10 21
1: 21 10
Before the patch:
> numactl -p0 -C0 ./transhuge-stress
transhuge-stress: 2.197 s/loop, 0.276 ms/page, 7249.168 MiB/s 7962 succeed, 0 failed, 1786 different pages
> numactl -p0 -C12 ./transhuge-stress
transhuge-stress: 2.962 s/loop, 0.372 ms/page, 5376.172 MiB/s 7962 succeed, 0 failed, 3873 different pages
Number of successful THP allocations corresponds to free memory on node 0 in
the first case and node 1 in the second case, i.e. -p parameter is ignored and
cpu binding "wins".
After the patch:
> numactl -p0 -C0 ./transhuge-stress
transhuge-stress: 2.183 s/loop, 0.274 ms/page, 7295.516 MiB/s 7962 succeed, 0 failed, 1760 different pages
> numactl -p0 -C12 ./transhuge-stress
transhuge-stress: 2.878 s/loop, 0.361 ms/page, 5533.638 MiB/s 7962 succeed, 0 failed, 1750 different pages
> numactl -p1 -C0 ./transhuge-stress
transhuge-stress: 4.628 s/loop, 0.581 ms/page, 3440.893 MiB/s 7962 succeed, 0 failed, 3918 different pages
The -p parameter is respected regardless of cpu binding.
> numactl -C0 ./transhuge-stress
transhuge-stress: 2.202 s/loop, 0.277 ms/page, 7230.003 MiB/s 7962 succeed, 0 failed, 1750 different pages
> numactl -C12 ./transhuge-stress
transhuge-stress: 3.020 s/loop, 0.379 ms/page, 5273.324 MiB/s 7962 succeed, 0 failed, 3916 different pages
Without -p parameter, hugepage restriction to CPU-local node works as before.
Fixes: 077fcf116c8c ("mm/thp: allocate transparent hugepages on local node")
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: David Rientjes <rientjes@google.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Michal Hocko <mhocko@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
mm/mempolicy.c | 38 ++++++++++++++++++++++----------------
1 file changed, 22 insertions(+), 16 deletions(-)
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1971,35 +1971,41 @@ retry_cpuset:
pol = get_vma_policy(vma, addr);
cpuset_mems_cookie = read_mems_allowed_begin();
- if (unlikely(IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && hugepage &&
- pol->mode != MPOL_INTERLEAVE)) {
+ if (pol->mode == MPOL_INTERLEAVE) {
+ unsigned nid;
+
+ nid = interleave_nid(pol, vma, addr, PAGE_SHIFT + order);
+ mpol_cond_put(pol);
+ page = alloc_page_interleave(gfp, order, nid);
+ goto out;
+ }
+
+ if (unlikely(IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && hugepage)) {
+ int hpage_node = node;
+
/*
* For hugepage allocation and non-interleave policy which
- * allows the current node, we only try to allocate from the
- * current node and don't fall back to other nodes, as the
- * cost of remote accesses would likely offset THP benefits.
+ * allows the current node (or other explicitly preferred
+ * node) we only try to allocate from the current/preferred
+ * node and don't fall back to other nodes, as the cost of
+ * remote accesses would likely offset THP benefits.
*
* If the policy is interleave, or does not allow the current
* node in its nodemask, we allocate the standard way.
*/
+ if (pol->mode == MPOL_PREFERRED &&
+ !(pol->flags & MPOL_F_LOCAL))
+ hpage_node = pol->v.preferred_node;
+
nmask = policy_nodemask(gfp, pol);
- if (!nmask || node_isset(node, *nmask)) {
+ if (!nmask || node_isset(hpage_node, *nmask)) {
mpol_cond_put(pol);
- page = alloc_pages_exact_node(node,
+ page = alloc_pages_exact_node(hpage_node,
gfp | __GFP_THISNODE, order);
goto out;
}
}
- if (pol->mode == MPOL_INTERLEAVE) {
- unsigned nid;
-
- nid = interleave_nid(pol, vma, addr, PAGE_SHIFT + order);
- mpol_cond_put(pol);
- page = alloc_page_interleave(gfp, order, nid);
- goto out;
- }
-
nmask = policy_nodemask(gfp, pol);
zl = policy_zonelist(gfp, pol, node);
mpol_cond_put(pol);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH 4.0 00/58] 4.0.9-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 36/58] regmap: Fix possible shift overflow in regmap_field_init() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 57/58] of/pci: Fix pci_address_to_pio() conversion of CPU address to I/O port Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 35/58] regmap: Fix regmap_bulk_read in BE mode Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 22/58] tools selftests: Fix clean target with make 3.81 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 09/58] sysfs: Create mountpoints with sysfs_create_mount_point Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 20/58] ACPI / PM: Add missing pm_generic_complete() invocation Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 33/58] mm, thp: respect MPOL_PREFERRED policy with non-local node Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 24/58] ARC: add compiler barrier to LLSC based cmpxchg Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 44/58] scsi_transport_srp: Fix a race condition Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 19/58] ACPI / init: Switch over platform to the ACPI mode later Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 32/58] mm: kmemleak_alloc_percpu() should follow the gfp from per_alloc() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 31/58] mm: kmemleak: allow safe memory scanning during kmemleak disabling Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 18/58] ALSA: hda - Fix the dock headphone output on Fujitsu Lifebook E780 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 23/58] ARC: add smp barriers around atomics per Documentation/atomic_ops.txt Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 07/58] mnt: Refactor the logic for mounting sysfs and proc in a user namespace Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 26/58] mei: txe: reduce suspend/resume time Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 15/58] ALSA: hda - Fix Dock Headphone on Thinkpad X250 seen as a Line Out Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 29/58] arm64: mm: Fix freeing of the wrong memmap entries with !SPARSEMEM_VMEMMAP Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 30/58] arm64: vdso: work-around broken ELF toolchains in Makefile Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 25/58] mei: me: wait for power gating exit confirmation Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 27/58] arm64: Do not attempt to use init_mm in reset_context() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 28/58] arm64: entry: fix context tracking for el0_sp_pc Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 21/58] iio: accel: kxcjk-1013: add the "KXCJ9000" ACPI id Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 17/58] ALSA: hda - Add headset support to Acer Aspire V5 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 08/58] mnt: Modify fs_fully_visible to deal with locked ro nodev and atime Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 16/58] ALSA: hda - set proper caps for newer AMD hda audio in KB/KV Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 58/58] Input: pixcir_i2c_ts - fix receive error Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:20 +0200 [PATCH 4.0 02/58] sysctl: Allow creating permanently empty directories that serve as mountpoints. Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 39/58] livepatch: add module locking around kallsyms calls Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 55/58] PCI: Add pci_bus_addr_t Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 41/58] spi: orion: Fix maximum baud rates for Armada 370/XP Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 52/58] mtd: fix: avoid race condition when accessing mtd->usecount Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 14/58] ALSA: pcm: Fix pcm_class sysfs output Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 38/58] regulator: core: fix constraints output buffer Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 05/58] sysfs: Add support for permanently empty directories to serve as mount points. Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 12/58] ipr: Increase default adapter init stage change timeout Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 37/58] regulator: max77686: fix gpio_enabled shift wrapping bug Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 47/58] IB/srp: Fix connection state tracking Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 56/58] PCI: pciehp: Wait for hotplug command completion where necessary Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 42/58] spi: pl022: Specify num-cs property as required in devicetree binding Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 43/58] scsi_transport_srp: Introduce srp_wait_for_queuecommand() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 48/58] IB/srp: Fix reconnection failure handling Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 45/58] IB/srp: Remove an extraneous scsi_host_put() from an error path Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 03/58] proc: Allow creating permanently empty directories that serve as mount points Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 13/58] Disable write buffering on Toshiba ToPIC95 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 46/58] IB/srp: Fix a connection setup race Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 54/58] PCI: Propagate the "ignore hotplug" setting to parent Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 04/58] kernfs: Add support for always empty directories. Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 50/58] video: mxsfb: Make sure axi clock is enabled when accessing registers Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 06/58] mnt: Update fs_fully_visible to test for permanently empty directories Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 01/58] fs: Add helper functions for permanently empty directories. Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 49/58] genirq: devres: Fix testing return value of request_any_context_irq() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 10/58] gpio: crystalcove: set IRQCHIP_SKIP_SET_WAKE for the irqchip Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 [PATCH 4.0 51/58] leds / PM: fix hibernation on arm when gpio-led used with CPU led trigger Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2015-07-19 21:30 +0200 Re: [PATCH 4.0 00/58] 4.0.9-stable review Shuah Khan <shuahkh@osg.samsung.com> - 2015-07-20 19:20 +0200
csiph-web