Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1349706 > unrolled thread

[PATCH v14 0/6] arm64, numa: Add numa support for arm64 platforms

Started byDavid Daney <ddaney.cavm@gmail.com>
First post2016-03-04 01:00 +0100
Last post2016-03-04 01:20 +0100
Articles 4 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v14 0/6] arm64, numa: Add numa support for arm64 platforms David Daney <ddaney.cavm@gmail.com> - 2016-03-04 01:00 +0100
    [PATCH v14 6/6] arm64, mm, numa: Add NUMA balancing support for arm64. David Daney <ddaney.cavm@gmail.com> - 2016-03-04 01:00 +0100
    [PATCH v14 3/6] of, numa: Add NUMA of binding implementation. David Daney <ddaney.cavm@gmail.com> - 2016-03-04 01:00 +0100
      Re: [PATCH v14 3/6] of, numa: Add NUMA of binding implementation. Rob Herring <robh+dt@kernel.org> - 2016-03-04 01:20 +0100

#1349706 — [PATCH v14 0/6] arm64, numa: Add numa support for arm64 platforms

FromDavid Daney <ddaney.cavm@gmail.com>
Date2016-03-04 01:00 +0100
Subject[PATCH v14 0/6] arm64, numa: Add numa support for arm64 platforms
Message-ID<r8LjX-6lK-7@gated-at.bofh.it>
From: David Daney <david.daney@cavium.com>

v14:
	- Revised patch to unflatten the device tree earlier.

	- Cleanups and added EXPORT_SYMBOL to of_numa.c as suggested
          by Rob Harring

v13:
	- Added patch to unflatten the device tree earlier.

	- Rewrote of_numa.c to work on unflattened the device tree.

	- Cleanup of EXPORTs in arch/arm64/mm/numa.c as suggested by
          Will Deacon.

v12:

	- Replaced 6 patches from Ard Biesheuvel with new simpler, and
          more correct, single patch, also from Ard.

v11:
	- Dropped cleanup patches for other architectures, they will be
          submitted as a separate set after more testing.

	- Added patch set from Ard Biesheuvel that are needed to make
          the whole thing actually work.  Previously this was a
          separate set.

	- Kconfig and other fixes and simplifications as suggested by
          Rob Herring.

	- Rearranged, refactored and reordered so that we don't patch
          new files multiple times.

	- Summary:

		o 6 patches from Ard Biesheuvel to allow use of
		  "memory" nodes with efi stub.

		o 2 patches to document and add of_numa.c

		o 1 patch to add arm64 NUMA support.

		o 1 patch to add NUMA balancing support for arm64.

v10:
	- Incorporated review comments from Rob Herring.
	- Moved numa binding and implementation to devicetree core.
	- Added cleanup patch to remove redundant NODE_DATA macro from asm header files
	- Include numa balancing support for arm64 patch in this series.
	- Fix tile build issue reported by the kbuild robot(patch 7)

v9:	- Added cleanup patch to reuse and avoid redefinition of cpumask_of_pcibus
	  as suggested from Will Deacon and Bjorn Helgaas.
	  - Including patch to Make pci-host-generic driver numa aware.
	  - Incorporated comment from Shannon Zhao.

v8:
	- Incorporated review comments of Mark Rutland and Will Deacon.
	- Added pci helper function and macro for numa.

v7:
	- managing numa memory mapping using memblock.
	- Incorporated review comments of Mark Rutland.

v6:
	- defined and implemented the numa dt binding using
	node property proximity and device node distance-map.
	- renamed dt_numa to of_numa

v5:
        - created base verion of numa.c which creates dummy numa without using dt
          on single socket platforms. Then added patches for dt support.
        - Incorporated review comments from Hanjun Guo.

v4:
done changes as per Arnd review comments.

v3:
Added changes to support numa on arm64 based platforms.
Tested these patches on cavium's multinode(2 node topology) platform.
In this patchset, defined and implemented dt bindings for numa mapping
for core and memory using device node property arm,associativity.

v2:
Defined and implemented numa map for memory, cores to node and
proximity distance matrix of nodes.

v1:
Initial patchset to support numa on arm64 platforms.

Note: 1. This patchset is tested for NUMA and without NUMA with dt
        (both with and without NUMA bindings) on thunderx single
        socket and dual socket boards.

Ard Biesheuvel (1):
  efi: ARM/arm64: ignore DT memory nodes instead of removing them

David Daney (2):
  of, numa: Add NUMA of binding implementation.
  arm64: Move unflatten_device_tree() call earlier.

Ganapatrao Kulkarni (3):
  Documentation, dt, numa: dt bindings for NUMA.
  arm64, numa: Add NUMA support for arm64 platforms.
  arm64, mm, numa: Add NUMA balancing support for arm64.

 Documentation/devicetree/bindings/numa.txt | 272 ++++++++++++++++++++
 arch/arm64/Kconfig                         |  27 ++
 arch/arm64/include/asm/mmu.h               |   3 +-
 arch/arm64/include/asm/mmzone.h            |  12 +
 arch/arm64/include/asm/numa.h              |  45 ++++
 arch/arm64/include/asm/pgtable.h           |  15 ++
 arch/arm64/include/asm/topology.h          |  10 +
 arch/arm64/kernel/pci.c                    |  10 +
 arch/arm64/kernel/setup.c                  |  19 +-
 arch/arm64/kernel/smp.c                    |   4 +
 arch/arm64/mm/Makefile                     |   1 +
 arch/arm64/mm/init.c                       |  34 ++-
 arch/arm64/mm/mmu.c                        |  18 +-
 arch/arm64/mm/numa.c                       | 396 +++++++++++++++++++++++++++++
 drivers/firmware/efi/arm-init.c            |   8 +
 drivers/firmware/efi/libstub/fdt.c         |  24 +-
 drivers/of/Kconfig                         |   3 +
 drivers/of/Makefile                        |   1 +
 drivers/of/of_numa.c                       | 202 +++++++++++++++
 include/linux/of.h                         |   9 +
 20 files changed, 1073 insertions(+), 40 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/numa.txt
 create mode 100644 arch/arm64/include/asm/mmzone.h
 create mode 100644 arch/arm64/include/asm/numa.h
 create mode 100644 arch/arm64/mm/numa.c
 create mode 100644 drivers/of/of_numa.c

-- 
1.8.3.1

[toc] | [next] | [standalone]

#1349716 — [PATCH v14 6/6] arm64, mm, numa: Add NUMA balancing support for arm64.

FromDavid Daney <ddaney.cavm@gmail.com>
Date2016-03-04 01:00 +0100
Subject[PATCH v14 6/6] arm64, mm, numa: Add NUMA balancing support for arm64.
Message-ID<r8LjZ-6lK-39@gated-at.bofh.it>
In reply to#1349706
From: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>

Enable NUMA balancing for arm64 platforms.
Add pte, pmd protnone helpers for use by automatic NUMA balancing.

Reviewed-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/Kconfig               |  1 +
 arch/arm64/include/asm/pgtable.h | 15 +++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 7013087..20f5192 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -11,6 +11,7 @@ config ARM64
 	select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
 	select ARCH_USE_CMPXCHG_LOCKREF
 	select ARCH_SUPPORTS_ATOMIC_RMW
+	select ARCH_SUPPORTS_NUMA_BALANCING
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select ARCH_WANT_COMPAT_IPC_PARSE_VERSION
 	select ARCH_WANT_FRAME_POINTERS
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index bf464de..5af9db2 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -346,6 +346,21 @@ static inline pgprot_t mk_sect_prot(pgprot_t prot)
 	return __pgprot(pgprot_val(prot) & ~PTE_TABLE_BIT);
 }
 
+#ifdef CONFIG_NUMA_BALANCING
+/*
+ * See the comment in include/asm-generic/pgtable.h
+ */
+static inline int pte_protnone(pte_t pte)
+{
+	return (pte_val(pte) & (PTE_VALID | PTE_PROT_NONE)) == PTE_PROT_NONE;
+}
+
+static inline int pmd_protnone(pmd_t pmd)
+{
+	return pte_protnone(pmd_pte(pmd));
+}
+#endif
+
 /*
  * THP definitions.
  */
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1349717 — [PATCH v14 3/6] of, numa: Add NUMA of binding implementation.

FromDavid Daney <ddaney.cavm@gmail.com>
Date2016-03-04 01:00 +0100
Subject[PATCH v14 3/6] of, numa: Add NUMA of binding implementation.
Message-ID<r8LjZ-6lK-41@gated-at.bofh.it>
In reply to#1349706
From: David Daney <david.daney@cavium.com>

Add device tree parsing for NUMA topology using device
"numa-node-id" property in distance-map and cpu nodes.

This is a complete rewrite of a previous patch by:
   Ganapatrao Kulkarni<gkulkarni@caviumnetworks.com>

Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/of/Kconfig   |   3 +
 drivers/of/Makefile  |   1 +
 drivers/of/of_numa.c | 202 +++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/of.h   |   9 +++
 4 files changed, 215 insertions(+)
 create mode 100644 drivers/of/of_numa.c

diff --git a/drivers/of/Kconfig b/drivers/of/Kconfig
index e2a4841..b3bec3a 100644
--- a/drivers/of/Kconfig
+++ b/drivers/of/Kconfig
@@ -112,4 +112,7 @@ config OF_OVERLAY
 	  While this option is selected automatically when needed, you can
 	  enable it manually to improve device tree unit test coverage.
 
+config OF_NUMA
+	bool
+
 endif # OF
diff --git a/drivers/of/Makefile b/drivers/of/Makefile
index 156c072..bee3fa9 100644
--- a/drivers/of/Makefile
+++ b/drivers/of/Makefile
@@ -14,5 +14,6 @@ obj-$(CONFIG_OF_MTD)	+= of_mtd.o
 obj-$(CONFIG_OF_RESERVED_MEM) += of_reserved_mem.o
 obj-$(CONFIG_OF_RESOLVE)  += resolver.o
 obj-$(CONFIG_OF_OVERLAY) += overlay.o
+obj-$(CONFIG_OF_NUMA) += of_numa.o
 
 obj-$(CONFIG_OF_UNITTEST) += unittest-data/
diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
new file mode 100644
index 0000000..f08dd69
--- /dev/null
+++ b/drivers/of/of_numa.c
@@ -0,0 +1,202 @@
+/*
+ * OF NUMA Parsing support.
+ *
+ * Copyright (C) 2015 - 2016 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/nodemask.h>
+
+#include <asm/numa.h>
+
+/* define default numa node to 0 */
+#define DEFAULT_NODE 0
+
+/*
+ * Even though we connect cpus to numa domains later in SMP
+ * init, we need to know the node ids now for all cpus.
+*/
+static void __init of_numa_parse_cpu_nodes(void)
+{
+	u32 nid;
+	int r;
+	struct device_node *cpus;
+	struct device_node *np = NULL;
+
+	cpus = of_find_node_by_path("/cpus");
+	if (!cpus)
+		return;
+
+	for_each_child_of_node(cpus, np) {
+		/* Skip things that are not CPUs */
+		if (of_node_cmp(np->type, "cpu") != 0)
+			continue;
+
+		r = of_property_read_u32(np, "numa-node-id", &nid);
+		if (r)
+			continue;
+
+		pr_debug("NUMA: CPU on %u\n", nid);
+		if (nid >= MAX_NUMNODES)
+			pr_warn("NUMA: Node id %u exceeds maximum value\n",
+				nid);
+		else
+			node_set(nid, numa_nodes_parsed);
+	}
+}
+
+static void __init of_numa_parse_memory_nodes(void)
+{
+	struct device_node *np = NULL;
+	struct resource rsrc;
+	u32 nid;
+	int r;
+
+	for (;;) {
+		np = of_find_node_by_type(np, "memory");
+		if (!np)
+			break;
+
+		r = of_property_read_u32(np, "numa-node-id", &nid);
+		if (r)
+			continue;
+
+		r = of_address_to_resource(np, 0, &rsrc);
+		if (r) {
+			pr_err("NUMA: bad reg property in memory node\n");
+			continue;
+		}
+
+		pr_debug("NUMA:  base = %llx len = %llx, node = %u\n",
+			 rsrc.start, rsrc.end - rsrc.start + 1, nid);
+
+		if (numa_add_memblk(nid, rsrc.start,
+				    rsrc.end - rsrc.start + 1) < 0)
+			break;
+	}
+
+	of_node_put(np);
+}
+
+static int __init of_numa_parse_distance_map_v1(struct device_node *map)
+{
+	const __be32 *matrix;
+	int entry_count;
+	int i;
+
+	pr_info("NUMA: parsing numa-distance-map-v1\n");
+
+	matrix = of_get_property(map, "distance-matrix", NULL);
+	if (!matrix) {
+		pr_err("NUMA: No distance-matrix property in distance-map\n");
+		return -EINVAL;
+	}
+
+	entry_count = of_property_count_u32_elems(map, "distance-matrix");
+	if (entry_count <= 0) {
+		pr_err("NUMA: Invalid distance-matrix\n");
+		return -EINVAL;
+	}
+
+	for (i = 0; i + 2 < entry_count; i += 3) {
+		u32 nodea, nodeb, distance;
+
+		nodea = of_read_number(matrix, 1);
+		matrix++;
+		nodeb = of_read_number(matrix, 1);
+		matrix++;
+		distance = of_read_number(matrix, 1);
+		matrix++;
+
+		numa_set_distance(nodea, nodeb, distance);
+		pr_debug("NUMA:  distance[node%d -> node%d] = %d\n",
+			 nodea, nodeb, distance);
+
+		/* Set default distance of node B->A same as A->B */
+		if (nodeb > nodea)
+			numa_set_distance(nodeb, nodea, distance);
+	}
+
+	return 0;
+}
+
+static int __init of_numa_parse_distance_map(void)
+{
+	int ret = -EINVAL;
+	struct device_node *np = of_find_node_by_path("/distance-map");
+
+	if (!np)
+		return ret;
+
+	if (of_device_is_compatible(np, "numa-distance-map-v1")) {
+		ret = of_numa_parse_distance_map_v1(np);
+		goto out;
+	}
+
+	pr_err("NUMA: invalid distance-map device node\n");
+out:
+	of_node_put(np);
+	return ret;
+}
+
+int of_node_to_nid(struct device_node *device)
+{
+	struct device_node *np;
+	u32 nid;
+	int r = -ENODATA;
+
+	np = of_node_get(device);
+
+	while (np) {
+		struct device_node *parent;
+
+		r = of_property_read_u32(np, "numa-node-id", &nid);
+		/*
+		 * -EINVAL indicates the property was not found, and
+		 *  we walk up the tree trying to find a parent with a
+		 *  "numa-node-id".  Any other type of error indicates
+		 *  a bad device tree and we give up.
+		 */
+		if (r != -EINVAL)
+			break;
+
+		parent = of_get_parent(np);
+		of_node_put(np);
+		np = parent;
+	}
+	if (np && r)
+		pr_warn("NUMA: Invalid \"numa-node-id\" property in node %s\n",
+			np->name);
+	of_node_put(np);
+
+	if (!r) {
+		if (nid >= MAX_NUMNODES)
+			pr_warn("NUMA: Node id %u exceeds maximum value\n",
+				nid);
+		else
+			return nid;
+	}
+
+	return NUMA_NO_NODE;
+}
+EXPORT_SYMBOL(of_node_to_nid);
+
+int __init of_numa_init(void)
+{
+	of_numa_parse_cpu_nodes();
+	of_numa_parse_memory_nodes();
+	return of_numa_parse_distance_map();
+}
diff --git a/include/linux/of.h b/include/linux/of.h
index dc6e396..fe67a4c 100644
--- a/include/linux/of.h
+++ b/include/linux/of.h
@@ -685,6 +685,15 @@ static inline int of_node_to_nid(struct device_node *device)
 }
 #endif
 
+#ifdef CONFIG_OF_NUMA
+extern int of_numa_init(void);
+#else
+static inline int of_numa_init(void)
+{
+	return -ENOSYS;
+}
+#endif
+
 static inline struct device_node *of_find_matching_node(
 	struct device_node *from,
 	const struct of_device_id *matches)
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1349721 — Re: [PATCH v14 3/6] of, numa: Add NUMA of binding implementation.

FromRob Herring <robh+dt@kernel.org>
Date2016-03-04 01:20 +0100
SubjectRe: [PATCH v14 3/6] of, numa: Add NUMA of binding implementation.
Message-ID<r8LDj-6HH-5@gated-at.bofh.it>
In reply to#1349717
On Thu, Mar 3, 2016 at 5:55 PM, David Daney <ddaney.cavm@gmail.com> wrote:
> From: David Daney <david.daney@cavium.com>
>
> Add device tree parsing for NUMA topology using device
> "numa-node-id" property in distance-map and cpu nodes.
>
> This is a complete rewrite of a previous patch by:
>    Ganapatrao Kulkarni<gkulkarni@caviumnetworks.com>
>
> Signed-off-by: David Daney <david.daney@cavium.com>

Acked-by: Rob Herring <robh@kernel.org>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web