Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1389733 > unrolled thread

[PATCH 00/21] X86_64, UV: Update kernel for SGI UV4 support

Back to article view | Back to linux.kernel

Contents

  [PATCH 00/21] X86_64, UV: Update kernel for SGI UV4 support Mike Travis <travis@sgi.com> - 2016-04-28 03:20 +0200
    [PATCH 20/21] X86_64, UV: Remove Obsolete GRU MMR address translation Mike Travis <travis@sgi.com> - 2016-04-28 03:20 +0200
      Re: [PATCH 20/21] X86_64, UV: Remove Obsolete GRU MMR address  translation Ingo Molnar <mingo@kernel.org> - 2016-04-28 11:00 +0200
    [PATCH 14/21] X86_64, UV: Fold blade info into per node hub info structs Mike Travis <travis@sgi.com> - 2016-04-28 03:20 +0200
    [PATCH 07/21] X86_64, UV: Remove Support for UV1 Mike Travis <travis@sgi.com> - 2016-04-28 03:20 +0200
      Re: [PATCH 07/21] X86_64, UV: Remove Support for UV1 Ingo Molnar <mingo@kernel.org> - 2016-04-28 10:50 +0200
        Re: [PATCH 07/21] X86_64, UV: Remove Support for UV1 Mike Travis <travis@sgi.com> - 2016-04-29 00:00 +0200

#1389733 — [PATCH 00/21] X86_64, UV: Update kernel for SGI UV4 support

This patch set primarily updates the Linux kernel to support the next
generation SGI Ultraviolet system, UV4.  This architecture change is a
larger incremental change than previous UV updates because of a major
change to the addressing scheme.  Previous UV architectures used a fixed
address width per node which was entirely appropriate, since the size
of RAM memory usually did not deviate very much between nodes.

Now on UV4 non-volatile memories can be added to any or all of the nodes.
The available sizes can be both smaller and much larger than that of
volatile memory on the same node.

To accommodate this configuration change, the architecture was updated
to support a varying per node memory width (or total per node memory).
And since this removed a key conversion factor known as "M" (the number
of address space bits per node), the kernel also had to change to support
the new address conversion functions.

Other changes in addition to this basic overhaul of the internal logic
include support for the increasing number of cores and thus CPU threads
per node.  Another inevitable architecture change will be the increase
of the system addressing space supported in future processors, upward
from the current 46 bits.

One other aspect, because these are somewhat extensive changes to the
UV kernel support system, we've also done extensive testing on current
UV architectures to insure that these changes do not break existing
functionality.  Since we do not have actual UV4 hardware yet, the
UV4 simulator has been used to test and debug UV4 specific changes.

Specific changes that support the updates are:

* Add support for UV4 specific hardware changes.  This was done in a
  way to maintain source code compatibility with supported external
  modules, for distros with kernel releases prior to the UV4 changes.

* Add new UV4 MMR definitions from automated verilog to MMR defines
  scripts.  Add control info to select UV4 in "UV common" functions.

* Migrate from using per cpu information structures to consolidating
  information common to nodes (move per cpu UVHUB info to per node info).

* Migrate more information look up from lists allocated on node 0 to
  structures allocated on the local node (move per blade info to per
  node info).

* Use more "soft" information that is collected by UV BIOS to control
  conversion routines instead of relying on information in MMR fields
  (gather more info from the EFI UVsystab system table.)

* Add high speed physical address to node/pnode/NASID conversion look ups.

* Add "soft" support for undefined MMR access to aid debugging the UV
  common functions.

* Remove support for UV1 going forward.  Remove obsolete GRU support from
  the GRU/XP drivers. Fix an ongoing problem determining the correct
  physical placement of nodes containing either no cpus or no memory
  ("headless/memoryless nodes").

-- 

[toc] | [next] | [standalone]

#1389734 — [PATCH 20/21] X86_64, UV: Remove Obsolete GRU MMR address translation

From: Dimitri Sivanich <sivanich@sgi.com>

Use no-op messages in place of cross-partition interrupts when nacking a
put message in the GRU.  This allows us to remove MMR's as a destination
from the GRU driver.

Signed-off-by: Dimitri Sivanich <sivanich@sgi.com>
Signed-off-by: Mike Travis <travis@sgi.com>
Tested-by: John Estabrook <estabrook@sgi.com>
Tested-by: Gary Kroening <gfk@sgi.com>
---
 arch/x86/include/asm/uv/uv_hub.h    |   10 ----------
 drivers/misc/sgi-gru/grukservices.c |   36 ++++++++++++++++++++----------------
 2 files changed, 20 insertions(+), 26 deletions(-)

--- linux.orig/arch/x86/include/asm/uv/uv_hub.h
+++ linux/arch/x86/include/asm/uv/uv_hub.h
@@ -642,16 +642,6 @@ static inline unsigned long uv_read_glob
 	return readq(uv_global_mmr64_address(pnode, offset));
 }
 
-/*
- * Global MMR space addresses when referenced by the GRU. (GRU does
- * NOT use socket addressing).
- */
-static inline unsigned long uv_global_gru_mmr_address(int pnode, unsigned long offset)
-{
-	return UV_GLOBAL_GRU_MMR_BASE | offset |
-		((unsigned long)pnode << uv_hub_info->m_val);
-}
-
 static inline void uv_write_global_mmr8(int pnode, unsigned long offset, unsigned char val)
 {
 	writeb(val, uv_global_mmr64_address(pnode, offset));
--- linux.orig/drivers/misc/sgi-gru/grukservices.c
+++ linux/drivers/misc/sgi-gru/grukservices.c
@@ -718,8 +718,8 @@ cberr:
 static int send_message_put_nacked(void *cb, struct gru_message_queue_desc *mqd,
 			void *mesg, int lines)
 {
-	unsigned long m, *val = mesg, gpa, save;
-	int ret;
+	unsigned long m;
+	int ret, loops = 200;	/* experimentally determined */
 
 	m = mqd->mq_gpa + (gru_get_amo_value_head(cb) << 6);
 	if (lines == 2) {
@@ -735,22 +735,26 @@ static int send_message_put_nacked(void
 		return MQE_OK;
 
 	/*
-	 * Send a cross-partition interrupt to the SSI that contains the target
-	 * message queue. Normally, the interrupt is automatically delivered by
-	 * hardware but some error conditions require explicit delivery.
-	 * Use the GRU to deliver the interrupt. Otherwise partition failures
+	 * Send a noop message in order to deliver a cross-partition interrupt
+	 * to the SSI that contains the target message queue. Normally, the
+	 * interrupt is automatically delivered by hardware following mesq
+	 * operations, but some error conditions require explicit delivery.
+	 * The noop message will trigger delivery. Otherwise partition failures
 	 * could cause unrecovered errors.
 	 */
-	gpa = uv_global_gru_mmr_address(mqd->interrupt_pnode, UVH_IPI_INT);
-	save = *val;
-	*val = uv_hub_ipi_value(mqd->interrupt_apicid, mqd->interrupt_vector,
-				dest_Fixed);
-	gru_vstore_phys(cb, gpa, gru_get_tri(mesg), IAA_REGISTER, IMA);
-	ret = gru_wait(cb);
-	*val = save;
-	if (ret != CBS_IDLE)
-		return MQE_UNEXPECTED_CB_ERR;
-	return MQE_OK;
+	do {
+		ret = send_noop_message(cb, mqd, mesg);
+	} while ((ret == MQIE_AGAIN || ret == MQE_CONGESTION) && (loops-- > 0));
+
+	if (ret == MQIE_AGAIN || ret == MQE_CONGESTION) {
+		/* Don't indicate to the app to resend the message, as it's */
+		/* already been successfully sent.  We simply send an OK */
+		/* (rather than fail the send with MQE_UNEXPECTED_CB_ERR), */
+		/* assuming that the other side is receiving enough */
+		/* interrupts to get this message processed anyway. */
+		ret = MQE_OK;
+	}
+	return ret;
 }
 
 /*

-- 

[toc] | [prev] | [next] | [standalone]

#1389903 — Re: [PATCH 20/21] X86_64, UV: Remove Obsolete GRU MMR address translation

* Mike Travis <travis@sgi.com> wrote:

>  	/*
> +	 * Send a noop message in order to deliver a cross-partition interrupt
> +	 * to the SSI that contains the target message queue. Normally, the
> +	 * interrupt is automatically delivered by hardware following mesq
> +	 * operations, but some error conditions require explicit delivery.
> +	 * The noop message will trigger delivery. Otherwise partition failures
>  	 * could cause unrecovered errors.
>  	 */

This is a properly structured comment.

> +		/* Don't indicate to the app to resend the message, as it's */
> +		/* already been successfully sent.  We simply send an OK */
> +		/* (rather than fail the send with MQE_UNEXPECTED_CB_ERR), */
> +		/* assuming that the other side is receiving enough */
> +		/* interrupts to get this message processed anyway. */

This is not...

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1389737 — [PATCH 14/21] X86_64, UV: Fold blade info into per node hub info structs

Migrate references from the blade info structs to the per node hub info
structs.  This phases out the allocation of the list of per blade info
structs on node 0, in favor of a per node hub info struct allocated on
the node's local memory.

There are also some minor cosemetic changes in the comments and whitespace
to clean things up a bit.

Signed-off-by: Mike Travis <travis@sgi.com>
Tested-by: Dimitri Sivanich <sivanich@sgi.com>
Reviewed-by: Andrew Banman <abanman@sgi.com>
Tested-by: John Estabrook <estabrook@sgi.com>
Tested-by: Gary Kroening <gfk@sgi.com>
---
 arch/x86/include/asm/uv/uv_hub.h   |   71 +++++-------
 arch/x86/kernel/apic/x2apic_uv_x.c |  212 +++++++++++++++++--------------------
 2 files changed, 133 insertions(+), 150 deletions(-)

--- linux.orig/arch/x86/include/asm/uv/uv_hub.h
+++ linux/arch/x86/include/asm/uv/uv_hub.h
@@ -16,6 +16,7 @@
 #include <linux/percpu.h>
 #include <linux/timer.h>
 #include <linux/io.h>
+#include <linux/topology.h>
 #include <asm/types.h>
 #include <asm/percpu.h>
 #include <asm/uv/uv_mmrs.h>
@@ -161,6 +162,9 @@ struct uv_hub_info_s {
 	unsigned short		numa_blade_id;
 	unsigned char		m_val;
 	unsigned char		n_val;
+	unsigned short		nr_possible_cpus;
+	unsigned short		nr_online_cpus;
+	short			memory_nid;
 };
 
 /* CPU specific info with a pointer to the hub common info struct */
@@ -444,7 +448,7 @@ static inline unsigned long uv_gpa_to_so
 }
 
 
-/* gpa -> pnode */
+/* gpa -> gnode */
 static inline unsigned long uv_gpa_to_gnode(unsigned long gpa)
 {
 	return gpa >> uv_hub_info->n_lshift;
@@ -453,12 +457,10 @@ static inline unsigned long uv_gpa_to_gn
 /* gpa -> pnode */
 static inline int uv_gpa_to_pnode(unsigned long gpa)
 {
-	unsigned long n_mask = (1UL << uv_hub_info->n_val) - 1;
-
-	return uv_gpa_to_gnode(gpa) & n_mask;
+	return uv_gpa_to_gnode(gpa) & uv_hub_info->pnode_mask;
 }
 
-/* gpa -> node offset*/
+/* gpa -> node offset */
 static inline unsigned long uv_gpa_to_offset(unsigned long gpa)
 {
 	return (gpa << uv_hub_info->m_shift) >> uv_hub_info->m_shift;
@@ -470,18 +472,13 @@ static inline void *uv_pnode_offset_to_v
 	return __va(((unsigned long)pnode << uv_hub_info->m_val) | offset);
 }
 
-
-/*
- * Extract a PNODE from an APICID (full apicid, not processor subset)
- */
+/* Extract a PNODE from an APICID (full apicid, not processor subset) */
 static inline int uv_apicid_to_pnode(int apicid)
 {
 	return (apicid >> uv_hub_info->apic_pnode_shift);
 }
 
-/*
- * Convert an apicid to the socket number on the blade
- */
+/* Convert an apicid to the socket number on the blade */
 static inline int uv_apicid_to_socket(int apicid)
 {
 		return 0;
@@ -576,21 +573,6 @@ static inline void uv_write_local_mmr8(u
 	writeb(val, uv_local_mmr_address(offset));
 }
 
-/*
- * Structures and definitions for converting between cpu, node, pnode, and blade
- * numbers.
- */
-struct uv_blade_info {
-	unsigned short	nr_possible_cpus;
-	unsigned short	nr_online_cpus;
-	unsigned short	pnode;
-	short		memory_nid;
-};
-extern struct uv_blade_info *uv_blade_info;
-extern short *uv_node_to_blade;
-extern short *uv_cpu_to_blade;
-extern short uv_possible_blades;
-
 /* Blade-local cpu number of current cpu. Numbered 0 .. <# cpus on the blade> */
 static inline int uv_blade_processor_id(void)
 {
@@ -604,61 +586,72 @@ static inline int uv_cpu_blade_processor
 }
 #define _uv_cpu_blade_processor_id 1	/* indicate function available */
 
+/* Blade number to Node number (UV1..UV4 is 1:1) */
+static inline int uv_blade_to_node(int blade)
+{
+	return blade;
+}
+
 /* Blade number of current cpu. Numnbered 0 .. <#blades -1> */
 static inline int uv_numa_blade_id(void)
 {
 	return uv_hub_info->numa_blade_id;
 }
 
-/* Convert a cpu number to the the UV blade number */
-static inline int uv_cpu_to_blade_id(int cpu)
+/*
+ * Convert linux node number to the UV blade number.
+ * .. Currently for UV1 thru UV4 the node and the blade are identical.
+ * .. If this changes then you MUST check references to this function!
+ */
+static inline int uv_node_to_blade_id(int nid)
 {
-	return uv_cpu_to_blade[cpu];
+	return nid;
 }
 
-/* Convert linux node number to the UV blade number */
-static inline int uv_node_to_blade_id(int nid)
+/* Convert a cpu number to the the UV blade number */
+static inline int uv_cpu_to_blade_id(int cpu)
 {
-	return uv_node_to_blade[nid];
+	return uv_node_to_blade_id(cpu_to_node(cpu));
 }
 
 /* Convert a blade id to the PNODE of the blade */
 static inline int uv_blade_to_pnode(int bid)
 {
-	return uv_blade_info[bid].pnode;
+	return uv_hub_info_list(uv_blade_to_node(bid))->pnode;
 }
 
 /* Nid of memory node on blade. -1 if no blade-local memory */
 static inline int uv_blade_to_memory_nid(int bid)
 {
-	return uv_blade_info[bid].memory_nid;
+	return uv_hub_info_list(uv_blade_to_node(bid))->memory_nid;
 }
 
 /* Determine the number of possible cpus on a blade */
 static inline int uv_blade_nr_possible_cpus(int bid)
 {
-	return uv_blade_info[bid].nr_possible_cpus;
+	return uv_hub_info_list(uv_blade_to_node(bid))->nr_possible_cpus;
 }
 
 /* Determine the number of online cpus on a blade */
 static inline int uv_blade_nr_online_cpus(int bid)
 {
-	return uv_blade_info[bid].nr_online_cpus;
+	return uv_hub_info_list(uv_blade_to_node(bid))->nr_online_cpus;
 }
 
 /* Convert a cpu id to the PNODE of the blade containing the cpu */
 static inline int uv_cpu_to_pnode(int cpu)
 {
-	return uv_blade_info[uv_cpu_to_blade_id(cpu)].pnode;
+	return uv_cpu_hub_info(cpu)->pnode;
 }
 
 /* Convert a linux node number to the PNODE of the blade */
 static inline int uv_node_to_pnode(int nid)
 {
-	return uv_blade_info[uv_node_to_blade_id(nid)].pnode;
+	return uv_hub_info_list(nid)->pnode;
 }
 
 /* Maximum possible number of blades */
+extern short uv_possible_blades;
 static inline int uv_num_possible_blades(void)
 {
 	return uv_possible_blades;
--- linux.orig/arch/x86/kernel/apic/x2apic_uv_x.c
+++ linux/arch/x86/kernel/apic/x2apic_uv_x.c
@@ -209,21 +209,14 @@ EXPORT_SYMBOL_GPL(__uv_hub_info_list);
 DEFINE_PER_CPU(struct uv_cpu_info_s, __uv_cpu_info);
 EXPORT_PER_CPU_SYMBOL_GPL(__uv_cpu_info);
 
-struct uv_blade_info *uv_blade_info;
-EXPORT_SYMBOL_GPL(uv_blade_info);
-
-short *uv_node_to_blade;
-EXPORT_SYMBOL_GPL(uv_node_to_blade);
-
-short *uv_cpu_to_blade;
-EXPORT_SYMBOL_GPL(uv_cpu_to_blade);
-
 short uv_possible_blades;
 EXPORT_SYMBOL_GPL(uv_possible_blades);
 
 unsigned long sn_rtc_cycles_per_second;
 EXPORT_SYMBOL(sn_rtc_cycles_per_second);
 
+static __initdata unsigned short *_node_to_pnode;
+
 extern int uv_hub_info_version(void)
 {
 	return UV_HUB_INFO_VERSION;
@@ -346,7 +339,6 @@ static unsigned long set_apic_id(unsigne
 
 static unsigned int uv_read_apic_id(void)
 {
-
 	return x2apic_get_apic_id(apic_read(APIC_ID));
 }
 
@@ -416,19 +408,6 @@ static struct apic __refdata apic_x2apic
 	.safe_wait_icr_idle		= native_safe_x2apic_wait_icr_idle,
 };
 
-/*
- * Called on boot cpu.
- */
-static __init int boot_pnode_to_blade(int pnode)
-{
-	int blade;
-
-	for (blade = 0; blade < uv_num_possible_blades(); blade++)
-		if (pnode == uv_blade_info[blade].pnode)
-			return blade;
-	BUG();
-}
-
 #define	UVH_RH_GAM_ALIAS210_REDIRECT_CONFIG_LENGTH	3
 #define DEST_SHIFT UVH_RH_GAM_ALIAS210_REDIRECT_CONFIG_0_MMR_DEST_BASE_SHFT
 
@@ -845,10 +824,10 @@ int uv_set_vga_state(struct pci_dev *pde
 void uv_cpu_init(void)
 {
 	/* CPU 0 initialization will be done via uv_system_init. */
-	if (!uv_blade_info)
+	if (smp_processor_id() == 0)
 		return;
 
-	uv_blade_info[uv_numa_blade_id()].nr_online_cpus++;
+	uv_hub_info->nr_online_cpus++;
 }
 
 struct mn {
@@ -929,11 +908,57 @@ void __init uv_init_hub_info(struct uv_h
 
 }
 
+/*
+ * Setup physical blade translations from UVH_NODE_PRESENT_TABLE
+ * .. NB: UVH_NODE_PRESENT_TABLE is going away,
+ * .. being replaced by GAM Range Table
+ */
+static __init void boot_init_possible_blades(struct uv_hub_info_s *hub_info)
+{
+	size_t bytes;
+	int blade, i, j, uv_pb = 0, num_nodes = num_possible_nodes();
+
+	pr_info("UV: NODE_PRESENT_DEPTH = %d\n", UVH_NODE_PRESENT_TABLE_DEPTH);
+	for (i = 0; i < UVH_NODE_PRESENT_TABLE_DEPTH; i++) {
+		unsigned long np;
+
+		np = uv_read_local_mmr(UVH_NODE_PRESENT_TABLE + i * 8);
+		if (np)
+			pr_info("UV: NODE_PRESENT(%d) = 0x%016lx\n", i, np);
+
+		uv_pb += hweight64(np);
+	}
+	if (uv_possible_blades != uv_pb)
+		uv_possible_blades = uv_pb;
+
+	bytes = num_nodes * sizeof(_node_to_pnode[0]);
+	_node_to_pnode = kmalloc(bytes, GFP_KERNEL);
+	BUG_ON(!_node_to_pnode);
+
+	for (blade = 0, i = 0; i < UVH_NODE_PRESENT_TABLE_DEPTH; i++) {
+		unsigned short pnode;
+		unsigned long present =
+			uv_read_local_mmr(UVH_NODE_PRESENT_TABLE + i * 8);
+
+		for (j = 0; j < 64; j++) {
+			if (!test_bit(j, &present))
+				continue;
+			pnode = (i * 64 + j) & hub_info->pnode_mask;
+			_node_to_pnode[blade++] = pnode;
+		}
+		if (blade > num_nodes) {
+			pr_err("UV: blade count(%d) exceeds node count(%d)!\n",
+				blade, num_nodes);
+			BUG();
+		}
+	}
+}
+
 void __init uv_system_init(void)
 {
 	struct uv_hub_info_s hub_info = {0};
-	int bytes, nid, cpu, pnode, blade, i, j;
-	int min_pnode = 999999, max_pnode = -1;
+	int bytes, cpu, nodeid;
+	unsigned short min_pnode = 9999, max_pnode = 0;
 	char *hub = is_uv4_hub() ? "UV400" :
 		    is_uv3_hub() ? "UV300" :
 		    is_uv2_hub() ? "UV2000/3000" :
@@ -950,16 +975,9 @@ void __init uv_system_init(void)
 		map_low_mmrs();
 
 	uv_init_hub_info(&hub_info);
-
-	pr_info("UV: NODE_PRESENT_DEPTH = %d\n", UVH_NODE_PRESENT_TABLE_DEPTH);
-	for (i = 0; i < UVH_NODE_PRESENT_TABLE_DEPTH; i++) {
-		unsigned long np;
-
-		np = uv_read_local_mmr(UVH_NODE_PRESENT_TABLE + i * 8);
-		if (np)
-			pr_info("UV: NODE_PRESENT(%d) = 0x%016lx\n", i, np);
-		uv_possible_blades += hweight64(np);
-	}
+	uv_possible_blades = num_possible_nodes();
+	if (!_node_to_pnode)
+		boot_init_possible_blades(&hub_info);
 
 	/* uv_num_possible_blades() is really the hub count */
 	pr_info("UV: Found %d hubs, %d nodes, %d cpus\n",
@@ -967,97 +985,69 @@ void __init uv_system_init(void)
 			num_possible_nodes(),
 			num_possible_cpus());
 
-	bytes = sizeof(struct uv_blade_info) * uv_num_possible_blades();
-	uv_blade_info = kzalloc(bytes, GFP_KERNEL);
-	BUG_ON(!uv_blade_info);
-
-	for (blade = 0; blade < uv_num_possible_blades(); blade++)
-		uv_blade_info[blade].memory_nid = -1;
-
-
-	bytes = sizeof(uv_node_to_blade[0]) * num_possible_nodes();
-	uv_node_to_blade = kmalloc(bytes, GFP_KERNEL);
-	BUG_ON(!uv_node_to_blade);
-	memset(uv_node_to_blade, 255, bytes);
-
-	bytes = sizeof(uv_cpu_to_blade[0]) * num_possible_cpus();
-	uv_cpu_to_blade = kmalloc(bytes, GFP_KERNEL);
-	BUG_ON(!uv_cpu_to_blade);
-	memset(uv_cpu_to_blade, 255, bytes);
-
-	bytes = sizeof(void *) * uv_num_possible_blades();
-	__uv_hub_info_list = kzalloc(bytes, GFP_KERNEL);
-	BUG_ON(!__uv_hub_info_list);
-
-	blade = 0;
-	for (i = 0; i < UVH_NODE_PRESENT_TABLE_DEPTH; i++) {
-		unsigned long present =
-			uv_read_local_mmr(UVH_NODE_PRESENT_TABLE + i * 8);
-		for (j = 0; j < 64; j++) {
-			if (!test_bit(j, &present))
-				continue;
-			pnode = (i * 64 + j) & hub_info.pnode_mask;
-			uv_blade_info[blade].pnode = pnode;
-			uv_blade_info[blade].nr_possible_cpus = 0;
-			uv_blade_info[blade].nr_online_cpus = 0;
-			min_pnode = min(pnode, min_pnode);
-			max_pnode = max(pnode, max_pnode);
-			blade++;
-		}
-	}
-
 	uv_bios_init();
 	uv_bios_get_sn_info(0, &uv_type, &sn_partition_id, &sn_coherency_id,
 			    &sn_region_size, &system_serial_number);
 	hub_info.coherency_domain_number = sn_coherency_id;
 	uv_rtc_init();
 
-	for_each_present_cpu(cpu) {
-		struct uv_hub_info_s *new_hub = NULL;
-		int apicid = per_cpu(x86_cpu_to_apicid, cpu);
-		int nodeid = cpu_to_node(cpu);
+	bytes = sizeof(void *) * uv_num_possible_blades();
+	__uv_hub_info_list = kzalloc(bytes, GFP_KERNEL);
+	BUG_ON(!__uv_hub_info_list);
 
-		/* Allocate new per hub info list */
-		if (uv_hub_info_list(nodeid) == NULL) {
-			if (cpu == 0)
-				__uv_hub_info_list[0] = &uv_hub_info_node0;
-			else
-				__uv_hub_info_list[nodeid] =
-					kzalloc_node(bytes, GFP_KERNEL, nodeid);
-
-			new_hub = uv_hub_info_list(nodeid);
-			BUG_ON(!new_hub);
-			*new_hub = hub_info;
-			blade = boot_pnode_to_blade(new_hub->pnode);
-			new_hub->pnode = uv_apicid_to_pnode(apicid);
-			new_hub->numa_blade_id = blade;
+	bytes = sizeof(struct uv_hub_info_s);
+	for_each_node(nodeid) {
+		struct uv_hub_info_s *new_hub;
+		unsigned short pnode;
+
+		if (__uv_hub_info_list[nodeid]) {
+			pr_err("UV: Node %d UV HUB already initialized!?\n",
+				nodeid);
+			BUG();
 		}
 
-		/* Any node on the blade, else will contain -1. */
-		uv_blade_info[blade].memory_nid = nodeid;
+		/* Allocate new per hub info list */
+		new_hub = (nodeid == 0) ?
+			&uv_hub_info_node0 :
+			kzalloc_node(bytes, GFP_KERNEL, nodeid);
+		BUG_ON(!new_hub);
+		__uv_hub_info_list[nodeid] = new_hub;
+		new_hub = uv_hub_info_list(nodeid);
+		BUG_ON(!new_hub);
+		*new_hub = hub_info;
+
+		pnode = _node_to_pnode[nodeid];
+		min_pnode = min(pnode, min_pnode);
+		max_pnode = max(pnode, max_pnode);
+		new_hub->pnode = pnode;
+		new_hub->numa_blade_id = uv_node_to_blade_id(nodeid);
+		new_hub->memory_nid = -1;
+		new_hub->nr_possible_cpus = 0;
+		new_hub->nr_online_cpus = 0;
+	}
 
-		uv_node_to_blade[nodeid] = blade;
-		uv_cpu_to_blade[cpu] = blade;
+	/* Initialize per cpu info */
+	for_each_possible_cpu(cpu) {
+		int apicid = per_cpu(x86_cpu_to_apicid, cpu);
 
-		/* Initialize per cpu info list */
+		nodeid = cpu_to_node(cpu);
 		uv_cpu_info_per(cpu)->p_uv_hub_info = uv_hub_info_list(nodeid);
-		uv_cpu_info_per(cpu)->scir.offset = uv_scir_offset(apicid);
 		uv_cpu_info_per(cpu)->blade_cpu_id =
-			uv_blade_info[blade].nr_possible_cpus++;
+			uv_cpu_hub_info(cpu)->nr_possible_cpus++;
+		if (uv_cpu_hub_info(cpu)->memory_nid == -1)
+			uv_cpu_hub_info(cpu)->memory_nid = cpu_to_node(cpu);
+		uv_cpu_scir_info(cpu)->offset = uv_scir_offset(apicid);
 	}
 
-	/* Add blade/pnode info for nodes without cpus */
-	for_each_online_node(nid) {
-		unsigned long paddr;
-
-		if (uv_node_to_blade[nid] >= 0)
-			continue;
-		paddr = node_start_pfn(nid) << PAGE_SHIFT;
-		pnode = uv_gpa_to_pnode(uv_soc_phys_ram_to_gpa(paddr));
-		blade = boot_pnode_to_blade(pnode);
-		uv_node_to_blade[nid] = blade;
+	/* Display per node info */
+	for_each_node(nodeid) {
+		pr_info("UV: UVHUB node:%2d pn:%02x nrcpus:%d\n",
+			nodeid,
+			uv_hub_info_list(nodeid)->pnode,
+			uv_hub_info_list(nodeid)->nr_possible_cpus);
 	}
 
+	pr_info("UV: min_pnode:%02x max_pnode:%02x\n", min_pnode, max_pnode);
 	map_gru_high(max_pnode);
 	map_mmr_high(max_pnode);
 	map_mmioh_high(min_pnode, max_pnode);

-- 

[toc] | [prev] | [next] | [standalone]

#1389738 — [PATCH 07/21] X86_64, UV: Remove Support for UV1

The UV1 architecture is no longer being supported in future Linux kernels, so
remove UV1 specific code.

Signed-off-by: Mike Travis <travis@sgi.com>
Reviewed-by: Dimitri Sivanich <sivanich@sgi.com>
Tested-by: John Estabrook <estabrook@sgi.com>
Tested-by: Gary Kroening <gfk@sgi.com>
---
 arch/x86/include/asm/uv/uv_hub.h   |    9 ------
 arch/x86/kernel/apic/x2apic_uv_x.c |   53 ++-----------------------------------
 arch/x86/platform/uv/tlb_uv.c      |    2 -
 arch/x86/platform/uv/uv_time.c     |    3 --
 4 files changed, 5 insertions(+), 62 deletions(-)

--- linux.orig/arch/x86/include/asm/uv/uv_hub.h
+++ linux/arch/x86/include/asm/uv/uv_hub.h
@@ -289,25 +289,21 @@ union uvh_apicid {
 #define UV4_GLOBAL_MMR32_SIZE		(16UL * 1024 * 1024)
 
 #define UV_LOCAL_MMR_BASE		(				\
-					is_uv1_hub() ? UV1_LOCAL_MMR_BASE : \
 					is_uv2_hub() ? UV2_LOCAL_MMR_BASE : \
 					is_uv3_hub() ? UV3_LOCAL_MMR_BASE : \
 					/*is_uv4_hub*/ UV4_LOCAL_MMR_BASE)
 
 #define UV_GLOBAL_MMR32_BASE		(				\
-					is_uv1_hub() ? UV1_GLOBAL_MMR32_BASE : \
 					is_uv2_hub() ? UV2_GLOBAL_MMR32_BASE : \
 					is_uv3_hub() ? UV3_GLOBAL_MMR32_BASE : \
 					/*is_uv4_hub*/ UV4_GLOBAL_MMR32_BASE)
 
 #define UV_LOCAL_MMR_SIZE		(				\
-					is_uv1_hub() ? UV1_LOCAL_MMR_SIZE : \
 					is_uv2_hub() ? UV2_LOCAL_MMR_SIZE : \
 					is_uv3_hub() ? UV3_LOCAL_MMR_SIZE : \
 					/*is_uv4_hub*/ UV4_LOCAL_MMR_SIZE)
 
 #define UV_GLOBAL_MMR32_SIZE		(				\
-					is_uv1_hub() ? UV1_GLOBAL_MMR32_SIZE : \
 					is_uv2_hub() ? UV2_GLOBAL_MMR32_SIZE : \
 					is_uv3_hub() ? UV3_GLOBAL_MMR32_SIZE : \
 					/*is_uv4_hub*/ UV4_GLOBAL_MMR32_SIZE)
@@ -445,9 +441,6 @@ static inline int uv_apicid_to_pnode(int
  */
 static inline int uv_apicid_to_socket(int apicid)
 {
-	if (is_uv1_hub())
-		return (apicid >> (uv_hub_info->apic_pnode_shift - 1)) & 1;
-	else
 		return 0;
 }
 
@@ -704,10 +697,8 @@ static inline void uv_set_cpu_scir_bits(
 	}
 }
 
-extern unsigned int uv_apicid_hibits;
 static unsigned long uv_hub_ipi_value(int apicid, int vector, int mode)
 {
-	apicid |= uv_apicid_hibits;
 	return (1UL << UVH_IPI_INT_SEND_SHFT) |
 			((apicid) << UVH_IPI_INT_APIC_ID_SHFT) |
 			(mode << UVH_IPI_INT_DELIVERY_MODE_SHFT) |
--- linux.orig/arch/x86/kernel/apic/x2apic_uv_x.c
+++ linux/arch/x86/kernel/apic/x2apic_uv_x.c
@@ -39,8 +39,6 @@
 #include <asm/x86_init.h>
 #include <asm/nmi.h>
 
-DEFINE_PER_CPU(int, x2apic_extra_bits);
-
 #define PR_DEVEL(fmt, args...)	pr_devel("%s: " fmt, __func__, args)
 
 static enum uv_system_type uv_system_type;
@@ -50,8 +48,6 @@ static u64 gru_dist_lmask, gru_dist_umas
 static union uvh_apicid uvh_apicid;
 int uv_min_hub_revision_id;
 EXPORT_SYMBOL_GPL(uv_min_hub_revision_id);
-unsigned int uv_apicid_hibits;
-EXPORT_SYMBOL_GPL(uv_apicid_hibits);
 
 static struct apic apic_x2apic_uv_x;
 
@@ -140,23 +136,6 @@ static void __init early_get_apic_pnode_
 		uvh_apicid.s.pnode_shift = UV_APIC_PNODE_SHIFT;
 }
 
-/*
- * Add an extra bit as dictated by bios to the destination apicid of
- * interrupts potentially passing through the UV HUB.  This prevents
- * a deadlock between interrupts and IO port operations.
- */
-static void __init uv_set_apicid_hibit(void)
-{
-	union uv1h_lb_target_physical_apic_id_mask_u apicid_mask;
-
-	if (is_uv1_hub()) {
-		apicid_mask.v =
-			uv_early_read_mmr(UV1H_LB_TARGET_PHYSICAL_APIC_ID_MASK);
-		uv_apicid_hibits =
-			apicid_mask.s1.bit_enables & UV_APICID_HIBIT_MASK;
-	}
-}
-
 static int __init uv_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	int pnodeid;
@@ -189,14 +168,6 @@ static int __init uv_acpi_madt_oem_check
 	if (!strcmp(oem_table_id, "UVX")) {		/* most common */
 		uv_system_type = UV_X2APIC;
 		uv_apic = 0;
-
-	} else if (!strcmp(oem_table_id, "UVH")) {	/* only UV1 systems */
-		uv_system_type = UV_NON_UNIQUE_APIC;
-		__this_cpu_write(x2apic_extra_bits,
-			pnodeid << uvh_apicid.s.pnode_shift);
-		uv_set_apicid_hibit();
-		uv_apic = 1;
-
 	} else	if (!strcmp(oem_table_id, "UVL")) {	/* only used for */
 		uv_system_type = UV_LEGACY_APIC;	/* very small systems */
 		uv_apic = 0;
@@ -252,7 +223,6 @@ static int uv_wakeup_secondary(int phys_
 	int pnode;
 
 	pnode = uv_apicid_to_pnode(phys_apicid);
-	phys_apicid |= uv_apicid_hibits;
 	val = (1UL << UVH_IPI_INT_SEND_SHFT) |
 	    (phys_apicid << UVH_IPI_INT_APIC_ID_SHFT) |
 	    ((start_rip << UVH_IPI_INT_VECTOR_SHFT) >> 12) |
@@ -344,7 +314,7 @@ uv_cpu_mask_to_apicid_and(const struct c
 	}
 
 	if (likely(cpu < nr_cpu_ids)) {
-		*apicid = per_cpu(x86_cpu_to_apicid, cpu) | uv_apicid_hibits;
+		*apicid = per_cpu(x86_cpu_to_apicid, cpu);
 		return 0;
 	}
 
@@ -353,21 +323,12 @@ uv_cpu_mask_to_apicid_and(const struct c
 
 static unsigned int x2apic_get_apic_id(unsigned long x)
 {
-	unsigned int id;
-
-	WARN_ON(preemptible() && num_online_cpus() > 1);
-	id = x | __this_cpu_read(x2apic_extra_bits);
-
-	return id;
+	return x;
 }
 
 static unsigned long set_apic_id(unsigned int id)
 {
-	unsigned long x;
-
-	/* maskout x2apic_extra_bits ? */
-	x = id;
-	return x;
+	return id;
 }
 
 static unsigned int uv_read_apic_id(void)
@@ -442,11 +403,6 @@ static struct apic __refdata apic_x2apic
 	.safe_wait_icr_idle		= native_safe_x2apic_wait_icr_idle,
 };
 
-static void set_x2apic_extra_bits(int pnode)
-{
-	__this_cpu_write(x2apic_extra_bits, pnode << uvh_apicid.s.pnode_shift);
-}
-
 /*
  * Called on boot cpu.
  */
@@ -878,9 +834,6 @@ void uv_cpu_init(void)
 		return;
 
 	uv_blade_info[uv_numa_blade_id()].nr_online_cpus++;
-
-	if (get_uv_system_type() == UV_NON_UNIQUE_APIC)
-		set_x2apic_extra_bits(uv_hub_info->pnode);
 }
 
 struct mn {
--- linux.orig/arch/x86/platform/uv/tlb_uv.c
+++ linux/arch/x86/platform/uv/tlb_uv.c
@@ -1834,7 +1834,7 @@ static void __init init_uvhub(int uvhub,
 	 * The below initialization can't be in firmware because the
 	 * messaging IRQ will be determined by the OS.
 	 */
-	apicid = uvhub_to_first_apicid(uvhub) | uv_apicid_hibits;
+	apicid = uvhub_to_first_apicid(uvhub);
 	write_mmr_data_config(pnode, ((apicid << 32) | vector));
 }
 
--- linux.orig/arch/x86/platform/uv/uv_time.c
+++ linux/arch/x86/platform/uv/uv_time.c
@@ -87,7 +87,6 @@ static void uv_rtc_send_IPI(int cpu)
 
 	apicid = cpu_physical_id(cpu);
 	pnode = uv_apicid_to_pnode(apicid);
-	apicid |= uv_apicid_hibits;
 	val = (1UL << UVH_IPI_INT_SEND_SHFT) |
 	      (apicid << UVH_IPI_INT_APIC_ID_SHFT) |
 	      (X86_PLATFORM_IPI_VECTOR << UVH_IPI_INT_VECTOR_SHFT);
@@ -111,7 +110,7 @@ static int uv_intr_pending(int pnode)
 static int uv_setup_intr(int cpu, u64 expires)
 {
 	u64 val;
-	unsigned long apicid = cpu_physical_id(cpu) | uv_apicid_hibits;
+	unsigned long apicid = cpu_physical_id(cpu);
 	int pnode = uv_cpu_to_pnode(cpu);
 
 	uv_write_global_mmr64(pnode, UVH_RTC1_INT_CONFIG,

-- 

[toc] | [prev] | [next] | [standalone]

#1389891 — Re: [PATCH 07/21] X86_64, UV: Remove Support for UV1

* Mike Travis <travis@sgi.com> wrote:

> The UV1 architecture is no longer being supported in future Linux kernels, so

Is such hardware still out there? If yes then please don't break it.

You might not want to test it, and if no-one else does it will eventually and 
inevitably bit rot, but please don't intentionally cripple it ..

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1390575 — Re: [PATCH 07/21] X86_64, UV: Remove Support for UV1

Hi Ingo,

I updated that patch to not remove any code, but instead there is now a
config option to enable or disable UV1 support, as well as an config help
message on why that might be done.  I also addressed the other issues
you raised in the following 2 emails.  And I found out that the newer
"checkpatches" script also had some new "concerns".  I was able to fix
all that were possible (or practical).

The patches follow...

Thanks!
Mike

On 4/28/2016 1:48 AM, Ingo Molnar wrote:
> 
> * Mike Travis <travis@sgi.com> wrote:
> 
>> The UV1 architecture is no longer being supported in future Linux kernels, so
> 
> Is such hardware still out there? If yes then please don't break it.
> 
> You might not want to test it, and if no-one else does it will eventually and 
> inevitably bit rot, but please don't intentionally cripple it ..
> 
> Thanks,
> 
> 	Ingo
> 

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web