Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1474995 > unrolled thread

[PATCH 0/6] MIPS: Remote processor driver

Started byMatt Redfearn <matt.redfearn@imgtec.com>
First post2016-09-02 12:10 +0200
Last post2016-09-02 12:10 +0200
Articles 7 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/6] MIPS: Remote processor driver Matt Redfearn <matt.redfearn@imgtec.com> - 2016-09-02 12:10 +0200
    [PATCH 3/6] MIPS: smp.c: Introduce mechanism for freeing and allocating IPIs Matt Redfearn <matt.redfearn@imgtec.com> - 2016-09-02 12:10 +0200
    [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC Matt Redfearn <matt.redfearn@imgtec.com> - 2016-09-02 12:10 +0200
      Re: [PATCH 1/6] irqchip: mips-gic: Add context saving for  MIPS_REMOTEPROC Matt Redfearn <matt.redfearn@imgtec.com> - 2016-09-02 12:30 +0200
      Re: [PATCH 1/6] irqchip: mips-gic: Add context saving for  MIPS_REMOTEPROC Marc Zyngier <marc.zyngier@arm.com> - 2016-09-02 13:00 +0200
        Re: [PATCH 1/6] irqchip: mips-gic: Add context saving for  MIPS_REMOTEPROC Matt Redfearn <matt.redfearn@imgtec.com> - 2016-09-02 16:20 +0200
    [PATCH 4/6] MIPS: CPS: Add VP(E) stealing Matt Redfearn <matt.redfearn@imgtec.com> - 2016-09-02 12:10 +0200

#1474995 — [PATCH 0/6] MIPS: Remote processor driver

FromMatt Redfearn <matt.redfearn@imgtec.com>
Date2016-09-02 12:10 +0200
Subject[PATCH 0/6] MIPS: Remote processor driver
Message-ID<scTA5-6xn-3@gated-at.bofh.it>
The MIPS remote processor driver allows non-Linux firmware to take
control of and execute on one of the systems VPEs. The CPU must be
offlined from Linux first. A sysfs interface is created which allows
firmware to be loaded and changed at runtime. A full description is
available at [1]. An example firmware that can be used with this driver
is available at [2].

This is useful to allow running bare metal code, or an RTOS, on one or
more CPUs while allowing Linux to continue running on those remaining.

The remote processor framework allows for firmwares to provide any
virtio device for communication between the firmware running on the
remote VPE and Linux. For example [2] demonstrates a simple firmware
which provides a virtual serial port. Any string sent to the port is
case inverted and returned.

This is conceptually similar to the VPE loader functionality, but is
more standard as it fits into the remoteproc subsystem.

The first patches in this series lay the groundwork for the driver
before it is added. The last series deprecates the VPE loader.

This functionality is supported on:
- MIPS32r2 devices implementing the MIPS MT ASE for multithreading, such
  as interAptiv.
- MIPS32r6 devices implementing VPs, such as I6400.

Limitations:
- The remoteproc core supports only 32bit ELFs. Therefore it is only
  possible to run 32bit firmware on the remote processor. Also, for
  virtio communication, pointers are passed from the kernel to firmware.
  There can be no mismatch in pointer sizes between the kernel and
  firmware, so this limits the host kernel to 32bit as well.

The functionality has been tested on the Ci40 board which has a 2 core 2
thread interAptiv.

This series is based on v4.8-rc4.

Depends on some patches from James Hogan's recent "MIPS: General EVA fixes &
cleanups" series:
MIPS: traps: 64bit kernels should read CP0_EBase 64bit
MIPS: traps: Convert ebase to KSeg0
MIPS: traps: Ensure full EBase is written

Without these patches, if firmware modifies ebase to allow handling
exceptions / interrupts, then when the VPE is returned to Linux the
kernel exception handlers won't be reinstated properly.

[1] http://wiki.prplfoundation.org/w/images/d/df/MIPS_OS_Remote_Processor_Driver_Whitepaper_1.0.9.pdf
[2] https://github.com/MIPS/mips-rproc-example



Lisa Parratt (1):
  MIPS: CPS: Add VP(E) stealing

Matt Redfearn (5):
  irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC
  MIPS: tlb-r4k: If there are wired entries, don't use TLBINVF
  MIPS: smp.c: Introduce mechanism for freeing and allocating IPIs
  remoteproc/MIPS: Add a remoteproc driver for MIPS
  MIPS: Deprecate VPE Loader

 Documentation/ABI/testing/sysfs-class-mips-rproc |  24 +
 arch/mips/Kconfig                                |  12 +-
 arch/mips/include/asm/smp-cps.h                  |   8 +
 arch/mips/include/asm/smp.h                      |  15 +
 arch/mips/kernel/smp-cps.c                       | 162 +++++-
 arch/mips/kernel/smp.c                           |  73 ++-
 arch/mips/mm/tlb-r4k.c                           |   7 +-
 drivers/irqchip/irq-mips-gic.c                   | 185 ++++++-
 drivers/remoteproc/Kconfig                       |  11 +
 drivers/remoteproc/Makefile                      |   1 +
 drivers/remoteproc/mips_remoteproc.c             | 651 +++++++++++++++++++++++
 11 files changed, 1124 insertions(+), 25 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-class-mips-rproc
 create mode 100644 drivers/remoteproc/mips_remoteproc.c

-- 
2.7.4

[toc] | [next] | [standalone]

#1474996 — [PATCH 3/6] MIPS: smp.c: Introduce mechanism for freeing and allocating IPIs

FromMatt Redfearn <matt.redfearn@imgtec.com>
Date2016-09-02 12:10 +0200
Subject[PATCH 3/6] MIPS: smp.c: Introduce mechanism for freeing and allocating IPIs
Message-ID<scTA6-6xn-59@gated-at.bofh.it>
In reply to#1474995
For the MIPS remote processor implementation, we need additional IPIs to
talk to the remote processor. Since MIPS GIC reserves exactly the right
number of IPI IRQs required by Linux for the number of VPs in the
system, this is not possible without releasing some recources.

This commit introduces mips_smp_ipi_allocate() which allocates IPIs to a
given cpumask. It is called as normal with the cpu_possible_mask at
bootup to initialise IPIs to all CPUs. mips_smp_ipi_free() may then be
used to free IPIs to a subset of those CPUs so that their hardware
resources can be reused.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
---

 arch/mips/include/asm/smp.h | 14 +++++++++++
 arch/mips/kernel/smp.c      | 61 +++++++++++++++++++++++++++++++++++++++------
 2 files changed, 67 insertions(+), 8 deletions(-)

diff --git a/arch/mips/include/asm/smp.h b/arch/mips/include/asm/smp.h
index 8bc6c70a4030..060f23ff1817 100644
--- a/arch/mips/include/asm/smp.h
+++ b/arch/mips/include/asm/smp.h
@@ -85,6 +85,20 @@ static inline void __cpu_die(unsigned int cpu)
 extern void play_dead(void);
 #endif
 
+/*
+ * This function will set up the necessary IPIs for Linux to communicate
+ * with the CPUs in mask.
+ * Return 0 on success.
+ */
+int mips_smp_ipi_allocate(const struct cpumask *mask);
+
+/*
+ * This function will free up IPIs allocated with mips_smp_ipi_allocate to the
+ * CPUs in mask, which must be a subset of the IPIs that have been configured.
+ * Return 0 on success.
+ */
+int mips_smp_ipi_free(const struct cpumask *mask);
+
 static inline void arch_send_call_function_single_ipi(int cpu)
 {
 	extern struct plat_smp_ops *mp_ops;	/* private */
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index f95f094f36e4..afa06c2bb019 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -229,7 +229,7 @@ static struct irqaction irq_call = {
 	.name		= "IPI call"
 };
 
-static __init void smp_ipi_init_one(unsigned int virq,
+static void smp_ipi_init_one(unsigned int virq,
 				    struct irqaction *action)
 {
 	int ret;
@@ -239,9 +239,11 @@ static __init void smp_ipi_init_one(unsigned int virq,
 	BUG_ON(ret);
 }
 
-static int __init mips_smp_ipi_init(void)
+static unsigned int call_virq, sched_virq;
+
+int mips_smp_ipi_allocate(const struct cpumask *mask)
 {
-	unsigned int call_virq, sched_virq;
+	int virq;
 	struct irq_domain *ipidomain;
 	struct device_node *node;
 
@@ -268,16 +270,20 @@ static int __init mips_smp_ipi_init(void)
 	if (!ipidomain)
 		return 0;
 
-	call_virq = irq_reserve_ipi(ipidomain, cpu_possible_mask);
-	BUG_ON(!call_virq);
+	virq = irq_reserve_ipi(ipidomain, mask);
+	BUG_ON(!virq);
+	if (!call_virq)
+		call_virq = virq;
 
-	sched_virq = irq_reserve_ipi(ipidomain, cpu_possible_mask);
-	BUG_ON(!sched_virq);
+	virq = irq_reserve_ipi(ipidomain, mask);
+	BUG_ON(!virq);
+	if (!sched_virq)
+		sched_virq = virq;
 
 	if (irq_domain_is_ipi_per_cpu(ipidomain)) {
 		int cpu;
 
-		for_each_cpu(cpu, cpu_possible_mask) {
+		for_each_cpu(cpu, mask) {
 			smp_ipi_init_one(call_virq + cpu, &irq_call);
 			smp_ipi_init_one(sched_virq + cpu, &irq_resched);
 		}
@@ -286,6 +292,45 @@ static int __init mips_smp_ipi_init(void)
 		smp_ipi_init_one(sched_virq, &irq_resched);
 	}
 
+	return 0;
+}
+
+int mips_smp_ipi_free(const struct cpumask *mask)
+{
+	struct irq_domain *ipidomain;
+	struct device_node *node;
+
+	node = of_irq_find_parent(of_root);
+	ipidomain = irq_find_matching_host(node, DOMAIN_BUS_IPI);
+
+	/*
+	 * Some platforms have half DT setup. So if we found irq node but
+	 * didn't find an ipidomain, try to search for one that is not in the
+	 * DT.
+	 */
+	if (node && !ipidomain)
+		ipidomain = irq_find_matching_host(NULL, DOMAIN_BUS_IPI);
+
+	BUG_ON(!ipidomain);
+
+	if (irq_domain_is_ipi_per_cpu(ipidomain)) {
+		int cpu;
+
+		for_each_cpu(cpu, mask) {
+			remove_irq(call_virq + cpu, &irq_call);
+			remove_irq(sched_virq + cpu, &irq_resched);
+		}
+	}
+	irq_destroy_ipi(call_virq, mask);
+	irq_destroy_ipi(sched_virq, mask);
+	return 0;
+}
+
+
+static int __init mips_smp_ipi_init(void)
+{
+	mips_smp_ipi_allocate(cpu_possible_mask);
+
 	call_desc = irq_to_desc(call_virq);
 	sched_desc = irq_to_desc(sched_virq);
 
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1474999 — [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC

FromMatt Redfearn <matt.redfearn@imgtec.com>
Date2016-09-02 12:10 +0200
Subject[PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC
Message-ID<scTA6-6xn-61@gated-at.bofh.it>
In reply to#1474995
The MIPS remote processor driver allows non-Linux firmware to take
control of and execute on one of the systems VPEs. If that VPE is
brought back under Linux, it is necessary to ensure that all GIC
interrupts are routed and masked as Linux expects them, as the firmware
can have done anything it likes with the GIC configuration (hopefully
just for that VPEs local interrupt sources, but allow for shared
external interrupts as well).

The configuration of shared and local CPU interrupts is maintained and
updated every time a change is made. When a CPU is brought online, the
saved configuration is restored.

These functions will also be useful for restoring GIC context after a
suspend to RAM.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
---

 drivers/irqchip/irq-mips-gic.c | 185 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 178 insertions(+), 7 deletions(-)

diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
index 83f498393a7f..5ba1fe1468ce 100644
--- a/drivers/irqchip/irq-mips-gic.c
+++ b/drivers/irqchip/irq-mips-gic.c
@@ -8,6 +8,7 @@
  */
 #include <linux/bitmap.h>
 #include <linux/clocksource.h>
+#include <linux/cpu.h>
 #include <linux/init.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
@@ -56,6 +57,47 @@ static unsigned int timer_cpu_pin;
 static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller;
 DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS);
 
+#if defined(CONFIG_MIPS_RPROC)
+#define CONTEXT_SAVING
+#endif
+
+#ifdef CONTEXT_SAVING
+static struct {
+	unsigned mask:		GIC_NUM_LOCAL_INTRS;
+} gic_local_state[NR_CPUS];
+
+#define gic_save_local_rmask(vpe, i)	(gic_local_state[vpe].mask &= i)
+#define gic_save_local_smask(vpe, i)	(gic_local_state[vpe].mask |= i)
+
+static struct {
+	unsigned vpe:		8;
+	unsigned pin:		4;
+
+	unsigned polarity:	1;
+	unsigned trigger:	1;
+	unsigned dual_edge:	1;
+	unsigned mask:		1;
+} gic_shared_state[GIC_MAX_INTRS];
+
+#define gic_save_shared_vpe(i, v)	gic_shared_state[i].vpe = v
+#define gic_save_shared_pin(i, p)	gic_shared_state[i].pin = p
+#define gic_save_shared_polarity(i, p)	gic_shared_state[i].polarity = p
+#define gic_save_shared_trigger(i, t)	gic_shared_state[i].trigger = t
+#define gic_save_shared_dual_edge(i, d)	gic_shared_state[i].dual_edge = d
+#define gic_save_shared_mask(i, m)	gic_shared_state[i].mask = m
+
+#else
+#define gic_save_local_rmask(vpe, i)
+#define gic_save_local_smask(vpe, i)
+
+#define gic_save_shared_vpe(i, v)
+#define gic_save_shared_pin(i, p)
+#define gic_save_shared_polarity(i, p)
+#define gic_save_shared_trigger(i, t)
+#define gic_save_shared_dual_edge(i, d)
+#define gic_save_shared_mask(i, m)
+#endif /* CONTEXT_SAVING */
+
 static void __gic_irq_dispatch(void);
 
 static inline u32 gic_read32(unsigned int reg)
@@ -105,52 +147,94 @@ static inline void gic_update_bits(unsigned int reg, unsigned long mask,
 	gic_write(reg, regval);
 }
 
-static inline void gic_reset_mask(unsigned int intr)
+static inline void gic_write_reset_mask(unsigned int intr)
 {
 	gic_write(GIC_REG(SHARED, GIC_SH_RMASK) + GIC_INTR_OFS(intr),
 		  1ul << GIC_INTR_BIT(intr));
 }
 
-static inline void gic_set_mask(unsigned int intr)
+static inline void gic_reset_mask(unsigned int intr)
+{
+	gic_save_shared_mask(intr, 0);
+	gic_write_reset_mask(intr);
+}
+
+static inline void gic_write_set_mask(unsigned int intr)
 {
 	gic_write(GIC_REG(SHARED, GIC_SH_SMASK) + GIC_INTR_OFS(intr),
 		  1ul << GIC_INTR_BIT(intr));
 }
 
-static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
+static inline void gic_set_mask(unsigned int intr)
+{
+	gic_save_shared_mask(intr, 1);
+	gic_write_set_mask(intr);
+}
+
+static inline void gic_write_polarity(unsigned int intr, unsigned int pol)
 {
 	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_POLARITY) +
 			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
 			(unsigned long)pol << GIC_INTR_BIT(intr));
 }
 
-static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
+static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
+{
+	gic_save_shared_polarity(intr, pol);
+	gic_write_polarity(intr, pol);
+}
+
+static inline void gic_write_trigger(unsigned int intr, unsigned int trig)
 {
 	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_TRIGGER) +
 			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
 			(unsigned long)trig << GIC_INTR_BIT(intr));
 }
 
-static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
+static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
+{
+	gic_save_shared_trigger(intr, trig);
+	gic_write_trigger(intr, trig);
+}
+
+static inline void gic_write_dual_edge(unsigned int intr, unsigned int dual)
 {
 	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_DUAL) + GIC_INTR_OFS(intr),
 			1ul << GIC_INTR_BIT(intr),
 			(unsigned long)dual << GIC_INTR_BIT(intr));
 }
 
-static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
+static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
+{
+	gic_save_shared_dual_edge(intr, dual);
+	gic_write_dual_edge(intr, dual);
+}
+
+static inline void gic_write_map_to_pin(unsigned int intr, unsigned int pin)
 {
 	gic_write32(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_PIN_BASE) +
 		    GIC_SH_MAP_TO_PIN(intr), GIC_MAP_TO_PIN_MSK | pin);
 }
 
-static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
+static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
+{
+	gic_save_shared_pin(intr, pin);
+	gic_write_map_to_pin(intr, pin);
+}
+
+static inline void gic_write_map_to_vpe(unsigned int intr, unsigned int vpe)
 {
 	gic_write(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_VPE_BASE) +
 		  GIC_SH_MAP_TO_VPE_REG_OFF(intr, vpe),
 		  GIC_SH_MAP_TO_VPE_REG_BIT(vpe));
 }
 
+static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
+{
+	gic_save_shared_vpe(intr, vpe);
+	gic_write_map_to_vpe(intr, vpe);
+}
+
 #ifdef CONFIG_CLKSRC_MIPS_GIC
 cycle_t gic_read_count(void)
 {
@@ -537,6 +621,7 @@ static void gic_mask_local_irq(struct irq_data *d)
 {
 	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
 
+	gic_save_local_rmask(smp_processor_id(), (1 << intr));
 	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_RMASK), 1 << intr);
 }
 
@@ -544,6 +629,7 @@ static void gic_unmask_local_irq(struct irq_data *d)
 {
 	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
 
+	gic_save_local_smask(smp_processor_id(), (1 << intr));
 	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_SMASK), 1 << intr);
 }
 
@@ -561,6 +647,7 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d)
 
 	spin_lock_irqsave(&gic_lock, flags);
 	for (i = 0; i < gic_vpes; i++) {
+		gic_save_local_rmask(mips_cm_vp_id(i), 1 << intr);
 		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
 			  mips_cm_vp_id(i));
 		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr);
@@ -576,6 +663,7 @@ static void gic_unmask_local_irq_all_vpes(struct irq_data *d)
 
 	spin_lock_irqsave(&gic_lock, flags);
 	for (i = 0; i < gic_vpes; i++) {
+		gic_save_local_smask(mips_cm_vp_id(i), 1 << intr);
 		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
 			  mips_cm_vp_id(i));
 		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr);
@@ -983,6 +1071,85 @@ static struct irq_domain_ops gic_ipi_domain_ops = {
 	.match = gic_ipi_domain_match,
 };
 
+#ifdef CONTEXT_SAVING
+static void gic_restore_shared(void)
+{
+	unsigned long flags;
+	int i;
+
+	spin_lock_irqsave(&gic_lock, flags);
+	for (i = 0; i < gic_shared_intrs; i++) {
+		gic_write_polarity(i, gic_shared_state[i].polarity);
+		gic_write_trigger(i, gic_shared_state[i].trigger);
+		gic_write_dual_edge(i, gic_shared_state[i].dual_edge);
+		gic_write_map_to_vpe(i, gic_shared_state[i].vpe);
+		gic_write_map_to_pin(i, gic_shared_state[i].pin);
+
+		if (gic_shared_state[i].mask)
+			gic_write_set_mask(i);
+		else
+			gic_write_reset_mask(i);
+	}
+	spin_unlock_irqrestore(&gic_lock, flags);
+}
+
+static void gic_restore_local(unsigned int vpe)
+{
+	int hw, virq, intr, mask;
+	unsigned long flags;
+
+	for (hw = 0; hw < GIC_NUM_LOCAL_INTRS; hw++) {
+		intr = GIC_LOCAL_TO_HWIRQ(hw);
+		virq = irq_linear_revmap(gic_irq_domain, intr);
+		gic_local_irq_domain_map(gic_irq_domain, virq, hw);
+	}
+
+	local_irq_save(flags);
+	gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), vpe);
+
+	/* Enable EIC mode if necessary */
+	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_CTL), cpu_has_veic);
+
+	/* Restore interrupt masks */
+	mask = gic_local_state[vpe].mask;
+	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), ~mask);
+	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), mask);
+
+	local_irq_restore(flags);
+}
+#endif /* CONTEXT_SAVING */
+
+#ifdef CONFIG_MIPS_RPROC
+/*
+ * The MIPS remote processor driver allows non-Linux firmware to take control
+ * of and execute on one of the systems VPEs. If that VPE is brought back under
+ * Linux, it is necessary to ensure that all GIC interrupts are routed and
+ * masked as Linux expects them, as the firmware can have done anything it
+ * likes with the GIC configuration (hopefully just for that VPEs local
+ * interrupt sources, but allow for shared external interrupts as well).
+ */
+static int gic_cpu_notify(struct notifier_block *nfb, unsigned long action,
+			       void *hcpu)
+{
+	unsigned int cpu = mips_cm_vp_id((unsigned long)hcpu);
+
+	switch (action) {
+	case CPU_UP_PREPARE:
+	case CPU_DOWN_FAILED:
+		gic_restore_shared();
+		gic_restore_local(cpu);
+	default:
+		break;
+	}
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block gic_cpu_notifier __refdata = {
+	.notifier_call = gic_cpu_notify
+};
+#endif /* CONFIG_MIPS_RPROC */
+
 static void __init __gic_init(unsigned long gic_base_addr,
 			      unsigned long gic_addrspace_size,
 			      unsigned int cpu_vec, unsigned int irqbase,
@@ -1082,6 +1249,10 @@ static void __init __gic_init(unsigned long gic_base_addr,
 	}
 
 	gic_basic_init();
+
+#ifdef CONFIG_MIPS_RPROC
+	register_hotcpu_notifier(&gic_cpu_notifier);
+#endif /* CONFIG_MIPS_RPROC */
 }
 
 void __init gic_init(unsigned long gic_base_addr,
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1475015 — Re: [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC

FromMatt Redfearn <matt.redfearn@imgtec.com>
Date2016-09-02 12:30 +0200
SubjectRe: [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC
Message-ID<scTTs-6I4-15@gated-at.bofh.it>
In reply to#1474999

On 02/09/16 10:59, Matt Redfearn wrote:
> The MIPS remote processor driver allows non-Linux firmware to take
> control of and execute on one of the systems VPEs. If that VPE is
> brought back under Linux, it is necessary to ensure that all GIC
> interrupts are routed and masked as Linux expects them, as the firmware
> can have done anything it likes with the GIC configuration (hopefully
> just for that VPEs local interrupt sources, but allow for shared
> external interrupts as well).
>
> The configuration of shared and local CPU interrupts is maintained and
> updated every time a change is made. When a CPU is brought online, the
> saved configuration is restored.
>
> These functions will also be useful for restoring GIC context after a
> suspend to RAM.
>
> Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
> ---
>
>   drivers/irqchip/irq-mips-gic.c | 185 +++++++++++++++++++++++++++++++++++++++--
>   1 file changed, 178 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
> index 83f498393a7f..5ba1fe1468ce 100644
> --- a/drivers/irqchip/irq-mips-gic.c
> +++ b/drivers/irqchip/irq-mips-gic.c
> @@ -8,6 +8,7 @@
>    */
>   #include <linux/bitmap.h>
>   #include <linux/clocksource.h>
> +#include <linux/cpu.h>
>   #include <linux/init.h>
>   #include <linux/interrupt.h>
>   #include <linux/irq.h>
> @@ -56,6 +57,47 @@ static unsigned int timer_cpu_pin;
>   static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller;
>   DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS);
>   
> +#if defined(CONFIG_MIPS_RPROC)

Spot the mistake - this should, of course, be CONFIG_MIPS_REMOTEPROC. 
I'll fix that in v2.

Matt

> +#define CONTEXT_SAVING
> +#endif
> +
> +#ifdef CONTEXT_SAVING
> +static struct {
> +	unsigned mask:		GIC_NUM_LOCAL_INTRS;
> +} gic_local_state[NR_CPUS];
> +
> +#define gic_save_local_rmask(vpe, i)	(gic_local_state[vpe].mask &= i)
> +#define gic_save_local_smask(vpe, i)	(gic_local_state[vpe].mask |= i)
> +
> +static struct {
> +	unsigned vpe:		8;
> +	unsigned pin:		4;
> +
> +	unsigned polarity:	1;
> +	unsigned trigger:	1;
> +	unsigned dual_edge:	1;
> +	unsigned mask:		1;
> +} gic_shared_state[GIC_MAX_INTRS];
> +
> +#define gic_save_shared_vpe(i, v)	gic_shared_state[i].vpe = v
> +#define gic_save_shared_pin(i, p)	gic_shared_state[i].pin = p
> +#define gic_save_shared_polarity(i, p)	gic_shared_state[i].polarity = p
> +#define gic_save_shared_trigger(i, t)	gic_shared_state[i].trigger = t
> +#define gic_save_shared_dual_edge(i, d)	gic_shared_state[i].dual_edge = d
> +#define gic_save_shared_mask(i, m)	gic_shared_state[i].mask = m
> +
> +#else
> +#define gic_save_local_rmask(vpe, i)
> +#define gic_save_local_smask(vpe, i)
> +
> +#define gic_save_shared_vpe(i, v)
> +#define gic_save_shared_pin(i, p)
> +#define gic_save_shared_polarity(i, p)
> +#define gic_save_shared_trigger(i, t)
> +#define gic_save_shared_dual_edge(i, d)
> +#define gic_save_shared_mask(i, m)
> +#endif /* CONTEXT_SAVING */
> +
>   static void __gic_irq_dispatch(void);
>   
>   static inline u32 gic_read32(unsigned int reg)
> @@ -105,52 +147,94 @@ static inline void gic_update_bits(unsigned int reg, unsigned long mask,
>   	gic_write(reg, regval);
>   }
>   
> -static inline void gic_reset_mask(unsigned int intr)
> +static inline void gic_write_reset_mask(unsigned int intr)
>   {
>   	gic_write(GIC_REG(SHARED, GIC_SH_RMASK) + GIC_INTR_OFS(intr),
>   		  1ul << GIC_INTR_BIT(intr));
>   }
>   
> -static inline void gic_set_mask(unsigned int intr)
> +static inline void gic_reset_mask(unsigned int intr)
> +{
> +	gic_save_shared_mask(intr, 0);
> +	gic_write_reset_mask(intr);
> +}
> +
> +static inline void gic_write_set_mask(unsigned int intr)
>   {
>   	gic_write(GIC_REG(SHARED, GIC_SH_SMASK) + GIC_INTR_OFS(intr),
>   		  1ul << GIC_INTR_BIT(intr));
>   }
>   
> -static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
> +static inline void gic_set_mask(unsigned int intr)
> +{
> +	gic_save_shared_mask(intr, 1);
> +	gic_write_set_mask(intr);
> +}
> +
> +static inline void gic_write_polarity(unsigned int intr, unsigned int pol)
>   {
>   	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_POLARITY) +
>   			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
>   			(unsigned long)pol << GIC_INTR_BIT(intr));
>   }
>   
> -static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
> +static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
> +{
> +	gic_save_shared_polarity(intr, pol);
> +	gic_write_polarity(intr, pol);
> +}
> +
> +static inline void gic_write_trigger(unsigned int intr, unsigned int trig)
>   {
>   	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_TRIGGER) +
>   			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
>   			(unsigned long)trig << GIC_INTR_BIT(intr));
>   }
>   
> -static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
> +static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
> +{
> +	gic_save_shared_trigger(intr, trig);
> +	gic_write_trigger(intr, trig);
> +}
> +
> +static inline void gic_write_dual_edge(unsigned int intr, unsigned int dual)
>   {
>   	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_DUAL) + GIC_INTR_OFS(intr),
>   			1ul << GIC_INTR_BIT(intr),
>   			(unsigned long)dual << GIC_INTR_BIT(intr));
>   }
>   
> -static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
> +static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
> +{
> +	gic_save_shared_dual_edge(intr, dual);
> +	gic_write_dual_edge(intr, dual);
> +}
> +
> +static inline void gic_write_map_to_pin(unsigned int intr, unsigned int pin)
>   {
>   	gic_write32(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_PIN_BASE) +
>   		    GIC_SH_MAP_TO_PIN(intr), GIC_MAP_TO_PIN_MSK | pin);
>   }
>   
> -static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
> +static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
> +{
> +	gic_save_shared_pin(intr, pin);
> +	gic_write_map_to_pin(intr, pin);
> +}
> +
> +static inline void gic_write_map_to_vpe(unsigned int intr, unsigned int vpe)
>   {
>   	gic_write(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_VPE_BASE) +
>   		  GIC_SH_MAP_TO_VPE_REG_OFF(intr, vpe),
>   		  GIC_SH_MAP_TO_VPE_REG_BIT(vpe));
>   }
>   
> +static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
> +{
> +	gic_save_shared_vpe(intr, vpe);
> +	gic_write_map_to_vpe(intr, vpe);
> +}
> +
>   #ifdef CONFIG_CLKSRC_MIPS_GIC
>   cycle_t gic_read_count(void)
>   {
> @@ -537,6 +621,7 @@ static void gic_mask_local_irq(struct irq_data *d)
>   {
>   	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
>   
> +	gic_save_local_rmask(smp_processor_id(), (1 << intr));
>   	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_RMASK), 1 << intr);
>   }
>   
> @@ -544,6 +629,7 @@ static void gic_unmask_local_irq(struct irq_data *d)
>   {
>   	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
>   
> +	gic_save_local_smask(smp_processor_id(), (1 << intr));
>   	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_SMASK), 1 << intr);
>   }
>   
> @@ -561,6 +647,7 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d)
>   
>   	spin_lock_irqsave(&gic_lock, flags);
>   	for (i = 0; i < gic_vpes; i++) {
> +		gic_save_local_rmask(mips_cm_vp_id(i), 1 << intr);
>   		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
>   			  mips_cm_vp_id(i));
>   		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr);
> @@ -576,6 +663,7 @@ static void gic_unmask_local_irq_all_vpes(struct irq_data *d)
>   
>   	spin_lock_irqsave(&gic_lock, flags);
>   	for (i = 0; i < gic_vpes; i++) {
> +		gic_save_local_smask(mips_cm_vp_id(i), 1 << intr);
>   		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
>   			  mips_cm_vp_id(i));
>   		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr);
> @@ -983,6 +1071,85 @@ static struct irq_domain_ops gic_ipi_domain_ops = {
>   	.match = gic_ipi_domain_match,
>   };
>   
> +#ifdef CONTEXT_SAVING
> +static void gic_restore_shared(void)
> +{
> +	unsigned long flags;
> +	int i;
> +
> +	spin_lock_irqsave(&gic_lock, flags);
> +	for (i = 0; i < gic_shared_intrs; i++) {
> +		gic_write_polarity(i, gic_shared_state[i].polarity);
> +		gic_write_trigger(i, gic_shared_state[i].trigger);
> +		gic_write_dual_edge(i, gic_shared_state[i].dual_edge);
> +		gic_write_map_to_vpe(i, gic_shared_state[i].vpe);
> +		gic_write_map_to_pin(i, gic_shared_state[i].pin);
> +
> +		if (gic_shared_state[i].mask)
> +			gic_write_set_mask(i);
> +		else
> +			gic_write_reset_mask(i);
> +	}
> +	spin_unlock_irqrestore(&gic_lock, flags);
> +}
> +
> +static void gic_restore_local(unsigned int vpe)
> +{
> +	int hw, virq, intr, mask;
> +	unsigned long flags;
> +
> +	for (hw = 0; hw < GIC_NUM_LOCAL_INTRS; hw++) {
> +		intr = GIC_LOCAL_TO_HWIRQ(hw);
> +		virq = irq_linear_revmap(gic_irq_domain, intr);
> +		gic_local_irq_domain_map(gic_irq_domain, virq, hw);
> +	}
> +
> +	local_irq_save(flags);
> +	gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), vpe);
> +
> +	/* Enable EIC mode if necessary */
> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_CTL), cpu_has_veic);
> +
> +	/* Restore interrupt masks */
> +	mask = gic_local_state[vpe].mask;
> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), ~mask);
> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), mask);
> +
> +	local_irq_restore(flags);
> +}
> +#endif /* CONTEXT_SAVING */
> +
> +#ifdef CONFIG_MIPS_RPROC
> +/*
> + * The MIPS remote processor driver allows non-Linux firmware to take control
> + * of and execute on one of the systems VPEs. If that VPE is brought back under
> + * Linux, it is necessary to ensure that all GIC interrupts are routed and
> + * masked as Linux expects them, as the firmware can have done anything it
> + * likes with the GIC configuration (hopefully just for that VPEs local
> + * interrupt sources, but allow for shared external interrupts as well).
> + */
> +static int gic_cpu_notify(struct notifier_block *nfb, unsigned long action,
> +			       void *hcpu)
> +{
> +	unsigned int cpu = mips_cm_vp_id((unsigned long)hcpu);
> +
> +	switch (action) {
> +	case CPU_UP_PREPARE:
> +	case CPU_DOWN_FAILED:
> +		gic_restore_shared();
> +		gic_restore_local(cpu);
> +	default:
> +		break;
> +	}
> +
> +	return NOTIFY_OK;
> +}
> +
> +static struct notifier_block gic_cpu_notifier __refdata = {
> +	.notifier_call = gic_cpu_notify
> +};
> +#endif /* CONFIG_MIPS_RPROC */
> +
>   static void __init __gic_init(unsigned long gic_base_addr,
>   			      unsigned long gic_addrspace_size,
>   			      unsigned int cpu_vec, unsigned int irqbase,
> @@ -1082,6 +1249,10 @@ static void __init __gic_init(unsigned long gic_base_addr,
>   	}
>   
>   	gic_basic_init();
> +
> +#ifdef CONFIG_MIPS_RPROC
> +	register_hotcpu_notifier(&gic_cpu_notifier);
> +#endif /* CONFIG_MIPS_RPROC */
>   }
>   
>   void __init gic_init(unsigned long gic_base_addr,

[toc] | [prev] | [next] | [standalone]

#1475035 — Re: [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC

FromMarc Zyngier <marc.zyngier@arm.com>
Date2016-09-02 13:00 +0200
SubjectRe: [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC
Message-ID<scUmu-6SO-33@gated-at.bofh.it>
In reply to#1474999
Hi Matt,

On 02/09/16 10:59, Matt Redfearn wrote:
> The MIPS remote processor driver allows non-Linux firmware to take
> control of and execute on one of the systems VPEs. If that VPE is
> brought back under Linux, it is necessary to ensure that all GIC
> interrupts are routed and masked as Linux expects them, as the firmware
> can have done anything it likes with the GIC configuration (hopefully
> just for that VPEs local interrupt sources, but allow for shared
> external interrupts as well).
> 
> The configuration of shared and local CPU interrupts is maintained and
> updated every time a change is made. When a CPU is brought online, the
> saved configuration is restored.
> 
> These functions will also be useful for restoring GIC context after a
> suspend to RAM.
> 
> Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
> ---
> 
>  drivers/irqchip/irq-mips-gic.c | 185 +++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 178 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
> index 83f498393a7f..5ba1fe1468ce 100644
> --- a/drivers/irqchip/irq-mips-gic.c
> +++ b/drivers/irqchip/irq-mips-gic.c
> @@ -8,6 +8,7 @@
>   */
>  #include <linux/bitmap.h>
>  #include <linux/clocksource.h>
> +#include <linux/cpu.h>
>  #include <linux/init.h>
>  #include <linux/interrupt.h>
>  #include <linux/irq.h>
> @@ -56,6 +57,47 @@ static unsigned int timer_cpu_pin;
>  static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller;
>  DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS);
>  
> +#if defined(CONFIG_MIPS_RPROC)
> +#define CONTEXT_SAVING
> +#endif
> +
> +#ifdef CONTEXT_SAVING

This looks really cumbersome. Why not make everything depend on
CONFIG_MIPS_RPROC instead?

> +static struct {
> +	unsigned mask:		GIC_NUM_LOCAL_INTRS;

nit/personal taste: can't you make this a normal type instead of a
bitfield? Considering that GIC_NUM_LOCAL_INTRS is hardcoded to 7, I'd
rather see a u8... or even an unsigned long if you have to use bitmap
operations on it.

> +} gic_local_state[NR_CPUS];

This looks like this really should be a percpu variable.

> +
> +#define gic_save_local_rmask(vpe, i)	(gic_local_state[vpe].mask &= i)
> +#define gic_save_local_smask(vpe, i)	(gic_local_state[vpe].mask |= i)
> +
> +static struct {
> +	unsigned vpe:		8;
> +	unsigned pin:		4;
> +
> +	unsigned polarity:	1;
> +	unsigned trigger:	1;
> +	unsigned dual_edge:	1;
> +	unsigned mask:		1;
> +} gic_shared_state[GIC_MAX_INTRS];
> +
> +#define gic_save_shared_vpe(i, v)	gic_shared_state[i].vpe = v
> +#define gic_save_shared_pin(i, p)	gic_shared_state[i].pin = p
> +#define gic_save_shared_polarity(i, p)	gic_shared_state[i].polarity = p
> +#define gic_save_shared_trigger(i, t)	gic_shared_state[i].trigger = t
> +#define gic_save_shared_dual_edge(i, d)	gic_shared_state[i].dual_edge = d
> +#define gic_save_shared_mask(i, m)	gic_shared_state[i].mask = m

Why don't you make these static functions? The compiler will inline them
nicely, and that will save you fixing them (they all miss proper
bracketing of arguments).

> +
> +#else
> +#define gic_save_local_rmask(vpe, i)
> +#define gic_save_local_smask(vpe, i)
> +
> +#define gic_save_shared_vpe(i, v)
> +#define gic_save_shared_pin(i, p)
> +#define gic_save_shared_polarity(i, p)
> +#define gic_save_shared_trigger(i, t)
> +#define gic_save_shared_dual_edge(i, d)
> +#define gic_save_shared_mask(i, m)

Please make those a "do { } while(0)" construct, so that the trailing
semi-colon is properly swallowed.

> +#endif /* CONTEXT_SAVING */
> +
>  static void __gic_irq_dispatch(void);
>  
>  static inline u32 gic_read32(unsigned int reg)
> @@ -105,52 +147,94 @@ static inline void gic_update_bits(unsigned int reg, unsigned long mask,
>  	gic_write(reg, regval);
>  }
>  
> -static inline void gic_reset_mask(unsigned int intr)
> +static inline void gic_write_reset_mask(unsigned int intr)
>  {
>  	gic_write(GIC_REG(SHARED, GIC_SH_RMASK) + GIC_INTR_OFS(intr),
>  		  1ul << GIC_INTR_BIT(intr));
>  }
>  
> -static inline void gic_set_mask(unsigned int intr)
> +static inline void gic_reset_mask(unsigned int intr)
> +{
> +	gic_save_shared_mask(intr, 0);
> +	gic_write_reset_mask(intr);
> +}
> +
> +static inline void gic_write_set_mask(unsigned int intr)
>  {
>  	gic_write(GIC_REG(SHARED, GIC_SH_SMASK) + GIC_INTR_OFS(intr),
>  		  1ul << GIC_INTR_BIT(intr));
>  }
>  
> -static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
> +static inline void gic_set_mask(unsigned int intr)
> +{
> +	gic_save_shared_mask(intr, 1);
> +	gic_write_set_mask(intr);
> +}
> +
> +static inline void gic_write_polarity(unsigned int intr, unsigned int pol)
>  {
>  	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_POLARITY) +
>  			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
>  			(unsigned long)pol << GIC_INTR_BIT(intr));
>  }
>  
> -static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
> +static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
> +{
> +	gic_save_shared_polarity(intr, pol);
> +	gic_write_polarity(intr, pol);
> +}
> +
> +static inline void gic_write_trigger(unsigned int intr, unsigned int trig)
>  {
>  	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_TRIGGER) +
>  			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
>  			(unsigned long)trig << GIC_INTR_BIT(intr));
>  }
>  
> -static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
> +static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
> +{
> +	gic_save_shared_trigger(intr, trig);
> +	gic_write_trigger(intr, trig);
> +}
> +
> +static inline void gic_write_dual_edge(unsigned int intr, unsigned int dual)
>  {
>  	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_DUAL) + GIC_INTR_OFS(intr),
>  			1ul << GIC_INTR_BIT(intr),
>  			(unsigned long)dual << GIC_INTR_BIT(intr));
>  }
>  
> -static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
> +static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
> +{
> +	gic_save_shared_dual_edge(intr, dual);
> +	gic_write_dual_edge(intr, dual);
> +}
> +
> +static inline void gic_write_map_to_pin(unsigned int intr, unsigned int pin)
>  {
>  	gic_write32(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_PIN_BASE) +
>  		    GIC_SH_MAP_TO_PIN(intr), GIC_MAP_TO_PIN_MSK | pin);
>  }
>  
> -static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
> +static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
> +{
> +	gic_save_shared_pin(intr, pin);
> +	gic_write_map_to_pin(intr, pin);
> +}
> +
> +static inline void gic_write_map_to_vpe(unsigned int intr, unsigned int vpe)
>  {
>  	gic_write(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_VPE_BASE) +
>  		  GIC_SH_MAP_TO_VPE_REG_OFF(intr, vpe),
>  		  GIC_SH_MAP_TO_VPE_REG_BIT(vpe));
>  }
>  
> +static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
> +{
> +	gic_save_shared_vpe(intr, vpe);
> +	gic_write_map_to_vpe(intr, vpe);
> +}
> +
>  #ifdef CONFIG_CLKSRC_MIPS_GIC
>  cycle_t gic_read_count(void)
>  {
> @@ -537,6 +621,7 @@ static void gic_mask_local_irq(struct irq_data *d)
>  {
>  	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
>  
> +	gic_save_local_rmask(smp_processor_id(), (1 << intr));
>  	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_RMASK), 1 << intr);
>  }
>  
> @@ -544,6 +629,7 @@ static void gic_unmask_local_irq(struct irq_data *d)
>  {
>  	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
>  
> +	gic_save_local_smask(smp_processor_id(), (1 << intr));
>  	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_SMASK), 1 << intr);
>  }
>  
> @@ -561,6 +647,7 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d)
>  
>  	spin_lock_irqsave(&gic_lock, flags);
>  	for (i = 0; i < gic_vpes; i++) {
> +		gic_save_local_rmask(mips_cm_vp_id(i), 1 << intr);
>  		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
>  			  mips_cm_vp_id(i));
>  		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr);
> @@ -576,6 +663,7 @@ static void gic_unmask_local_irq_all_vpes(struct irq_data *d)
>  
>  	spin_lock_irqsave(&gic_lock, flags);
>  	for (i = 0; i < gic_vpes; i++) {
> +		gic_save_local_smask(mips_cm_vp_id(i), 1 << intr);
>  		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
>  			  mips_cm_vp_id(i));
>  		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr);
> @@ -983,6 +1071,85 @@ static struct irq_domain_ops gic_ipi_domain_ops = {
>  	.match = gic_ipi_domain_match,
>  };
>  
> +#ifdef CONTEXT_SAVING
> +static void gic_restore_shared(void)
> +{
> +	unsigned long flags;
> +	int i;
> +
> +	spin_lock_irqsave(&gic_lock, flags);
> +	for (i = 0; i < gic_shared_intrs; i++) {
> +		gic_write_polarity(i, gic_shared_state[i].polarity);
> +		gic_write_trigger(i, gic_shared_state[i].trigger);
> +		gic_write_dual_edge(i, gic_shared_state[i].dual_edge);
> +		gic_write_map_to_vpe(i, gic_shared_state[i].vpe);
> +		gic_write_map_to_pin(i, gic_shared_state[i].pin);
> +
> +		if (gic_shared_state[i].mask)
> +			gic_write_set_mask(i);
> +		else
> +			gic_write_reset_mask(i);
> +	}
> +	spin_unlock_irqrestore(&gic_lock, flags);
> +}
> +
> +static void gic_restore_local(unsigned int vpe)
> +{
> +	int hw, virq, intr, mask;
> +	unsigned long flags;
> +
> +	for (hw = 0; hw < GIC_NUM_LOCAL_INTRS; hw++) {
> +		intr = GIC_LOCAL_TO_HWIRQ(hw);
> +		virq = irq_linear_revmap(gic_irq_domain, intr);
> +		gic_local_irq_domain_map(gic_irq_domain, virq, hw);
> +	}
> +
> +	local_irq_save(flags);
> +	gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), vpe);
> +
> +	/* Enable EIC mode if necessary */
> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_CTL), cpu_has_veic);
> +
> +	/* Restore interrupt masks */
> +	mask = gic_local_state[vpe].mask;
> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), ~mask);
> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), mask);
> +
> +	local_irq_restore(flags);
> +}
> +#endif /* CONTEXT_SAVING */
> +
> +#ifdef CONFIG_MIPS_RPROC

I'm even more confused now. How can you not have both CONFIG_MIPS_RPROC
and CONTEXT_SAVING defined at the same time?

> +/*
> + * The MIPS remote processor driver allows non-Linux firmware to take control
> + * of and execute on one of the systems VPEs. If that VPE is brought back under
> + * Linux, it is necessary to ensure that all GIC interrupts are routed and
> + * masked as Linux expects them, as the firmware can have done anything it
> + * likes with the GIC configuration (hopefully just for that VPEs local
> + * interrupt sources, but allow for shared external interrupts as well).
> + */
> +static int gic_cpu_notify(struct notifier_block *nfb, unsigned long action,
> +			       void *hcpu)
> +{
> +	unsigned int cpu = mips_cm_vp_id((unsigned long)hcpu);
> +
> +	switch (action) {
> +	case CPU_UP_PREPARE:
> +	case CPU_DOWN_FAILED:
> +		gic_restore_shared();
> +		gic_restore_local(cpu);
> +	default:
> +		break;
> +	}
> +
> +	return NOTIFY_OK;
> +}
> +
> +static struct notifier_block gic_cpu_notifier __refdata = {
> +	.notifier_call = gic_cpu_notify
> +};
> +#endif /* CONFIG_MIPS_RPROC */
> +
>  static void __init __gic_init(unsigned long gic_base_addr,
>  			      unsigned long gic_addrspace_size,
>  			      unsigned int cpu_vec, unsigned int irqbase,
> @@ -1082,6 +1249,10 @@ static void __init __gic_init(unsigned long gic_base_addr,
>  	}
>  
>  	gic_basic_init();
> +
> +#ifdef CONFIG_MIPS_RPROC
> +	register_hotcpu_notifier(&gic_cpu_notifier);
> +#endif /* CONFIG_MIPS_RPROC */
>  }
>  
>  void __init gic_init(unsigned long gic_base_addr,
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[toc] | [prev] | [next] | [standalone]

#1475186 — Re: [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC

FromMatt Redfearn <matt.redfearn@imgtec.com>
Date2016-09-02 16:20 +0200
SubjectRe: [PATCH 1/6] irqchip: mips-gic: Add context saving for MIPS_REMOTEPROC
Message-ID<scXu1-wL-7@gated-at.bofh.it>
In reply to#1475035
Hi Marc,

Thanks for the review!


On 02/09/16 11:54, Marc Zyngier wrote:
> Hi Matt,
>
> On 02/09/16 10:59, Matt Redfearn wrote:
>> The MIPS remote processor driver allows non-Linux firmware to take
>> control of and execute on one of the systems VPEs. If that VPE is
>> brought back under Linux, it is necessary to ensure that all GIC
>> interrupts are routed and masked as Linux expects them, as the firmware
>> can have done anything it likes with the GIC configuration (hopefully
>> just for that VPEs local interrupt sources, but allow for shared
>> external interrupts as well).
>>
>> The configuration of shared and local CPU interrupts is maintained and
>> updated every time a change is made. When a CPU is brought online, the
>> saved configuration is restored.
>>
>> These functions will also be useful for restoring GIC context after a
>> suspend to RAM.
>>
>> Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
>> ---
>>
>>   drivers/irqchip/irq-mips-gic.c | 185 +++++++++++++++++++++++++++++++++++++++--
>>   1 file changed, 178 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
>> index 83f498393a7f..5ba1fe1468ce 100644
>> --- a/drivers/irqchip/irq-mips-gic.c
>> +++ b/drivers/irqchip/irq-mips-gic.c
>> @@ -8,6 +8,7 @@
>>    */
>>   #include <linux/bitmap.h>
>>   #include <linux/clocksource.h>
>> +#include <linux/cpu.h>
>>   #include <linux/init.h>
>>   #include <linux/interrupt.h>
>>   #include <linux/irq.h>
>> @@ -56,6 +57,47 @@ static unsigned int timer_cpu_pin;
>>   static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller;
>>   DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS);
>>   
>> +#if defined(CONFIG_MIPS_RPROC)
>> +#define CONTEXT_SAVING
>> +#endif
>> +
>> +#ifdef CONTEXT_SAVING
> This looks really cumbersome. Why not make everything depend on
> CONFIG_MIPS_RPROC instead?

The idea was that the context saving is useful for more than just the 
remote processor driver, for example suspending the state to ram would 
need this. The plan was to have additional things turn on context saving 
here, but with just he one user I agree it looks a bit odd. I could 
perhaps just make everything selected by CONFIG_MIPS_RPROC to start off 
with and then change that in the future when additional users get merged.

>
>> +static struct {
>> +	unsigned mask:		GIC_NUM_LOCAL_INTRS;
> nit/personal taste: can't you make this a normal type instead of a
> bitfield? Considering that GIC_NUM_LOCAL_INTRS is hardcoded to 7, I'd
> rather see a u8... or even an unsigned long if you have to use bitmap
> operations on it.

Sure, no problem.

>
>> +} gic_local_state[NR_CPUS];
> This looks like this really should be a percpu variable.

Yes, it probably can be.

>
>> +
>> +#define gic_save_local_rmask(vpe, i)	(gic_local_state[vpe].mask &= i)
>> +#define gic_save_local_smask(vpe, i)	(gic_local_state[vpe].mask |= i)
>> +
>> +static struct {
>> +	unsigned vpe:		8;
>> +	unsigned pin:		4;
>> +
>> +	unsigned polarity:	1;
>> +	unsigned trigger:	1;
>> +	unsigned dual_edge:	1;
>> +	unsigned mask:		1;
>> +} gic_shared_state[GIC_MAX_INTRS];
>> +
>> +#define gic_save_shared_vpe(i, v)	gic_shared_state[i].vpe = v
>> +#define gic_save_shared_pin(i, p)	gic_shared_state[i].pin = p
>> +#define gic_save_shared_polarity(i, p)	gic_shared_state[i].polarity = p
>> +#define gic_save_shared_trigger(i, t)	gic_shared_state[i].trigger = t
>> +#define gic_save_shared_dual_edge(i, d)	gic_shared_state[i].dual_edge = d
>> +#define gic_save_shared_mask(i, m)	gic_shared_state[i].mask = m
> Why don't you make these static functions? The compiler will inline them
> nicely, and that will save you fixing them (they all miss proper
> bracketing of arguments).

Sure, makes sense.

>
>> +
>> +#else
>> +#define gic_save_local_rmask(vpe, i)
>> +#define gic_save_local_smask(vpe, i)
>> +
>> +#define gic_save_shared_vpe(i, v)
>> +#define gic_save_shared_pin(i, p)
>> +#define gic_save_shared_polarity(i, p)
>> +#define gic_save_shared_trigger(i, t)
>> +#define gic_save_shared_dual_edge(i, d)
>> +#define gic_save_shared_mask(i, m)
> Please make those a "do { } while(0)" construct, so that the trailing
> semi-colon is properly swallowed.

OK.

>
>> +#endif /* CONTEXT_SAVING */
>> +
>>   static void __gic_irq_dispatch(void);
>>   
>>   static inline u32 gic_read32(unsigned int reg)
>> @@ -105,52 +147,94 @@ static inline void gic_update_bits(unsigned int reg, unsigned long mask,
>>   	gic_write(reg, regval);
>>   }
>>   
>> -static inline void gic_reset_mask(unsigned int intr)
>> +static inline void gic_write_reset_mask(unsigned int intr)
>>   {
>>   	gic_write(GIC_REG(SHARED, GIC_SH_RMASK) + GIC_INTR_OFS(intr),
>>   		  1ul << GIC_INTR_BIT(intr));
>>   }
>>   
>> -static inline void gic_set_mask(unsigned int intr)
>> +static inline void gic_reset_mask(unsigned int intr)
>> +{
>> +	gic_save_shared_mask(intr, 0);
>> +	gic_write_reset_mask(intr);
>> +}
>> +
>> +static inline void gic_write_set_mask(unsigned int intr)
>>   {
>>   	gic_write(GIC_REG(SHARED, GIC_SH_SMASK) + GIC_INTR_OFS(intr),
>>   		  1ul << GIC_INTR_BIT(intr));
>>   }
>>   
>> -static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
>> +static inline void gic_set_mask(unsigned int intr)
>> +{
>> +	gic_save_shared_mask(intr, 1);
>> +	gic_write_set_mask(intr);
>> +}
>> +
>> +static inline void gic_write_polarity(unsigned int intr, unsigned int pol)
>>   {
>>   	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_POLARITY) +
>>   			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
>>   			(unsigned long)pol << GIC_INTR_BIT(intr));
>>   }
>>   
>> -static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
>> +static inline void gic_set_polarity(unsigned int intr, unsigned int pol)
>> +{
>> +	gic_save_shared_polarity(intr, pol);
>> +	gic_write_polarity(intr, pol);
>> +}
>> +
>> +static inline void gic_write_trigger(unsigned int intr, unsigned int trig)
>>   {
>>   	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_TRIGGER) +
>>   			GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr),
>>   			(unsigned long)trig << GIC_INTR_BIT(intr));
>>   }
>>   
>> -static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
>> +static inline void gic_set_trigger(unsigned int intr, unsigned int trig)
>> +{
>> +	gic_save_shared_trigger(intr, trig);
>> +	gic_write_trigger(intr, trig);
>> +}
>> +
>> +static inline void gic_write_dual_edge(unsigned int intr, unsigned int dual)
>>   {
>>   	gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_DUAL) + GIC_INTR_OFS(intr),
>>   			1ul << GIC_INTR_BIT(intr),
>>   			(unsigned long)dual << GIC_INTR_BIT(intr));
>>   }
>>   
>> -static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
>> +static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual)
>> +{
>> +	gic_save_shared_dual_edge(intr, dual);
>> +	gic_write_dual_edge(intr, dual);
>> +}
>> +
>> +static inline void gic_write_map_to_pin(unsigned int intr, unsigned int pin)
>>   {
>>   	gic_write32(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_PIN_BASE) +
>>   		    GIC_SH_MAP_TO_PIN(intr), GIC_MAP_TO_PIN_MSK | pin);
>>   }
>>   
>> -static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
>> +static inline void gic_map_to_pin(unsigned int intr, unsigned int pin)
>> +{
>> +	gic_save_shared_pin(intr, pin);
>> +	gic_write_map_to_pin(intr, pin);
>> +}
>> +
>> +static inline void gic_write_map_to_vpe(unsigned int intr, unsigned int vpe)
>>   {
>>   	gic_write(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_VPE_BASE) +
>>   		  GIC_SH_MAP_TO_VPE_REG_OFF(intr, vpe),
>>   		  GIC_SH_MAP_TO_VPE_REG_BIT(vpe));
>>   }
>>   
>> +static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe)
>> +{
>> +	gic_save_shared_vpe(intr, vpe);
>> +	gic_write_map_to_vpe(intr, vpe);
>> +}
>> +
>>   #ifdef CONFIG_CLKSRC_MIPS_GIC
>>   cycle_t gic_read_count(void)
>>   {
>> @@ -537,6 +621,7 @@ static void gic_mask_local_irq(struct irq_data *d)
>>   {
>>   	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
>>   
>> +	gic_save_local_rmask(smp_processor_id(), (1 << intr));
>>   	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_RMASK), 1 << intr);
>>   }
>>   
>> @@ -544,6 +629,7 @@ static void gic_unmask_local_irq(struct irq_data *d)
>>   {
>>   	int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq);
>>   
>> +	gic_save_local_smask(smp_processor_id(), (1 << intr));
>>   	gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_SMASK), 1 << intr);
>>   }
>>   
>> @@ -561,6 +647,7 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d)
>>   
>>   	spin_lock_irqsave(&gic_lock, flags);
>>   	for (i = 0; i < gic_vpes; i++) {
>> +		gic_save_local_rmask(mips_cm_vp_id(i), 1 << intr);
>>   		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
>>   			  mips_cm_vp_id(i));
>>   		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr);
>> @@ -576,6 +663,7 @@ static void gic_unmask_local_irq_all_vpes(struct irq_data *d)
>>   
>>   	spin_lock_irqsave(&gic_lock, flags);
>>   	for (i = 0; i < gic_vpes; i++) {
>> +		gic_save_local_smask(mips_cm_vp_id(i), 1 << intr);
>>   		gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
>>   			  mips_cm_vp_id(i));
>>   		gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr);
>> @@ -983,6 +1071,85 @@ static struct irq_domain_ops gic_ipi_domain_ops = {
>>   	.match = gic_ipi_domain_match,
>>   };
>>   
>> +#ifdef CONTEXT_SAVING
>> +static void gic_restore_shared(void)
>> +{
>> +	unsigned long flags;
>> +	int i;
>> +
>> +	spin_lock_irqsave(&gic_lock, flags);
>> +	for (i = 0; i < gic_shared_intrs; i++) {
>> +		gic_write_polarity(i, gic_shared_state[i].polarity);
>> +		gic_write_trigger(i, gic_shared_state[i].trigger);
>> +		gic_write_dual_edge(i, gic_shared_state[i].dual_edge);
>> +		gic_write_map_to_vpe(i, gic_shared_state[i].vpe);
>> +		gic_write_map_to_pin(i, gic_shared_state[i].pin);
>> +
>> +		if (gic_shared_state[i].mask)
>> +			gic_write_set_mask(i);
>> +		else
>> +			gic_write_reset_mask(i);
>> +	}
>> +	spin_unlock_irqrestore(&gic_lock, flags);
>> +}
>> +
>> +static void gic_restore_local(unsigned int vpe)
>> +{
>> +	int hw, virq, intr, mask;
>> +	unsigned long flags;
>> +
>> +	for (hw = 0; hw < GIC_NUM_LOCAL_INTRS; hw++) {
>> +		intr = GIC_LOCAL_TO_HWIRQ(hw);
>> +		virq = irq_linear_revmap(gic_irq_domain, intr);
>> +		gic_local_irq_domain_map(gic_irq_domain, virq, hw);
>> +	}
>> +
>> +	local_irq_save(flags);
>> +	gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), vpe);
>> +
>> +	/* Enable EIC mode if necessary */
>> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_CTL), cpu_has_veic);
>> +
>> +	/* Restore interrupt masks */
>> +	mask = gic_local_state[vpe].mask;
>> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), ~mask);
>> +	gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), mask);
>> +
>> +	local_irq_restore(flags);
>> +}
>> +#endif /* CONTEXT_SAVING */
>> +
>> +#ifdef CONFIG_MIPS_RPROC
> I'm even more confused now. How can you not have both CONFIG_MIPS_RPROC
> and CONTEXT_SAVING defined at the same time?

This is fallout of there potentially being multiple users of the context 
saving, where the context saving could be turned on by something that's 
not MIPS remote processor, so it wouldn't need the CPU notifier below.

Thanks,
Matt

>
>> +/*
>> + * The MIPS remote processor driver allows non-Linux firmware to take control
>> + * of and execute on one of the systems VPEs. If that VPE is brought back under
>> + * Linux, it is necessary to ensure that all GIC interrupts are routed and
>> + * masked as Linux expects them, as the firmware can have done anything it
>> + * likes with the GIC configuration (hopefully just for that VPEs local
>> + * interrupt sources, but allow for shared external interrupts as well).
>> + */
>> +static int gic_cpu_notify(struct notifier_block *nfb, unsigned long action,
>> +			       void *hcpu)
>> +{
>> +	unsigned int cpu = mips_cm_vp_id((unsigned long)hcpu);
>> +
>> +	switch (action) {
>> +	case CPU_UP_PREPARE:
>> +	case CPU_DOWN_FAILED:
>> +		gic_restore_shared();
>> +		gic_restore_local(cpu);
>> +	default:
>> +		break;
>> +	}
>> +
>> +	return NOTIFY_OK;
>> +}
>> +
>> +static struct notifier_block gic_cpu_notifier __refdata = {
>> +	.notifier_call = gic_cpu_notify
>> +};
>> +#endif /* CONFIG_MIPS_RPROC */
>> +
>>   static void __init __gic_init(unsigned long gic_base_addr,
>>   			      unsigned long gic_addrspace_size,
>>   			      unsigned int cpu_vec, unsigned int irqbase,
>> @@ -1082,6 +1249,10 @@ static void __init __gic_init(unsigned long gic_base_addr,
>>   	}
>>   
>>   	gic_basic_init();
>> +
>> +#ifdef CONFIG_MIPS_RPROC
>> +	register_hotcpu_notifier(&gic_cpu_notifier);
>> +#endif /* CONFIG_MIPS_RPROC */
>>   }
>>   
>>   void __init gic_init(unsigned long gic_base_addr,
>>
> Thanks,
>
> 	M.

[toc] | [prev] | [next] | [standalone]

#1475001 — [PATCH 4/6] MIPS: CPS: Add VP(E) stealing

FromMatt Redfearn <matt.redfearn@imgtec.com>
Date2016-09-02 12:10 +0200
Subject[PATCH 4/6] MIPS: CPS: Add VP(E) stealing
Message-ID<scTA7-6xn-69@gated-at.bofh.it>
In reply to#1474995
From: Lisa Parratt <Lisa.Parratt@imgtec.com>

VP(E) stealing provides a mechanism for removing an offline Virtual
Processor from the Linux kernel such that it is available to run bare
metal code.
Once the CPU has been offlined from Linux, the CPU can be given a task
to run via mips_cps_steal_cpu_and_execute(). The CPU is removed from the
cpu_present mask and is set up to execute from address entry_fn. Stack
space is assigned via the tsk task_struct so that C initialisation code
may be used.
To return the CPU back to Linux control, mips_cps_halt_and_return_cpu
will arrange to halt the CPU and return it to the cpu_present mask. It
is then available to be brought online again via CPU hotplug.

This mechanism is used by the MIPS remote processor driver to allow
CPUs within the system to execute bare metal code, not under control of
the kernel.

Signed-off-by: Lisa Parratt <Lisa.Parratt@imgtec.com>
Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
---

 arch/mips/Kconfig               |   7 ++
 arch/mips/include/asm/smp-cps.h |   8 ++
 arch/mips/include/asm/smp.h     |   1 +
 arch/mips/kernel/smp-cps.c      | 162 ++++++++++++++++++++++++++++++++++++++--
 arch/mips/kernel/smp.c          |  12 +++
 5 files changed, 183 insertions(+), 7 deletions(-)

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 26388562e300..2094cbcea0d4 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -2341,6 +2341,13 @@ config MIPS_CPS
 	  no external assistance. It is safe to enable this when hardware
 	  support is unavailable.
 
+config MIPS_STEAL
+	bool "VPE stealing"
+	depends on HOTPLUG_CPU && MIPS_CPS
+	help
+	  Select this is you wish to be able to run bare metal code on offline
+	  VPEs.
+
 config MIPS_CPS_PM
 	depends on MIPS_CPS
 	select MIPS_CPC
diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h
index 2ae1f61a4a95..4f6cd5b14185 100644
--- a/arch/mips/include/asm/smp-cps.h
+++ b/arch/mips/include/asm/smp-cps.h
@@ -34,6 +34,14 @@ extern void mips_cps_boot_vpes(struct core_boot_config *cfg, unsigned vpe);
 extern void mips_cps_pm_save(void);
 extern void mips_cps_pm_restore(void);
 
+#ifdef CONFIG_MIPS_STEAL
+
+extern int mips_cps_steal_cpu_and_execute(unsigned int cpu, void *entry_fn,
+					  struct task_struct *tsk);
+extern int mips_cps_halt_and_return_cpu(unsigned int cpu);
+
+#endif /* CONFIG_MIPS_STEAL */
+
 #ifdef CONFIG_MIPS_CPS
 
 extern bool mips_cps_smp_in_use(void);
diff --git a/arch/mips/include/asm/smp.h b/arch/mips/include/asm/smp.h
index 060f23ff1817..3c62a1958af5 100644
--- a/arch/mips/include/asm/smp.h
+++ b/arch/mips/include/asm/smp.h
@@ -117,4 +117,5 @@ static inline void arch_send_call_function_ipi_mask(const struct cpumask *mask)
 extern void (*dump_ipi_function_ptr)(void *);
 void dump_send_ipi(void (*dump_ipi_callback)(void *));
 #endif
+
 #endif /* __ASM_SMP_H */
diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c
index e9d9fc6c754c..bcb9b62816b1 100644
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -8,6 +8,7 @@
  * option) any later version.
  */
 
+#include <linux/cpu.h>
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/irqchip/mips-gic.h>
@@ -39,6 +40,31 @@ static int __init setup_nothreads(char *s)
 }
 early_param("nothreads", setup_nothreads);
 
+#ifdef CONFIG_MIPS_STEAL
+struct cpumask cpu_stolen_mask;
+
+static inline bool cpu_stolen(int cpu)
+{
+	return cpumask_test_cpu(cpu, &cpu_stolen_mask);
+}
+
+static inline void set_cpu_stolen(int cpu, bool state)
+{
+	if (state)
+		cpumask_set_cpu(cpu, &cpu_stolen_mask);
+	else
+		cpumask_clear_cpu(cpu, &cpu_stolen_mask);
+}
+#else
+static inline bool cpu_stolen(int cpu)
+{
+	return false;
+}
+
+static inline void set_cpu_stolen(int cpu, bool state) { }
+
+#endif /* CONFIG_MIPS_STEAL */
+
 static unsigned core_vpe_count(unsigned core)
 {
 	unsigned cfg;
@@ -109,6 +135,10 @@ static void __init cps_smp_setup(void)
 		write_gcr_bev_base(core_entry);
 	}
 
+#ifdef CONFIG_MIPS_STEAL
+	cpumask_clear(&cpu_stolen_mask);
+#endif /* CONFIG_MIPS_STEAL */
+
 #ifdef CONFIG_MIPS_MT_FPAFF
 	/* If we have an FPU, enroll ourselves in the FPU-full mask */
 	if (cpu_has_fpu)
@@ -287,7 +317,7 @@ static void remote_vpe_boot(void *dummy)
 	mips_cps_boot_vpes(core_cfg, cpu_vpe_id(&current_cpu_data));
 }
 
-static void cps_boot_secondary(int cpu, struct task_struct *idle)
+static void cps_start_secondary(int cpu, void *entry_fn, struct task_struct *tsk)
 {
 	unsigned core = cpu_data[cpu].core;
 	unsigned vpe_id = cpu_vpe_id(&cpu_data[cpu]);
@@ -297,9 +327,9 @@ static void cps_boot_secondary(int cpu, struct task_struct *idle)
 	unsigned int remote;
 	int err;
 
-	vpe_cfg->pc = (unsigned long)&smp_bootstrap;
-	vpe_cfg->sp = __KSTK_TOS(idle);
-	vpe_cfg->gp = (unsigned long)task_thread_info(idle);
+	vpe_cfg->pc = (unsigned long)entry_fn;
+	vpe_cfg->sp = __KSTK_TOS(tsk);
+	vpe_cfg->gp = (unsigned long)task_thread_info(tsk);
 
 	atomic_or(1 << cpu_vpe_id(&cpu_data[cpu]), &core_cfg->vpe_mask);
 
@@ -343,6 +373,11 @@ out:
 	preempt_enable();
 }
 
+static void cps_boot_secondary(int cpu, struct task_struct *idle)
+{
+	cps_start_secondary(cpu, &smp_bootstrap, idle);
+}
+
 static void cps_init_secondary(void)
 {
 	/* Disable MT - we only want to run 1 TC per VPE */
@@ -394,6 +429,28 @@ static int cps_cpu_disable(void)
 	if (!cps_pm_support_state(CPS_PM_POWER_GATED))
 		return -EINVAL;
 
+#ifdef CONFIG_MIPS_STEAL
+	/*
+	 * With the MT ASE only VPEs in the same core may read / write the
+	 * control registers of other VPEs. Therefore to maintain control of
+	 * any stolen VPEs at least one sibling VPE must be kept online.
+	 */
+	if (cpu_has_mipsmt) {
+		int stolen, siblings = 0;
+
+		for_each_cpu((stolen), &cpu_stolen_mask)
+			if (cpu_data[stolen].core == cpu_data[cpu].core)
+				siblings++;
+
+		if (siblings == 1)
+			/*
+			 * When a VPE has been stolen, keep at least one of it's
+			 * siblings around in order to control it.
+			 */
+			return -EBUSY;
+	}
+#endif /* CONFIG_MIPS_STEAL */
+
 	core_cfg = &mips_cps_core_bootcfg[current_cpu_data.core];
 	atomic_sub(1 << cpu_vpe_id(&current_cpu_data), &core_cfg->vpe_mask);
 	smp_mb__after_atomic();
@@ -426,7 +483,7 @@ void play_dead(void)
 		core = cpu_data[cpu].core;
 
 		/* Look for another online VPE within the core */
-		for_each_online_cpu(cpu_death_sibling) {
+		for_each_possible_cpu(cpu_death_sibling) {
 			if (cpu_data[cpu_death_sibling].core != core)
 				continue;
 
@@ -434,8 +491,11 @@ void play_dead(void)
 			 * There is an online VPE within the core. Just halt
 			 * this TC and leave the core alone.
 			 */
-			cpu_death = CPU_DEATH_HALT;
-			break;
+			if (cpu_online(cpu_death_sibling) ||
+			    cpu_stolen(cpu_death_sibling))
+				cpu_death = CPU_DEATH_HALT;
+			if (cpu_online(cpu_death_sibling))
+				break;
 		}
 	}
 
@@ -466,6 +526,94 @@ void play_dead(void)
 	panic("Failed to offline CPU %u", cpu);
 }
 
+#ifdef CONFIG_MIPS_STEAL
+
+/* Find an online sibling CPU (another VPE in the same core) */
+static inline int mips_cps_get_online_sibling(unsigned int cpu)
+{
+	int sibling;
+
+	for_each_online_cpu(sibling)
+		if (cpu_data[sibling].core == cpu_data[cpu].core)
+			return sibling;
+
+	return -1;
+}
+
+int mips_cps_steal_cpu_and_execute(unsigned int cpu, void *entry_fn,
+				   struct task_struct *tsk)
+{
+	int err = -EINVAL;
+
+	preempt_disable();
+
+	if (!cpu_present(cpu) || cpu_online(cpu) || cpu_stolen(cpu))
+		goto out;
+
+	if (cpu_has_mipsmt && (mips_cps_get_online_sibling(cpu) < 0))
+		pr_warn("CPU%d has no online siblings to control it\n", cpu);
+	else {
+		set_cpu_present(cpu, false);
+		set_cpu_stolen(cpu, true);
+
+		cps_start_secondary(cpu, entry_fn, tsk);
+		err = 0;
+	}
+out:
+	preempt_enable();
+	return err;
+}
+
+static void mips_cps_halt_sibling(void *ptr_cpu)
+{
+	unsigned int cpu = (unsigned long)ptr_cpu;
+	unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]);
+	unsigned long flags;
+	int vpflags;
+
+	local_irq_save(flags);
+	vpflags = dvpe();
+	settc(vpe_id);
+	write_tc_c0_tchalt(TCHALT_H);
+	evpe(vpflags);
+	local_irq_restore(flags);
+}
+
+int mips_cps_halt_and_return_cpu(unsigned int cpu)
+{
+	unsigned int core = cpu_data[cpu].core;
+	unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]);
+
+	if (!cpu_stolen(cpu))
+		return -EINVAL;
+
+	if (cpu_has_mipsmt && (core == cpu_data[smp_processor_id()].core))
+		mips_cps_halt_sibling((void *)(unsigned long)cpu);
+	else if (cpu_has_mipsmt) {
+		int sibling = mips_cps_get_online_sibling(cpu);
+
+		if (sibling < 0) {
+			pr_warn("CPU%d has no online siblings\n", cpu);
+			return -EINVAL;
+		}
+
+		if (smp_call_function_single(sibling, mips_cps_halt_sibling,
+						(void *)(unsigned long)cpu, 1))
+			panic("Failed to call sibling CPU\n");
+
+	} else if (cpu_has_vp) {
+		mips_cm_lock_other(core, vpe_id);
+		write_cpc_co_vp_stop(1 << vpe_id);
+		mips_cm_unlock_other();
+	}
+
+	set_cpu_stolen(cpu, false);
+	set_cpu_present(cpu, true);
+	return 0;
+}
+
+#endif /* CONFIG_MIPS_STEAL */
+
 static void wait_for_sibling_halt(void *ptr_cpu)
 {
 	unsigned cpu = (unsigned long)ptr_cpu;
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index afa06c2bb019..f3d01f556fe2 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -233,6 +233,18 @@ static void smp_ipi_init_one(unsigned int virq,
 				    struct irqaction *action)
 {
 	int ret;
+#ifdef CONFIG_MIPS_STEAL
+	struct irq_data *data;
+	/*
+	 * A bit of a hack to ensure that the ipi_offset is 0.
+	 * This is to deal with removing / reallocating IPIs
+	 * to subsets of the possible CPUs, where the IPI IRQ domain
+	 * will set ipi_offset to the first cpu in the cpumask when the
+	 * IPI is reallocated.
+	 */
+	data = irq_get_irq_data(virq);
+	data->common->ipi_offset = 0;
+#endif /* CONFIG_MIPS_STEAL */
 
 	irq_set_handler(virq, handle_percpu_irq);
 	ret = setup_irq(virq, action);
-- 
2.7.4

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web