Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1293126 > unrolled thread

[PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs Daniel Thompson <daniel.thompson@linaro.org> - 2015-12-16 18:10 +0100
    Re: [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI  IDs Marc Zyngier <marc.zyngier@arm.com> - 2015-12-16 18:50 +0100
      Re: [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs Daniel Thompson <daniel.thompson@linaro.org> - 2015-12-17 20:30 +0100
        Re: [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI  IDs Marc Zyngier <marc.zyngier@arm.com> - 2015-12-18 08:40 +0100
          Re: [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs Daniel Thompson <daniel.thompson@linaro.org> - 2015-12-18 12:30 +0100

#1293126 — [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs

It is possible for the secure world to reserve certain SGI IDs for itself.
Currently we have limited visibility of which IDs are safe to use for IPIs.

Modify the GIC initialization code to actively search for reserved SGI IDs
and report if any are found. Warn even more loudly if the reserved SGIs
overlap with the normal IPI range.

When run on an Inforce IFC6410 (Snapdragon 600) this code produces the
following messages:
~~~ cut here ~~~
CPU0: Detected reserved SGI IDs: 14-15
CPU1: Detected reserved SGI IDs: 15
CPU2: Detected reserved SGI IDs: 15
CPU3: Detected reserved SGI IDs: 15
~~~ cut here ~~~

Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
---
 drivers/irqchip/irq-gic.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index abf2ffaed392..541622da7049 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -490,6 +490,7 @@ static void gic_cpu_init(struct gic_chip_data *gic)
 	void __iomem *base = gic_data_cpu_base(gic);
 	unsigned int cpu_mask, cpu = smp_processor_id();
 	int i;
+	DECLARE_BITMAP(sgi_mask, 16);
 
 	/*
 	 * Setting up the CPU map is only relevant for the primary GIC
@@ -511,6 +512,58 @@ static void gic_cpu_init(struct gic_chip_data *gic)
 		for (i = 0; i < NR_GIC_CPU_IF; i++)
 			if (i != cpu)
 				gic_cpu_map[i] &= ~cpu_mask;
+
+		/*
+		 * Fiddle with the SGI set/clear registers to try identify
+		 * any IPIs that are reserved for secure world.
+		 */
+		bitmap_fill(sgi_mask, 16);
+
+		for (i = 0; i < 16; i++) {
+			void __iomem *set_reg =
+			    dist_base + GIC_DIST_SGI_PENDING_SET + (i & ~3);
+			void __iomem *clear_reg =
+			    dist_base + GIC_DIST_SGI_PENDING_CLEAR + (i & ~3);
+			unsigned long mask = cpu_mask << (8*(i%4));
+			unsigned long flags, pending, after_clear, after_set;
+
+			local_irq_save(flags);
+
+			/* record original value */
+			pending = readl_relaxed(set_reg);
+
+			/* clear, test, set, and test again */
+			writel_relaxed(mask, clear_reg);
+			after_clear = readl_relaxed(set_reg);
+			writel_relaxed(mask, set_reg);
+			after_set = readl_relaxed(set_reg);
+
+			/* restore original value */
+			writel_relaxed(mask & ~pending, clear_reg);
+
+			local_irq_restore(flags);
+
+			if (mask & ~after_clear && mask & after_set)
+				clear_bit(i, sgi_mask);
+		}
+
+		/*
+		 * Show the SGI mask if it is "interesting". Here interesting
+		 * means that the set/clear register is implemented
+		 * (mask is not full) and it tells us that the secure world
+		 * has reserved some SGIs (mask is not empty).
+		 */
+		if (!bitmap_full(sgi_mask, 16) && !bitmap_empty(sgi_mask, 16))
+			pr_info("CPU%d: Detected reserved SGI IDs: %*pbl\n",
+				cpu, 16, sgi_mask);
+
+		/*
+		 * Yell if the reserved IDs make the system unviable.
+		 */
+		if (!bitmap_full(sgi_mask, 16) &&
+		    find_first_bit(sgi_mask, 16) < NR_IPI)
+			pr_crit("CPU%d: Not enough SGI IDs; expect failure\n",
+				cpu);
 	}
 
 	gic_cpu_config(dist_base, NULL);
-- 
2.5.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1293152 — Re: [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs

Hi Daniel,

On 16/12/15 17:08, Daniel Thompson wrote:
> It is possible for the secure world to reserve certain SGI IDs for itself.
> Currently we have limited visibility of which IDs are safe to use for IPIs.
> 
> Modify the GIC initialization code to actively search for reserved SGI IDs
> and report if any are found. Warn even more loudly if the reserved SGIs
> overlap with the normal IPI range.
> 
> When run on an Inforce IFC6410 (Snapdragon 600) this code produces the
> following messages:
> ~~~ cut here ~~~
> CPU0: Detected reserved SGI IDs: 14-15
> CPU1: Detected reserved SGI IDs: 15
> CPU2: Detected reserved SGI IDs: 15
> CPU3: Detected reserved SGI IDs: 15
> ~~~ cut here ~~~
> 
> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
> ---
>  drivers/irqchip/irq-gic.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 53 insertions(+)
> 
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index abf2ffaed392..541622da7049 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -490,6 +490,7 @@ static void gic_cpu_init(struct gic_chip_data *gic)
>  	void __iomem *base = gic_data_cpu_base(gic);
>  	unsigned int cpu_mask, cpu = smp_processor_id();
>  	int i;
> +	DECLARE_BITMAP(sgi_mask, 16);
>  
>  	/*
>  	 * Setting up the CPU map is only relevant for the primary GIC
> @@ -511,6 +512,58 @@ static void gic_cpu_init(struct gic_chip_data *gic)
>  		for (i = 0; i < NR_GIC_CPU_IF; i++)
>  			if (i != cpu)
>  				gic_cpu_map[i] &= ~cpu_mask;
> +
> +		/*
> +		 * Fiddle with the SGI set/clear registers to try identify
> +		 * any IPIs that are reserved for secure world.
> +		 */
> +		bitmap_fill(sgi_mask, 16);
> +
> +		for (i = 0; i < 16; i++) {
> +			void __iomem *set_reg =
> +			    dist_base + GIC_DIST_SGI_PENDING_SET + (i & ~3);
> +			void __iomem *clear_reg =
> +			    dist_base + GIC_DIST_SGI_PENDING_CLEAR + (i & ~3);
> +			unsigned long mask = cpu_mask << (8*(i%4));
> +			unsigned long flags, pending, after_clear, after_set;

Please make these u32, as unsigned long is 64bit on arm64. Another thing
to note is that GICD_CPEND{S,C}SGIRn are byte accessible, so you can
save yourself some this hassle shifting things around and just write a
single byte. You're already writing 16 times anyway...

Another thing to consider is that these locations are only defined on
GICv2 and not GICv1, so this patch is likely to cause trouble on older HW.

> +
> +			local_irq_save(flags);

Why do you need to do this? The CPU interface is not enabled yet, so I
can't see how you could get an interrupt on this CPU.

> +
> +			/* record original value */
> +			pending = readl_relaxed(set_reg);
> +
> +			/* clear, test, set, and test again */
> +			writel_relaxed(mask, clear_reg);
> +			after_clear = readl_relaxed(set_reg);
> +			writel_relaxed(mask, set_reg);
> +			after_set = readl_relaxed(set_reg);

It should be enough to write to the SET register, and read back, as the
bit is RAZ/WI when the interrupt is Group-0.

> +
> +			/* restore original value */
> +			writel_relaxed(mask & ~pending, clear_reg);
> +
> +			local_irq_restore(flags);
> +
> +			if (mask & ~after_clear && mask & after_set)
> +				clear_bit(i, sgi_mask);
> +		}
> +
> +		/*
> +		 * Show the SGI mask if it is "interesting". Here interesting
> +		 * means that the set/clear register is implemented
> +		 * (mask is not full) and it tells us that the secure world
> +		 * has reserved some SGIs (mask is not empty).
> +		 */
> +		if (!bitmap_full(sgi_mask, 16) && !bitmap_empty(sgi_mask, 16))
> +			pr_info("CPU%d: Detected reserved SGI IDs: %*pbl\n",
> +				cpu, 16, sgi_mask);
> +
> +		/*
> +		 * Yell if the reserved IDs make the system unviable.
> +		 */
> +		if (!bitmap_full(sgi_mask, 16) &&
> +		    find_first_bit(sgi_mask, 16) < NR_IPI)
> +			pr_crit("CPU%d: Not enough SGI IDs; expect failure\n",
> +				cpu);
>  	}
>  
>  	gic_cpu_config(dist_base, NULL);
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1294167

On Wed, Dec 16, 2015 at 05:47:09PM +0000, Marc Zyngier wrote:
> Hi Daniel,

Hi Marc

Thanks for the review.


> On 16/12/15 17:08, Daniel Thompson wrote:
> > It is possible for the secure world to reserve certain SGI IDs for itself.
> > Currently we have limited visibility of which IDs are safe to use for IPIs.
> > 
> > Modify the GIC initialization code to actively search for reserved SGI IDs
> > and report if any are found. Warn even more loudly if the reserved SGIs
> > overlap with the normal IPI range.
> > 
> > When run on an Inforce IFC6410 (Snapdragon 600) this code produces the
> > following messages:
> > ~~~ cut here ~~~
> > CPU0: Detected reserved SGI IDs: 14-15
> > CPU1: Detected reserved SGI IDs: 15
> > CPU2: Detected reserved SGI IDs: 15
> > CPU3: Detected reserved SGI IDs: 15
> > ~~~ cut here ~~~
> > 
> > Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>

BTW you *didn't* say "this code is pointless and I hate it"...

Does that mean I should be looking at adding similar code for GICv3+? I 
wanted to guage reactions to this sort of diagnostics before getting
carried away!


> > +
> > +		/*
> > +		 * Fiddle with the SGI set/clear registers to try identify
> > +		 * any IPIs that are reserved for secure world.
> > +		 */
> > +		bitmap_fill(sgi_mask, 16);
> > +
> > +		for (i = 0; i < 16; i++) {
> > +			void __iomem *set_reg =
> > +			    dist_base + GIC_DIST_SGI_PENDING_SET + (i & ~3);
> > +			void __iomem *clear_reg =
> > +			    dist_base + GIC_DIST_SGI_PENDING_CLEAR + (i & ~3);
> > +			unsigned long mask = cpu_mask << (8*(i%4));
> > +			unsigned long flags, pending, after_clear, after_set;
> 
> Please make these u32, as unsigned long is 64bit on arm64. Another thing
> to note is that GICD_CPEND{S,C}SGIRn are byte accessible, so you can
> save yourself some this hassle shifting things around and just write a
> single byte. You're already writing 16 times anyway...

Will do both.


> Another thing to consider is that these locations are only defined on
> GICv2 and not GICv1, so this patch is likely to cause trouble on older HW.

As presented the code relies on the RAZ/WI property of reserved
registers to avoid issues on GICv1; it does not report anything if there
appear to be know working SGIs on the assumption we are actually running
on a GICv1.

You'd prefer an explicit version check?


> > +
> > +			local_irq_save(flags);
> 
> Why do you need to do this? The CPU interface is not enabled yet, so I
> can't see how you could get an interrupt on this CPU.

Agreed. Can get rid of these.


> > +
> > +			/* record original value */
> > +			pending = readl_relaxed(set_reg);
> > +
> > +			/* clear, test, set, and test again */
> > +			writel_relaxed(mask, clear_reg);
> > +			after_clear = readl_relaxed(set_reg);
> > +			writel_relaxed(mask, set_reg);
> > +			after_set = readl_relaxed(set_reg);
> 
> It should be enough to write to the SET register, and read back, as the
> bit is RAZ/WI when the interrupt is Group-0.

Good point. Will simplify.


Daniel.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1294469 — Re: [PATCH 2/2] irqchip/gic: Identify and report any reserved SGI IDs

On Thu, 17 Dec 2015 19:26:10 +0000
Daniel Thompson <daniel.thompson@linaro.org> wrote:

> On Wed, Dec 16, 2015 at 05:47:09PM +0000, Marc Zyngier wrote:
> > Hi Daniel,
> 
> Hi Marc
> 
> Thanks for the review.
> 
> 
> > On 16/12/15 17:08, Daniel Thompson wrote:
> > > It is possible for the secure world to reserve certain SGI IDs for itself.
> > > Currently we have limited visibility of which IDs are safe to use for IPIs.
> > > 
> > > Modify the GIC initialization code to actively search for reserved SGI IDs
> > > and report if any are found. Warn even more loudly if the reserved SGIs
> > > overlap with the normal IPI range.
> > > 
> > > When run on an Inforce IFC6410 (Snapdragon 600) this code produces the
> > > following messages:
> > > ~~~ cut here ~~~
> > > CPU0: Detected reserved SGI IDs: 14-15
> > > CPU1: Detected reserved SGI IDs: 15
> > > CPU2: Detected reserved SGI IDs: 15
> > > CPU3: Detected reserved SGI IDs: 15
> > > ~~~ cut here ~~~
> > > 
> > > Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
> 
> BTW you *didn't* say "this code is pointless and I hate it"...

Well, this was obviously used to detect another issue (patch #1), so I
can't see any harm in trying to sanitize things, as long as it doesn't
break anything else. I imagine this patch will also trigger on my
sunxi platforms, which use a SGI to implement PSCI in secure mode.

> Does that mean I should be looking at adding similar code for GICv3+? I 
> wanted to guage reactions to this sort of diagnostics before getting
> carried away!

Could be useful as well.

> 
> 
> > > +
> > > +		/*
> > > +		 * Fiddle with the SGI set/clear registers to try identify
> > > +		 * any IPIs that are reserved for secure world.
> > > +		 */
> > > +		bitmap_fill(sgi_mask, 16);
> > > +
> > > +		for (i = 0; i < 16; i++) {
> > > +			void __iomem *set_reg =
> > > +			    dist_base + GIC_DIST_SGI_PENDING_SET + (i & ~3);
> > > +			void __iomem *clear_reg =
> > > +			    dist_base + GIC_DIST_SGI_PENDING_CLEAR + (i & ~3);
> > > +			unsigned long mask = cpu_mask << (8*(i%4));
> > > +			unsigned long flags, pending, after_clear, after_set;
> > 
> > Please make these u32, as unsigned long is 64bit on arm64. Another thing
> > to note is that GICD_CPEND{S,C}SGIRn are byte accessible, so you can
> > save yourself some this hassle shifting things around and just write a
> > single byte. You're already writing 16 times anyway...
> 
> Will do both.
> 
> 
> > Another thing to consider is that these locations are only defined on
> > GICv2 and not GICv1, so this patch is likely to cause trouble on older HW.
> 
> As presented the code relies on the RAZ/WI property of reserved
> registers to avoid issues on GICv1; it does not report anything if there
> appear to be know working SGIs on the assumption we are actually running
> on a GICv1.
> 
> You'd prefer an explicit version check?

I'd rather be cautious and check for the architecture version,
specially if you settle for the byte access mentioned above (a GICv1
may not support byte access and explode unexpectedly). ICPIDR2.ArchRev
should be the right thing to check.

> 
> 
> > > +
> > > +			local_irq_save(flags);
> > 
> > Why do you need to do this? The CPU interface is not enabled yet, so I
> > can't see how you could get an interrupt on this CPU.
> 
> Agreed. Can get rid of these.
> 
> 
> > > +
> > > +			/* record original value */
> > > +			pending = readl_relaxed(set_reg);
> > > +
> > > +			/* clear, test, set, and test again */
> > > +			writel_relaxed(mask, clear_reg);
> > > +			after_clear = readl_relaxed(set_reg);
> > > +			writel_relaxed(mask, set_reg);
> > > +			after_set = readl_relaxed(set_reg);
> > 
> > It should be enough to write to the SET register, and read back, as the
> > bit is RAZ/WI when the interrupt is Group-0.
> 
> Good point. Will simplify.

I'd also suggest moving the whole thing to a separate function that'd
get called from gic_cpu_init().

Thanks,

	M.
-- 
Without deviation from the norm, progress is not possible.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1294705

On 18/12/15 07:39, Marc Zyngier wrote:
>>> On 16/12/15 17:08, Daniel Thompson wrote:
>>>> It is possible for the secure world to reserve certain SGI IDs for itself.
>>>> Currently we have limited visibility of which IDs are safe to use for IPIs.
>>>>
>>>> Modify the GIC initialization code to actively search for reserved SGI IDs
>>>> and report if any are found. Warn even more loudly if the reserved SGIs
>>>> overlap with the normal IPI range.
>>>>
>>>> When run on an Inforce IFC6410 (Snapdragon 600) this code produces the
>>>> following messages:
>>>> ~~~ cut here ~~~
>>>> CPU0: Detected reserved SGI IDs: 14-15
>>>> CPU1: Detected reserved SGI IDs: 15
>>>> CPU2: Detected reserved SGI IDs: 15
>>>> CPU3: Detected reserved SGI IDs: 15
>>>> ~~~ cut here ~~~
>>>>
>>>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
...

>>> Another thing to consider is that these locations are only defined on
>>> GICv2 and not GICv1, so this patch is likely to cause trouble on older HW.
>>
>> As presented the code relies on the RAZ/WI property of reserved
>> registers to avoid issues on GICv1; it does not report anything if there
>> appear to be know working SGIs on the assumption we are actually running
>> on a GICv1.
>>
>> You'd prefer an explicit version check?
>
> I'd rather be cautious and check for the architecture version,
> specially if you settle for the byte access mentioned above (a GICv1
> may not support byte access and explode unexpectedly). ICPIDR2.ArchRev
> should be the right thing to check.

Will do.


>>>> +
>>>> +			/* record original value */
>>>> +			pending = readl_relaxed(set_reg);
>>>> +
>>>> +			/* clear, test, set, and test again */
>>>> +			writel_relaxed(mask, clear_reg);
>>>> +			after_clear = readl_relaxed(set_reg);
>>>> +			writel_relaxed(mask, set_reg);
>>>> +			after_set = readl_relaxed(set_reg);
>>>
>>> It should be enough to write to the SET register, and read back, as the
>>> bit is RAZ/WI when the interrupt is Group-0.
>>
>> Good point. Will simplify.
>
> I'd also suggest moving the whole thing to a separate function that'd
> get called from gic_cpu_init().

Will do.

I think I will also upgrade the pr_crit() to a WARN_ON() and hard code 
the check to 8 rather than NR_IPI.

Currently NR_IPI is small on arm64 so it would be good for us to shout 
loudly about latent firmware/secure-zone misconfiguration.


Daniel.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web