Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1675963 > unrolled thread

[PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr() Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-06-27 18:20 +0200
    Re: [PATCH 2/3] iommu/iova: don't disable preempt around  this_cpu_ptr() Joerg Roedel <joro@8bytes.org> - 2017-06-28 11:30 +0200
      Re: [PATCH 2/3] iommu/iova: don't disable preempt around  this_cpu_ptr() Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-06-28 11:40 +0200
        Re: [PATCH 2/3] iommu/iova: don't disable preempt around  this_cpu_ptr() Joerg Roedel <joro@8bytes.org> - 2017-06-28 12:30 +0200

#1675963 — [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()

Commit 583248e6620a ("iommu/iova: Disable preemption around use of
this_cpu_ptr()") disables preemption while accessing a per-CPU variable.
This does keep lockdep quiet. However I don't see the point why it is
bad if we get migrated after its access to another CPU.
__iova_rcache_insert() and __iova_rcache_get() immediately locks the
variable after obtaining it - before accessing its members.
_If_ we get migrated away after retrieving the address of cpu_rcache
before taking the lock then the *other* task on the same CPU will
retrieve the same address of cpu_rcache and will spin on the lock.

alloc_iova_fast() disables preemption while invoking
free_cpu_cached_iovas() on each CPU. The function itself uses
per_cpu_ptr() which does not trigger a warning (like this_cpu_ptr()
does). It _could_ make sense to use get_online_cpus() instead but the we
have a hotplug notifier for CPU down (and none for up) so we are good.

Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/iommu/iova.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c
index 5c88ba70e4e0..f0ff0aa04081 100644
--- a/drivers/iommu/iova.c
+++ b/drivers/iommu/iova.c
@@ -22,6 +22,7 @@
 #include <linux/slab.h>
 #include <linux/smp.h>
 #include <linux/bitops.h>
+#include <linux/cpu.h>
 
 static bool iova_rcache_insert(struct iova_domain *iovad,
 			       unsigned long pfn,
@@ -398,10 +399,8 @@ alloc_iova_fast(struct iova_domain *iovad, unsigned long size,
 
 		/* Try replenishing IOVAs by flushing rcache. */
 		flushed_rcache = true;
-		preempt_disable();
 		for_each_online_cpu(cpu)
 			free_cpu_cached_iovas(cpu, iovad);
-		preempt_enable();
 		goto retry;
 	}
 
@@ -729,7 +728,7 @@ static bool __iova_rcache_insert(struct iova_domain *iovad,
 	bool can_insert = false;
 	unsigned long flags;
 
-	cpu_rcache = get_cpu_ptr(rcache->cpu_rcaches);
+	cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches);
 	spin_lock_irqsave(&cpu_rcache->lock, flags);
 
 	if (!iova_magazine_full(cpu_rcache->loaded)) {
@@ -759,7 +758,6 @@ static bool __iova_rcache_insert(struct iova_domain *iovad,
 		iova_magazine_push(cpu_rcache->loaded, iova_pfn);
 
 	spin_unlock_irqrestore(&cpu_rcache->lock, flags);
-	put_cpu_ptr(rcache->cpu_rcaches);
 
 	if (mag_to_free) {
 		iova_magazine_free_pfns(mag_to_free, iovad);
@@ -793,7 +791,7 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache,
 	bool has_pfn = false;
 	unsigned long flags;
 
-	cpu_rcache = get_cpu_ptr(rcache->cpu_rcaches);
+	cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches);
 	spin_lock_irqsave(&cpu_rcache->lock, flags);
 
 	if (!iova_magazine_empty(cpu_rcache->loaded)) {
@@ -815,7 +813,6 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache,
 		iova_pfn = iova_magazine_pop(cpu_rcache->loaded, limit_pfn);
 
 	spin_unlock_irqrestore(&cpu_rcache->lock, flags);
-	put_cpu_ptr(rcache->cpu_rcaches);
 
 	return iova_pfn;
 }
-- 
2.13.1

[toc] | [next] | [standalone]

#1676481 — Re: [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()

On Tue, Jun 27, 2017 at 06:16:47PM +0200, Sebastian Andrzej Siewior wrote:
> Commit 583248e6620a ("iommu/iova: Disable preemption around use of
> this_cpu_ptr()") disables preemption while accessing a per-CPU variable.
> This does keep lockdep quiet. However I don't see the point why it is
> bad if we get migrated after its access to another CPU.
> __iova_rcache_insert() and __iova_rcache_get() immediately locks the
> variable after obtaining it - before accessing its members.
> _If_ we get migrated away after retrieving the address of cpu_rcache
> before taking the lock then the *other* task on the same CPU will
> retrieve the same address of cpu_rcache and will spin on the lock.
> 
> alloc_iova_fast() disables preemption while invoking
> free_cpu_cached_iovas() on each CPU. The function itself uses
> per_cpu_ptr() which does not trigger a warning (like this_cpu_ptr()
> does). It _could_ make sense to use get_online_cpus() instead but the we
> have a hotplug notifier for CPU down (and none for up) so we are good.

Does that really matter? The spin_lock disables irqs and thus avoids
preemption too. We also can't get rid of the irqsave lock here because
these locks are taken in the dma-api path which is used from interrupt
context.



	Joerg

[toc] | [prev] | [next] | [standalone]

#1676493 — Re: [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()

On 2017-06-28 11:22:05 [+0200], Joerg Roedel wrote:
> On Tue, Jun 27, 2017 at 06:16:47PM +0200, Sebastian Andrzej Siewior wrote:
> > Commit 583248e6620a ("iommu/iova: Disable preemption around use of
> > this_cpu_ptr()") disables preemption while accessing a per-CPU variable.
> > This does keep lockdep quiet. However I don't see the point why it is
> > bad if we get migrated after its access to another CPU.
> > __iova_rcache_insert() and __iova_rcache_get() immediately locks the
> > variable after obtaining it - before accessing its members.
> > _If_ we get migrated away after retrieving the address of cpu_rcache
> > before taking the lock then the *other* task on the same CPU will
> > retrieve the same address of cpu_rcache and will spin on the lock.
> > 
> > alloc_iova_fast() disables preemption while invoking
> > free_cpu_cached_iovas() on each CPU. The function itself uses
> > per_cpu_ptr() which does not trigger a warning (like this_cpu_ptr()
> > does). It _could_ make sense to use get_online_cpus() instead but the we
> > have a hotplug notifier for CPU down (and none for up) so we are good.
> 
> Does that really matter? The spin_lock disables irqs and thus avoids
> preemption too. We also can't get rid of the irqsave lock here because
> these locks are taken in the dma-api path which is used from interrupt
> context.

It really does. The spin_lock() does disable preemption but this is not
the problem. The thing is that the preempt_disable() is superfluous and
it hurts Preempt-RT (and this is how I noticed it). Also the
get_cpu_ptr() is not requited and was only added to keep lockdep quiet
(according to the history).
Everything else here can stay as-is, I am just asking for the removal of
the redundant preempt_disable() where it is not required.

> 	Joerg

Sebastian

[toc] | [prev] | [next] | [standalone]

#1676529 — Re: [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()

On Wed, Jun 28, 2017 at 11:31:55AM +0200, Sebastian Andrzej Siewior wrote:
> It really does. The spin_lock() does disable preemption but this is not
> the problem. The thing is that the preempt_disable() is superfluous and
> it hurts Preempt-RT (and this is how I noticed it). Also the
> get_cpu_ptr() is not requited and was only added to keep lockdep quiet
> (according to the history).
> Everything else here can stay as-is, I am just asking for the removal of
> the redundant preempt_disable() where it is not required.

Okay, makes sense, I applied both patches.


Thanks,

	Joerg

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web