Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1241371 > unrolled thread
| Started by | Will Deacon <will.deacon@arm.com> |
|---|---|
| First post | 2015-10-07 13:10 +0200 |
| Last post | 2015-10-09 19:30 +0200 |
| Articles | 20 on this page of 26 — 4 participants |
Back to article view | Back to linux.kernel
[PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-07 13:10 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 13:20 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-07 15:30 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 17:00 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 17:30 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Michael Ellerman <mpe@ellerman.id.au> - 2015-10-08 06:00 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-08 13:20 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-08 15:00 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 00:20 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-09 12:00 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 13:30 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 19:50 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 19:50 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-09 20:40 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-13 01:40 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-08 23:50 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 09:30 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 10:40 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-09 11:50 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 13:10 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-09 14:50 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 13:20 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Will Deacon <will.deacon@arm.com> - 2015-10-09 15:00 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 15:10 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 13:20 +0200
Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 19:30 +0200
Page 1 of 2 [1] 2 Next page →
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-10-07 13:10 +0200 |
| Subject | [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qgULF-2Y1-29@gated-at.bofh.it> |
As much as we'd like to live in a world where RELEASE -> ACQUIRE is
always cheaply ordered and can be used to construct UNLOCK -> LOCK
definitions with similar guarantees, the grim reality is that this isn't
even possible on x86 (thanks to Paul for bringing us crashing down to
Earth).
This patch handles the issue by introducing a new barrier macro,
smp_mb__release_acquire, that can be placed between a RELEASE and a
subsequent ACQUIRE operation in order to upgrade them to a full memory
barrier. At the moment, it doesn't have any users, so its existence
serves mainly as a documentation aid.
Documentation/memory-barriers.txt is updated to describe more clearly
the ACQUIRE and RELEASE ordering in this area and to show an example of
the new barrier in action.
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
v1 -> v2: - Clarified that this barrier affects only access performed by
the executing CPU
- Definitions for all TSO architectures
Documentation/memory-barriers.txt | 26 +++++++++++++++++++++++++-
arch/ia64/include/asm/barrier.h | 1 +
arch/powerpc/include/asm/barrier.h | 1 +
arch/s390/include/asm/barrier.h | 2 ++
arch/sparc/include/asm/barrier_64.h | 5 +++--
arch/x86/include/asm/barrier.h | 2 ++
include/asm-generic/barrier.h | 4 ++++
7 files changed, 38 insertions(+), 3 deletions(-)
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 2ba8461b0631..0f69f1dc74fc 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -459,11 +459,18 @@ And a couple of implicit varieties:
RELEASE on that same variable are guaranteed to be visible. In other
words, within a given variable's critical section, all accesses of all
previous critical sections for that variable are guaranteed to have
- completed.
+ completed. If the RELEASE and ACQUIRE operations act on independent
+ variables, an smp_mb__release_acquire() barrier can be placed between
+ them to upgrade the sequence to a full barrier.
This means that ACQUIRE acts as a minimal "acquire" operation and
RELEASE acts as a minimal "release" operation.
+A subset of the atomic operations described in atomic_ops.txt have ACQUIRE
+and RELEASE variants in addition to fully-ordered and relaxed (no barrier
+semantics) definitions. For compound atomics performing both a load and
+a store, ACQUIRE semantics apply only to the load and RELEASE semantics
+only to the store portion of the operation.
Memory barriers are only required where there's a possibility of interaction
between two CPUs or between a CPU and a device. If it can be guaranteed that
@@ -1895,6 +1902,23 @@ the RELEASE would simply complete, thereby avoiding the deadlock.
a sleep-unlock race, but the locking primitive needs to resolve
such races properly in any case.
+Where the RELEASE and ACQUIRE operations are performed by the same CPU,
+ordering can be enforced by use of an smp_mb__release_acquire() barrier:
+
+ *A = a;
+ RELEASE M
+ smp_mb__release_acquire();
+ ACQUIRE N
+ *B = b;
+
+in which case, the only permitted sequences are:
+
+ STORE *A, RELEASE M, ACQUIRE N, STORE *B
+ STORE *A, ACQUIRE N, RELEASE M, STORE *B
+
+Note that smp_mb__release_acquire() has no effect on ACQUIRE or RELEASE
+operations performed by other CPUs.
+
Locks and semaphores may not provide any guarantee of ordering on UP compiled
systems, and so cannot be counted on in such a situation to actually achieve
anything at all - especially with respect to I/O accesses - unless combined
diff --git a/arch/ia64/include/asm/barrier.h b/arch/ia64/include/asm/barrier.h
index df896a1c41d3..9dceee6c2f20 100644
--- a/arch/ia64/include/asm/barrier.h
+++ b/arch/ia64/include/asm/barrier.h
@@ -77,6 +77,7 @@ do { \
___p1; \
})
+#define smp_mb__release_acquire() smp_mb()
#define smp_store_mb(var, value) do { WRITE_ONCE(var, value); mb(); } while (0)
/*
diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h
index 0eca6efc0631..919624634d0a 100644
--- a/arch/powerpc/include/asm/barrier.h
+++ b/arch/powerpc/include/asm/barrier.h
@@ -87,6 +87,7 @@ do { \
___p1; \
})
+#define smp_mb__release_acquire() smp_mb()
#define smp_mb__before_atomic() smp_mb()
#define smp_mb__after_atomic() smp_mb()
#define smp_mb__before_spinlock() smp_mb()
diff --git a/arch/s390/include/asm/barrier.h b/arch/s390/include/asm/barrier.h
index d48fe0162331..0c150b5fdd1c 100644
--- a/arch/s390/include/asm/barrier.h
+++ b/arch/s390/include/asm/barrier.h
@@ -53,4 +53,6 @@ do { \
___p1; \
})
+#define smp_mb__release_acquire() smp_mb()
+
#endif /* __ASM_BARRIER_H */
diff --git a/arch/sparc/include/asm/barrier_64.h b/arch/sparc/include/asm/barrier_64.h
index 14a928601657..4ae875cd9e78 100644
--- a/arch/sparc/include/asm/barrier_64.h
+++ b/arch/sparc/include/asm/barrier_64.h
@@ -71,7 +71,8 @@ do { \
___p1; \
})
-#define smp_mb__before_atomic() barrier()
-#define smp_mb__after_atomic() barrier()
+#define smp_mb__release_acquire() smp_mb()
+#define smp_mb__before_atomic() barrier()
+#define smp_mb__after_atomic() barrier()
#endif /* !(__SPARC64_BARRIER_H) */
diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h
index 0681d2532527..1c61ad251e0e 100644
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -85,6 +85,8 @@ do { \
___p1; \
})
+#define smp_mb__release_acquire() smp_mb()
+
#endif
/* Atomic operations are already serializing on x86 */
diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index b42afada1280..61ae95199397 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -119,5 +119,9 @@ do { \
___p1; \
})
+#ifndef smp_mb__release_acquire
+#define smp_mb__release_acquire() do { } while (0)
+#endif
+
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_GENERIC_BARRIER_H */
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-07 13:20 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qgUVj-39p-1@gated-at.bofh.it> |
| In reply to | #1241371 |
On Wed, Oct 07, 2015 at 11:59:28AM +0100, Will Deacon wrote:
> As much as we'd like to live in a world where RELEASE -> ACQUIRE is
> always cheaply ordered and can be used to construct UNLOCK -> LOCK
> definitions with similar guarantees, the grim reality is that this isn't
> even possible on x86 (thanks to Paul for bringing us crashing down to
> Earth).
>
> This patch handles the issue by introducing a new barrier macro,
> smp_mb__release_acquire, that can be placed between a RELEASE and a
> subsequent ACQUIRE operation in order to upgrade them to a full memory
> barrier. At the moment, it doesn't have any users, so its existence
> serves mainly as a documentation aid.
Does we want to go revert 12d560f4ea87 ("rcu,locking: Privatize
smp_mb__after_unlock_lock()") for that same reason?
> Documentation/memory-barriers.txt is updated to describe more clearly
> the ACQUIRE and RELEASE ordering in this area and to show an example of
> the new barrier in action.
The only nit I have is that if we revert the above it might be make
sense to more clearly call out the distinction between the two.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-10-07 15:30 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qgWX8-65g-17@gated-at.bofh.it> |
| In reply to | #1241377 |
Hi Peter,
Thanks for the headache ;)
On Wed, Oct 07, 2015 at 01:19:15PM +0200, Peter Zijlstra wrote:
> On Wed, Oct 07, 2015 at 11:59:28AM +0100, Will Deacon wrote:
> > As much as we'd like to live in a world where RELEASE -> ACQUIRE is
> > always cheaply ordered and can be used to construct UNLOCK -> LOCK
> > definitions with similar guarantees, the grim reality is that this isn't
> > even possible on x86 (thanks to Paul for bringing us crashing down to
> > Earth).
> >
> > This patch handles the issue by introducing a new barrier macro,
> > smp_mb__release_acquire, that can be placed between a RELEASE and a
> > subsequent ACQUIRE operation in order to upgrade them to a full memory
> > barrier. At the moment, it doesn't have any users, so its existence
> > serves mainly as a documentation aid.
>
> Does we want to go revert 12d560f4ea87 ("rcu,locking: Privatize
> smp_mb__after_unlock_lock()") for that same reason?
I don't think we want a straight revert. smp_mb__after_unlock_lock could
largely die if PPC strengthened its locks, whereas smp_mb__release_acquire
is needed by quite a few architectures.
> > Documentation/memory-barriers.txt is updated to describe more clearly
> > the ACQUIRE and RELEASE ordering in this area and to show an example of
> > the new barrier in action.
>
> The only nit I have is that if we revert the above it might be make
> sense to more clearly call out the distinction between the two.
Right. Where I think we'd like to get to is:
- RELEASE -> ACQUIRE acts as a full barrier if they operate on the same
variable and the ACQUIRE reads from the RELEASE
- RELEASE -> ACQUIRE acts as a full barrier if they execute on the same
CPU and are interleaved with an smp_mb__release_acquire barrier.
- RELEASE -> ACQUIRE ordering is transitive
[only the transitivity part is missing in this patch, because I lost
track of that discussion]
We could then use these same guarantees for UNLOCK -> LOCK in RCU,
defining smp_mb__after_unlock_lock to be the same as
smp_mb__release_acquire, but only applying to UNLOCK -> LOCK. That's a
slight relaxation of how it's defined at the moment (and I guess would
need some work on PPC?), but it keeps things consistent which is
especially important as core locking primitives are ported over to the
ACQUIRE/RELEASE primitives.
Thoughts?
Will
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-07 17:00 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qgYme-7YI-7@gated-at.bofh.it> |
| In reply to | #1241485 |
On Wed, Oct 07, 2015 at 02:23:17PM +0100, Will Deacon wrote:
> Hi Peter,
>
> Thanks for the headache ;)
Most welcome :-)
> > Does we want to go revert 12d560f4ea87 ("rcu,locking: Privatize
> > smp_mb__after_unlock_lock()") for that same reason?
>
> I don't think we want a straight revert. smp_mb__after_unlock_lock could
> largely die if PPC strengthened its locks, whereas smp_mb__release_acquire
> is needed by quite a few architectures.
Fair enough, lets wait for the benchmark results from the PPC people
doing that.
> > > Documentation/memory-barriers.txt is updated to describe more clearly
> > > the ACQUIRE and RELEASE ordering in this area and to show an example of
> > > the new barrier in action.
> >
> > The only nit I have is that if we revert the above it might be make
> > sense to more clearly call out the distinction between the two.
>
> Right. Where I think we'd like to get to is:
>
> - RELEASE -> ACQUIRE acts as a full barrier if they operate on the same
> variable and the ACQUIRE reads from the RELEASE
>
> - RELEASE -> ACQUIRE acts as a full barrier if they execute on the same
> CPU and are interleaved with an smp_mb__release_acquire barrier.
>
> - RELEASE -> ACQUIRE ordering is transitive
>
> [only the transitivity part is missing in this patch, because I lost
> track of that discussion]
>
> We could then use these same guarantees for UNLOCK -> LOCK in RCU,
> defining smp_mb__after_unlock_lock to be the same as
> smp_mb__release_acquire, but only applying to UNLOCK -> LOCK. That's a
> slight relaxation of how it's defined at the moment (and I guess would
> need some work on PPC?), but it keeps things consistent which is
> especially important as core locking primitives are ported over to the
> ACQUIRE/RELEASE primitives.
>
> Thoughts?
/me like, although I'm too tired to see how those 3 rules combine to
something weaker than the current after_unlock_lock thing for PPC.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> |
|---|---|
| Date | 2015-10-07 17:30 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qgYPh-kO-53@gated-at.bofh.it> |
| In reply to | #1241485 |
On Wed, Oct 07, 2015 at 02:23:17PM +0100, Will Deacon wrote:
> Hi Peter,
>
> Thanks for the headache ;)
>
> On Wed, Oct 07, 2015 at 01:19:15PM +0200, Peter Zijlstra wrote:
> > On Wed, Oct 07, 2015 at 11:59:28AM +0100, Will Deacon wrote:
> > > As much as we'd like to live in a world where RELEASE -> ACQUIRE is
> > > always cheaply ordered and can be used to construct UNLOCK -> LOCK
> > > definitions with similar guarantees, the grim reality is that this isn't
> > > even possible on x86 (thanks to Paul for bringing us crashing down to
> > > Earth).
> > >
> > > This patch handles the issue by introducing a new barrier macro,
> > > smp_mb__release_acquire, that can be placed between a RELEASE and a
> > > subsequent ACQUIRE operation in order to upgrade them to a full memory
> > > barrier. At the moment, it doesn't have any users, so its existence
> > > serves mainly as a documentation aid.
> >
> > Does we want to go revert 12d560f4ea87 ("rcu,locking: Privatize
> > smp_mb__after_unlock_lock()") for that same reason?
>
> I don't think we want a straight revert. smp_mb__after_unlock_lock could
> largely die if PPC strengthened its locks, whereas smp_mb__release_acquire
> is needed by quite a few architectures.
>
> > > Documentation/memory-barriers.txt is updated to describe more clearly
> > > the ACQUIRE and RELEASE ordering in this area and to show an example of
> > > the new barrier in action.
> >
> > The only nit I have is that if we revert the above it might be make
> > sense to more clearly call out the distinction between the two.
>
> Right. Where I think we'd like to get to is:
>
> - RELEASE -> ACQUIRE acts as a full barrier if they operate on the same
> variable and the ACQUIRE reads from the RELEASE
>
> - RELEASE -> ACQUIRE acts as a full barrier if they execute on the same
> CPU and are interleaved with an smp_mb__release_acquire barrier.
>
> - RELEASE -> ACQUIRE ordering is transitive
I believe that these three are good.
> [only the transitivity part is missing in this patch, because I lost
> track of that discussion]
>
> We could then use these same guarantees for UNLOCK -> LOCK in RCU,
> defining smp_mb__after_unlock_lock to be the same as
> smp_mb__release_acquire, but only applying to UNLOCK -> LOCK. That's a
> slight relaxation of how it's defined at the moment (and I guess would
> need some work on PPC?), but it keeps things consistent which is
> especially important as core locking primitives are ported over to the
> ACQUIRE/RELEASE primitives.
Currently, we do need smp_mb__after_unlock_lock() to be after the
acquisition on PPC -- putting it between the unlock and the lock
of course doesn't cut it for the cross-thread unlock/lock case.
I am with Peter -- we do need the benchmark results for PPC.
Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Michael Ellerman <mpe@ellerman.id.au> |
|---|---|
| Date | 2015-10-08 06:00 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhax4-en-5@gated-at.bofh.it> |
| In reply to | #1241601 |
On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote:
> On Wed, Oct 07, 2015 at 02:23:17PM +0100, Will Deacon wrote:
> > Hi Peter,
> >
> > Thanks for the headache ;)
> >
> > On Wed, Oct 07, 2015 at 01:19:15PM +0200, Peter Zijlstra wrote:
> > > On Wed, Oct 07, 2015 at 11:59:28AM +0100, Will Deacon wrote:
> > > > As much as we'd like to live in a world where RELEASE -> ACQUIRE is
> > > > always cheaply ordered and can be used to construct UNLOCK -> LOCK
> > > > definitions with similar guarantees, the grim reality is that this isn't
> > > > even possible on x86 (thanks to Paul for bringing us crashing down to
> > > > Earth).
> > > >
> > > > This patch handles the issue by introducing a new barrier macro,
> > > > smp_mb__release_acquire, that can be placed between a RELEASE and a
> > > > subsequent ACQUIRE operation in order to upgrade them to a full memory
> > > > barrier. At the moment, it doesn't have any users, so its existence
> > > > serves mainly as a documentation aid.
> > >
> > > Does we want to go revert 12d560f4ea87 ("rcu,locking: Privatize
> > > smp_mb__after_unlock_lock()") for that same reason?
> >
> > I don't think we want a straight revert. smp_mb__after_unlock_lock could
> > largely die if PPC strengthened its locks, whereas smp_mb__release_acquire
> > is needed by quite a few architectures.
>
> Currently, we do need smp_mb__after_unlock_lock() to be after the
> acquisition on PPC -- putting it between the unlock and the lock
> of course doesn't cut it for the cross-thread unlock/lock case.
>
> I am with Peter -- we do need the benchmark results for PPC.
Urgh, sorry guys. I have been slowly doing some benchmarks, but time is not
plentiful at the moment.
If we do a straight lwsync -> sync conversion for unlock it looks like that
will cost us ~4.2% on Anton's standard context switch benchmark.
So that's not all that nice. But we also don't want to be the only arch that
has the weird lock semantics and has to find all the bugs.
cheers
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-08 13:20 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhhoS-1PL-3@gated-at.bofh.it> |
| In reply to | #1241953 |
On Thu, Oct 08, 2015 at 02:50:36PM +1100, Michael Ellerman wrote: > On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote: > > Currently, we do need smp_mb__after_unlock_lock() to be after the > > acquisition on PPC -- putting it between the unlock and the lock > > of course doesn't cut it for the cross-thread unlock/lock case. This ^, that makes me think I don't understand smp_mb__after_unlock_lock. How is: UNLOCK x smp_mb__after_unlock_lock() LOCK y a problem? That's still a full barrier. > > I am with Peter -- we do need the benchmark results for PPC. > > Urgh, sorry guys. I have been slowly doing some benchmarks, but time is not > plentiful at the moment. > > If we do a straight lwsync -> sync conversion for unlock it looks like that > will cost us ~4.2% on Anton's standard context switch benchmark. And that does not seem to agree with Paul's smp_mb__after_unlock_lock() usage and would not be sufficient for the same (as of yet unexplained) reason. Why does it matter which of the LOCK or UNLOCK gets promoted to full barrier on PPC in order to become RCsc? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-10-08 15:00 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhiXF-3Z7-21@gated-at.bofh.it> |
| In reply to | #1242204 |
On Thu, Oct 08, 2015 at 01:16:38PM +0200, Peter Zijlstra wrote:
> On Thu, Oct 08, 2015 at 02:50:36PM +1100, Michael Ellerman wrote:
> > On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote:
>
> > > Currently, we do need smp_mb__after_unlock_lock() to be after the
> > > acquisition on PPC -- putting it between the unlock and the lock
> > > of course doesn't cut it for the cross-thread unlock/lock case.
>
> This ^, that makes me think I don't understand
> smp_mb__after_unlock_lock.
>
> How is:
>
> UNLOCK x
> smp_mb__after_unlock_lock()
> LOCK y
>
> a problem? That's still a full barrier.
I thought Paul was talking about something like this case:
CPU A CPU B CPU C
foo = 1
UNLOCK x
LOCK x
(RELEASE) bar = 1
ACQUIRE bar = 1
READ_ONCE foo = 0
but this looks the same as ISA2+lwsyncs/ISA2+lwsync+ctrlisync+lwsync,
which are both forbidden on PPC, so now I'm also confused.
The different-lock, same thread case is more straight-forward, I think.
> > > I am with Peter -- we do need the benchmark results for PPC.
> >
> > Urgh, sorry guys. I have been slowly doing some benchmarks, but time is not
> > plentiful at the moment.
> >
> > If we do a straight lwsync -> sync conversion for unlock it looks like that
> > will cost us ~4.2% on Anton's standard context switch benchmark.
Thanks Michael!
> And that does not seem to agree with Paul's smp_mb__after_unlock_lock()
> usage and would not be sufficient for the same (as of yet unexplained)
> reason.
>
> Why does it matter which of the LOCK or UNLOCK gets promoted to full
> barrier on PPC in order to become RCsc?
I think we need a PPC litmus test illustrating the inter-thread, same
lock failure case when smp_mb__after_unlock_lock is not present so that
we can reason about this properly. Paul?
Will
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> |
|---|---|
| Date | 2015-10-09 00:20 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhrHA-8et-25@gated-at.bofh.it> |
| In reply to | #1242320 |
On Thu, Oct 08, 2015 at 01:59:38PM +0100, Will Deacon wrote:
> On Thu, Oct 08, 2015 at 01:16:38PM +0200, Peter Zijlstra wrote:
> > On Thu, Oct 08, 2015 at 02:50:36PM +1100, Michael Ellerman wrote:
> > > On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote:
> >
> > > > Currently, we do need smp_mb__after_unlock_lock() to be after the
> > > > acquisition on PPC -- putting it between the unlock and the lock
> > > > of course doesn't cut it for the cross-thread unlock/lock case.
> >
> > This ^, that makes me think I don't understand
> > smp_mb__after_unlock_lock.
> >
> > How is:
> >
> > UNLOCK x
> > smp_mb__after_unlock_lock()
> > LOCK y
> >
> > a problem? That's still a full barrier.
>
> I thought Paul was talking about something like this case:
>
> CPU A CPU B CPU C
> foo = 1
> UNLOCK x
> LOCK x
> (RELEASE) bar = 1
> ACQUIRE bar = 1
> READ_ONCE foo = 0
More like this:
CPU A CPU B CPU C
WRITE_ONCE(foo, 1);
UNLOCK x
LOCK x
r1 = READ_ONCE(bar);
WRITE_ONCE(bar, 1);
smp_mb();
r2 = READ_ONCE(foo);
This can result in r1==0 && r2==0.
> but this looks the same as ISA2+lwsyncs/ISA2+lwsync+ctrlisync+lwsync,
> which are both forbidden on PPC, so now I'm also confused.
>
> The different-lock, same thread case is more straight-forward, I think.
Indeed it is:
CPU A CPU B
WRITE_ONCE(foo, 1);
UNLOCK x
LOCK x
r1 = READ_ONCE(bar);
WRITE_ONCE(bar, 1);
smp_mb();
r2 = READ_ONCE(foo);
This also can result in r1==0 && r2==0.
> > > > I am with Peter -- we do need the benchmark results for PPC.
> > >
> > > Urgh, sorry guys. I have been slowly doing some benchmarks, but time is not
> > > plentiful at the moment.
> > >
> > > If we do a straight lwsync -> sync conversion for unlock it looks like that
> > > will cost us ~4.2% on Anton's standard context switch benchmark.
>
> Thanks Michael!
>
> > And that does not seem to agree with Paul's smp_mb__after_unlock_lock()
> > usage and would not be sufficient for the same (as of yet unexplained)
> > reason.
> >
> > Why does it matter which of the LOCK or UNLOCK gets promoted to full
> > barrier on PPC in order to become RCsc?
>
> I think we need a PPC litmus test illustrating the inter-thread, same
> lock failure case when smp_mb__after_unlock_lock is not present so that
> we can reason about this properly. Paul?
Please see above. ;-)
The corresponding litmus tests are below.
Thanx, Paul
------------------------------------------------------------------------
PPC lock-2thread-WR-barrier.litmus
""
(*
* Does 3.0 Linux-kernel Power lock-unlock provide local
* barrier that orders prior stores against subsequent loads,
* if the unlock and lock happen on different threads?
* This version uses lwsync instead of isync.
*)
(* 23-July-2013: ppcmem says "Sometimes" *)
{
l=1;
0:r1=1; 0:r4=x; 0:r10=0; 0:r12=l;
1:r1=1; 1:r3=42; 1:r4=x; 1:r5=y; 1:r10=0; 1:r11=0; 1:r12=l;
2:r1=1; 2:r4=x; 2:r5=y;
}
P0 | P1 | P2;
stw r1,0(r4) | lwarx r11,r10,r12 | stw r1,0(r5) ;
lwsync | cmpwi r11,0 | lwsync ;
stw r10,0(r12) | bne Fail1 | lwz r7,0(r4) ;
| stwcx. r1,r10,r12 | ;
| bne Fail1 | ;
| isync | ;
| lwz r3,0(r5) | ;
| Fail1: | ;
exists
(1:r3=0 /\ 2:r7=0)
------------------------------------------------------------------------
PPC lock-1thread-WR-barrier.litmus
""
(*
* Does 3.0 Linux-kernel Power lock-unlock provide local
* barrier that orders prior stores against subsequent loads,
* if the unlock and lock happen in the same thread?
* This version uses lwsync instead of isync.
*)
(* 8-Oct-2015: ppcmem says "Sometimes" *)
{
l=1;
0:r1=1; 0:r3=42; 0:r4=x; 0:r5=y; 0:r10=0; 0:r11=0; 0:r12=l;
1:r1=1; 1:r4=x; 1:r5=y;
}
P0 | P1 ;
stw r1,0(r4) | stw r1,0(r5) ;
lwsync | lwsync ;
stw r10,0(r12) | lwz r7,0(r4) ;
lwarx r11,r10,r12 | ;
cmpwi r11,0 | ;
bne Fail1 | ;
stwcx. r1,r10,r12 | ;
bne Fail1 | ;
isync | ;
lwz r3,0(r5) | ;
Fail1: | ;
exists
(0:r3=0 /\ 1:r7=0)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-10-09 12:00 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhCCZ-6Yo-5@gated-at.bofh.it> |
| In reply to | #1242819 |
Hi Paul,
On Thu, Oct 08, 2015 at 03:17:16PM -0700, Paul E. McKenney wrote:
> On Thu, Oct 08, 2015 at 01:59:38PM +0100, Will Deacon wrote:
> > I thought Paul was talking about something like this case:
> >
> > CPU A CPU B CPU C
> > foo = 1
> > UNLOCK x
> > LOCK x
> > (RELEASE) bar = 1
> > ACQUIRE bar = 1
> > READ_ONCE foo = 0
>
> More like this:
>
> CPU A CPU B CPU C
> WRITE_ONCE(foo, 1);
> UNLOCK x
> LOCK x
> r1 = READ_ONCE(bar);
> WRITE_ONCE(bar, 1);
> smp_mb();
> r2 = READ_ONCE(foo);
>
> This can result in r1==0 && r2==0.
Thank you, that is extremely enlightening :)
> > I think we need a PPC litmus test illustrating the inter-thread, same
> > lock failure case when smp_mb__after_unlock_lock is not present so that
> > we can reason about this properly. Paul?
>
> Please see above. ;-)
>
> The corresponding litmus tests are below.
How do people feel about including these in memory-barriers.txt? I find
them considerably easier to read than our current kernel code + list of
possible orderings + wall of text, but there's a good chance that my
brain has been corrupted from staring at this stuff for too long.
The only snag is the ppc assembly code, but it's not *too* horrific ;)
> PPC lock-2thread-WR-barrier.litmus
> ""
> (*
> * Does 3.0 Linux-kernel Power lock-unlock provide local
> * barrier that orders prior stores against subsequent loads,
> * if the unlock and lock happen on different threads?
> * This version uses lwsync instead of isync.
> *)
> (* 23-July-2013: ppcmem says "Sometimes" *)
> {
> l=1;
> 0:r1=1; 0:r4=x; 0:r10=0; 0:r12=l;
> 1:r1=1; 1:r3=42; 1:r4=x; 1:r5=y; 1:r10=0; 1:r11=0; 1:r12=l;
> 2:r1=1; 2:r4=x; 2:r5=y;
> }
> P0 | P1 | P2;
> stw r1,0(r4) | lwarx r11,r10,r12 | stw r1,0(r5) ;
> lwsync | cmpwi r11,0 | lwsync ;
> stw r10,0(r12) | bne Fail1 | lwz r7,0(r4) ;
> | stwcx. r1,r10,r12 | ;
> | bne Fail1 | ;
> | isync | ;
> | lwz r3,0(r5) | ;
> | Fail1: | ;
>
>
> exists
> (1:r3=0 /\ 2:r7=0)
We could also include a link to the ppcmem/herd web frontends and your
lwn.net article. (ppcmem is already linked, but it's not obvious that
you can run litmus tests in your browser).
Will
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-09 13:30 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhE25-Fi-1@gated-at.bofh.it> |
| In reply to | #1243204 |
On Fri, Oct 09, 2015 at 10:51:29AM +0100, Will Deacon wrote: > > The corresponding litmus tests are below. > > How do people feel about including these in memory-barriers.txt? I find > them considerably easier to read than our current kernel code + list of > possible orderings + wall of text, but there's a good chance that my > brain has been corrupted from staring at this stuff for too long. Your brain is corrupt (but then, so probably is mine, just differently so). I've not yet mastered the knack of reading those things; then again, I suspect its not too hard, just not something I've had the time to play with. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> |
|---|---|
| Date | 2015-10-09 19:50 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhJXP-HL-9@gated-at.bofh.it> |
| In reply to | #1243259 |
On Fri, Oct 09, 2015 at 01:25:54PM +0200, Peter Zijlstra wrote: > On Fri, Oct 09, 2015 at 10:51:29AM +0100, Will Deacon wrote: > > > The corresponding litmus tests are below. > > > > How do people feel about including these in memory-barriers.txt? I find > > them considerably easier to read than our current kernel code + list of > > possible orderings + wall of text, but there's a good chance that my > > brain has been corrupted from staring at this stuff for too long. > > Your brain is corrupt (but then, so probably is mine, just differently > so). > > I've not yet mastered the knack of reading those things; then again, I > suspect its not too hard, just not something I've had the time to play > with. It does take a bit of practice, but is worth the effort. Of course, I -would- say that. ;-) Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> |
|---|---|
| Date | 2015-10-09 19:50 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhJXQ-HL-27@gated-at.bofh.it> |
| In reply to | #1243204 |
On Fri, Oct 09, 2015 at 10:51:29AM +0100, Will Deacon wrote:
> Hi Paul,
>
> On Thu, Oct 08, 2015 at 03:17:16PM -0700, Paul E. McKenney wrote:
> > On Thu, Oct 08, 2015 at 01:59:38PM +0100, Will Deacon wrote:
> > > I thought Paul was talking about something like this case:
> > >
> > > CPU A CPU B CPU C
> > > foo = 1
> > > UNLOCK x
> > > LOCK x
> > > (RELEASE) bar = 1
> > > ACQUIRE bar = 1
> > > READ_ONCE foo = 0
> >
> > More like this:
> >
> > CPU A CPU B CPU C
> > WRITE_ONCE(foo, 1);
> > UNLOCK x
> > LOCK x
> > r1 = READ_ONCE(bar);
> > WRITE_ONCE(bar, 1);
> > smp_mb();
> > r2 = READ_ONCE(foo);
> >
> > This can result in r1==0 && r2==0.
>
> Thank you, that is extremely enlightening :)
>
> > > I think we need a PPC litmus test illustrating the inter-thread, same
> > > lock failure case when smp_mb__after_unlock_lock is not present so that
> > > we can reason about this properly. Paul?
> >
> > Please see above. ;-)
> >
> > The corresponding litmus tests are below.
>
> How do people feel about including these in memory-barriers.txt? I find
> them considerably easier to read than our current kernel code + list of
> possible orderings + wall of text, but there's a good chance that my
> brain has been corrupted from staring at this stuff for too long.
>
> The only snag is the ppc assembly code, but it's not *too* horrific ;)
Maybe we should include them as separate files in Documentation/litmus
or some such. We could then use a litmus-test style with Linux-kernel
C code, and reference litmus tests for various architectures. Maybe
Documentation/litmus/{arm,arm64,powerpc,x86}/ and so on.
For example, the first example in memory-barriers.txt is this:
------------------------------------------------------------------------
CPU 1 CPU 2
=============== ===============
{ A == 1; B == 2 }
A = 3; x = B;
B = 4; y = A;
The set of accesses as seen by the memory system in the middle can be arranged
in 24 different combinations:
STORE A=3, STORE B=4, y=LOAD A->3, x=LOAD B->4
STORE A=3, STORE B=4, x=LOAD B->4, y=LOAD A->3
STORE A=3, y=LOAD A->3, STORE B=4, x=LOAD B->4
STORE A=3, y=LOAD A->3, x=LOAD B->2, STORE B=4
STORE A=3, x=LOAD B->2, STORE B=4, y=LOAD A->3
STORE A=3, x=LOAD B->2, y=LOAD A->3, STORE B=4
STORE B=4, STORE A=3, y=LOAD A->3, x=LOAD B->4
STORE B=4, ...
...
and can thus result in four different combinations of values:
x == 2, y == 1
x == 2, y == 3
x == 4, y == 1
x == 4, y == 3
------------------------------------------------------------------------
Maybe this changes to:
------------------------------------------------------------------------
Linux MP
""
{
a=1; b=2;
}
P0 | P1 ;
WRITE_ONCE(a, 3) | r1 = READ_ONCE(b) ;
WRITE_ONCE(b, 4) | r2 = READ_ONCE(a) ;
exists (2:r1=4 /\ 2:r2=3)
------------------------------------------------------------------------
We can then state that this assertion can fail. We could include
either ppcmem or herd output along with the litmus tests, which would
allow the curious to see a full list of the possible outcomes.
> > PPC lock-2thread-WR-barrier.litmus
> > ""
> > (*
> > * Does 3.0 Linux-kernel Power lock-unlock provide local
> > * barrier that orders prior stores against subsequent loads,
> > * if the unlock and lock happen on different threads?
> > * This version uses lwsync instead of isync.
> > *)
> > (* 23-July-2013: ppcmem says "Sometimes" *)
> > {
> > l=1;
> > 0:r1=1; 0:r4=x; 0:r10=0; 0:r12=l;
> > 1:r1=1; 1:r3=42; 1:r4=x; 1:r5=y; 1:r10=0; 1:r11=0; 1:r12=l;
> > 2:r1=1; 2:r4=x; 2:r5=y;
> > }
> > P0 | P1 | P2;
> > stw r1,0(r4) | lwarx r11,r10,r12 | stw r1,0(r5) ;
> > lwsync | cmpwi r11,0 | lwsync ;
> > stw r10,0(r12) | bne Fail1 | lwz r7,0(r4) ;
> > | stwcx. r1,r10,r12 | ;
> > | bne Fail1 | ;
> > | isync | ;
> > | lwz r3,0(r5) | ;
> > | Fail1: | ;
> >
> >
> > exists
> > (1:r3=0 /\ 2:r7=0)
>
> We could also include a link to the ppcmem/herd web frontends and your
> lwn.net article. (ppcmem is already linked, but it's not obvious that
> you can run litmus tests in your browser).
I bet that the URLs for the web frontends are not stable long term.
Don't get me wrong, PPCMEM/ARMMEM has been there for me for a goodly
number of years, but professors do occasionally move from one institution
to another. For but one example, Susmit Sarkar is now at University
of St. Andrews rather than at Cambridge.
So to make this work, we probably need to be thinking in terms of
asking the researchers for permission to include their ocaml code in the
Linux-kernel source tree. I would be strongly in favor of this, actually.
Thoughts?
Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-10-09 20:40 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhKKe-1RL-11@gated-at.bofh.it> |
| In reply to | #1243557 |
On Fri, Oct 09, 2015 at 10:43:27AM -0700, Paul E. McKenney wrote:
> On Fri, Oct 09, 2015 at 10:51:29AM +0100, Will Deacon wrote:
> > How do people feel about including these in memory-barriers.txt? I find
> > them considerably easier to read than our current kernel code + list of
> > possible orderings + wall of text, but there's a good chance that my
> > brain has been corrupted from staring at this stuff for too long.
> >
> > The only snag is the ppc assembly code, but it's not *too* horrific ;)
>
> Maybe we should include them as separate files in Documentation/litmus
> or some such. We could then use a litmus-test style with Linux-kernel
> C code, and reference litmus tests for various architectures. Maybe
> Documentation/litmus/{arm,arm64,powerpc,x86}/ and so on.
I think that would be useful, particularly if we had a way to "compile"
a kernel litmus test into one for a particular architecture.
> For example, the first example in memory-barriers.txt is this:
>
> ------------------------------------------------------------------------
> CPU 1 CPU 2
> =============== ===============
> { A == 1; B == 2 }
> A = 3; x = B;
> B = 4; y = A;
>
> The set of accesses as seen by the memory system in the middle can be arranged
> in 24 different combinations:
>
> STORE A=3, STORE B=4, y=LOAD A->3, x=LOAD B->4
> STORE A=3, STORE B=4, x=LOAD B->4, y=LOAD A->3
> STORE A=3, y=LOAD A->3, STORE B=4, x=LOAD B->4
> STORE A=3, y=LOAD A->3, x=LOAD B->2, STORE B=4
> STORE A=3, x=LOAD B->2, STORE B=4, y=LOAD A->3
> STORE A=3, x=LOAD B->2, y=LOAD A->3, STORE B=4
> STORE B=4, STORE A=3, y=LOAD A->3, x=LOAD B->4
> STORE B=4, ...
> ...
>
> and can thus result in four different combinations of values:
>
> x == 2, y == 1
> x == 2, y == 3
> x == 4, y == 1
> x == 4, y == 3
> ------------------------------------------------------------------------
>
> Maybe this changes to:
>
> ------------------------------------------------------------------------
> Linux MP
> ""
> {
> a=1; b=2;
> }
> P0 | P1 ;
> WRITE_ONCE(a, 3) | r1 = READ_ONCE(b) ;
> WRITE_ONCE(b, 4) | r2 = READ_ONCE(a) ;
>
> exists (2:r1=4 /\ 2:r2=3)
> ------------------------------------------------------------------------
>
> We can then state that this assertion can fail. We could include
> either ppcmem or herd output along with the litmus tests, which would
> allow the curious to see a full list of the possible outcomes.
More importantly, it would allow them to make small changes to the test
and see what the outcome is, without us having to spawn another
thread-of-death on LKML :)
> > We could also include a link to the ppcmem/herd web frontends and your
> > lwn.net article. (ppcmem is already linked, but it's not obvious that
> > you can run litmus tests in your browser).
>
> I bet that the URLs for the web frontends are not stable long term.
> Don't get me wrong, PPCMEM/ARMMEM has been there for me for a goodly
> number of years, but professors do occasionally move from one institution
> to another. For but one example, Susmit Sarkar is now at University
> of St. Andrews rather than at Cambridge.
>
> So to make this work, we probably need to be thinking in terms of
> asking the researchers for permission to include their ocaml code in the
> Linux-kernel source tree. I would be strongly in favor of this, actually.
>
> Thoughts?
I'm extremely hesitant to import a bunch of dubiously licensed, academic
ocaml code into the kernel. Even if we did, who would maintain it?
A better solution might be to host a mirror of the code on kernel.org,
along with a web front-end for people to play with (the tests we're talking
about here do seem to run ok in my browser).
Will
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> |
|---|---|
| Date | 2015-10-13 01:40 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qiURb-5EZ-5@gated-at.bofh.it> |
| In reply to | #1243594 |
On Fri, Oct 09, 2015 at 07:33:28PM +0100, Will Deacon wrote:
> On Fri, Oct 09, 2015 at 10:43:27AM -0700, Paul E. McKenney wrote:
> > On Fri, Oct 09, 2015 at 10:51:29AM +0100, Will Deacon wrote:
> > > How do people feel about including these in memory-barriers.txt? I find
> > > them considerably easier to read than our current kernel code + list of
> > > possible orderings + wall of text, but there's a good chance that my
> > > brain has been corrupted from staring at this stuff for too long.
> > >
> > > The only snag is the ppc assembly code, but it's not *too* horrific ;)
> >
> > Maybe we should include them as separate files in Documentation/litmus
> > or some such. We could then use a litmus-test style with Linux-kernel
> > C code, and reference litmus tests for various architectures. Maybe
> > Documentation/litmus/{arm,arm64,powerpc,x86}/ and so on.
>
> I think that would be useful, particularly if we had a way to "compile"
> a kernel litmus test into one for a particular architecture.
Very useful! Probably some limitations, especially for loops and lock
acquisition, but what else is new?
> > For example, the first example in memory-barriers.txt is this:
> >
> > ------------------------------------------------------------------------
> > CPU 1 CPU 2
> > =============== ===============
> > { A == 1; B == 2 }
> > A = 3; x = B;
> > B = 4; y = A;
> >
> > The set of accesses as seen by the memory system in the middle can be arranged
> > in 24 different combinations:
> >
> > STORE A=3, STORE B=4, y=LOAD A->3, x=LOAD B->4
> > STORE A=3, STORE B=4, x=LOAD B->4, y=LOAD A->3
> > STORE A=3, y=LOAD A->3, STORE B=4, x=LOAD B->4
> > STORE A=3, y=LOAD A->3, x=LOAD B->2, STORE B=4
> > STORE A=3, x=LOAD B->2, STORE B=4, y=LOAD A->3
> > STORE A=3, x=LOAD B->2, y=LOAD A->3, STORE B=4
> > STORE B=4, STORE A=3, y=LOAD A->3, x=LOAD B->4
> > STORE B=4, ...
> > ...
> >
> > and can thus result in four different combinations of values:
> >
> > x == 2, y == 1
> > x == 2, y == 3
> > x == 4, y == 1
> > x == 4, y == 3
> > ------------------------------------------------------------------------
> > > > Maybe this changes to:
> >
> > ------------------------------------------------------------------------
> > Linux MP
> > ""
> > {
> > a=1; b=2;
> > }
> > P0 | P1 ;
> > WRITE_ONCE(a, 3) | r1 = READ_ONCE(b) ;
> > WRITE_ONCE(b, 4) | r2 = READ_ONCE(a) ;
> >
> > exists (2:r1=4 /\ 2:r2=3)
> > ------------------------------------------------------------------------
> >
> > We can then state that this assertion can fail. We could include
> > either ppcmem or herd output along with the litmus tests, which would
> > allow the curious to see a full list of the possible outcomes.
>
> More importantly, it would allow them to make small changes to the test
> and see what the outcome is, without us having to spawn another
> thread-of-death on LKML :)
Or perhaps we could at least hope for thread-of-tool-supported-death
on LKML. ;-)
> > > We could also include a link to the ppcmem/herd web frontends and your
> > > lwn.net article. (ppcmem is already linked, but it's not obvious that
> > > you can run litmus tests in your browser).
> >
> > I bet that the URLs for the web frontends are not stable long term.
> > Don't get me wrong, PPCMEM/ARMMEM has been there for me for a goodly
> > number of years, but professors do occasionally move from one institution
> > to another. For but one example, Susmit Sarkar is now at University
> > of St. Andrews rather than at Cambridge.
> >
> > So to make this work, we probably need to be thinking in terms of
> > asking the researchers for permission to include their ocaml code in the
> > Linux-kernel source tree. I would be strongly in favor of this, actually.
> >
> > Thoughts?
>
> I'm extremely hesitant to import a bunch of dubiously licensed, academic
> ocaml code into the kernel. Even if we did, who would maintain it?
>
> A better solution might be to host a mirror of the code on kernel.org,
> along with a web front-end for people to play with (the tests we're talking
> about here do seem to run ok in my browser).
I am not too worried about how this happens, but we should avoid
constraining the work of our academic partners. The reason I was thinking
in terms of in the kernel was to avoid version-synchronization issues.
"Wait, this is Linux kernel v4.17, which means that you need to use
version 8.3.5.1 of the tooling... And with these four patches as well."
Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> |
|---|---|
| Date | 2015-10-08 23:50 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhreB-7rq-87@gated-at.bofh.it> |
| In reply to | #1242204 |
On Thu, Oct 08, 2015 at 01:16:38PM +0200, Peter Zijlstra wrote: > On Thu, Oct 08, 2015 at 02:50:36PM +1100, Michael Ellerman wrote: > > On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote: > > > > Currently, we do need smp_mb__after_unlock_lock() to be after the > > > acquisition on PPC -- putting it between the unlock and the lock > > > of course doesn't cut it for the cross-thread unlock/lock case. > > This ^, that makes me think I don't understand > smp_mb__after_unlock_lock. > > How is: > > UNLOCK x > smp_mb__after_unlock_lock() > LOCK y > > a problem? That's still a full barrier. The problem is that I need smp_mb__after_unlock_lock() to give me transitivity even if the UNLOCK happened on one CPU and the LOCK on another. For that to work, the smp_mb__after_unlock_lock() needs to be either immediately after the acquire (the current choice) or immediately before the release (which would also work from a purely technical viewpoint, but I much prefer the current choice). Or am I missing your point? > > > I am with Peter -- we do need the benchmark results for PPC. > > > > Urgh, sorry guys. I have been slowly doing some benchmarks, but time is not > > plentiful at the moment. > > > > If we do a straight lwsync -> sync conversion for unlock it looks like that > > will cost us ~4.2% on Anton's standard context switch benchmark. > > And that does not seem to agree with Paul's smp_mb__after_unlock_lock() > usage and would not be sufficient for the same (as of yet unexplained) > reason. > > Why does it matter which of the LOCK or UNLOCK gets promoted to full > barrier on PPC in order to become RCsc? You could do either. However, as I understand it, there is hardware for which bc;isync is faster than lwsync. For such hardware, it is cheaper to upgrade the unlock from lwsync to sync than to upgrade the lock from bc;isync to sync. If I recall correctly, the kernel rewrites itself at boot to select whichever of lwsync or bc;isync is better for the hardware at hand. Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-09 09:30 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhAhP-3Jz-1@gated-at.bofh.it> |
| In reply to | #1242781 |
On Thu, Oct 08, 2015 at 02:44:39PM -0700, Paul E. McKenney wrote: > > > > I am with Peter -- we do need the benchmark results for PPC. > > > > > > Urgh, sorry guys. I have been slowly doing some benchmarks, but time is not > > > plentiful at the moment. > > > > > > If we do a straight lwsync -> sync conversion for unlock it looks like that > > > will cost us ~4.2% on Anton's standard context switch benchmark. > > > > And that does not seem to agree with Paul's smp_mb__after_unlock_lock() > > usage and would not be sufficient for the same (as of yet unexplained) > > reason. > > > > Why does it matter which of the LOCK or UNLOCK gets promoted to full > > barrier on PPC in order to become RCsc? > > You could do either. However, as I understand it, there is hardware for > which bc;isync is faster than lwsync. For such hardware, it is cheaper > to upgrade the unlock from lwsync to sync than to upgrade the lock from > bc;isync to sync. If I recall correctly, the kernel rewrites itself at > boot to select whichever of lwsync or bc;isync is better for the hardware > at hand. Fair enough. I'll go wake up and think about the other issue ;-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-09 10:40 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhBnz-5g1-1@gated-at.bofh.it> |
| In reply to | #1242781 |
On Thu, Oct 08, 2015 at 02:44:39PM -0700, Paul E. McKenney wrote:
> On Thu, Oct 08, 2015 at 01:16:38PM +0200, Peter Zijlstra wrote:
> > On Thu, Oct 08, 2015 at 02:50:36PM +1100, Michael Ellerman wrote:
> > > On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote:
> >
> > > > Currently, we do need smp_mb__after_unlock_lock() to be after the
> > > > acquisition on PPC -- putting it between the unlock and the lock
> > > > of course doesn't cut it for the cross-thread unlock/lock case.
> >
> > This ^, that makes me think I don't understand
> > smp_mb__after_unlock_lock.
> >
> > How is:
> >
> > UNLOCK x
> > smp_mb__after_unlock_lock()
> > LOCK y
> >
> > a problem? That's still a full barrier.
>
> The problem is that I need smp_mb__after_unlock_lock() to give me
> transitivity even if the UNLOCK happened on one CPU and the LOCK
> on another. For that to work, the smp_mb__after_unlock_lock() needs
> to be either immediately after the acquire (the current choice) or
> immediately before the release (which would also work from a purely
> technical viewpoint, but I much prefer the current choice).
>
> Or am I missing your point?
So lots of little confusions added up to complete fail :-{
Mostly I think it was the UNLOCK x + LOCK x are fully ordered (where I
forgot: but not against uninvolved CPUs) and RELEASE/ACQUIRE are
transitive (where I forgot: RELEASE/ACQUIRE _chains_ are transitive, but
again not against uninvolved CPUs).
Which leads me to think I would like to suggest alternative rules for
RELEASE/ACQUIRE (to replace those Will suggested; as I think those are
partly responsible for my confusion).
- RELEASE -> ACQUIRE is fully ordered (but not a full barrier) when
they operate on the same variable and the ACQUIRE reads from the
RELEASE. Notable, RELEASE/ACQUIRE are RCpc and lack transitivity.
- RELEASE -> ACQUIRE can be upgraded to a full barrier (including
transitivity) using smp_mb__release_acquire(), either before RELEASE
or after ACQUIRE (but consistently [*]).
- RELEASE -> ACQUIRE _chains_ (on shared variables) preserve causality,
(because each link is fully ordered) but are not transitive.
And I think that in the past few weeks we've been using transitive
ambiguously, the definition we have in Documentation/memory-barriers.txt
is a _strong_ transitivity, where we can make guarantees about CPUs not
directly involved.
What we have here (due to RCpc) is a weak form of transitivity, which,
while it preserves the natural concept of causality, does not extend to
other CPUs.
So we could go around and call them 'strong' and 'weak' transitivity,
but I suspect its easier for everyone involved if we come up with
separate terms (less room for error if we accidentally omit the
'strong/weak' qualifier).
[*] Do we want to take that choice away and go for:
smp_mb__after_release_acquire() ?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-10-09 11:50 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhCtk-6Ne-25@gated-at.bofh.it> |
| In reply to | #1243125 |
On Fri, Oct 09, 2015 at 10:31:38AM +0200, Peter Zijlstra wrote:
> On Thu, Oct 08, 2015 at 02:44:39PM -0700, Paul E. McKenney wrote:
> > On Thu, Oct 08, 2015 at 01:16:38PM +0200, Peter Zijlstra wrote:
> > > On Thu, Oct 08, 2015 at 02:50:36PM +1100, Michael Ellerman wrote:
> > > > On Wed, 2015-10-07 at 08:25 -0700, Paul E. McKenney wrote:
> > >
> > > > > Currently, we do need smp_mb__after_unlock_lock() to be after the
> > > > > acquisition on PPC -- putting it between the unlock and the lock
> > > > > of course doesn't cut it for the cross-thread unlock/lock case.
> > >
> > > This ^, that makes me think I don't understand
> > > smp_mb__after_unlock_lock.
> > >
> > > How is:
> > >
> > > UNLOCK x
> > > smp_mb__after_unlock_lock()
> > > LOCK y
> > >
> > > a problem? That's still a full barrier.
> >
> > The problem is that I need smp_mb__after_unlock_lock() to give me
> > transitivity even if the UNLOCK happened on one CPU and the LOCK
> > on another. For that to work, the smp_mb__after_unlock_lock() needs
> > to be either immediately after the acquire (the current choice) or
> > immediately before the release (which would also work from a purely
> > technical viewpoint, but I much prefer the current choice).
> >
> > Or am I missing your point?
>
> So lots of little confusions added up to complete fail :-{
>
> Mostly I think it was the UNLOCK x + LOCK x are fully ordered (where I
> forgot: but not against uninvolved CPUs) and RELEASE/ACQUIRE are
> transitive (where I forgot: RELEASE/ACQUIRE _chains_ are transitive, but
> again not against uninvolved CPUs).
>
> Which leads me to think I would like to suggest alternative rules for
> RELEASE/ACQUIRE (to replace those Will suggested; as I think those are
> partly responsible for my confusion).
Yeah, sorry. I originally used the phrase "fully ordered" but changed it
to "full barrier", which has stronger transitivity (newly understood
definition) requirements that I didn't intend.
RELEASE -> ACQUIRE should be used for message passing between two CPUs
and not have ordering effects on other observers unless they're part of
the RELEASE -> ACQUIRE chain.
> - RELEASE -> ACQUIRE is fully ordered (but not a full barrier) when
> they operate on the same variable and the ACQUIRE reads from the
> RELEASE. Notable, RELEASE/ACQUIRE are RCpc and lack transitivity.
Are we explicit about the difference between "fully ordered" and "full
barrier" somewhere else, because this looks like it will confuse people.
> - RELEASE -> ACQUIRE can be upgraded to a full barrier (including
> transitivity) using smp_mb__release_acquire(), either before RELEASE
> or after ACQUIRE (but consistently [*]).
Hmm, but we don't actually need this for RELEASE -> ACQUIRE, afaict. This
is just needed for UNLOCK -> LOCK, and is exactly what RCU is currently
using (for PPC only).
Stepping back a second, I believe that there are three cases:
RELEASE X -> ACQUIRE Y (same CPU)
* Needs a barrier on TSO architectures for full ordering
UNLOCK X -> LOCK Y (same CPU)
* Needs a barrier on PPC for full ordering
RELEASE X -> ACQUIRE X (different CPUs)
UNLOCK X -> ACQUIRE X (different CPUs)
* Fully ordered everywhere...
* ... but needs a barrier on PPC to become a full barrier
so maybe it makes more sense to split out the local and inter-cpu ordering
with something like:
smp_mb__after_release_acquire()
smp_mb__after_release_acquire_local()
then the first one directly replaces smp_mb__after_unlock_lock, and is
only defined for PPC, whereas the second one is also defined for TSO archs.
> - RELEASE -> ACQUIRE _chains_ (on shared variables) preserve causality,
> (because each link is fully ordered) but are not transitive.
Yup, and that's the same for UNLOCK -> LOCK, too.
> And I think that in the past few weeks we've been using transitive
> ambiguously, the definition we have in Documentation/memory-barriers.txt
> is a _strong_ transitivity, where we can make guarantees about CPUs not
> directly involved.
>
> What we have here (due to RCpc) is a weak form of transitivity, which,
> while it preserves the natural concept of causality, does not extend to
> other CPUs.
>
> So we could go around and call them 'strong' and 'weak' transitivity,
> but I suspect its easier for everyone involved if we come up with
> separate terms (less room for error if we accidentally omit the
> 'strong/weak' qualifier).
Surely the general case is message passing and so "transitivity" should
just refer to chains of RELEASE -> ACQUIRE? Then "strong transitivity"
could refer to the far more complicated (imo) case that is synonymous
with "full barrier".
Will
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-10-09 13:10 +0200 |
| Subject | Re: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation |
| Message-ID | <qhDIJ-i3-7@gated-at.bofh.it> |
| In reply to | #1243198 |
On Fri, Oct 09, 2015 at 10:40:39AM +0100, Will Deacon wrote:
> Stepping back a second, I believe that there are three cases:
>
>
> RELEASE X -> ACQUIRE Y (same CPU)
> * Needs a barrier on TSO architectures for full ordering
+PPC
> UNLOCK X -> LOCK Y (same CPU)
> * Needs a barrier on PPC for full ordering
> RELEASE X -> ACQUIRE X (different CPUs)
* Fully ordered everywhere...
* ... but needs a barrier on TSO + PPC to become a full barrier
> UNLOCK X -> ACQUIRE X (different CPUs)
s/ACQUIRE/LOCK/ ?
> * Fully ordered everywhere...
> * ... but needs a barrier on PPC to become a full barrier
If you really meant ACQUIRE, then x86 also needs a barrier in order to
upgrade, seeing how our unlock is equivalent to smp_store_release(). Our
LOCK otoh is far heavier than smp_load_acquire() and would result in
different rules.
And I'm not sure the "(different CPUs)" bit makes sense, as the same is
true if they're on the same CPU.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
Page 1 of 2 [1] 2 Next page →
Back to top | Article view | linux.kernel
csiph-web