Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1186130 > unrolled thread
| Started by | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| First post | 2015-07-16 19:40 +0200 |
| Last post | 2015-07-17 15:00 +0200 |
| Articles | 17 — 2 participants |
Back to article view | Back to linux.kernel
[PATCH 00/28] arch: Provide atomic logic ops Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:40 +0200
[PATCH 19/28] xtensa: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:40 +0200
[PATCH 02/28] alpha: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:40 +0200
[PATCH 20/28] s390: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 22/28] frv: Rewrite atomic implementation Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 25/28] atomic: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 05/28] arm64: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 13/28] mips: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 14/28] mn10300: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 12/28] metag: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 26/28] atomic: Collapse all atomic_{set,clear}_mask definitions Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 04/28] arm: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 08/28] hexagon: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 11/28] m68k: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 17/28] sh: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
[PATCH 24/28] tile: Provide atomic_{or,xor,and} Peter Zijlstra <peterz@infradead.org> - 2015-07-16 19:50 +0200
Re: [PATCH 00/28] arch: Provide atomic logic ops Will Deacon <will.deacon@arm.com> - 2015-07-17 15:00 +0200
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:40 +0200 |
| Subject | [PATCH 00/28] arch: Provide atomic logic ops |
| Message-ID | <pMVix-57c-3@gated-at.bofh.it> |
Currently there is an incoherent mess of atomic_{set,clear}_mask() and
atomic_or() (but no atomic_{and,nand,xor}()) in the tree.
Those archs that implement atomic_{set,clear}_mask() are not even consistent on
its signature.
Implement atomic_{or,and,xor}() on all archs and deprecate
atomic_{set,clear}_mask().
The series has been compile tested by the build-bot, no known failures at this time.
I would like to take this through tip; I'll ask Ingo to put it in a separate
branch such that people can pull in that branch if there is conflicting work
elsewhere (Will has some for Argh64).
Changes since lst time:
- Quite a few build fails fixed, most notable would be that each arch
now defines CONFIG_ARCH_HAS_ATOMIC_OR for a little while -- we take it
out again later.
- FRV has some extra magic that allows the inline assembly to correctly
work as inline functions.
- TILE patch contributed by Chris -- Thanks!
- A little extention to lib/atomic64_test.c that compile tests the new
primitives.
- Alpha, fixed ASM due to fallout from actually trying to compile things.
- H8300, somehow I had overlooked it previously.
- Added atomic_andnot() to a few archs that have that instruction to test that.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:40 +0200 |
| Subject | [PATCH 19/28] xtensa: Provide atomic_{or,xor,and} |
| Message-ID | <pMViA-57c-53@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/xtensa/include/asm/atomic.h | 85 +++++++--------------------------------
1 file changed, 16 insertions(+), 69 deletions(-)
--- a/arch/xtensa/include/asm/atomic.h
+++ b/arch/xtensa/include/asm/atomic.h
@@ -145,10 +145,26 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_or(mask, v);
+}
+
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_and(~mask, v);
+}
+
/**
* atomic_sub_and_test - subtract value from variable and test result
* @i: integer value to subtract
@@ -250,75 +266,6 @@ static __inline__ int __atomic_add_unles
return c;
}
-
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
-#if XCHAL_HAVE_S32C1I
- unsigned long tmp;
- int result;
-
- __asm__ __volatile__(
- "1: l32i %1, %3, 0\n"
- " wsr %1, scompare1\n"
- " and %0, %1, %2\n"
- " s32c1i %0, %3, 0\n"
- " bne %0, %1, 1b\n"
- : "=&a" (result), "=&a" (tmp)
- : "a" (~mask), "a" (v)
- : "memory"
- );
-#else
- unsigned int all_f = -1;
- unsigned int vval;
-
- __asm__ __volatile__(
- " rsil a15,"__stringify(LOCKLEVEL)"\n"
- " l32i %0, %2, 0\n"
- " xor %1, %4, %3\n"
- " and %0, %0, %4\n"
- " s32i %0, %2, 0\n"
- " wsr a15, ps\n"
- " rsync\n"
- : "=&a" (vval), "=a" (mask)
- : "a" (v), "a" (all_f), "1" (mask)
- : "a15", "memory"
- );
-#endif
-}
-
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
-#if XCHAL_HAVE_S32C1I
- unsigned long tmp;
- int result;
-
- __asm__ __volatile__(
- "1: l32i %1, %3, 0\n"
- " wsr %1, scompare1\n"
- " or %0, %1, %2\n"
- " s32c1i %0, %3, 0\n"
- " bne %0, %1, 1b\n"
- : "=&a" (result), "=&a" (tmp)
- : "a" (mask), "a" (v)
- : "memory"
- );
-#else
- unsigned int vval;
-
- __asm__ __volatile__(
- " rsil a15,"__stringify(LOCKLEVEL)"\n"
- " l32i %0, %2, 0\n"
- " or %0, %0, %1\n"
- " s32i %0, %2, 0\n"
- " wsr a15, ps\n"
- " rsync\n"
- : "=&a" (vval)
- : "a" (mask), "a" (v)
- : "a15", "memory"
- );
-#endif
-}
-
#endif /* __KERNEL__ */
#endif /* _XTENSA_ATOMIC_H */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:40 +0200 |
| Subject | [PATCH 02/28] alpha: Provide atomic_{or,xor,and} |
| Message-ID | <pMViA-57c-57@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/alpha/include/asm/atomic.h | 43 ++++++++++++++++++++++++++--------------
1 file changed, 28 insertions(+), 15 deletions(-)
--- a/arch/alpha/include/asm/atomic.h
+++ b/arch/alpha/include/asm/atomic.h
@@ -29,13 +29,13 @@
* branch back to restart the operation.
*/
-#define ATOMIC_OP(op) \
+#define ATOMIC_OP(op, asm_op) \
static __inline__ void atomic_##op(int i, atomic_t * v) \
{ \
unsigned long temp; \
__asm__ __volatile__( \
"1: ldl_l %0,%1\n" \
- " " #op "l %0,%2,%0\n" \
+ " " #asm_op " %0,%2,%0\n" \
" stl_c %0,%1\n" \
" beq %0,2f\n" \
".subsection 2\n" \
@@ -45,15 +45,15 @@ static __inline__ void atomic_##op(int i
:"Ir" (i), "m" (v->counter)); \
} \
-#define ATOMIC_OP_RETURN(op) \
+#define ATOMIC_OP_RETURN(op, asm_op) \
static inline int atomic_##op##_return(int i, atomic_t *v) \
{ \
long temp, result; \
smp_mb(); \
__asm__ __volatile__( \
"1: ldl_l %0,%1\n" \
- " " #op "l %0,%3,%2\n" \
- " " #op "l %0,%3,%0\n" \
+ " " #asm_op " %0,%3,%2\n" \
+ " " #asm_op " %0,%3,%0\n" \
" stl_c %0,%1\n" \
" beq %0,2f\n" \
".subsection 2\n" \
@@ -65,13 +65,13 @@ static inline int atomic_##op##_return(i
return result; \
}
-#define ATOMIC64_OP(op) \
+#define ATOMIC64_OP(op, asm_op) \
static __inline__ void atomic64_##op(long i, atomic64_t * v) \
{ \
unsigned long temp; \
__asm__ __volatile__( \
"1: ldq_l %0,%1\n" \
- " " #op "q %0,%2,%0\n" \
+ " " #asm_op " %0,%2,%0\n" \
" stq_c %0,%1\n" \
" beq %0,2f\n" \
".subsection 2\n" \
@@ -81,15 +81,15 @@ static __inline__ void atomic64_##op(lon
:"Ir" (i), "m" (v->counter)); \
} \
-#define ATOMIC64_OP_RETURN(op) \
+#define ATOMIC64_OP_RETURN(op, asm_op) \
static __inline__ long atomic64_##op##_return(long i, atomic64_t * v) \
{ \
long temp, result; \
smp_mb(); \
__asm__ __volatile__( \
"1: ldq_l %0,%1\n" \
- " " #op "q %0,%3,%2\n" \
- " " #op "q %0,%3,%0\n" \
+ " " #asm_op " %0,%3,%2\n" \
+ " " #asm_op " %0,%3,%0\n" \
" stq_c %0,%1\n" \
" beq %0,2f\n" \
".subsection 2\n" \
@@ -101,15 +101,28 @@ static __inline__ long atomic64_##op##_r
return result; \
}
-#define ATOMIC_OPS(opg) \
- ATOMIC_OP(opg) \
- ATOMIC_OP_RETURN(opg) \
- ATOMIC64_OP(opg) \
- ATOMIC64_OP_RETURN(opg)
+#define ATOMIC_OPS(op) \
+ ATOMIC_OP(op, op##l) \
+ ATOMIC_OP_RETURN(op, op##l) \
+ ATOMIC64_OP(op, op##q) \
+ ATOMIC64_OP_RETURN(op, op##q)
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+#define atomic_andnot atomic_andnot
+#define atomic64_andnot atomic64_andnot
+
+ATOMIC_OP(and, and)
+ATOMIC_OP(andnot, bic)
+ATOMIC_OP(or, bis)
+ATOMIC_OP(xor, xor)
+ATOMIC64_OP(and, and)
+ATOMIC64_OP(andnot, bic)
+ATOMIC64_OP(or, bis)
+ATOMIC64_OP(xor, xor)
+
#undef ATOMIC_OPS
#undef ATOMIC64_OP_RETURN
#undef ATOMIC64_OP
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 20/28] s390: Provide atomic_{or,xor,and} |
| Message-ID | <pMVse-5iT-3@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/s390/include/asm/atomic.h | 47 ++++++++++++++++++++++++++++-------------
1 file changed, 33 insertions(+), 14 deletions(-)
--- a/arch/s390/include/asm/atomic.h
+++ b/arch/s390/include/asm/atomic.h
@@ -27,6 +27,7 @@
#define __ATOMIC_OR "lao"
#define __ATOMIC_AND "lan"
#define __ATOMIC_ADD "laa"
+#define __ATOMIC_XOR "lax"
#define __ATOMIC_BARRIER "bcr 14,0\n"
#define __ATOMIC_LOOP(ptr, op_val, op_string, __barrier) \
@@ -49,6 +50,7 @@
#define __ATOMIC_OR "or"
#define __ATOMIC_AND "nr"
#define __ATOMIC_ADD "ar"
+#define __ATOMIC_XOR "xr"
#define __ATOMIC_BARRIER "\n"
#define __ATOMIC_LOOP(ptr, op_val, op_string, __barrier) \
@@ -118,14 +120,26 @@ static inline void atomic_add(int i, ato
#define atomic_dec_return(_v) atomic_sub_return(1, _v)
#define atomic_dec_and_test(_v) (atomic_sub_return(1, _v) == 0)
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
+#define ATOMIC_OP(op, OP) \
+static inline void atomic_##op(int i, atomic_t *v) \
+{ \
+ __ATOMIC_LOOP(v, i, __ATOMIC_##OP, __ATOMIC_NO_BARRIER); \
+}
+
+ATOMIC_OP(and, AND)
+ATOMIC_OP(or, OR)
+ATOMIC_OP(xor, XOR)
+
+#undef ATOMIC_OP
+
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
{
- __ATOMIC_LOOP(v, ~mask, __ATOMIC_AND, __ATOMIC_NO_BARRIER);
+ atomic_and(~mask, v);
}
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
{
- __ATOMIC_LOOP(v, mask, __ATOMIC_OR, __ATOMIC_NO_BARRIER);
+ atomic_or(mask, v);
}
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
@@ -167,6 +181,7 @@ static inline int __atomic_add_unless(at
#define __ATOMIC64_OR "laog"
#define __ATOMIC64_AND "lang"
#define __ATOMIC64_ADD "laag"
+#define __ATOMIC64_XOR "laxg"
#define __ATOMIC64_BARRIER "bcr 14,0\n"
#define __ATOMIC64_LOOP(ptr, op_val, op_string, __barrier) \
@@ -189,6 +204,7 @@ static inline int __atomic_add_unless(at
#define __ATOMIC64_OR "ogr"
#define __ATOMIC64_AND "ngr"
#define __ATOMIC64_ADD "agr"
+#define __ATOMIC64_XOR "xgr"
#define __ATOMIC64_BARRIER "\n"
#define __ATOMIC64_LOOP(ptr, op_val, op_string, __barrier) \
@@ -247,16 +263,6 @@ static inline void atomic64_add(long lon
__ATOMIC64_LOOP(v, i, __ATOMIC64_ADD, __ATOMIC64_NO_BARRIER);
}
-static inline void atomic64_clear_mask(unsigned long mask, atomic64_t *v)
-{
- __ATOMIC64_LOOP(v, ~mask, __ATOMIC64_AND, __ATOMIC64_NO_BARRIER);
-}
-
-static inline void atomic64_set_mask(unsigned long mask, atomic64_t *v)
-{
- __ATOMIC64_LOOP(v, mask, __ATOMIC64_OR, __ATOMIC64_NO_BARRIER);
-}
-
#define atomic64_xchg(v, new) (xchg(&((v)->counter), new))
static inline long long atomic64_cmpxchg(atomic64_t *v,
@@ -270,6 +276,19 @@ static inline long long atomic64_cmpxchg
return old;
}
+#define ATOMIC64_OP(op, OP) \
+static inline void atomic64_##op(long i, atomic64_t *v) \
+{ \
+ __ATOMIC64_LOOP(v, i, __ATOMIC64_##OP, __ATOMIC64_NO_BARRIER); \
+}
+
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC64_OP(and, AND)
+ATOMIC64_OP(or, OR)
+ATOMIC64_OP(xor, XOR)
+
+#undef ATOMIC64_OP
#undef __ATOMIC64_LOOP
static inline int atomic64_add_unless(atomic64_t *v, long long i, long long u)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 22/28] frv: Rewrite atomic implementation |
| Message-ID | <pMVse-5iT-5@gated-at.bofh.it> |
| In reply to | #1186130 |
Mostly complete rewrite of the FRV atomic implementation, instead of
using assembly files, use inline assembler.
The out-of-line CONFIG option makes a bit of a mess of things, but a
little CPP trickery gets that done too.
FRV already had the atomic logic ops but under a non standard name,
the reimplementation provides the generic names and provides the
intermediate form required for the bitops implementation.
The slightly inconsistent __atomic32_fetch_##op naming is because
__atomic_fetch_##op conlicts with GCC builtin functions.
The 64bit atomic ops use the inline assembly %Ln construct to access
the low word register (r+1), afaik this construct was not previously
used in the kernel and is completely undocumented, but I found it in
the FRV GCC code and it seems to work.
FRV had a non-standard definition of atomic_{clear,set}_mask() which
would work types other than atomic_t, the one user relying on that
(arch/frv/kernel/dma.c) got converted to use the new intermediate
form.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/frv/include/asm/atomic.h | 119 ++++++++++++++-----------
arch/frv/include/asm/atomic_defs.h | 172 +++++++++++++++++++++++++++++++++++++
arch/frv/include/asm/bitops.h | 99 ++-------------------
arch/frv/kernel/dma.c | 6 -
arch/frv/kernel/frv_ksyms.c | 5 -
arch/frv/lib/Makefile | 2
arch/frv/lib/atomic-lib.c | 7 +
arch/frv/lib/atomic-ops.S | 110 -----------------------
arch/frv/lib/atomic64-ops.S | 94 --------------------
9 files changed, 259 insertions(+), 355 deletions(-)
--- a/arch/frv/include/asm/atomic.h
+++ b/arch/frv/include/asm/atomic.h
@@ -15,7 +15,6 @@
#define _ASM_ATOMIC_H
#include <linux/types.h>
-#include <asm/spr-regs.h>
#include <asm/cmpxchg.h>
#include <asm/barrier.h>
@@ -23,6 +22,8 @@
#error not SMP safe
#endif
+#include <asm/atomic_defs.h>
+
/*
* Atomic operations that C can't guarantee us. Useful for
* resource counting etc..
@@ -34,56 +35,26 @@
#define atomic_read(v) ACCESS_ONCE((v)->counter)
#define atomic_set(v, i) (((v)->counter) = (i))
-#ifndef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS
-static inline int atomic_add_return(int i, atomic_t *v)
+static inline int atomic_inc_return(atomic_t *v)
{
- unsigned long val;
+ return __atomic_add_return(1, &v->counter);
+}
- asm("0: \n"
- " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */
- " ckeq icc3,cc7 \n"
- " ld.p %M0,%1 \n" /* LD.P/ORCR must be atomic */
- " orcr cc7,cc7,cc3 \n" /* set CC3 to true */
- " add%I2 %1,%2,%1 \n"
- " cst.p %1,%M0 ,cc3,#1 \n"
- " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* clear ICC3.Z if store happens */
- " beq icc3,#0,0b \n"
- : "+U"(v->counter), "=&r"(val)
- : "NPr"(i)
- : "memory", "cc7", "cc3", "icc3"
- );
+static inline int atomic_dec_return(atomic_t *v)
+{
+ return __atomic_sub_return(1, &v->counter);
+}
- return val;
+static inline int atomic_add_return(int i, atomic_t *v)
+{
+ return __atomic_add_return(i, &v->counter);
}
static inline int atomic_sub_return(int i, atomic_t *v)
{
- unsigned long val;
-
- asm("0: \n"
- " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */
- " ckeq icc3,cc7 \n"
- " ld.p %M0,%1 \n" /* LD.P/ORCR must be atomic */
- " orcr cc7,cc7,cc3 \n" /* set CC3 to true */
- " sub%I2 %1,%2,%1 \n"
- " cst.p %1,%M0 ,cc3,#1 \n"
- " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* clear ICC3.Z if store happens */
- " beq icc3,#0,0b \n"
- : "+U"(v->counter), "=&r"(val)
- : "NPr"(i)
- : "memory", "cc7", "cc3", "icc3"
- );
-
- return val;
+ return __atomic_sub_return(i, &v->counter);
}
-#else
-
-extern int atomic_add_return(int i, atomic_t *v);
-extern int atomic_sub_return(int i, atomic_t *v);
-
-#endif
-
static inline int atomic_add_negative(int i, atomic_t *v)
{
return atomic_add_return(i, v) < 0;
@@ -101,17 +72,14 @@ static inline void atomic_sub(int i, ato
static inline void atomic_inc(atomic_t *v)
{
- atomic_add_return(1, v);
+ atomic_inc_return(v);
}
static inline void atomic_dec(atomic_t *v)
{
- atomic_sub_return(1, v);
+ atomic_dec_return(v);
}
-#define atomic_dec_return(v) atomic_sub_return(1, (v))
-#define atomic_inc_return(v) atomic_add_return(1, (v))
-
#define atomic_sub_and_test(i,v) (atomic_sub_return((i), (v)) == 0)
#define atomic_dec_and_test(v) (atomic_sub_return(1, (v)) == 0)
#define atomic_inc_and_test(v) (atomic_add_return(1, (v)) == 0)
@@ -120,18 +88,19 @@ static inline void atomic_dec(atomic_t *
* 64-bit atomic ops
*/
typedef struct {
- volatile long long counter;
+ long long counter;
} atomic64_t;
#define ATOMIC64_INIT(i) { (i) }
-static inline long long atomic64_read(atomic64_t *v)
+static inline long long atomic64_read(const atomic64_t *v)
{
long long counter;
asm("ldd%I1 %M1,%0"
: "=e"(counter)
: "m"(v->counter));
+
return counter;
}
@@ -142,10 +111,25 @@ static inline void atomic64_set(atomic64
: "e"(i));
}
-extern long long atomic64_inc_return(atomic64_t *v);
-extern long long atomic64_dec_return(atomic64_t *v);
-extern long long atomic64_add_return(long long i, atomic64_t *v);
-extern long long atomic64_sub_return(long long i, atomic64_t *v);
+static inline long long atomic64_inc_return(atomic64_t *v)
+{
+ return __atomic64_add_return(1, &v->counter);
+}
+
+static inline long long atomic64_dec_return(atomic64_t *v)
+{
+ return __atomic64_sub_return(1, &v->counter);
+}
+
+static inline long long atomic64_add_return(long long i, atomic64_t *v)
+{
+ return __atomic64_add_return(i, &v->counter);
+}
+
+static inline long long atomic64_sub_return(long long i, atomic64_t *v)
+{
+ return __atomic64_sub_return(i, &v->counter);
+}
static inline long long atomic64_add_negative(long long i, atomic64_t *v)
{
@@ -176,6 +160,7 @@ static inline void atomic64_dec(atomic64
#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
#define atomic64_inc_and_test(v) (atomic64_inc_return((v)) == 0)
+
#define atomic_cmpxchg(v, old, new) (cmpxchg(&(v)->counter, old, new))
#define atomic_xchg(v, new) (xchg(&(v)->counter, new))
#define atomic64_cmpxchg(v, old, new) (__cmpxchg_64(old, new, &(v)->counter))
@@ -196,5 +181,33 @@ static __inline__ int __atomic_add_unles
return c;
}
+#define ATOMIC_OP(op) \
+static inline void atomic_##op(int i, atomic_t *v) \
+{ \
+ (void)__atomic32_fetch_##op(i, &v->counter); \
+} \
+ \
+static inline void atomic64_##op(long long i, atomic64_t *v) \
+{ \
+ (void)__atomic64_fetch_##op(i, &v->counter); \
+}
+
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(or)
+ATOMIC_OP(and)
+ATOMIC_OP(xor)
+
+#undef ATOMIC_OP
+
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_and(~mask, v);
+}
+
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_or(mask, v);
+}
#endif /* _ASM_ATOMIC_H */
--- /dev/null
+++ b/arch/frv/include/asm/atomic_defs.h
@@ -0,0 +1,172 @@
+
+#include <asm/spr-regs.h>
+
+#ifdef __ATOMIC_LIB__
+
+#ifdef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS
+
+#define ATOMIC_QUALS
+#define ATOMIC_EXPORT(x) EXPORT_SYMBOL(x)
+
+#else /* !OUTOFLINE && LIB */
+
+#define ATOMIC_OP_RETURN(op)
+#define ATOMIC_FETCH_OP(op)
+
+#endif /* OUTOFLINE */
+
+#else /* !__ATOMIC_LIB__ */
+
+#define ATOMIC_EXPORT(x)
+
+#ifdef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS
+
+#define ATOMIC_OP_RETURN(op) \
+extern int __atomic_##op##_return(int i, int *v); \
+extern long long __atomic64_##op##_return(long long i, long long *v);
+
+#define ATOMIC_FETCH_OP(op) \
+extern int __atomic32_fetch_##op(int i, int *v); \
+extern long long __atomic64_fetch_##op(long long i, long long *v);
+
+#else /* !OUTOFLINE && !LIB */
+
+#define ATOMIC_QUALS static inline
+
+#endif /* OUTOFLINE */
+#endif /* __ATOMIC_LIB__ */
+
+
+/*
+ * Note on the 64 bit inline asm variants...
+ *
+ * CSTD is a conditional instruction and needs a constrained memory reference.
+ * Normally 'U' provides the correct constraints for conditional instructions
+ * and this is used for the 32 bit version, however 'U' does not appear to work
+ * for 64 bit values (gcc-4.9)
+ *
+ * The exact constraint is that conditional instructions cannot deal with an
+ * immediate displacement in the memory reference, so what we do is we read the
+ * address through a volatile cast into a local variable in order to insure we
+ * _have_ to compute the correct address without displacement. This allows us
+ * to use the regular 'm' for the memory address.
+ *
+ * Furthermore, the %Ln operand, which prints the low word register (r+1),
+ * really only works for registers, this means we cannot allow immediate values
+ * for the 64 bit versions -- like we do for the 32 bit ones.
+ *
+ */
+
+#ifndef ATOMIC_OP_RETURN
+#define ATOMIC_OP_RETURN(op) \
+ATOMIC_QUALS int __atomic_##op##_return(int i, int *v) \
+{ \
+ int val; \
+ \
+ asm volatile( \
+ "0: \n" \
+ " orcc gr0,gr0,gr0,icc3 \n" \
+ " ckeq icc3,cc7 \n" \
+ " ld.p %M0,%1 \n" \
+ " orcr cc7,cc7,cc3 \n" \
+ " "#op"%I2 %1,%2,%1 \n" \
+ " cst.p %1,%M0 ,cc3,#1 \n" \
+ " corcc gr29,gr29,gr0 ,cc3,#1 \n" \
+ " beq icc3,#0,0b \n" \
+ : "+U"(*v), "=&r"(val) \
+ : "NPr"(i) \
+ : "memory", "cc7", "cc3", "icc3" \
+ ); \
+ \
+ return val; \
+} \
+ATOMIC_EXPORT(__atomic_##op##_return); \
+ \
+ATOMIC_QUALS long long __atomic64_##op##_return(long long i, long long *v) \
+{ \
+ long long *__v = READ_ONCE(v); \
+ long long val; \
+ \
+ asm volatile( \
+ "0: \n" \
+ " orcc gr0,gr0,gr0,icc3 \n" \
+ " ckeq icc3,cc7 \n" \
+ " ldd.p %M0,%1 \n" \
+ " orcr cc7,cc7,cc3 \n" \
+ " "#op"cc %L1,%L2,%L1,icc0 \n" \
+ " "#op"x %1,%2,%1,icc0 \n" \
+ " cstd.p %1,%M0 ,cc3,#1 \n" \
+ " corcc gr29,gr29,gr0 ,cc3,#1 \n" \
+ " beq icc3,#0,0b \n" \
+ : "+m"(*__v), "=&e"(val) \
+ : "e"(i) \
+ : "memory", "cc7", "cc3", "icc0", "icc3" \
+ ); \
+ \
+ return val; \
+} \
+ATOMIC_EXPORT(__atomic64_##op##_return);
+#endif
+
+#ifndef ATOMIC_FETCH_OP
+#define ATOMIC_FETCH_OP(op) \
+ATOMIC_QUALS int __atomic32_fetch_##op(int i, int *v) \
+{ \
+ int old, tmp; \
+ \
+ asm volatile( \
+ "0: \n" \
+ " orcc gr0,gr0,gr0,icc3 \n" \
+ " ckeq icc3,cc7 \n" \
+ " ld.p %M0,%1 \n" \
+ " orcr cc7,cc7,cc3 \n" \
+ " "#op"%I3 %1,%3,%2 \n" \
+ " cst.p %2,%M0 ,cc3,#1 \n" \
+ " corcc gr29,gr29,gr0 ,cc3,#1 \n" \
+ " beq icc3,#0,0b \n" \
+ : "+U"(*v), "=&r"(old), "=r"(tmp) \
+ : "NPr"(i) \
+ : "memory", "cc7", "cc3", "icc3" \
+ ); \
+ \
+ return old; \
+} \
+ATOMIC_EXPORT(__atomic32_fetch_##op); \
+ \
+ATOMIC_QUALS long long __atomic64_fetch_##op(long long i, long long *v) \
+{ \
+ long long *__v = READ_ONCE(v); \
+ long long old, tmp; \
+ \
+ asm volatile( \
+ "0: \n" \
+ " orcc gr0,gr0,gr0,icc3 \n" \
+ " ckeq icc3,cc7 \n" \
+ " ldd.p %M0,%1 \n" \
+ " orcr cc7,cc7,cc3 \n" \
+ " "#op" %L1,%L3,%L2 \n" \
+ " "#op" %1,%3,%2 \n" \
+ " cstd.p %2,%M0 ,cc3,#1 \n" \
+ " corcc gr29,gr29,gr0 ,cc3,#1 \n" \
+ " beq icc3,#0,0b \n" \
+ : "+m"(*__v), "=&e"(old), "=e"(tmp) \
+ : "e"(i) \
+ : "memory", "cc7", "cc3", "icc3" \
+ ); \
+ \
+ return old; \
+} \
+ATOMIC_EXPORT(__atomic64_fetch_##op);
+#endif
+
+ATOMIC_FETCH_OP(or)
+ATOMIC_FETCH_OP(and)
+ATOMIC_FETCH_OP(xor)
+
+ATOMIC_OP_RETURN(add)
+ATOMIC_OP_RETURN(sub)
+
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+#undef ATOMIC_QUALS
+#undef ATOMIC_EXPORT
--- a/arch/frv/include/asm/bitops.h
+++ b/arch/frv/include/asm/bitops.h
@@ -25,109 +25,30 @@
#include <asm-generic/bitops/ffz.h>
-#ifndef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS
-static inline
-unsigned long atomic_test_and_ANDNOT_mask(unsigned long mask, volatile unsigned long *v)
-{
- unsigned long old, tmp;
-
- asm volatile(
- "0: \n"
- " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */
- " ckeq icc3,cc7 \n"
- " ld.p %M0,%1 \n" /* LD.P/ORCR are atomic */
- " orcr cc7,cc7,cc3 \n" /* set CC3 to true */
- " and%I3 %1,%3,%2 \n"
- " cst.p %2,%M0 ,cc3,#1 \n" /* if store happens... */
- " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* ... clear ICC3.Z */
- " beq icc3,#0,0b \n"
- : "+U"(*v), "=&r"(old), "=r"(tmp)
- : "NPr"(~mask)
- : "memory", "cc7", "cc3", "icc3"
- );
-
- return old;
-}
-
-static inline
-unsigned long atomic_test_and_OR_mask(unsigned long mask, volatile unsigned long *v)
-{
- unsigned long old, tmp;
-
- asm volatile(
- "0: \n"
- " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */
- " ckeq icc3,cc7 \n"
- " ld.p %M0,%1 \n" /* LD.P/ORCR are atomic */
- " orcr cc7,cc7,cc3 \n" /* set CC3 to true */
- " or%I3 %1,%3,%2 \n"
- " cst.p %2,%M0 ,cc3,#1 \n" /* if store happens... */
- " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* ... clear ICC3.Z */
- " beq icc3,#0,0b \n"
- : "+U"(*v), "=&r"(old), "=r"(tmp)
- : "NPr"(mask)
- : "memory", "cc7", "cc3", "icc3"
- );
-
- return old;
-}
-
-static inline
-unsigned long atomic_test_and_XOR_mask(unsigned long mask, volatile unsigned long *v)
-{
- unsigned long old, tmp;
-
- asm volatile(
- "0: \n"
- " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */
- " ckeq icc3,cc7 \n"
- " ld.p %M0,%1 \n" /* LD.P/ORCR are atomic */
- " orcr cc7,cc7,cc3 \n" /* set CC3 to true */
- " xor%I3 %1,%3,%2 \n"
- " cst.p %2,%M0 ,cc3,#1 \n" /* if store happens... */
- " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* ... clear ICC3.Z */
- " beq icc3,#0,0b \n"
- : "+U"(*v), "=&r"(old), "=r"(tmp)
- : "NPr"(mask)
- : "memory", "cc7", "cc3", "icc3"
- );
-
- return old;
-}
-
-#else
-
-extern unsigned long atomic_test_and_ANDNOT_mask(unsigned long mask, volatile unsigned long *v);
-extern unsigned long atomic_test_and_OR_mask(unsigned long mask, volatile unsigned long *v);
-extern unsigned long atomic_test_and_XOR_mask(unsigned long mask, volatile unsigned long *v);
-
-#endif
-
-#define atomic_clear_mask(mask, v) atomic_test_and_ANDNOT_mask((mask), (v))
-#define atomic_set_mask(mask, v) atomic_test_and_OR_mask((mask), (v))
+#include <asm/atomic.h>
static inline int test_and_clear_bit(unsigned long nr, volatile void *addr)
{
- volatile unsigned long *ptr = addr;
- unsigned long mask = 1UL << (nr & 31);
+ unsigned int *ptr = (void *)addr;
+ unsigned int mask = 1UL << (nr & 31);
ptr += nr >> 5;
- return (atomic_test_and_ANDNOT_mask(mask, ptr) & mask) != 0;
+ return (__atomic32_fetch_and(~mask, ptr) & mask) != 0;
}
static inline int test_and_set_bit(unsigned long nr, volatile void *addr)
{
- volatile unsigned long *ptr = addr;
- unsigned long mask = 1UL << (nr & 31);
+ unsigned int *ptr = (void *)addr;
+ unsigned int mask = 1UL << (nr & 31);
ptr += nr >> 5;
- return (atomic_test_and_OR_mask(mask, ptr) & mask) != 0;
+ return (__atomic32_fetch_or(mask, ptr) & mask) != 0;
}
static inline int test_and_change_bit(unsigned long nr, volatile void *addr)
{
- volatile unsigned long *ptr = addr;
- unsigned long mask = 1UL << (nr & 31);
+ unsigned int *ptr = (void *)addr;
+ unsigned int mask = 1UL << (nr & 31);
ptr += nr >> 5;
- return (atomic_test_and_XOR_mask(mask, ptr) & mask) != 0;
+ return (__atomic32_fetch_xor(mask, ptr) & mask) != 0;
}
static inline void clear_bit(unsigned long nr, volatile void *addr)
--- a/arch/frv/kernel/dma.c
+++ b/arch/frv/kernel/dma.c
@@ -109,13 +109,13 @@ static struct frv_dma_channel frv_dma_ch
static DEFINE_RWLOCK(frv_dma_channels_lock);
-unsigned long frv_dma_inprogress;
+unsigned int frv_dma_inprogress;
#define frv_clear_dma_inprogress(channel) \
- atomic_clear_mask(1 << (channel), &frv_dma_inprogress);
+ (void)__atomic32_fetch_and(~(1 << (channel)), &frv_dma_inprogress);
#define frv_set_dma_inprogress(channel) \
- atomic_set_mask(1 << (channel), &frv_dma_inprogress);
+ (void)__atomic32_fetch_or(1 << (channel), &frv_dma_inprogress);
/*****************************************************************************/
/*
--- a/arch/frv/kernel/frv_ksyms.c
+++ b/arch/frv/kernel/frv_ksyms.c
@@ -58,11 +58,6 @@ EXPORT_SYMBOL(__outsl_ns);
EXPORT_SYMBOL(__insl_ns);
#ifdef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS
-EXPORT_SYMBOL(atomic_test_and_ANDNOT_mask);
-EXPORT_SYMBOL(atomic_test_and_OR_mask);
-EXPORT_SYMBOL(atomic_test_and_XOR_mask);
-EXPORT_SYMBOL(atomic_add_return);
-EXPORT_SYMBOL(atomic_sub_return);
EXPORT_SYMBOL(__xchg_32);
EXPORT_SYMBOL(__cmpxchg_32);
#endif
--- a/arch/frv/lib/Makefile
+++ b/arch/frv/lib/Makefile
@@ -5,4 +5,4 @@
lib-y := \
__ashldi3.o __lshrdi3.o __muldi3.o __ashrdi3.o __negdi2.o __ucmpdi2.o \
checksum.o memcpy.o memset.o atomic-ops.o atomic64-ops.o \
- outsl_ns.o outsl_sw.o insl_ns.o insl_sw.o cache.o
+ outsl_ns.o outsl_sw.o insl_ns.o insl_sw.o cache.o atomic-lib.o
--- /dev/null
+++ b/arch/frv/lib/atomic-lib.c
@@ -0,0 +1,7 @@
+
+#include <linux/export.h>
+#include <asm/atomic.h>
+
+#define __ATOMIC_LIB__
+
+#include <asm/atomic_defs.h>
--- a/arch/frv/lib/atomic-ops.S
+++ b/arch/frv/lib/atomic-ops.S
@@ -19,116 +19,6 @@
###############################################################################
#
-# unsigned long atomic_test_and_ANDNOT_mask(unsigned long mask, volatile unsigned long *v);
-#
-###############################################################################
- .globl atomic_test_and_ANDNOT_mask
- .type atomic_test_and_ANDNOT_mask,@function
-atomic_test_and_ANDNOT_mask:
- not.p gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- and gr8,gr10,gr11
- cst.p gr11,@(gr9,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic_test_and_ANDNOT_mask, .-atomic_test_and_ANDNOT_mask
-
-###############################################################################
-#
-# unsigned long atomic_test_and_OR_mask(unsigned long mask, volatile unsigned long *v);
-#
-###############################################################################
- .globl atomic_test_and_OR_mask
- .type atomic_test_and_OR_mask,@function
-atomic_test_and_OR_mask:
- or.p gr8,gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- or gr8,gr10,gr11
- cst.p gr11,@(gr9,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic_test_and_OR_mask, .-atomic_test_and_OR_mask
-
-###############################################################################
-#
-# unsigned long atomic_test_and_XOR_mask(unsigned long mask, volatile unsigned long *v);
-#
-###############################################################################
- .globl atomic_test_and_XOR_mask
- .type atomic_test_and_XOR_mask,@function
-atomic_test_and_XOR_mask:
- or.p gr8,gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- xor gr8,gr10,gr11
- cst.p gr11,@(gr9,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic_test_and_XOR_mask, .-atomic_test_and_XOR_mask
-
-###############################################################################
-#
-# int atomic_add_return(int i, atomic_t *v)
-#
-###############################################################################
- .globl atomic_add_return
- .type atomic_add_return,@function
-atomic_add_return:
- or.p gr8,gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- add gr8,gr10,gr8
- cst.p gr8,@(gr9,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic_add_return, .-atomic_add_return
-
-###############################################################################
-#
-# int atomic_sub_return(int i, atomic_t *v)
-#
-###############################################################################
- .globl atomic_sub_return
- .type atomic_sub_return,@function
-atomic_sub_return:
- or.p gr8,gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- sub gr8,gr10,gr8
- cst.p gr8,@(gr9,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic_sub_return, .-atomic_sub_return
-
-###############################################################################
-#
# uint32_t __xchg_32(uint32_t i, uint32_t *v)
#
###############################################################################
--- a/arch/frv/lib/atomic64-ops.S
+++ b/arch/frv/lib/atomic64-ops.S
@@ -20,100 +20,6 @@
###############################################################################
#
-# long long atomic64_inc_return(atomic64_t *v)
-#
-###############################################################################
- .globl atomic64_inc_return
- .type atomic64_inc_return,@function
-atomic64_inc_return:
- or.p gr8,gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- addicc gr9,#1,gr9,icc0
- addxi gr8,#0,gr8,icc0
- cstd.p gr8,@(gr10,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic64_inc_return, .-atomic64_inc_return
-
-###############################################################################
-#
-# long long atomic64_dec_return(atomic64_t *v)
-#
-###############################################################################
- .globl atomic64_dec_return
- .type atomic64_dec_return,@function
-atomic64_dec_return:
- or.p gr8,gr8,gr10
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- subicc gr9,#1,gr9,icc0
- subxi gr8,#0,gr8,icc0
- cstd.p gr8,@(gr10,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic64_dec_return, .-atomic64_dec_return
-
-###############################################################################
-#
-# long long atomic64_add_return(long long i, atomic64_t *v)
-#
-###############################################################################
- .globl atomic64_add_return
- .type atomic64_add_return,@function
-atomic64_add_return:
- or.p gr8,gr8,gr4
- or gr9,gr9,gr5
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- addcc gr9,gr5,gr9,icc0
- addx gr8,gr4,gr8,icc0
- cstd.p gr8,@(gr10,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic64_add_return, .-atomic64_add_return
-
-###############################################################################
-#
-# long long atomic64_sub_return(long long i, atomic64_t *v)
-#
-###############################################################################
- .globl atomic64_sub_return
- .type atomic64_sub_return,@function
-atomic64_sub_return:
- or.p gr8,gr8,gr4
- or gr9,gr9,gr5
-0:
- orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */
- ckeq icc3,cc7
- ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */
- orcr cc7,cc7,cc3 /* set CC3 to true */
- subcc gr9,gr5,gr9,icc0
- subx gr8,gr4,gr8,icc0
- cstd.p gr8,@(gr10,gr0) ,cc3,#1
- corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */
- beq icc3,#0,0b
- bralr
-
- .size atomic64_sub_return, .-atomic64_sub_return
-
-###############################################################################
-#
# uint64_t __xchg_64(uint64_t i, uint64_t *v)
#
###############################################################################
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 25/28] atomic: Provide atomic_{or,xor,and} |
| Message-ID | <pMVse-5iT-13@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/alpha/include/asm/atomic.h | 1 -
arch/arc/include/asm/atomic.h | 1 -
arch/arm/include/asm/atomic.h | 1 -
arch/arm64/include/asm/atomic.h | 1 -
arch/avr32/include/asm/atomic.h | 2 --
arch/blackfin/include/asm/atomic.h | 2 --
arch/frv/include/asm/atomic.h | 2 --
arch/h8300/include/asm/atomic.h | 2 --
arch/hexagon/include/asm/atomic.h | 2 --
arch/ia64/include/asm/atomic.h | 2 --
arch/m32r/include/asm/atomic.h | 2 --
arch/m68k/include/asm/atomic.h | 2 --
arch/metag/include/asm/atomic_lnkget.h | 2 --
arch/mips/include/asm/atomic.h | 2 --
arch/mn10300/include/asm/atomic.h | 2 --
arch/parisc/include/asm/atomic.h | 2 --
arch/powerpc/include/asm/atomic.h | 2 --
arch/s390/include/asm/atomic.h | 2 --
arch/sh/include/asm/atomic-grb.h | 2 --
arch/sparc/include/asm/atomic_32.h | 2 --
arch/sparc/include/asm/atomic_64.h | 2 --
arch/tile/include/asm/atomic_32.h | 2 --
arch/tile/include/asm/atomic_64.h | 2 --
arch/x86/include/asm/atomic.h | 2 --
arch/xtensa/include/asm/atomic.h | 2 --
include/asm-generic/atomic.h | 21 ++++++++++++---------
include/asm-generic/atomic64.h | 4 ++++
include/linux/atomic.h | 13 -------------
lib/atomic64.c | 3 +++
29 files changed, 19 insertions(+), 68 deletions(-)
--- a/arch/alpha/include/asm/atomic.h
+++ b/arch/alpha/include/asm/atomic.h
@@ -110,7 +110,6 @@ static __inline__ long atomic64_##op##_r
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
#define atomic_andnot atomic_andnot
#define atomic64_andnot atomic64_andnot
--- a/arch/arc/include/asm/atomic.h
+++ b/arch/arc/include/asm/atomic.h
@@ -144,7 +144,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, +=, add)
ATOMIC_OPS(sub, -=, sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
#define atomic_andnot atomic_andnot
ATOMIC_OP(and, &=, and)
--- a/arch/arm/include/asm/atomic.h
+++ b/arch/arm/include/asm/atomic.h
@@ -194,7 +194,6 @@ static inline int __atomic_add_unless(at
ATOMIC_OPS(add, +=, add)
ATOMIC_OPS(sub, -=, sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
#define atomic_andnot atomic_andnot
ATOMIC_OP(and, &=, and)
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -85,7 +85,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, add)
ATOMIC_OPS(sub, sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
#define atomic_andnot atomic_andnot
ATOMIC_OP(and, and)
--- a/arch/avr32/include/asm/atomic.h
+++ b/arch/avr32/include/asm/atomic.h
@@ -51,8 +51,6 @@ static inline void atomic_##op(int i, at
(void)__atomic_##op##_return(i, v); \
}
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, and)
ATOMIC_OP(or, or)
ATOMIC_OP(xor, eor)
--- a/arch/blackfin/include/asm/atomic.h
+++ b/arch/blackfin/include/asm/atomic.h
@@ -28,8 +28,6 @@ asmlinkage int __raw_atomic_test_asm(con
#define atomic_add_return(i, v) __raw_atomic_add_asm(&(v)->counter, i)
#define atomic_sub_return(i, v) __raw_atomic_add_asm(&(v)->counter, -(i))
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
#define atomic_or(i, v) (void)__raw_atomic_or_asm(&(v)->counter, i)
#define atomic_and(i, v) (void)__raw_atomic_and_asm(&(v)->counter, i)
#define atomic_xor(i, v) (void)__raw_atomic_xor_asm(&(v)->counter, i)
--- a/arch/frv/include/asm/atomic.h
+++ b/arch/frv/include/asm/atomic.h
@@ -192,8 +192,6 @@ static inline void atomic64_##op(long lo
(void)__atomic64_fetch_##op(i, &v->counter); \
}
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(or)
ATOMIC_OP(and)
ATOMIC_OP(xor)
--- a/arch/h8300/include/asm/atomic.h
+++ b/arch/h8300/include/asm/atomic.h
@@ -41,8 +41,6 @@ static inline void atomic_##op(int i, at
ATOMIC_OP_RETURN(add, +=)
ATOMIC_OP_RETURN(sub, -=)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, &=)
ATOMIC_OP(or, |=)
ATOMIC_OP(xor, ^=)
--- a/arch/hexagon/include/asm/atomic.h
+++ b/arch/hexagon/include/asm/atomic.h
@@ -132,8 +132,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/ia64/include/asm/atomic.h
+++ b/arch/ia64/include/asm/atomic.h
@@ -69,8 +69,6 @@ ATOMIC_OP(sub, -)
: ia64_atomic_sub(__ia64_asr_i, v); \
})
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, &)
ATOMIC_OP(or, |)
ATOMIC_OP(xor, ^)
--- a/arch/m32r/include/asm/atomic.h
+++ b/arch/m32r/include/asm/atomic.h
@@ -94,8 +94,6 @@ static __inline__ int atomic_##op##_retu
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/m68k/include/asm/atomic.h
+++ b/arch/m68k/include/asm/atomic.h
@@ -77,8 +77,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, +=, add)
ATOMIC_OPS(sub, -=, sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, &=, and)
ATOMIC_OP(or, |=, or)
ATOMIC_OP(xor, ^=, eor)
--- a/arch/metag/include/asm/atomic_lnkget.h
+++ b/arch/metag/include/asm/atomic_lnkget.h
@@ -74,8 +74,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/mips/include/asm/atomic.h
+++ b/arch/mips/include/asm/atomic.h
@@ -137,8 +137,6 @@ static __inline__ int atomic_##op##_retu
ATOMIC_OPS(add, +=, addu)
ATOMIC_OPS(sub, -=, subu)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, &=, and)
ATOMIC_OP(or, |=, or)
ATOMIC_OP(xor, ^=, xor)
--- a/arch/mn10300/include/asm/atomic.h
+++ b/arch/mn10300/include/asm/atomic.h
@@ -89,8 +89,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/parisc/include/asm/atomic.h
+++ b/arch/parisc/include/asm/atomic.h
@@ -126,8 +126,6 @@ static __inline__ int atomic_##op##_retu
ATOMIC_OPS(add, +=)
ATOMIC_OPS(sub, -=)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, &=)
ATOMIC_OP(or, |=)
ATOMIC_OP(xor, ^=)
--- a/arch/powerpc/include/asm/atomic.h
+++ b/arch/powerpc/include/asm/atomic.h
@@ -67,8 +67,6 @@ static __inline__ int atomic_##op##_retu
ATOMIC_OPS(add, add)
ATOMIC_OPS(sub, subf)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and, and)
ATOMIC_OP(or, or)
ATOMIC_OP(xor, xor)
--- a/arch/s390/include/asm/atomic.h
+++ b/arch/s390/include/asm/atomic.h
@@ -282,8 +282,6 @@ static inline void atomic64_##op(long i,
__ATOMIC64_LOOP(v, i, __ATOMIC64_##OP, __ATOMIC64_NO_BARRIER); \
}
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC64_OP(and, AND)
ATOMIC64_OP(or, OR)
ATOMIC64_OP(xor, XOR)
--- a/arch/sh/include/asm/atomic-grb.h
+++ b/arch/sh/include/asm/atomic-grb.h
@@ -48,8 +48,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/sparc/include/asm/atomic_32.h
+++ b/arch/sparc/include/asm/atomic_32.h
@@ -17,8 +17,6 @@
#include <asm/barrier.h>
#include <asm-generic/atomic64.h>
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
#define ATOMIC_INIT(i) { (i) }
int atomic_add_return(int, atomic_t *);
--- a/arch/sparc/include/asm/atomic_64.h
+++ b/arch/sparc/include/asm/atomic_64.h
@@ -33,8 +33,6 @@ long atomic64_##op##_return(long, atomic
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/tile/include/asm/atomic_32.h
+++ b/arch/tile/include/asm/atomic_32.h
@@ -41,8 +41,6 @@ static inline void atomic_##op(int i, at
_atomic_##op((unsigned long *)&v->counter, i); \
}
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/tile/include/asm/atomic_64.h
+++ b/arch/tile/include/asm/atomic_64.h
@@ -58,8 +58,6 @@ static inline int __atomic_add_unless(at
return oldval;
}
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
static inline void atomic_and(int i, atomic_t *v)
{
__insn_fetchand4((void *)&v->counter, i);
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@ -191,8 +191,6 @@ static inline void atomic_##op(int i, at
: "memory"); \
}
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/arch/xtensa/include/asm/atomic.h
+++ b/arch/xtensa/include/asm/atomic.h
@@ -145,8 +145,6 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-
ATOMIC_OP(and)
ATOMIC_OP(or)
ATOMIC_OP(xor)
--- a/include/asm-generic/atomic.h
+++ b/include/asm-generic/atomic.h
@@ -102,24 +102,27 @@ ATOMIC_OP_RETURN(sub, -)
ATOMIC_OP(and, &)
#endif
-#ifndef atomic_clear_mask
-#define atomic_clear_mask(i, v) atomic_and(~(i), (v))
-#endif
-
#ifndef atomic_or
-#ifndef CONFIG_ARCH_HAS_ATOMIC_OR
-#define CONFIG_ARCH_HAS_ATOMIC_OR
-#endif
ATOMIC_OP(or, |)
#endif
-#ifndef atomic_set_mask
-#define atomic_set_mask(i, v) atomic_or((i), (v))
+#ifndef atomic_xor
+ATOMIC_OP(xor, ^)
#endif
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_and(~mask, v);
+}
+
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_or(mask, v);
+}
+
/*
* Atomic operations that C can't guarantee us. Useful for
* resource counting etc..
--- a/include/asm-generic/atomic64.h
+++ b/include/asm-generic/atomic64.h
@@ -32,6 +32,10 @@ extern long long atomic64_##op##_return(
ATOMIC64_OPS(add)
ATOMIC64_OPS(sub)
+ATOMIC64_OP(and)
+ATOMIC64_OP(or)
+ATOMIC64_OP(xor)
+
#undef ATOMIC64_OPS
#undef ATOMIC64_OP_RETURN
#undef ATOMIC64_OP
--- a/include/linux/atomic.h
+++ b/include/linux/atomic.h
@@ -111,19 +111,6 @@ static inline int atomic_dec_if_positive
}
#endif
-#ifndef CONFIG_ARCH_HAS_ATOMIC_OR
-static inline void atomic_or(int i, atomic_t *v)
-{
- int old;
- int new;
-
- do {
- old = atomic_read(v);
- new = old | i;
- } while (atomic_cmpxchg(v, old, new) != old);
-}
-#endif /* #ifndef CONFIG_ARCH_HAS_ATOMIC_OR */
-
#include <asm-generic/atomic-long.h>
#ifdef CONFIG_GENERIC_ATOMIC64
#include <asm-generic/atomic64.h>
--- a/lib/atomic64.c
+++ b/lib/atomic64.c
@@ -102,6 +102,9 @@ EXPORT_SYMBOL(atomic64_##op##_return);
ATOMIC64_OPS(add, +=)
ATOMIC64_OPS(sub, -=)
+ATOMIC64_OP(and, &=)
+ATOMIC64_OP(or, |=)
+ATOMIC64_OP(xor, ^=)
#undef ATOMIC64_OPS
#undef ATOMIC64_OP_RETURN
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 05/28] arm64: Provide atomic_{or,xor,and} |
| Message-ID | <pMVse-5iT-17@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/arm64/include/asm/atomic.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
--- a/arch/arm64/include/asm/atomic.h
+++ b/arch/arm64/include/asm/atomic.h
@@ -85,6 +85,14 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, add)
ATOMIC_OPS(sub, sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+#define atomic_andnot atomic_andnot
+
+ATOMIC_OP(and, and)
+ATOMIC_OP(andnot, bic)
+ATOMIC_OP(or, orr)
+ATOMIC_OP(xor, eor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
@@ -183,6 +191,13 @@ static inline long atomic64_##op##_retur
ATOMIC64_OPS(add, add)
ATOMIC64_OPS(sub, sub)
+#define atomic64_andnot atomic64_andnot
+
+ATOMIC64_OP(and, and)
+ATOMIC64_OP(andnot, bic)
+ATOMIC64_OP(or, orr)
+ATOMIC64_OP(xor, eor)
+
#undef ATOMIC64_OPS
#undef ATOMIC64_OP_RETURN
#undef ATOMIC64_OP
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 13/28] mips: Provide atomic_{or,xor,and} |
| Message-ID | <pMVse-5iT-19@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Acked-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/mips/include/asm/atomic.h | 9 +++++++++
1 file changed, 9 insertions(+)
--- a/arch/mips/include/asm/atomic.h
+++ b/arch/mips/include/asm/atomic.h
@@ -137,6 +137,12 @@ static __inline__ int atomic_##op##_retu
ATOMIC_OPS(add, +=, addu)
ATOMIC_OPS(sub, -=, subu)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and, &=, and)
+ATOMIC_OP(or, |=, or)
+ATOMIC_OP(xor, ^=, xor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
@@ -416,6 +422,9 @@ static __inline__ long atomic64_##op##_r
ATOMIC64_OPS(add, +=, daddu)
ATOMIC64_OPS(sub, -=, dsubu)
+ATOMIC64_OP(and, &=, and)
+ATOMIC64_OP(or, |=, or)
+ATOMIC64_OP(xor, ^=, xor)
#undef ATOMIC64_OPS
#undef ATOMIC64_OP_RETURN
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 14/28] mn10300: Provide atomic_{or,xor,and} |
| Message-ID | <pMVse-5iT-23@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/mn10300/include/asm/atomic.h | 57 ++++++--------------------------------
1 file changed, 10 insertions(+), 47 deletions(-)
--- a/arch/mn10300/include/asm/atomic.h
+++ b/arch/mn10300/include/asm/atomic.h
@@ -89,6 +89,12 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
@@ -134,31 +140,9 @@ static inline void atomic_dec(atomic_t *
*
* Atomically clears the bits set in mask from the memory word specified.
*/
-static inline void atomic_clear_mask(unsigned long mask, unsigned long *addr)
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
{
-#ifdef CONFIG_SMP
- int status;
-
- asm volatile(
- "1: mov %3,(_AAR,%2) \n"
- " mov (_ADR,%2),%0 \n"
- " and %4,%0 \n"
- " mov %0,(_ADR,%2) \n"
- " mov (_ADR,%2),%0 \n" /* flush */
- " mov (_ASR,%2),%0 \n"
- " or %0,%0 \n"
- " bne 1b \n"
- : "=&r"(status), "=m"(*addr)
- : "a"(ATOMIC_OPS_BASE_ADDR), "r"(addr), "r"(~mask)
- : "memory", "cc");
-#else
- unsigned long flags;
-
- mask = ~mask;
- flags = arch_local_cli_save();
- *addr &= mask;
- arch_local_irq_restore(flags);
-#endif
+ atomic_and(~mask, v);
}
/**
@@ -168,30 +152,9 @@ static inline void atomic_clear_mask(uns
*
* Atomically sets the bits set in mask from the memory word specified.
*/
-static inline void atomic_set_mask(unsigned long mask, unsigned long *addr)
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
{
-#ifdef CONFIG_SMP
- int status;
-
- asm volatile(
- "1: mov %3,(_AAR,%2) \n"
- " mov (_ADR,%2),%0 \n"
- " or %4,%0 \n"
- " mov %0,(_ADR,%2) \n"
- " mov (_ADR,%2),%0 \n" /* flush */
- " mov (_ASR,%2),%0 \n"
- " or %0,%0 \n"
- " bne 1b \n"
- : "=&r"(status), "=m"(*addr)
- : "a"(ATOMIC_OPS_BASE_ADDR), "r"(addr), "r"(mask)
- : "memory", "cc");
-#else
- unsigned long flags;
-
- flags = arch_local_cli_save();
- *addr |= mask;
- arch_local_irq_restore(flags);
-#endif
+ atomic_or(mask, v);
}
#endif /* __KERNEL__ */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 12/28] metag: Provide atomic_{or,xor,and} |
| Message-ID | <pMVse-5iT-27@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/metag/include/asm/atomic_lnkget.h | 38 ++++++++-------------------------
arch/metag/include/asm/atomic_lock1.h | 21 ++++++------------
2 files changed, 17 insertions(+), 42 deletions(-)
--- a/arch/metag/include/asm/atomic_lnkget.h
+++ b/arch/metag/include/asm/atomic_lnkget.h
@@ -74,42 +74,24 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
{
- int temp;
-
- asm volatile (
- "1: LNKGETD %0, [%1]\n"
- " AND %0, %0, %2\n"
- " LNKSETD [%1] %0\n"
- " DEFR %0, TXSTAT\n"
- " ANDT %0, %0, #HI(0x3f000000)\n"
- " CMPT %0, #HI(0x02000000)\n"
- " BNZ 1b\n"
- : "=&d" (temp)
- : "da" (&v->counter), "bd" (~mask)
- : "cc");
+ atomic_and(~mask, v);
}
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
{
- int temp;
-
- asm volatile (
- "1: LNKGETD %0, [%1]\n"
- " OR %0, %0, %2\n"
- " LNKSETD [%1], %0\n"
- " DEFR %0, TXSTAT\n"
- " ANDT %0, %0, #HI(0x3f000000)\n"
- " CMPT %0, #HI(0x02000000)\n"
- " BNZ 1b\n"
- : "=&d" (temp)
- : "da" (&v->counter), "bd" (mask)
- : "cc");
+ atomic_or(mask, v);
}
static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
--- a/arch/metag/include/asm/atomic_lock1.h
+++ b/arch/metag/include/asm/atomic_lock1.h
@@ -68,29 +68,22 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, +=)
ATOMIC_OPS(sub, -=)
+ATOMIC_OP(and, &=)
+ATOMIC_OP(or, |=)
+ATOMIC_OP(xor, ^=)
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
{
- unsigned long flags;
-
- __global_lock1(flags);
- fence();
- v->counter &= ~mask;
- __global_unlock1(flags);
+ atomic_and(~mask, v);
}
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
{
- unsigned long flags;
-
- __global_lock1(flags);
- fence();
- v->counter |= mask;
- __global_unlock1(flags);
+ atomic_or(mask, v);
}
static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 26/28] atomic: Collapse all atomic_{set,clear}_mask definitions |
| Message-ID | <pMVsf-5iT-31@gated-at.bofh.it> |
| In reply to | #1186130 |
Move the now generic definitions of atomic_{set,clear}_mask() into
linux/atomic.h to avoid endless and pointless repetition.
Also, provide an atomic_andnot() wrapper for those few archs that can
implement that.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/arc/include/asm/atomic.h | 10 ----------
arch/blackfin/include/asm/atomic.h | 10 ----------
arch/frv/include/asm/atomic.h | 10 ----------
arch/h8300/include/asm/atomic.h | 10 ----------
arch/m32r/include/asm/atomic.h | 11 -----------
arch/m68k/include/asm/atomic.h | 10 ----------
arch/metag/include/asm/atomic_lnkget.h | 10 ----------
arch/metag/include/asm/atomic_lock1.h | 10 ----------
arch/mn10300/include/asm/atomic.h | 24 ------------------------
arch/powerpc/kernel/misc_32.S | 19 -------------------
arch/s390/include/asm/atomic.h | 10 ----------
arch/sh/include/asm/atomic.h | 10 ----------
arch/x86/include/asm/atomic.h | 10 ----------
arch/xtensa/include/asm/atomic.h | 10 ----------
include/asm-generic/atomic.h | 10 ----------
include/linux/atomic.h | 25 +++++++++++++++++++++++++
16 files changed, 25 insertions(+), 174 deletions(-)
--- a/arch/arc/include/asm/atomic.h
+++ b/arch/arc/include/asm/atomic.h
@@ -155,16 +155,6 @@ ATOMIC_OP(xor, ^=, xor)
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
/**
* __atomic_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
--- a/arch/blackfin/include/asm/atomic.h
+++ b/arch/blackfin/include/asm/atomic.h
@@ -32,16 +32,6 @@ asmlinkage int __raw_atomic_test_asm(con
#define atomic_and(i, v) (void)__raw_atomic_and_asm(&(v)->counter, i)
#define atomic_xor(i, v) (void)__raw_atomic_xor_asm(&(v)->counter, i)
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#endif
#include <asm-generic/atomic.h>
--- a/arch/frv/include/asm/atomic.h
+++ b/arch/frv/include/asm/atomic.h
@@ -198,14 +198,4 @@ ATOMIC_OP(xor)
#undef ATOMIC_OP
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#endif /* _ASM_ATOMIC_H */
--- a/arch/h8300/include/asm/atomic.h
+++ b/arch/h8300/include/asm/atomic.h
@@ -89,14 +89,4 @@ static inline int __atomic_add_unless(at
return ret;
}
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#endif /* __ARCH_H8300_ATOMIC __ */
--- a/arch/m32r/include/asm/atomic.h
+++ b/arch/m32r/include/asm/atomic.h
@@ -243,15 +243,4 @@ static __inline__ int __atomic_add_unles
return c;
}
-
-static __inline__ __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static __inline__ __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#endif /* _ASM_M32R_ATOMIC_H */
--- a/arch/m68k/include/asm/atomic.h
+++ b/arch/m68k/include/asm/atomic.h
@@ -174,16 +174,6 @@ static inline int atomic_add_negative(in
return c != 0;
}
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
{
int c, old;
--- a/arch/metag/include/asm/atomic_lnkget.h
+++ b/arch/metag/include/asm/atomic_lnkget.h
@@ -82,16 +82,6 @@ ATOMIC_OP(xor)
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
{
int result, temp;
--- a/arch/metag/include/asm/atomic_lock1.h
+++ b/arch/metag/include/asm/atomic_lock1.h
@@ -76,16 +76,6 @@ ATOMIC_OP(xor, ^=)
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
{
int ret;
--- a/arch/mn10300/include/asm/atomic.h
+++ b/arch/mn10300/include/asm/atomic.h
@@ -131,30 +131,6 @@ static inline void atomic_dec(atomic_t *
#define atomic_xchg(ptr, v) (xchg(&(ptr)->counter, (v)))
#define atomic_cmpxchg(v, old, new) (cmpxchg(&((v)->counter), (old), (new)))
-/**
- * atomic_clear_mask - Atomically clear bits in memory
- * @mask: Mask of the bits to be cleared
- * @v: pointer to word in memory
- *
- * Atomically clears the bits set in mask from the memory word specified.
- */
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-/**
- * atomic_set_mask - Atomically set bits in memory
- * @mask: Mask of the bits to be set
- * @v: pointer to word in memory
- *
- * Atomically sets the bits set in mask from the memory word specified.
- */
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#endif /* __KERNEL__ */
#endif /* CONFIG_SMP */
#endif /* _ASM_ATOMIC_H */
--- a/arch/powerpc/kernel/misc_32.S
+++ b/arch/powerpc/kernel/misc_32.S
@@ -596,25 +596,6 @@ _GLOBAL(copy_page)
b 2b
/*
- * void atomic_clear_mask(atomic_t mask, atomic_t *addr)
- * void atomic_set_mask(atomic_t mask, atomic_t *addr);
- */
-_GLOBAL(atomic_clear_mask)
-10: lwarx r5,0,r4
- andc r5,r5,r3
- PPC405_ERR77(0,r4)
- stwcx. r5,0,r4
- bne- 10b
- blr
-_GLOBAL(atomic_set_mask)
-10: lwarx r5,0,r4
- or r5,r5,r3
- PPC405_ERR77(0,r4)
- stwcx. r5,0,r4
- bne- 10b
- blr
-
-/*
* Extended precision shifts.
*
* Updated to be valid for shift counts from 0 to 63 inclusive.
--- a/arch/s390/include/asm/atomic.h
+++ b/arch/s390/include/asm/atomic.h
@@ -132,16 +132,6 @@ ATOMIC_OP(xor, XOR)
#undef ATOMIC_OP
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
--- a/arch/sh/include/asm/atomic.h
+++ b/arch/sh/include/asm/atomic.h
@@ -25,16 +25,6 @@
#include <asm/atomic-irq.h>
#endif
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#define atomic_add_negative(a, v) (atomic_add_return((a), (v)) < 0)
#define atomic_dec_return(v) atomic_sub_return(1, (v))
#define atomic_inc_return(v) atomic_add_return(1, (v))
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@ -234,16 +234,6 @@ static __always_inline short int atomic_
return *v;
}
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
#ifdef CONFIG_X86_32
# include <asm/atomic64_32.h>
#else
--- a/arch/xtensa/include/asm/atomic.h
+++ b/arch/xtensa/include/asm/atomic.h
@@ -153,16 +153,6 @@ ATOMIC_OP(xor)
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
/**
* atomic_sub_and_test - subtract value from variable and test result
* @i: integer value to subtract
--- a/include/asm-generic/atomic.h
+++ b/include/asm-generic/atomic.h
@@ -113,16 +113,6 @@ ATOMIC_OP(xor, ^)
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- atomic_and(~mask, v);
-}
-
-static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- atomic_or(mask, v);
-}
-
/*
* Atomic operations that C can't guarantee us. Useful for
* resource counting etc..
--- a/include/linux/atomic.h
+++ b/include/linux/atomic.h
@@ -28,6 +28,23 @@ static inline int atomic_add_unless(atom
#define atomic_inc_not_zero(v) atomic_add_unless((v), 1, 0)
#endif
+#ifndef atomic_andnot
+static inline void atomic_andnot(int i, atomic_t *v)
+{
+ atomic_and(~i, v);
+}
+#endif
+
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_andnot(mask, v);
+}
+
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_or(mask, v);
+}
+
/**
* atomic_inc_not_zero_hint - increment if not null
* @v: pointer of type atomic_t
@@ -115,4 +132,12 @@ static inline int atomic_dec_if_positive
#ifdef CONFIG_GENERIC_ATOMIC64
#include <asm-generic/atomic64.h>
#endif
+
+#ifndef atomic64_andnot
+static inline void atomic64_andnot(long long i, atomic64_t *v)
+{
+ atomic64_and(~i, v);
+}
+#endif
+
#endif /* _LINUX_ATOMIC_H */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 04/28] arm: Provide atomic_{or,xor,and} |
| Message-ID | <pMVsf-5iT-35@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/arm/include/asm/atomic.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
--- a/arch/arm/include/asm/atomic.h
+++ b/arch/arm/include/asm/atomic.h
@@ -194,6 +194,14 @@ static inline int __atomic_add_unless(at
ATOMIC_OPS(add, +=, add)
ATOMIC_OPS(sub, -=, sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+#define atomic_andnot atomic_andnot
+
+ATOMIC_OP(and, &=, and)
+ATOMIC_OP(andnot, &= ~, bic)
+ATOMIC_OP(or, |=, orr)
+ATOMIC_OP(xor, ^=, eor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
@@ -321,6 +329,13 @@ static inline long long atomic64_##op##_
ATOMIC64_OPS(add, adds, adc)
ATOMIC64_OPS(sub, subs, sbc)
+#define atomic64_andnot atomic64_andnot
+
+ATOMIC64_OP(and, and, and)
+ATOMIC64_OP(andnot, bic, bic)
+ATOMIC64_OP(or, orr, orr)
+ATOMIC64_OP(xor, eor, eor)
+
#undef ATOMIC64_OPS
#undef ATOMIC64_OP_RETURN
#undef ATOMIC64_OP
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 08/28] hexagon: Provide atomic_{or,xor,and} |
| Message-ID | <pMVsf-5iT-39@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/hexagon/include/asm/atomic.h | 6 ++++++
1 file changed, 6 insertions(+)
--- a/arch/hexagon/include/asm/atomic.h
+++ b/arch/hexagon/include/asm/atomic.h
@@ -132,6 +132,12 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 11/28] m68k: Provide atomic_{or,xor,and} |
| Message-ID | <pMVsf-5iT-43@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/m68k/include/asm/atomic.h | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
--- a/arch/m68k/include/asm/atomic.h
+++ b/arch/m68k/include/asm/atomic.h
@@ -77,6 +77,12 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, +=, add)
ATOMIC_OPS(sub, -=, sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and, &=, and)
+ATOMIC_OP(or, |=, or)
+ATOMIC_OP(xor, ^=, eor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
@@ -170,14 +176,14 @@ static inline int atomic_add_negative(in
return c != 0;
}
-static inline void atomic_clear_mask(unsigned long mask, unsigned long *v)
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
{
- __asm__ __volatile__("andl %1,%0" : "+m" (*v) : ASM_DI (~(mask)));
+ atomic_and(~mask, v);
}
-static inline void atomic_set_mask(unsigned long mask, unsigned long *v)
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
{
- __asm__ __volatile__("orl %1,%0" : "+m" (*v) : ASM_DI (mask));
+ atomic_or(mask, v);
}
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 17/28] sh: Provide atomic_{or,xor,and} |
| Message-ID | <pMVsf-5iT-53@gated-at.bofh.it> |
| In reply to | #1186130 |
Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/sh/include/asm/atomic-grb.h | 45 +++++---------------------------------
arch/sh/include/asm/atomic-irq.h | 21 ++---------------
arch/sh/include/asm/atomic-llsc.h | 31 ++------------------------
arch/sh/include/asm/atomic.h | 10 ++++++++
4 files changed, 22 insertions(+), 85 deletions(-)
--- a/arch/sh/include/asm/atomic-grb.h
+++ b/arch/sh/include/asm/atomic-grb.h
@@ -48,47 +48,14 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
+
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- int tmp;
- unsigned int _mask = ~mask;
-
- __asm__ __volatile__ (
- " .align 2 \n\t"
- " mova 1f, r0 \n\t" /* r0 = end point */
- " mov r15, r1 \n\t" /* r1 = saved sp */
- " mov #-6, r15 \n\t" /* LOGIN: r15 = size */
- " mov.l @%1, %0 \n\t" /* load old value */
- " and %2, %0 \n\t" /* add */
- " mov.l %0, @%1 \n\t" /* store new value */
- "1: mov r1, r15 \n\t" /* LOGOUT */
- : "=&r" (tmp),
- "+r" (v)
- : "r" (_mask)
- : "memory" , "r0", "r1");
-}
-
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- int tmp;
-
- __asm__ __volatile__ (
- " .align 2 \n\t"
- " mova 1f, r0 \n\t" /* r0 = end point */
- " mov r15, r1 \n\t" /* r1 = saved sp */
- " mov #-6, r15 \n\t" /* LOGIN: r15 = size */
- " mov.l @%1, %0 \n\t" /* load old value */
- " or %2, %0 \n\t" /* or */
- " mov.l %0, @%1 \n\t" /* store new value */
- "1: mov r1, r15 \n\t" /* LOGOUT */
- : "=&r" (tmp),
- "+r" (v)
- : "r" (mask)
- : "memory" , "r0", "r1");
-}
-
#endif /* __ASM_SH_ATOMIC_GRB_H */
--- a/arch/sh/include/asm/atomic-irq.h
+++ b/arch/sh/include/asm/atomic-irq.h
@@ -37,27 +37,12 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add, +=)
ATOMIC_OPS(sub, -=)
+ATOMIC_OP(and, &=)
+ATOMIC_OP(or, |=)
+ATOMIC_OP(xor, ^=)
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- unsigned long flags;
-
- raw_local_irq_save(flags);
- v->counter &= ~mask;
- raw_local_irq_restore(flags);
-}
-
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- unsigned long flags;
-
- raw_local_irq_save(flags);
- v->counter |= mask;
- raw_local_irq_restore(flags);
-}
-
#endif /* __ASM_SH_ATOMIC_IRQ_H */
--- a/arch/sh/include/asm/atomic-llsc.h
+++ b/arch/sh/include/asm/atomic-llsc.h
@@ -52,37 +52,12 @@ static inline int atomic_##op##_return(i
ATOMIC_OPS(add)
ATOMIC_OPS(sub)
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
#undef ATOMIC_OPS
#undef ATOMIC_OP_RETURN
#undef ATOMIC_OP
-static inline void atomic_clear_mask(unsigned int mask, atomic_t *v)
-{
- unsigned long tmp;
-
- __asm__ __volatile__ (
-"1: movli.l @%2, %0 ! atomic_clear_mask \n"
-" and %1, %0 \n"
-" movco.l %0, @%2 \n"
-" bf 1b \n"
- : "=&z" (tmp)
- : "r" (~mask), "r" (&v->counter)
- : "t");
-}
-
-static inline void atomic_set_mask(unsigned int mask, atomic_t *v)
-{
- unsigned long tmp;
-
- __asm__ __volatile__ (
-"1: movli.l @%2, %0 ! atomic_set_mask \n"
-" or %1, %0 \n"
-" movco.l %0, @%2 \n"
-" bf 1b \n"
- : "=&z" (tmp)
- : "r" (mask), "r" (&v->counter)
- : "t");
-}
-
#endif /* __ASM_SH_ATOMIC_LLSC_H */
--- a/arch/sh/include/asm/atomic.h
+++ b/arch/sh/include/asm/atomic.h
@@ -25,6 +25,16 @@
#include <asm/atomic-irq.h>
#endif
+static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_and(~mask, v);
+}
+
+static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v)
+{
+ atomic_or(mask, v);
+}
+
#define atomic_add_negative(a, v) (atomic_add_return((a), (v)) < 0)
#define atomic_dec_return(v) atomic_sub_return(1, (v))
#define atomic_inc_return(v) atomic_add_return(1, (v))
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Date | 2015-07-16 19:50 +0200 |
| Subject | [PATCH 24/28] tile: Provide atomic_{or,xor,and} |
| Message-ID | <pMVsg-5iT-57@gated-at.bofh.it> |
| In reply to | #1186130 |
From: Chris Metcalf <cmetcalf@ezchip.com>
Implement atomic logic ops -- atomic_{or,xor,and}.
For tilegx, these are relatively straightforward; the architecture
provides atomic "or" and "and", both 32-bit and 64-bit. To support
xor we provide a loop using "cmpexch".
For the older 32-bit tilepro architecture, we have to extend
the set of low-level assembly routines to include 32-bit "and",
as well as all three 64-bit routines. Somewhat confusingly,
some 32-bit versions are already used by the bitops inlines, with
parameter types appropriate for bitops, so we have to do a bit of
casting to match "int" to "unsigned long".
Signed-off-by: Chris Metcalf <cmetcalf@ezchip.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1436474297-32187-1-git-send-email-cmetcalf@ezchip.com
---
arch/tile/include/asm/atomic_32.h | 30 +++++++++++++++++++++++++++
arch/tile/include/asm/atomic_64.h | 42 ++++++++++++++++++++++++++++++++++++++
arch/tile/lib/atomic_32.c | 23 ++++++++++++++++++++
arch/tile/lib/atomic_asm_32.S | 4 +++
4 files changed, 99 insertions(+)
--- a/arch/tile/include/asm/atomic_32.h
+++ b/arch/tile/include/asm/atomic_32.h
@@ -34,6 +34,21 @@ static inline void atomic_add(int i, ato
_atomic_xchg_add(&v->counter, i);
}
+#define ATOMIC_OP(op) \
+unsigned long _atomic_##op(volatile unsigned long *p, unsigned long mask); \
+static inline void atomic_##op(int i, atomic_t *v) \
+{ \
+ _atomic_##op((unsigned long *)&v->counter, i); \
+}
+
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+ATOMIC_OP(and)
+ATOMIC_OP(or)
+ATOMIC_OP(xor)
+
+#undef ATOMIC_OP
+
/**
* atomic_add_return - add integer and return
* @v: pointer of type atomic_t
@@ -113,6 +128,17 @@ static inline void atomic64_add(long lon
_atomic64_xchg_add(&v->counter, i);
}
+#define ATOMIC64_OP(op) \
+long long _atomic64_##op(long long *v, long long n); \
+static inline void atomic64_##op(long long i, atomic64_t *v) \
+{ \
+ _atomic64_##op(&v->counter, i); \
+}
+
+ATOMIC64_OP(and)
+ATOMIC64_OP(or)
+ATOMIC64_OP(xor)
+
/**
* atomic64_add_return - add integer and return
* @v: pointer of type atomic64_t
@@ -225,6 +251,7 @@ extern struct __get_user __atomic_xchg_a
extern struct __get_user __atomic_xchg_add_unless(volatile int *p,
int *lock, int o, int n);
extern struct __get_user __atomic_or(volatile int *p, int *lock, int n);
+extern struct __get_user __atomic_and(volatile int *p, int *lock, int n);
extern struct __get_user __atomic_andn(volatile int *p, int *lock, int n);
extern struct __get_user __atomic_xor(volatile int *p, int *lock, int n);
extern long long __atomic64_cmpxchg(volatile long long *p, int *lock,
@@ -234,6 +261,9 @@ extern long long __atomic64_xchg_add(vol
long long n);
extern long long __atomic64_xchg_add_unless(volatile long long *p,
int *lock, long long o, long long n);
+extern long long __atomic64_and(volatile long long *p, int *lock, long long n);
+extern long long __atomic64_or(volatile long long *p, int *lock, long long n);
+extern long long __atomic64_xor(volatile long long *p, int *lock, long long n);
/* Return failure from the atomic wrappers. */
struct __get_user __atomic_bad_address(int __user *addr);
--- a/arch/tile/include/asm/atomic_64.h
+++ b/arch/tile/include/asm/atomic_64.h
@@ -58,6 +58,28 @@ static inline int __atomic_add_unless(at
return oldval;
}
+#define CONFIG_ARCH_HAS_ATOMIC_OR
+
+static inline void atomic_and(int i, atomic_t *v)
+{
+ __insn_fetchand4((void *)&v->counter, i);
+}
+
+static inline void atomic_or(int i, atomic_t *v)
+{
+ __insn_fetchor4((void *)&v->counter, i);
+}
+
+static inline void atomic_xor(int i, atomic_t *v)
+{
+ int guess, oldval = v->counter;
+ do {
+ guess = oldval;
+ __insn_mtspr(SPR_CMPEXCH_VALUE, guess);
+ oldval = __insn_cmpexch4(&v->counter, guess ^ i);
+ } while (guess != oldval);
+}
+
/* Now the true 64-bit operations. */
#define ATOMIC64_INIT(i) { (i) }
@@ -91,6 +113,26 @@ static inline long atomic64_add_unless(a
return oldval != u;
}
+static inline void atomic64_and(long i, atomic64_t *v)
+{
+ __insn_fetchand((void *)&v->counter, i);
+}
+
+static inline void atomic64_or(long i, atomic64_t *v)
+{
+ __insn_fetchor((void *)&v->counter, i);
+}
+
+static inline void atomic64_xor(long i, atomic64_t *v)
+{
+ long guess, oldval = v->counter;
+ do {
+ guess = oldval;
+ __insn_mtspr(SPR_CMPEXCH_VALUE, guess);
+ oldval = __insn_cmpexch(&v->counter, guess ^ i);
+ } while (guess != oldval);
+}
+
#define atomic64_sub_return(i, v) atomic64_add_return(-(i), (v))
#define atomic64_sub(i, v) atomic64_add(-(i), (v))
#define atomic64_inc_return(v) atomic64_add_return(1, (v))
--- a/arch/tile/lib/atomic_32.c
+++ b/arch/tile/lib/atomic_32.c
@@ -94,6 +94,12 @@ unsigned long _atomic_or(volatile unsign
}
EXPORT_SYMBOL(_atomic_or);
+unsigned long _atomic_and(volatile unsigned long *p, unsigned long mask)
+{
+ return __atomic_and((int *)p, __atomic_setup(p), mask).val;
+}
+EXPORT_SYMBOL(_atomic_and);
+
unsigned long _atomic_andn(volatile unsigned long *p, unsigned long mask)
{
return __atomic_andn((int *)p, __atomic_setup(p), mask).val;
@@ -136,6 +142,23 @@ long long _atomic64_cmpxchg(long long *v
}
EXPORT_SYMBOL(_atomic64_cmpxchg);
+long long _atomic64_and(long long *v, long long n)
+{
+ return __atomic64_and(v, __atomic_setup(v), n);
+}
+EXPORT_SYMBOL(_atomic64_and);
+
+long long _atomic64_or(long long *v, long long n)
+{
+ return __atomic64_or(v, __atomic_setup(v), n);
+}
+EXPORT_SYMBOL(_atomic64_or);
+
+long long _atomic64_xor(long long *v, long long n)
+{
+ return __atomic64_xor(v, __atomic_setup(v), n);
+}
+EXPORT_SYMBOL(_atomic64_xor);
/*
* If any of the atomic or futex routines hit a bad address (not in
--- a/arch/tile/lib/atomic_asm_32.S
+++ b/arch/tile/lib/atomic_asm_32.S
@@ -178,6 +178,7 @@ atomic_op _xchg_add, 32, "add r24, r22,
atomic_op _xchg_add_unless, 32, \
"sne r26, r22, r2; { bbns r26, 3f; add r24, r22, r3 }"
atomic_op _or, 32, "or r24, r22, r2"
+atomic_op _and, 32, "and r24, r22, r2"
atomic_op _andn, 32, "nor r2, r2, zero; and r24, r22, r2"
atomic_op _xor, 32, "xor r24, r22, r2"
@@ -191,6 +192,9 @@ atomic_op 64_xchg_add_unless, 64, \
{ bbns r26, 3f; add r24, r22, r4 }; \
{ bbns r27, 3f; add r25, r23, r5 }; \
slt_u r26, r24, r22; add r25, r25, r26"
+atomic_op 64_or, 64, "{ or r24, r22, r2; or r25, r23, r3 }"
+atomic_op 64_and, 64, "{ and r24, r22, r2; and r25, r23, r3 }"
+atomic_op 64_xor, 64, "{ xor r24, r22, r2; xor r25, r23, r3 }"
jrp lr /* happy backtracer */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Date | 2015-07-17 15:00 +0200 |
| Message-ID | <pNdp9-5Tq-35@gated-at.bofh.it> |
| In reply to | #1186130 |
On Thu, Jul 16, 2015 at 06:21:10PM +0100, Peter Zijlstra wrote:
> Currently there is an incoherent mess of atomic_{set,clear}_mask() and
> atomic_or() (but no atomic_{and,nand,xor}()) in the tree.
>
> Those archs that implement atomic_{set,clear}_mask() are not even consistent on
> its signature.
>
> Implement atomic_{or,and,xor}() on all archs and deprecate
> atomic_{set,clear}_mask().
>
> The series has been compile tested by the build-bot, no known failures at this time.
>
> I would like to take this through tip; I'll ask Ingo to put it in a separate
> branch such that people can pull in that branch if there is conflicting work
> elsewhere (Will has some for Argh64).
Cheers, Peter. FWIW, the ARM/arm64 bits look fine to me:
Acked-by: Will Deacon <will.deacon@arm.com>
Will
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web