Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1451335 > unrolled thread

Re: [RFC PATCH v7 1/7] Restartable sequences system call

Started byBoqun Feng <boqun.feng@gmail.com>
First post2016-07-27 17:10 +0200
Last post2016-07-28 05:20 +0200
Articles 12 — 2 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC PATCH v7 1/7] Restartable sequences system call Boqun Feng <boqun.feng@gmail.com> - 2016-07-27 17:10 +0200
    [RFC 4/4] Restartable sequences: Add self-tests for PPC Boqun Feng <boqun.feng@gmail.com> - 2016-07-27 17:10 +0200
      Re: [RFC 4/4] Restartable sequences: Add self-tests for PPC Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-07-28 05:10 +0200
        Re: [RFC 4/4] Restartable sequences: Add self-tests for PPC Boqun Feng <boqun.feng@gmail.com> - 2016-07-28 06:50 +0200
          [RFC v2] Restartable sequences: Add self-tests for PPC Boqun Feng <boqun.feng@gmail.com> - 2016-07-28 09:40 +0200
            Re: [RFC v2] Restartable sequences: Add self-tests for PPC Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-07-28 16:10 +0200
          Re: [RFC 4/4] Restartable sequences: Add self-tests for PPC Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-07-28 15:50 +0200
    [RFC 1/4] rseq/param_test: Convert test_data_entry::count to intptr_t Boqun Feng <boqun.feng@gmail.com> - 2016-07-27 17:10 +0200
      [RFC 3/4] Restartable sequences: Wire up powerpc system call Boqun Feng <boqun.feng@gmail.com> - 2016-07-27 17:10 +0200
        Re: [RFC 3/4] Restartable sequences: Wire up powerpc system call Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-07-28 05:20 +0200
      Re: [RFC 1/4] rseq/param_test: Convert test_data_entry::count to  intptr_t Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-07-28 05:10 +0200
    Re: [RFC PATCH v7 1/7] Restartable sequences system call Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-07-28 05:20 +0200

#1451335 — Re: [RFC PATCH v7 1/7] Restartable sequences system call

FromBoqun Feng <boqun.feng@gmail.com>
Date2016-07-27 17:10 +0200
SubjectRe: [RFC PATCH v7 1/7] Restartable sequences system call
Message-ID<rZyD7-12s-15@gated-at.bofh.it>

[Multipart message — attachments visible in raw view] — view raw

Hi Mathieu,

On Thu, Jul 21, 2016 at 05:14:16PM -0400, Mathieu Desnoyers wrote:
> Expose a new system call allowing each thread to register one userspace
> memory area to be used as an ABI between kernel and user-space for two
> purposes: user-space restartable sequences and quick access to read the
> current CPU number value from user-space.
> 
> * Restartable sequences (per-cpu atomics)
> 
> The restartable critical sections (percpu atomics) work has been started
> by Paul Turner and Andrew Hunter. It lets the kernel handle restart of
> critical sections. [1] [2] The re-implementation proposed here brings a
> few simplifications to the ABI which facilitates porting to other

Agreed ;-)

> architectures and speeds up the user-space fast path. A locking-based
> fall-back, purely implemented in user-space, is proposed here to deal
> with debugger single-stepping. This fallback interacts with rseq_start()
> and rseq_finish(), which force retries in response to concurrent
> lock-based activity.
> 

So I have enabled this on powerpc, thanks to your nice work to make
things easy for porting ;-)

A patchset will follow in-reply-to this email, which includes patches
enabling this on powerpc and a patch that improves the portability of
the selftests, which I think it's not necessary to be a standalone
patch, so it's OK to be merged into your patch #7.

I did some tests on 64bit little/big endian pSeries(guest) kernel with
selftest cases(64bit LE selftest on 64bit LE kernel, 64/32bit BE
selftest on 64bit BE kernel), things seemingly went well ;-)

Here are some benchmark results I got on a little endian guest with 64
VCPUs:

Benchmarking various approaches for reading the current CPU number:

Power8 PSeries Guest(64 VCPUs, the host has 16 cores, 128 hardware
threads):
							
- Baseline (empty loop):                                   1.56 ns
- Read CPU from rseq cpu_id:                               1.56 ns
- Read CPU from rseq cpu_id (lazy register):               2.08 ns
- glibc 2.23-0ubuntu3 getcpu:                              7.72 ns
- getcpu system call:                                     91.80 ns


Benchmarking various approaches for counter increment:

Power8 PSeries KVM Guest(64 VCPUs, the host has 16 cores, 128 hardware
threads):

                                 Counter increment speed (ns/increment)
                              1 thread   2 threads   4 threads   8 threads   16 threads   32 threads
global increment (baseline)     6.5          N/A         N/A         N/A         N/A           N/A
percpu rseq increment           6.9          6.9         7.2         7.3        15.4          35.5
percpu rseq spinlock           19.0         18.9        19.4        19.4        35.5          71.8
global atomic increment        25.8        111.0       261.0       905.2      2319.5        4170.5 (__sync_add_and_fetch_4)
global atomic CAS              26.2        119.0       341.6      1183.0      3951.3        9312.5 (__sync_val_compare_and_swap_4)
global pthread mutex           40.0        238.1       644.0      2052.2      4272.5        8612.2


I surely need to run more tests for my patches in different
environments, and will try to adjust the patchset according to whatever
change you make(e.g. rseq_finish2) in the future.

(Add PPC maintainers in Cc)

Regards,
Boqun

> Here are benchmarks of counter increment in various scenarios compared
> to restartable sequences:
> 
> ARMv7 Processor rev 4 (v7l)
> Machine model: Cubietruck
> 
>                       Counter increment speed (ns/increment)
>                              1 thread    2 threads
> global increment (baseline)      6           N/A
> percpu rseq increment           50            52
> percpu rseq spinlock            94            94
> global atomic increment         48            74 (__sync_add_and_fetch_4)
> global atomic CAS               50           172 (__sync_val_compare_and_swap_4)
> global pthread mutex           148           862
> 
> ARMv7 Processor rev 10 (v7l)
> Machine model: Wandboard
> 
>                       Counter increment speed (ns/increment)
>                              1 thread    4 threads
> global increment (baseline)      7           N/A
> percpu rseq increment           50            50
> percpu rseq spinlock            82            84
> global atomic increment         44           262 (__sync_add_and_fetch_4)
> global atomic CAS               46           316 (__sync_val_compare_and_swap_4)
> global pthread mutex           146          1400
> 
> x86-64 Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz:
> 
>                       Counter increment speed (ns/increment)
>                               1 thread           8 threads
> global increment (baseline)      3.0                N/A
> percpu rseq increment            3.6                3.8
> percpu rseq spinlock             5.6                6.2
> global LOCK; inc                 8.0              166.4
> global LOCK; cmpxchg            13.4              435.2
> global pthread mutex            25.2             1363.6
> 
> * Reading the current CPU number
> 
> Speeding up reading the current CPU number on which the caller thread is
> running is done by keeping the current CPU number up do date within the
> cpu_id field of the memory area registered by the thread. This is done
> by making scheduler migration set the TIF_NOTIFY_RESUME flag on the
> current thread. Upon return to user-space, a notify-resume handler
> updates the current CPU value within the registered user-space memory
> area. User-space can then read the current CPU number directly from
> memory.
> 
> Keeping the current cpu id in a memory area shared between kernel and
> user-space is an improvement over current mechanisms available to read
> the current CPU number, which has the following benefits over
> alternative approaches:
> 
> - 35x speedup on ARM vs system call through glibc
> - 20x speedup on x86 compared to calling glibc, which calls vdso
>   executing a "lsl" instruction,
> - 14x speedup on x86 compared to inlined "lsl" instruction,
> - Unlike vdso approaches, this cpu_id value can be read from an inline
>   assembly, which makes it a useful building block for restartable
>   sequences.
> - The approach of reading the cpu id through memory mapping shared
>   between kernel and user-space is portable (e.g. ARM), which is not the
>   case for the lsl-based x86 vdso.
> 
> On x86, yet another possible approach would be to use the gs segment
> selector to point to user-space per-cpu data. This approach performs
> similarly to the cpu id cache, but it has two disadvantages: it is
> not portable, and it is incompatible with existing applications already
> using the gs segment selector for other purposes.
> 
> Benchmarking various approaches for reading the current CPU number:
> 
> ARMv7 Processor rev 4 (v7l)
> Machine model: Cubietruck
> - Baseline (empty loop):                                    8.4 ns
> - Read CPU from rseq cpu_id:                               16.7 ns
> - Read CPU from rseq cpu_id (lazy register):               19.8 ns
> - glibc 2.19-0ubuntu6.6 getcpu:                           301.8 ns
> - getcpu system call:                                     234.9 ns
> 
> x86-64 Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz:
> - Baseline (empty loop):                                    0.8 ns
> - Read CPU from rseq cpu_id:                                0.8 ns
> - Read CPU from rseq cpu_id (lazy register):                0.8 ns
> - Read using gs segment selector:                           0.8 ns
> - "lsl" inline assembly:                                   13.0 ns
> - glibc 2.19-0ubuntu6 getcpu:                              16.6 ns
> - getcpu system call:                                      53.9 ns
> 
> - Speed
> 
> Running 10 runs of hackbench -l 100000 seems to indicate, contrary to
> expectations, that enabling CONFIG_RSEQ slightly accelerates the
> scheduler:
> 
> Configuration: 2 sockets * 8-core Intel(R) Xeon(R) CPU E5-2630 v3 @
> 2.40GHz (directly on hardware, hyperthreading disabled in BIOS, energy
> saving disabled in BIOS, turboboost disabled in BIOS, cpuidle.off=1
> kernel parameter), with a Linux v4.6 defconfig+localyesconfig,
> restartable sequences series applied.
> 

[snip]

[toc] | [next] | [standalone]

#1451336 — [RFC 4/4] Restartable sequences: Add self-tests for PPC

FromBoqun Feng <boqun.feng@gmail.com>
Date2016-07-27 17:10 +0200
Subject[RFC 4/4] Restartable sequences: Add self-tests for PPC
Message-ID<rZyD8-12s-31@gated-at.bofh.it>
In reply to#1451335
As rseq syscall is enabled on PPC, implement the self-tests on PPC to
verify the implementation of the syscall.

Please note we only support 32bit userspace on BE kernel.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 tools/testing/selftests/rseq/param_test.c |  14 ++++
 tools/testing/selftests/rseq/rseq.h       | 120 ++++++++++++++++++++++++++++++
 2 files changed, 134 insertions(+)

diff --git a/tools/testing/selftests/rseq/param_test.c b/tools/testing/selftests/rseq/param_test.c
index db25e0a818e5..e2cb1b165f81 100644
--- a/tools/testing/selftests/rseq/param_test.c
+++ b/tools/testing/selftests/rseq/param_test.c
@@ -75,6 +75,20 @@ static __thread unsigned int yield_mod_cnt, nr_retry;
 	"bne 222b\n\t" \
 	"333:\n\t"
 
+#elif __PPC__
+#define INJECT_ASM_REG	"r18"
+
+#define RSEQ_INJECT_CLOBBER \
+	, INJECT_ASM_REG
+
+#define RSEQ_INJECT_ASM(n) \
+	"lwz %%" INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \
+	"cmpwi %%" INJECT_ASM_REG ", 0\n\t" \
+	"beq 333f\n\t" \
+	"222:\n\t" \
+	"subic. %%" INJECT_ASM_REG ", %%" INJECT_ASM_REG ", 1\n\t" \
+	"bne 222b\n\t" \
+	"333:\n\t"
 #else
 #error unsupported target
 #endif
diff --git a/tools/testing/selftests/rseq/rseq.h b/tools/testing/selftests/rseq/rseq.h
index 791e14cf42ae..dea0bea52566 100644
--- a/tools/testing/selftests/rseq/rseq.h
+++ b/tools/testing/selftests/rseq/rseq.h
@@ -138,6 +138,35 @@ do {									\
 #define has_fast_acquire_release()	0
 #define has_single_copy_load_64()	1
 
+#elif __PPC__
+#define smp_mb()	__asm__ __volatile__ ("sync" : : : "memory")
+#define smp_lwsync()	__asm__ __volatile__ ("lwsync" : : : "memory")
+#define smp_rmb()	smp_lwsync()
+#define smp_wmb()	smp_lwsync()
+
+#define smp_load_acquire(p)						\
+__extension__ ({							\
+	__typeof(*p) ____p1 = READ_ONCE(*p);				\
+	smp_lwsync();							\
+	____p1;								\
+})
+
+#define smp_acquire__after_ctrl_dep()	smp_lwsync()
+
+#define smp_store_release(p, v)						\
+do {									\
+	smp_lwsync();							\
+	WRITE_ONCE(*p, v);						\
+} while (0)
+
+#define has_fast_acquire_release()	1
+
+# if __PPC64__
+# define has_single_copy_load_64()	1
+# else
+# define has_single_copy_load_64()	0
+# endif
+
 #else
 #error unsupported target
 #endif
@@ -404,6 +433,97 @@ bool rseq_finish(struct rseq_lock *rlock,
 			: succeed
 		);
 	}
+#elif __PPC64__
+	{
+		/*
+		 * The __rseq_table section can be used by debuggers to better
+		 * handle single-stepping through the restartable critical
+		 * sections.
+		 */
+		__asm__ __volatile__ goto (
+			".pushsection __rseq_table, \"aw\"\n\t"
+			".balign 8\n\t"
+			"4:\n\t"
+			".quad 1f, 2f, 3f\n\t"
+			".popsection\n\t"
+			"1:\n\t"
+			RSEQ_INJECT_ASM(1)
+			"lis %%r17, (4b)@highest\n\t"
+			"ori %%r17, %%r17, (4b)@higher\n\t"
+			"rldicr %%r17, %%r17, 32, 31\n\t"
+			"oris %%r17, %%r17, (4b)@h\n\t"
+			"ori %%r17, %%r17, (4b)@l\n\t"
+			"std %%r17, 0(%[rseq_cs])\n\t"
+			RSEQ_INJECT_ASM(2)
+			"lwz %%r17, %[current_event_counter]\n\t"
+			"li %%r16, 0\n\t"
+			"cmpw cr7, %[start_event_counter], %%r17\n\t"
+			"bne cr7, 3f\n\t"
+			RSEQ_INJECT_ASM(3)
+			"std %[to_write], 0(%[target])\n\t"
+			"2:\n\t"
+			RSEQ_INJECT_ASM(4)
+			"std %%r16, 0(%[rseq_cs])\n\t"
+			"b %l[succeed]\n\t"
+			"3:\n\t"
+			"li %%r16, 0\n\t"
+			"std %%r16, 0(%[rseq_cs])\n\t"
+			: /* no outputs */
+			: [start_event_counter]"r"(start_value.event_counter),
+			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
+			  [to_write]"r"(to_write),
+			  [target]"b"(p),
+			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
+			  RSEQ_INJECT_INPUT
+			: "r16", "r17", "memory", "cc"
+			  RSEQ_INJECT_CLOBBER
+			: succeed
+		);
+	}
+#elif __PPC__
+	{
+		/*
+		 * The __rseq_table section can be used by debuggers to better
+		 * handle single-stepping through the restartable critical
+		 * sections.
+		 */
+		__asm__ __volatile__ goto (
+			".pushsection __rseq_table, \"aw\"\n\t"
+			".balign 8\n\t"
+			"4:\n\t"
+			".long 0x0, 1f, 0x0, 2f, 0x0, 3f\n\t" /* 32 bit only supported on BE */
+			".popsection\n\t"
+			"1:\n\t"
+			RSEQ_INJECT_ASM(1)
+			"lis %%r17, (4b)@ha\n\t"
+			"addi %%r17, %%r17, (4b)@l\n\t"
+			"stw %%r17, 0(%[rseq_cs])\n\t"
+			RSEQ_INJECT_ASM(2)
+			"lwz %%r17, %[current_event_counter]\n\t"
+			"li %%r16, 0\n\t"
+			"cmpw cr7, %[start_event_counter], %%r17\n\t"
+			"bne cr7, 3f\n\t"
+			RSEQ_INJECT_ASM(3)
+			"stw %[to_write], 0(%[target])\n\t"
+			"2:\n\t"
+			RSEQ_INJECT_ASM(4)
+			"stw %%r16, 0(%[rseq_cs])\n\t"
+			"b %l[succeed]\n\t"
+			"3:\n\t"
+			"li %%r16, 0\n\t"
+			"stw %%r16, 0(%[rseq_cs])\n\t"
+			: /* no outputs */
+			: [start_event_counter]"r"(start_value.event_counter),
+			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
+			  [to_write]"r"(to_write),
+			  [target]"b"(p),
+			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
+			  RSEQ_INJECT_INPUT
+			: "r16", "r17", "memory", "cc"
+			  RSEQ_INJECT_CLOBBER
+			: succeed
+		);
+	}
 #else
 #error unsupported target
 #endif
-- 
2.9.0

[toc] | [prev] | [next] | [standalone]

#1451678 — Re: [RFC 4/4] Restartable sequences: Add self-tests for PPC

FromMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date2016-07-28 05:10 +0200
SubjectRe: [RFC 4/4] Restartable sequences: Add self-tests for PPC
Message-ID<rZJRV-h8-51@gated-at.bofh.it>
In reply to#1451336
----- On Jul 27, 2016, at 11:05 AM, Boqun Feng boqun.feng@gmail.com wrote:

> As rseq syscall is enabled on PPC, implement the self-tests on PPC to
> verify the implementation of the syscall.
> 
> Please note we only support 32bit userspace on BE kernel.
> 
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
> tools/testing/selftests/rseq/param_test.c |  14 ++++
> tools/testing/selftests/rseq/rseq.h       | 120 ++++++++++++++++++++++++++++++
> 2 files changed, 134 insertions(+)
> 
> diff --git a/tools/testing/selftests/rseq/param_test.c
> b/tools/testing/selftests/rseq/param_test.c
> index db25e0a818e5..e2cb1b165f81 100644
> --- a/tools/testing/selftests/rseq/param_test.c
> +++ b/tools/testing/selftests/rseq/param_test.c
> @@ -75,6 +75,20 @@ static __thread unsigned int yield_mod_cnt, nr_retry;
> 	"bne 222b\n\t" \
> 	"333:\n\t"
> 
> +#elif __PPC__
> +#define INJECT_ASM_REG	"r18"
> +
> +#define RSEQ_INJECT_CLOBBER \
> +	, INJECT_ASM_REG
> +
> +#define RSEQ_INJECT_ASM(n) \
> +	"lwz %%" INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \
> +	"cmpwi %%" INJECT_ASM_REG ", 0\n\t" \
> +	"beq 333f\n\t" \
> +	"222:\n\t" \
> +	"subic. %%" INJECT_ASM_REG ", %%" INJECT_ASM_REG ", 1\n\t" \
> +	"bne 222b\n\t" \
> +	"333:\n\t"
> #else
> #error unsupported target
> #endif
> diff --git a/tools/testing/selftests/rseq/rseq.h
> b/tools/testing/selftests/rseq/rseq.h
> index 791e14cf42ae..dea0bea52566 100644
> --- a/tools/testing/selftests/rseq/rseq.h
> +++ b/tools/testing/selftests/rseq/rseq.h
> @@ -138,6 +138,35 @@ do {									\
> #define has_fast_acquire_release()	0
> #define has_single_copy_load_64()	1
> 
> +#elif __PPC__
> +#define smp_mb()	__asm__ __volatile__ ("sync" : : : "memory")
> +#define smp_lwsync()	__asm__ __volatile__ ("lwsync" : : : "memory")
> +#define smp_rmb()	smp_lwsync()
> +#define smp_wmb()	smp_lwsync()
> +
> +#define smp_load_acquire(p)						\
> +__extension__ ({							\
> +	__typeof(*p) ____p1 = READ_ONCE(*p);				\
> +	smp_lwsync();							\
> +	____p1;								\
> +})
> +
> +#define smp_acquire__after_ctrl_dep()	smp_lwsync()
> +
> +#define smp_store_release(p, v)						\
> +do {									\
> +	smp_lwsync();							\
> +	WRITE_ONCE(*p, v);						\
> +} while (0)
> +
> +#define has_fast_acquire_release()	1

Can you check if defining has_fast_acquire_release() to 0 speeds up
performance significantly ? It turns the smp_lwsync() into a
compiler barrier() on the smp_load_acquire() side (fast-path), and
turn the smp_lwsync() into a membarrier system call instead of the
matching smp_store_release() (slow path).

Thanks,

Mathieu

> +
> +# if __PPC64__
> +# define has_single_copy_load_64()	1
> +# else
> +# define has_single_copy_load_64()	0
> +# endif
> +
> #else
> #error unsupported target
> #endif
> @@ -404,6 +433,97 @@ bool rseq_finish(struct rseq_lock *rlock,
> 			: succeed
> 		);
> 	}
> +#elif __PPC64__
> +	{
> +		/*
> +		 * The __rseq_table section can be used by debuggers to better
> +		 * handle single-stepping through the restartable critical
> +		 * sections.
> +		 */
> +		__asm__ __volatile__ goto (
> +			".pushsection __rseq_table, \"aw\"\n\t"
> +			".balign 8\n\t"
> +			"4:\n\t"
> +			".quad 1f, 2f, 3f\n\t"
> +			".popsection\n\t"
> +			"1:\n\t"
> +			RSEQ_INJECT_ASM(1)
> +			"lis %%r17, (4b)@highest\n\t"
> +			"ori %%r17, %%r17, (4b)@higher\n\t"
> +			"rldicr %%r17, %%r17, 32, 31\n\t"
> +			"oris %%r17, %%r17, (4b)@h\n\t"
> +			"ori %%r17, %%r17, (4b)@l\n\t"
> +			"std %%r17, 0(%[rseq_cs])\n\t"
> +			RSEQ_INJECT_ASM(2)
> +			"lwz %%r17, %[current_event_counter]\n\t"
> +			"li %%r16, 0\n\t"
> +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
> +			"bne cr7, 3f\n\t"
> +			RSEQ_INJECT_ASM(3)
> +			"std %[to_write], 0(%[target])\n\t"
> +			"2:\n\t"
> +			RSEQ_INJECT_ASM(4)
> +			"std %%r16, 0(%[rseq_cs])\n\t"
> +			"b %l[succeed]\n\t"
> +			"3:\n\t"
> +			"li %%r16, 0\n\t"
> +			"std %%r16, 0(%[rseq_cs])\n\t"
> +			: /* no outputs */
> +			: [start_event_counter]"r"(start_value.event_counter),
> +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
> +			  [to_write]"r"(to_write),
> +			  [target]"b"(p),
> +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
> +			  RSEQ_INJECT_INPUT
> +			: "r16", "r17", "memory", "cc"
> +			  RSEQ_INJECT_CLOBBER
> +			: succeed
> +		);
> +	}
> +#elif __PPC__
> +	{
> +		/*
> +		 * The __rseq_table section can be used by debuggers to better
> +		 * handle single-stepping through the restartable critical
> +		 * sections.
> +		 */
> +		__asm__ __volatile__ goto (
> +			".pushsection __rseq_table, \"aw\"\n\t"
> +			".balign 8\n\t"
> +			"4:\n\t"
> +			".long 0x0, 1f, 0x0, 2f, 0x0, 3f\n\t" /* 32 bit only supported on BE */
> +			".popsection\n\t"
> +			"1:\n\t"
> +			RSEQ_INJECT_ASM(1)
> +			"lis %%r17, (4b)@ha\n\t"
> +			"addi %%r17, %%r17, (4b)@l\n\t"
> +			"stw %%r17, 0(%[rseq_cs])\n\t"
> +			RSEQ_INJECT_ASM(2)
> +			"lwz %%r17, %[current_event_counter]\n\t"
> +			"li %%r16, 0\n\t"
> +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
> +			"bne cr7, 3f\n\t"
> +			RSEQ_INJECT_ASM(3)
> +			"stw %[to_write], 0(%[target])\n\t"
> +			"2:\n\t"
> +			RSEQ_INJECT_ASM(4)
> +			"stw %%r16, 0(%[rseq_cs])\n\t"
> +			"b %l[succeed]\n\t"
> +			"3:\n\t"
> +			"li %%r16, 0\n\t"
> +			"stw %%r16, 0(%[rseq_cs])\n\t"
> +			: /* no outputs */
> +			: [start_event_counter]"r"(start_value.event_counter),
> +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
> +			  [to_write]"r"(to_write),
> +			  [target]"b"(p),
> +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
> +			  RSEQ_INJECT_INPUT
> +			: "r16", "r17", "memory", "cc"
> +			  RSEQ_INJECT_CLOBBER
> +			: succeed
> +		);
> +	}
> #else
> #error unsupported target
> #endif
> --
> 2.9.0

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[toc] | [prev] | [next] | [standalone]

#1451715 — Re: [RFC 4/4] Restartable sequences: Add self-tests for PPC

FromBoqun Feng <boqun.feng@gmail.com>
Date2016-07-28 06:50 +0200
SubjectRe: [RFC 4/4] Restartable sequences: Add self-tests for PPC
Message-ID<rZLqG-1bP-1@gated-at.bofh.it>
In reply to#1451678

[Multipart message — attachments visible in raw view] — view raw

On Thu, Jul 28, 2016 at 02:59:45AM +0000, Mathieu Desnoyers wrote:
> ----- On Jul 27, 2016, at 11:05 AM, Boqun Feng boqun.feng@gmail.com wrote:
> 
> > As rseq syscall is enabled on PPC, implement the self-tests on PPC to
> > verify the implementation of the syscall.
> > 
> > Please note we only support 32bit userspace on BE kernel.
> > 
> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > ---
> > tools/testing/selftests/rseq/param_test.c |  14 ++++
> > tools/testing/selftests/rseq/rseq.h       | 120 ++++++++++++++++++++++++++++++
> > 2 files changed, 134 insertions(+)
> > 
> > diff --git a/tools/testing/selftests/rseq/param_test.c
> > b/tools/testing/selftests/rseq/param_test.c
> > index db25e0a818e5..e2cb1b165f81 100644
> > --- a/tools/testing/selftests/rseq/param_test.c
> > +++ b/tools/testing/selftests/rseq/param_test.c
> > @@ -75,6 +75,20 @@ static __thread unsigned int yield_mod_cnt, nr_retry;
> > 	"bne 222b\n\t" \
> > 	"333:\n\t"
> > 
> > +#elif __PPC__
> > +#define INJECT_ASM_REG	"r18"
> > +
> > +#define RSEQ_INJECT_CLOBBER \
> > +	, INJECT_ASM_REG
> > +
> > +#define RSEQ_INJECT_ASM(n) \
> > +	"lwz %%" INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \
> > +	"cmpwi %%" INJECT_ASM_REG ", 0\n\t" \
> > +	"beq 333f\n\t" \
> > +	"222:\n\t" \
> > +	"subic. %%" INJECT_ASM_REG ", %%" INJECT_ASM_REG ", 1\n\t" \
> > +	"bne 222b\n\t" \
> > +	"333:\n\t"
> > #else
> > #error unsupported target
> > #endif
> > diff --git a/tools/testing/selftests/rseq/rseq.h
> > b/tools/testing/selftests/rseq/rseq.h
> > index 791e14cf42ae..dea0bea52566 100644
> > --- a/tools/testing/selftests/rseq/rseq.h
> > +++ b/tools/testing/selftests/rseq/rseq.h
> > @@ -138,6 +138,35 @@ do {									\
> > #define has_fast_acquire_release()	0
> > #define has_single_copy_load_64()	1
> > 
> > +#elif __PPC__
> > +#define smp_mb()	__asm__ __volatile__ ("sync" : : : "memory")
> > +#define smp_lwsync()	__asm__ __volatile__ ("lwsync" : : : "memory")
> > +#define smp_rmb()	smp_lwsync()
> > +#define smp_wmb()	smp_lwsync()
> > +
> > +#define smp_load_acquire(p)						\
> > +__extension__ ({							\
> > +	__typeof(*p) ____p1 = READ_ONCE(*p);				\
> > +	smp_lwsync();							\
> > +	____p1;								\
> > +})
> > +
> > +#define smp_acquire__after_ctrl_dep()	smp_lwsync()
> > +
> > +#define smp_store_release(p, v)						\
> > +do {									\
> > +	smp_lwsync();							\
> > +	WRITE_ONCE(*p, v);						\
> > +} while (0)
> > +
> > +#define has_fast_acquire_release()	1
> 
> Can you check if defining has_fast_acquire_release() to 0 speeds up
> performance significantly ? It turns the smp_lwsync() into a
> compiler barrier() on the smp_load_acquire() side (fast-path), and
> turn the smp_lwsync() into a membarrier system call instead of the
> matching smp_store_release() (slow path).
> 

Good point. Here are the numbers:

Power8 PSeries KVM Guest(64 VCPUs, the host has 16 cores, 128 hardware
threads):

                                 Counter increment speed (ns/increment)
                              1 thread   2 threads   4 threads   8 threads   16 threads   32 threads
global increment (baseline)     6.5          N/A         N/A         N/A       N/A           N/A
percpu rseq increment           7.0          7.0         7.2         7.2       9.3          14.5
percpu rseq spinlock           18.5         18.5        18.6        18.8      25.5          52.7

So looks like defining has_fast_acquire_release() to 0 could benefit the
cases with more threads in current benchmark. I will send a updated
patch doing this.

And as discussed in IRC, I will also remove jump from rseq_finish()
fast-path in powerpc asm in the updated patch as you did for x86 and
ARM.

Regards,
Boqun


> Thanks,
> 
> Mathieu
> 
> > +
> > +# if __PPC64__
> > +# define has_single_copy_load_64()	1
> > +# else
> > +# define has_single_copy_load_64()	0
> > +# endif
> > +
> > #else
> > #error unsupported target
> > #endif
> > @@ -404,6 +433,97 @@ bool rseq_finish(struct rseq_lock *rlock,
> > 			: succeed
> > 		);
> > 	}
> > +#elif __PPC64__
> > +	{
> > +		/*
> > +		 * The __rseq_table section can be used by debuggers to better
> > +		 * handle single-stepping through the restartable critical
> > +		 * sections.
> > +		 */
> > +		__asm__ __volatile__ goto (
> > +			".pushsection __rseq_table, \"aw\"\n\t"
> > +			".balign 8\n\t"
> > +			"4:\n\t"
> > +			".quad 1f, 2f, 3f\n\t"
> > +			".popsection\n\t"
> > +			"1:\n\t"
> > +			RSEQ_INJECT_ASM(1)
> > +			"lis %%r17, (4b)@highest\n\t"
> > +			"ori %%r17, %%r17, (4b)@higher\n\t"
> > +			"rldicr %%r17, %%r17, 32, 31\n\t"
> > +			"oris %%r17, %%r17, (4b)@h\n\t"
> > +			"ori %%r17, %%r17, (4b)@l\n\t"
> > +			"std %%r17, 0(%[rseq_cs])\n\t"
> > +			RSEQ_INJECT_ASM(2)
> > +			"lwz %%r17, %[current_event_counter]\n\t"
> > +			"li %%r16, 0\n\t"
> > +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
> > +			"bne cr7, 3f\n\t"
> > +			RSEQ_INJECT_ASM(3)
> > +			"std %[to_write], 0(%[target])\n\t"
> > +			"2:\n\t"
> > +			RSEQ_INJECT_ASM(4)
> > +			"std %%r16, 0(%[rseq_cs])\n\t"
> > +			"b %l[succeed]\n\t"
> > +			"3:\n\t"
> > +			"li %%r16, 0\n\t"
> > +			"std %%r16, 0(%[rseq_cs])\n\t"
> > +			: /* no outputs */
> > +			: [start_event_counter]"r"(start_value.event_counter),
> > +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
> > +			  [to_write]"r"(to_write),
> > +			  [target]"b"(p),
> > +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
> > +			  RSEQ_INJECT_INPUT
> > +			: "r16", "r17", "memory", "cc"
> > +			  RSEQ_INJECT_CLOBBER
> > +			: succeed
> > +		);
> > +	}
> > +#elif __PPC__
> > +	{
> > +		/*
> > +		 * The __rseq_table section can be used by debuggers to better
> > +		 * handle single-stepping through the restartable critical
> > +		 * sections.
> > +		 */
> > +		__asm__ __volatile__ goto (
> > +			".pushsection __rseq_table, \"aw\"\n\t"
> > +			".balign 8\n\t"
> > +			"4:\n\t"
> > +			".long 0x0, 1f, 0x0, 2f, 0x0, 3f\n\t" /* 32 bit only supported on BE */
> > +			".popsection\n\t"
> > +			"1:\n\t"
> > +			RSEQ_INJECT_ASM(1)
> > +			"lis %%r17, (4b)@ha\n\t"
> > +			"addi %%r17, %%r17, (4b)@l\n\t"
> > +			"stw %%r17, 0(%[rseq_cs])\n\t"
> > +			RSEQ_INJECT_ASM(2)
> > +			"lwz %%r17, %[current_event_counter]\n\t"
> > +			"li %%r16, 0\n\t"
> > +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
> > +			"bne cr7, 3f\n\t"
> > +			RSEQ_INJECT_ASM(3)
> > +			"stw %[to_write], 0(%[target])\n\t"
> > +			"2:\n\t"
> > +			RSEQ_INJECT_ASM(4)
> > +			"stw %%r16, 0(%[rseq_cs])\n\t"
> > +			"b %l[succeed]\n\t"
> > +			"3:\n\t"
> > +			"li %%r16, 0\n\t"
> > +			"stw %%r16, 0(%[rseq_cs])\n\t"
> > +			: /* no outputs */
> > +			: [start_event_counter]"r"(start_value.event_counter),
> > +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
> > +			  [to_write]"r"(to_write),
> > +			  [target]"b"(p),
> > +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
> > +			  RSEQ_INJECT_INPUT
> > +			: "r16", "r17", "memory", "cc"
> > +			  RSEQ_INJECT_CLOBBER
> > +			: succeed
> > +		);
> > +	}
> > #else
> > #error unsupported target
> > #endif
> > --
> > 2.9.0
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

[toc] | [prev] | [next] | [standalone]

#1451762 — [RFC v2] Restartable sequences: Add self-tests for PPC

FromBoqun Feng <boqun.feng@gmail.com>
Date2016-07-28 09:40 +0200
Subject[RFC v2] Restartable sequences: Add self-tests for PPC
Message-ID<rZO5b-32s-1@gated-at.bofh.it>
In reply to#1451715
As rseq syscall is enabled on PPC, implement the self-tests on PPC to
verify the implementation of the syscall.

Please note we only support 32bit userspace on BE kernel.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
v1-->v2:
	1. Remove branch in rseq_finish() fastpath

	2. Use bne- instead of bne to jump when failure.

	3. Use r17 instead of r16 for storing zero to rseq_cs, which
	could save a register in rseq_finish() asm block.

 tools/testing/selftests/rseq/param_test.c |  14 ++++
 tools/testing/selftests/rseq/rseq.h       | 112 ++++++++++++++++++++++++++++++
 2 files changed, 126 insertions(+)

diff --git a/tools/testing/selftests/rseq/param_test.c b/tools/testing/selftests/rseq/param_test.c
index db25e0a818e5..e2cb1b165f81 100644
--- a/tools/testing/selftests/rseq/param_test.c
+++ b/tools/testing/selftests/rseq/param_test.c
@@ -75,6 +75,20 @@ static __thread unsigned int yield_mod_cnt, nr_retry;
 	"bne 222b\n\t" \
 	"333:\n\t"
 
+#elif __PPC__
+#define INJECT_ASM_REG	"r18"
+
+#define RSEQ_INJECT_CLOBBER \
+	, INJECT_ASM_REG
+
+#define RSEQ_INJECT_ASM(n) \
+	"lwz %%" INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \
+	"cmpwi %%" INJECT_ASM_REG ", 0\n\t" \
+	"beq 333f\n\t" \
+	"222:\n\t" \
+	"subic. %%" INJECT_ASM_REG ", %%" INJECT_ASM_REG ", 1\n\t" \
+	"bne 222b\n\t" \
+	"333:\n\t"
 #else
 #error unsupported target
 #endif
diff --git a/tools/testing/selftests/rseq/rseq.h b/tools/testing/selftests/rseq/rseq.h
index 35b60ee3bb02..b5336cf54788 100644
--- a/tools/testing/selftests/rseq/rseq.h
+++ b/tools/testing/selftests/rseq/rseq.h
@@ -138,6 +138,35 @@ do {									\
 #define has_fast_acquire_release()	0
 #define has_single_copy_load_64()	1
 
+#elif __PPC__
+#define smp_mb()	__asm__ __volatile__ ("sync" : : : "memory")
+#define smp_lwsync()	__asm__ __volatile__ ("lwsync" : : : "memory")
+#define smp_rmb()	smp_lwsync()
+#define smp_wmb()	smp_lwsync()
+
+#define smp_load_acquire(p)						\
+__extension__ ({							\
+	__typeof(*p) ____p1 = READ_ONCE(*p);				\
+	smp_lwsync();							\
+	____p1;								\
+})
+
+#define smp_acquire__after_ctrl_dep()	smp_lwsync()
+
+#define smp_store_release(p, v)						\
+do {									\
+	smp_lwsync();							\
+	WRITE_ONCE(*p, v);						\
+} while (0)
+
+#define has_fast_acquire_release()	0
+
+# if __PPC64__
+# define has_single_copy_load_64()	1
+# else
+# define has_single_copy_load_64()	0
+# endif
+
 #else
 #error unsupported target
 #endif
@@ -398,6 +427,89 @@ bool rseq_finish(struct rseq_lock *rlock,
 			: failure
 		);
 	}
+#elif __PPC64__
+	{
+		/*
+		 * The __rseq_table section can be used by debuggers to better
+		 * handle single-stepping through the restartable critical
+		 * sections.
+		 */
+		__asm__ __volatile__ goto (
+			".pushsection __rseq_table, \"aw\"\n\t"
+			".balign 8\n\t"
+			"3:\n\t"
+			".quad 1f, 2f, %l[failure]\n\t"
+			".popsection\n\t"
+			"1:\n\t"
+			RSEQ_INJECT_ASM(1)
+			"lis %%r17, (3b)@highest\n\t"
+			"ori %%r17, %%r17, (3b)@higher\n\t"
+			"rldicr %%r17, %%r17, 32, 31\n\t"
+			"oris %%r17, %%r17, (3b)@h\n\t"
+			"ori %%r17, %%r17, (3b)@l\n\t"
+			"std %%r17, 0(%[rseq_cs])\n\t"
+			RSEQ_INJECT_ASM(2)
+			"lwz %%r17, %[current_event_counter]\n\t"
+			"cmpw cr7, %[start_event_counter], %%r17\n\t"
+			"bne- cr7, %l[failure]\n\t"
+			RSEQ_INJECT_ASM(3)
+			"std %[to_write], 0(%[target])\n\t"
+			"2:\n\t"
+			RSEQ_INJECT_ASM(4)
+			"li %%r17, 0\n\t"
+			"std %%r17, 0(%[rseq_cs])\n\t"
+			: /* no outputs */
+			: [start_event_counter]"r"(start_value.event_counter),
+			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
+			  [to_write]"r"(to_write),
+			  [target]"b"(p),
+			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
+			  RSEQ_INJECT_INPUT
+			: "r17", "memory", "cc"
+			  RSEQ_INJECT_CLOBBER
+			: failure
+		);
+	}
+#elif __PPC__
+	{
+		/*
+		 * The __rseq_table section can be used by debuggers to better
+		 * handle single-stepping through the restartable critical
+		 * sections.
+		 */
+		__asm__ __volatile__ goto (
+			".pushsection __rseq_table, \"aw\"\n\t"
+			".balign 8\n\t"
+			"3:\n\t"
+			".long 0x0, 1f, 0x0, 2f, 0x0, %l[failure]\n\t" /* 32 bit only supported on BE */
+			".popsection\n\t"
+			"1:\n\t"
+			RSEQ_INJECT_ASM(1)
+			"lis %%r17, (3b)@ha\n\t"
+			"addi %%r17, %%r17, (3b)@l\n\t"
+			"stw %%r17, 0(%[rseq_cs])\n\t"
+			RSEQ_INJECT_ASM(2)
+			"lwz %%r17, %[current_event_counter]\n\t"
+			"cmpw cr7, %[start_event_counter], %%r17\n\t"
+			"bne- cr7, %l[failure]\n\t"
+			RSEQ_INJECT_ASM(3)
+			"stw %[to_write], 0(%[target])\n\t"
+			"2:\n\t"
+			RSEQ_INJECT_ASM(4)
+			"li %%r17, 0\n\t"
+			"stw %%r17, 0(%[rseq_cs])\n\t"
+			: /* no outputs */
+			: [start_event_counter]"r"(start_value.event_counter),
+			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
+			  [to_write]"r"(to_write),
+			  [target]"b"(p),
+			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
+			  RSEQ_INJECT_INPUT
+			: "r17", "memory", "cc"
+			  RSEQ_INJECT_CLOBBER
+			: failure
+		);
+	}
 #else
 #error unsupported target
 #endif
-- 
2.9.0

[toc] | [prev] | [next] | [standalone]

#1451950 — Re: [RFC v2] Restartable sequences: Add self-tests for PPC

FromMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date2016-07-28 16:10 +0200
SubjectRe: [RFC v2] Restartable sequences: Add self-tests for PPC
Message-ID<rZUaC-7jd-13@gated-at.bofh.it>
In reply to#1451762
----- On Jul 28, 2016, at 3:37 AM, Boqun Feng boqun.feng@gmail.com wrote:

> As rseq syscall is enabled on PPC, implement the self-tests on PPC to
> verify the implementation of the syscall.
> 
> Please note we only support 32bit userspace on BE kernel.

Picked into my rseq-fallback dev branch, thanks!

Mathieu

> 
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
> v1-->v2:
>	1. Remove branch in rseq_finish() fastpath
> 
>	2. Use bne- instead of bne to jump when failure.
> 
>	3. Use r17 instead of r16 for storing zero to rseq_cs, which
>	could save a register in rseq_finish() asm block.
> 
> tools/testing/selftests/rseq/param_test.c |  14 ++++
> tools/testing/selftests/rseq/rseq.h       | 112 ++++++++++++++++++++++++++++++
> 2 files changed, 126 insertions(+)
> 
> diff --git a/tools/testing/selftests/rseq/param_test.c
> b/tools/testing/selftests/rseq/param_test.c
> index db25e0a818e5..e2cb1b165f81 100644
> --- a/tools/testing/selftests/rseq/param_test.c
> +++ b/tools/testing/selftests/rseq/param_test.c
> @@ -75,6 +75,20 @@ static __thread unsigned int yield_mod_cnt, nr_retry;
> 	"bne 222b\n\t" \
> 	"333:\n\t"
> 
> +#elif __PPC__
> +#define INJECT_ASM_REG	"r18"
> +
> +#define RSEQ_INJECT_CLOBBER \
> +	, INJECT_ASM_REG
> +
> +#define RSEQ_INJECT_ASM(n) \
> +	"lwz %%" INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \
> +	"cmpwi %%" INJECT_ASM_REG ", 0\n\t" \
> +	"beq 333f\n\t" \
> +	"222:\n\t" \
> +	"subic. %%" INJECT_ASM_REG ", %%" INJECT_ASM_REG ", 1\n\t" \
> +	"bne 222b\n\t" \
> +	"333:\n\t"
> #else
> #error unsupported target
> #endif
> diff --git a/tools/testing/selftests/rseq/rseq.h
> b/tools/testing/selftests/rseq/rseq.h
> index 35b60ee3bb02..b5336cf54788 100644
> --- a/tools/testing/selftests/rseq/rseq.h
> +++ b/tools/testing/selftests/rseq/rseq.h
> @@ -138,6 +138,35 @@ do {									\
> #define has_fast_acquire_release()	0
> #define has_single_copy_load_64()	1
> 
> +#elif __PPC__
> +#define smp_mb()	__asm__ __volatile__ ("sync" : : : "memory")
> +#define smp_lwsync()	__asm__ __volatile__ ("lwsync" : : : "memory")
> +#define smp_rmb()	smp_lwsync()
> +#define smp_wmb()	smp_lwsync()
> +
> +#define smp_load_acquire(p)						\
> +__extension__ ({							\
> +	__typeof(*p) ____p1 = READ_ONCE(*p);				\
> +	smp_lwsync();							\
> +	____p1;								\
> +})
> +
> +#define smp_acquire__after_ctrl_dep()	smp_lwsync()
> +
> +#define smp_store_release(p, v)						\
> +do {									\
> +	smp_lwsync();							\
> +	WRITE_ONCE(*p, v);						\
> +} while (0)
> +
> +#define has_fast_acquire_release()	0
> +
> +# if __PPC64__
> +# define has_single_copy_load_64()	1
> +# else
> +# define has_single_copy_load_64()	0
> +# endif
> +
> #else
> #error unsupported target
> #endif
> @@ -398,6 +427,89 @@ bool rseq_finish(struct rseq_lock *rlock,
> 			: failure
> 		);
> 	}
> +#elif __PPC64__
> +	{
> +		/*
> +		 * The __rseq_table section can be used by debuggers to better
> +		 * handle single-stepping through the restartable critical
> +		 * sections.
> +		 */
> +		__asm__ __volatile__ goto (
> +			".pushsection __rseq_table, \"aw\"\n\t"
> +			".balign 8\n\t"
> +			"3:\n\t"
> +			".quad 1f, 2f, %l[failure]\n\t"
> +			".popsection\n\t"
> +			"1:\n\t"
> +			RSEQ_INJECT_ASM(1)
> +			"lis %%r17, (3b)@highest\n\t"
> +			"ori %%r17, %%r17, (3b)@higher\n\t"
> +			"rldicr %%r17, %%r17, 32, 31\n\t"
> +			"oris %%r17, %%r17, (3b)@h\n\t"
> +			"ori %%r17, %%r17, (3b)@l\n\t"
> +			"std %%r17, 0(%[rseq_cs])\n\t"
> +			RSEQ_INJECT_ASM(2)
> +			"lwz %%r17, %[current_event_counter]\n\t"
> +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
> +			"bne- cr7, %l[failure]\n\t"
> +			RSEQ_INJECT_ASM(3)
> +			"std %[to_write], 0(%[target])\n\t"
> +			"2:\n\t"
> +			RSEQ_INJECT_ASM(4)
> +			"li %%r17, 0\n\t"
> +			"std %%r17, 0(%[rseq_cs])\n\t"
> +			: /* no outputs */
> +			: [start_event_counter]"r"(start_value.event_counter),
> +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
> +			  [to_write]"r"(to_write),
> +			  [target]"b"(p),
> +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
> +			  RSEQ_INJECT_INPUT
> +			: "r17", "memory", "cc"
> +			  RSEQ_INJECT_CLOBBER
> +			: failure
> +		);
> +	}
> +#elif __PPC__
> +	{
> +		/*
> +		 * The __rseq_table section can be used by debuggers to better
> +		 * handle single-stepping through the restartable critical
> +		 * sections.
> +		 */
> +		__asm__ __volatile__ goto (
> +			".pushsection __rseq_table, \"aw\"\n\t"
> +			".balign 8\n\t"
> +			"3:\n\t"
> +			".long 0x0, 1f, 0x0, 2f, 0x0, %l[failure]\n\t" /* 32 bit only supported on
> BE */
> +			".popsection\n\t"
> +			"1:\n\t"
> +			RSEQ_INJECT_ASM(1)
> +			"lis %%r17, (3b)@ha\n\t"
> +			"addi %%r17, %%r17, (3b)@l\n\t"
> +			"stw %%r17, 0(%[rseq_cs])\n\t"
> +			RSEQ_INJECT_ASM(2)
> +			"lwz %%r17, %[current_event_counter]\n\t"
> +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
> +			"bne- cr7, %l[failure]\n\t"
> +			RSEQ_INJECT_ASM(3)
> +			"stw %[to_write], 0(%[target])\n\t"
> +			"2:\n\t"
> +			RSEQ_INJECT_ASM(4)
> +			"li %%r17, 0\n\t"
> +			"stw %%r17, 0(%[rseq_cs])\n\t"
> +			: /* no outputs */
> +			: [start_event_counter]"r"(start_value.event_counter),
> +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
> +			  [to_write]"r"(to_write),
> +			  [target]"b"(p),
> +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
> +			  RSEQ_INJECT_INPUT
> +			: "r17", "memory", "cc"
> +			  RSEQ_INJECT_CLOBBER
> +			: failure
> +		);
> +	}
> #else
> #error unsupported target
> #endif
> --
> 2.9.0

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[toc] | [prev] | [next] | [standalone]

#1451937 — Re: [RFC 4/4] Restartable sequences: Add self-tests for PPC

FromMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date2016-07-28 15:50 +0200
SubjectRe: [RFC 4/4] Restartable sequences: Add self-tests for PPC
Message-ID<rZTRf-6WK-3@gated-at.bofh.it>
In reply to#1451715
----- On Jul 28, 2016, at 12:43 AM, Boqun Feng boqun.feng@gmail.com wrote:

> On Thu, Jul 28, 2016 at 02:59:45AM +0000, Mathieu Desnoyers wrote:
>> ----- On Jul 27, 2016, at 11:05 AM, Boqun Feng boqun.feng@gmail.com wrote:
>> 
>> > As rseq syscall is enabled on PPC, implement the self-tests on PPC to
>> > verify the implementation of the syscall.
>> > 
>> > Please note we only support 32bit userspace on BE kernel.
>> > 
>> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
>> > ---
>> > tools/testing/selftests/rseq/param_test.c |  14 ++++
>> > tools/testing/selftests/rseq/rseq.h       | 120 ++++++++++++++++++++++++++++++
>> > 2 files changed, 134 insertions(+)
>> > 
>> > diff --git a/tools/testing/selftests/rseq/param_test.c
>> > b/tools/testing/selftests/rseq/param_test.c
>> > index db25e0a818e5..e2cb1b165f81 100644
>> > --- a/tools/testing/selftests/rseq/param_test.c
>> > +++ b/tools/testing/selftests/rseq/param_test.c
>> > @@ -75,6 +75,20 @@ static __thread unsigned int yield_mod_cnt, nr_retry;
>> > 	"bne 222b\n\t" \
>> > 	"333:\n\t"
>> > 
>> > +#elif __PPC__
>> > +#define INJECT_ASM_REG	"r18"
>> > +
>> > +#define RSEQ_INJECT_CLOBBER \
>> > +	, INJECT_ASM_REG
>> > +
>> > +#define RSEQ_INJECT_ASM(n) \
>> > +	"lwz %%" INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \
>> > +	"cmpwi %%" INJECT_ASM_REG ", 0\n\t" \
>> > +	"beq 333f\n\t" \
>> > +	"222:\n\t" \
>> > +	"subic. %%" INJECT_ASM_REG ", %%" INJECT_ASM_REG ", 1\n\t" \
>> > +	"bne 222b\n\t" \
>> > +	"333:\n\t"
>> > #else
>> > #error unsupported target
>> > #endif
>> > diff --git a/tools/testing/selftests/rseq/rseq.h
>> > b/tools/testing/selftests/rseq/rseq.h
>> > index 791e14cf42ae..dea0bea52566 100644
>> > --- a/tools/testing/selftests/rseq/rseq.h
>> > +++ b/tools/testing/selftests/rseq/rseq.h
>> > @@ -138,6 +138,35 @@ do {									\
>> > #define has_fast_acquire_release()	0
>> > #define has_single_copy_load_64()	1
>> > 
>> > +#elif __PPC__
>> > +#define smp_mb()	__asm__ __volatile__ ("sync" : : : "memory")
>> > +#define smp_lwsync()	__asm__ __volatile__ ("lwsync" : : : "memory")
>> > +#define smp_rmb()	smp_lwsync()
>> > +#define smp_wmb()	smp_lwsync()
>> > +
>> > +#define smp_load_acquire(p)						\
>> > +__extension__ ({							\
>> > +	__typeof(*p) ____p1 = READ_ONCE(*p);				\
>> > +	smp_lwsync();							\
>> > +	____p1;								\
>> > +})
>> > +
>> > +#define smp_acquire__after_ctrl_dep()	smp_lwsync()
>> > +
>> > +#define smp_store_release(p, v)						\
>> > +do {									\
>> > +	smp_lwsync();							\
>> > +	WRITE_ONCE(*p, v);						\
>> > +} while (0)
>> > +
>> > +#define has_fast_acquire_release()	1
>> 
>> Can you check if defining has_fast_acquire_release() to 0 speeds up
>> performance significantly ? It turns the smp_lwsync() into a
>> compiler barrier() on the smp_load_acquire() side (fast-path), and
>> turn the smp_lwsync() into a membarrier system call instead of the
>> matching smp_store_release() (slow path).
>> 
> 
> Good point. Here are the numbers:
> 
> Power8 PSeries KVM Guest(64 VCPUs, the host has 16 cores, 128 hardware
> threads):
> 
>                                 Counter increment speed (ns/increment)
>                              1 thread   2 threads   4 threads   8 threads   16 threads   32 threads
> global increment (baseline)     6.5          N/A         N/A         N/A
> N/A           N/A
> percpu rseq increment           7.0          7.0         7.2         7.2
> 9.3          14.5
> percpu rseq spinlock           18.5         18.5        18.6        18.8
> 25.5          52.7
> 
> So looks like defining has_fast_acquire_release() to 0 could benefit the
> cases with more threads in current benchmark. I will send a updated
> patch doing this.

Good to know the lwsync barrier overhead kicks in at that level of
workload on Power8.

> 
> And as discussed in IRC, I will also remove jump from rseq_finish()
> fast-path in powerpc asm in the updated patch as you did for x86 and
> ARM.

Allright, thanks!

Mathieu

> 
> Regards,
> Boqun
> 
> 
>> Thanks,
>> 
>> Mathieu
>> 
>> > +
>> > +# if __PPC64__
>> > +# define has_single_copy_load_64()	1
>> > +# else
>> > +# define has_single_copy_load_64()	0
>> > +# endif
>> > +
>> > #else
>> > #error unsupported target
>> > #endif
>> > @@ -404,6 +433,97 @@ bool rseq_finish(struct rseq_lock *rlock,
>> > 			: succeed
>> > 		);
>> > 	}
>> > +#elif __PPC64__
>> > +	{
>> > +		/*
>> > +		 * The __rseq_table section can be used by debuggers to better
>> > +		 * handle single-stepping through the restartable critical
>> > +		 * sections.
>> > +		 */
>> > +		__asm__ __volatile__ goto (
>> > +			".pushsection __rseq_table, \"aw\"\n\t"
>> > +			".balign 8\n\t"
>> > +			"4:\n\t"
>> > +			".quad 1f, 2f, 3f\n\t"
>> > +			".popsection\n\t"
>> > +			"1:\n\t"
>> > +			RSEQ_INJECT_ASM(1)
>> > +			"lis %%r17, (4b)@highest\n\t"
>> > +			"ori %%r17, %%r17, (4b)@higher\n\t"
>> > +			"rldicr %%r17, %%r17, 32, 31\n\t"
>> > +			"oris %%r17, %%r17, (4b)@h\n\t"
>> > +			"ori %%r17, %%r17, (4b)@l\n\t"
>> > +			"std %%r17, 0(%[rseq_cs])\n\t"
>> > +			RSEQ_INJECT_ASM(2)
>> > +			"lwz %%r17, %[current_event_counter]\n\t"
>> > +			"li %%r16, 0\n\t"
>> > +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
>> > +			"bne cr7, 3f\n\t"
>> > +			RSEQ_INJECT_ASM(3)
>> > +			"std %[to_write], 0(%[target])\n\t"
>> > +			"2:\n\t"
>> > +			RSEQ_INJECT_ASM(4)
>> > +			"std %%r16, 0(%[rseq_cs])\n\t"
>> > +			"b %l[succeed]\n\t"
>> > +			"3:\n\t"
>> > +			"li %%r16, 0\n\t"
>> > +			"std %%r16, 0(%[rseq_cs])\n\t"
>> > +			: /* no outputs */
>> > +			: [start_event_counter]"r"(start_value.event_counter),
>> > +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
>> > +			  [to_write]"r"(to_write),
>> > +			  [target]"b"(p),
>> > +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
>> > +			  RSEQ_INJECT_INPUT
>> > +			: "r16", "r17", "memory", "cc"
>> > +			  RSEQ_INJECT_CLOBBER
>> > +			: succeed
>> > +		);
>> > +	}
>> > +#elif __PPC__
>> > +	{
>> > +		/*
>> > +		 * The __rseq_table section can be used by debuggers to better
>> > +		 * handle single-stepping through the restartable critical
>> > +		 * sections.
>> > +		 */
>> > +		__asm__ __volatile__ goto (
>> > +			".pushsection __rseq_table, \"aw\"\n\t"
>> > +			".balign 8\n\t"
>> > +			"4:\n\t"
>> > +			".long 0x0, 1f, 0x0, 2f, 0x0, 3f\n\t" /* 32 bit only supported on BE */
>> > +			".popsection\n\t"
>> > +			"1:\n\t"
>> > +			RSEQ_INJECT_ASM(1)
>> > +			"lis %%r17, (4b)@ha\n\t"
>> > +			"addi %%r17, %%r17, (4b)@l\n\t"
>> > +			"stw %%r17, 0(%[rseq_cs])\n\t"
>> > +			RSEQ_INJECT_ASM(2)
>> > +			"lwz %%r17, %[current_event_counter]\n\t"
>> > +			"li %%r16, 0\n\t"
>> > +			"cmpw cr7, %[start_event_counter], %%r17\n\t"
>> > +			"bne cr7, 3f\n\t"
>> > +			RSEQ_INJECT_ASM(3)
>> > +			"stw %[to_write], 0(%[target])\n\t"
>> > +			"2:\n\t"
>> > +			RSEQ_INJECT_ASM(4)
>> > +			"stw %%r16, 0(%[rseq_cs])\n\t"
>> > +			"b %l[succeed]\n\t"
>> > +			"3:\n\t"
>> > +			"li %%r16, 0\n\t"
>> > +			"stw %%r16, 0(%[rseq_cs])\n\t"
>> > +			: /* no outputs */
>> > +			: [start_event_counter]"r"(start_value.event_counter),
>> > +			  [current_event_counter]"m"(start_value.rseqp->abi.u.e.event_counter),
>> > +			  [to_write]"r"(to_write),
>> > +			  [target]"b"(p),
>> > +			  [rseq_cs]"b"(&start_value.rseqp->abi.rseq_cs)
>> > +			  RSEQ_INJECT_INPUT
>> > +			: "r16", "r17", "memory", "cc"
>> > +			  RSEQ_INJECT_CLOBBER
>> > +			: succeed
>> > +		);
>> > +	}
>> > #else
>> > #error unsupported target
>> > #endif
>> > --
>> > 2.9.0
>> 
>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
> > http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[toc] | [prev] | [next] | [standalone]

#1451337 — [RFC 1/4] rseq/param_test: Convert test_data_entry::count to intptr_t

FromBoqun Feng <boqun.feng@gmail.com>
Date2016-07-27 17:10 +0200
Subject[RFC 1/4] rseq/param_test: Convert test_data_entry::count to intptr_t
Message-ID<rZyD7-12s-17@gated-at.bofh.it>
In reply to#1451335
The current semantics of do_resq() is to do a intptr_t type store in
successful cases, however, in test_percpu_{inc,spinlock}, we use
test_data_entry::count as the location to store, whose type is int.

intptr_t and int have different size on LP64 systems, and despite the
inconsistency of types, having test_data_entry::count as int needs more
care on endian handling.

To make things simpler and more consistent, convert
test_data_entry::count to type intptr_t, which also makes the coming
tests for ppc64le and ppc64 share the same code.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 tools/testing/selftests/rseq/param_test.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/rseq/param_test.c b/tools/testing/selftests/rseq/param_test.c
index f95fba5a1b2a..db25e0a818e5 100644
--- a/tools/testing/selftests/rseq/param_test.c
+++ b/tools/testing/selftests/rseq/param_test.c
@@ -124,7 +124,7 @@ struct percpu_lock {
 };
 
 struct test_data_entry {
-	int count;
+	intptr_t count;
 } __attribute__((aligned(128)));
 
 struct spinlock_test_data {
@@ -234,7 +234,8 @@ void *test_percpu_spinlock_thread(void *arg)
 void test_percpu_spinlock(void)
 {
 	const int num_threads = opt_threads;
-	int i, sum, ret;
+	int i, ret;
+	intptr_t sum;
 	pthread_t test_threads[num_threads];
 	struct spinlock_test_data data;
 	struct spinlock_thread_test_data thread_data[num_threads];
@@ -308,7 +309,8 @@ void *test_percpu_inc_thread(void *arg)
 void test_percpu_inc(void)
 {
 	const int num_threads = opt_threads;
-	int i, sum, ret;
+	int i, ret;
+	intptr_t sum;
 	pthread_t test_threads[num_threads];
 	struct inc_test_data data;
 	struct inc_thread_test_data thread_data[num_threads];
-- 
2.9.0

[toc] | [prev] | [next] | [standalone]

#1451338 — [RFC 3/4] Restartable sequences: Wire up powerpc system call

FromBoqun Feng <boqun.feng@gmail.com>
Date2016-07-27 17:10 +0200
Subject[RFC 3/4] Restartable sequences: Wire up powerpc system call
Message-ID<rZyD8-12s-33@gated-at.bofh.it>
In reply to#1451337
Wire up the rseq system call on powerpc.

This provides an ABI improving the speed of a user-space getcpu
operation on powerpc by skipping the getcpu system call on the fast
path, as well as improving the speed of user-space operations on per-cpu
data compared to using load-reservation/store-conditional atomics.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 arch/powerpc/include/asm/systbl.h      | 1 +
 arch/powerpc/include/asm/unistd.h      | 2 +-
 arch/powerpc/include/uapi/asm/unistd.h | 1 +
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/systbl.h b/arch/powerpc/include/asm/systbl.h
index 2fc5d4db503c..c68f4d0d00b2 100644
--- a/arch/powerpc/include/asm/systbl.h
+++ b/arch/powerpc/include/asm/systbl.h
@@ -386,3 +386,4 @@ SYSCALL(mlock2)
 SYSCALL(copy_file_range)
 COMPAT_SYS_SPU(preadv2)
 COMPAT_SYS_SPU(pwritev2)
+SYSCALL(rseq)
diff --git a/arch/powerpc/include/asm/unistd.h b/arch/powerpc/include/asm/unistd.h
index cf12c580f6b2..a01e97d3f305 100644
--- a/arch/powerpc/include/asm/unistd.h
+++ b/arch/powerpc/include/asm/unistd.h
@@ -12,7 +12,7 @@
 #include <uapi/asm/unistd.h>
 
 
-#define NR_syscalls		382
+#define NR_syscalls		383
 
 #define __NR__exit __NR_exit
 
diff --git a/arch/powerpc/include/uapi/asm/unistd.h b/arch/powerpc/include/uapi/asm/unistd.h
index e9f5f41aa55a..d1849d64c8ef 100644
--- a/arch/powerpc/include/uapi/asm/unistd.h
+++ b/arch/powerpc/include/uapi/asm/unistd.h
@@ -392,5 +392,6 @@
 #define __NR_copy_file_range	379
 #define __NR_preadv2		380
 #define __NR_pwritev2		381
+#define __NR_rseq		382
 
 #endif /* _UAPI_ASM_POWERPC_UNISTD_H_ */
-- 
2.9.0

[toc] | [prev] | [next] | [standalone]

#1451682 — Re: [RFC 3/4] Restartable sequences: Wire up powerpc system call

FromMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date2016-07-28 05:20 +0200
SubjectRe: [RFC 3/4] Restartable sequences: Wire up powerpc system call
Message-ID<rZK1z-kW-7@gated-at.bofh.it>
In reply to#1451338
----- On Jul 27, 2016, at 11:05 AM, Boqun Feng boqun.feng@gmail.com wrote:

> Wire up the rseq system call on powerpc.
> 
> This provides an ABI improving the speed of a user-space getcpu
> operation on powerpc by skipping the getcpu system call on the fast
> path, as well as improving the speed of user-space operations on per-cpu
> data compared to using load-reservation/store-conditional atomics.

Picked up in my dev branch too, thanks!

Mathieu

> 
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
> arch/powerpc/include/asm/systbl.h      | 1 +
> arch/powerpc/include/asm/unistd.h      | 2 +-
> arch/powerpc/include/uapi/asm/unistd.h | 1 +
> 3 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/powerpc/include/asm/systbl.h
> b/arch/powerpc/include/asm/systbl.h
> index 2fc5d4db503c..c68f4d0d00b2 100644
> --- a/arch/powerpc/include/asm/systbl.h
> +++ b/arch/powerpc/include/asm/systbl.h
> @@ -386,3 +386,4 @@ SYSCALL(mlock2)
> SYSCALL(copy_file_range)
> COMPAT_SYS_SPU(preadv2)
> COMPAT_SYS_SPU(pwritev2)
> +SYSCALL(rseq)
> diff --git a/arch/powerpc/include/asm/unistd.h
> b/arch/powerpc/include/asm/unistd.h
> index cf12c580f6b2..a01e97d3f305 100644
> --- a/arch/powerpc/include/asm/unistd.h
> +++ b/arch/powerpc/include/asm/unistd.h
> @@ -12,7 +12,7 @@
> #include <uapi/asm/unistd.h>
> 
> 
> -#define NR_syscalls		382
> +#define NR_syscalls		383
> 
> #define __NR__exit __NR_exit
> 
> diff --git a/arch/powerpc/include/uapi/asm/unistd.h
> b/arch/powerpc/include/uapi/asm/unistd.h
> index e9f5f41aa55a..d1849d64c8ef 100644
> --- a/arch/powerpc/include/uapi/asm/unistd.h
> +++ b/arch/powerpc/include/uapi/asm/unistd.h
> @@ -392,5 +392,6 @@
> #define __NR_copy_file_range	379
> #define __NR_preadv2		380
> #define __NR_pwritev2		381
> +#define __NR_rseq		382
> 
> #endif /* _UAPI_ASM_POWERPC_UNISTD_H_ */
> --
> 2.9.0

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[toc] | [prev] | [next] | [standalone]

#1451668 — Re: [RFC 1/4] rseq/param_test: Convert test_data_entry::count to intptr_t

FromMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date2016-07-28 05:10 +0200
SubjectRe: [RFC 1/4] rseq/param_test: Convert test_data_entry::count to intptr_t
Message-ID<rZJRU-h8-33@gated-at.bofh.it>
In reply to#1451337
----- On Jul 27, 2016, at 11:05 AM, Boqun Feng boqun.feng@gmail.com wrote:

> The current semantics of do_resq() is to do a intptr_t type store in
> successful cases, however, in test_percpu_{inc,spinlock}, we use
> test_data_entry::count as the location to store, whose type is int.
> 
> intptr_t and int have different size on LP64 systems, and despite the
> inconsistency of types, having test_data_entry::count as int needs more
> care on endian handling.
> 
> To make things simpler and more consistent, convert
> test_data_entry::count to type intptr_t, which also makes the coming
> tests for ppc64le and ppc64 share the same code.

Folded into my rseq tests patch for next round, thanks!

I also took care of basic_percpu_ops_test.c which had the
same issue.

Thanks!

Mathieu

> 
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
> tools/testing/selftests/rseq/param_test.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/testing/selftests/rseq/param_test.c
> b/tools/testing/selftests/rseq/param_test.c
> index f95fba5a1b2a..db25e0a818e5 100644
> --- a/tools/testing/selftests/rseq/param_test.c
> +++ b/tools/testing/selftests/rseq/param_test.c
> @@ -124,7 +124,7 @@ struct percpu_lock {
> };
> 
> struct test_data_entry {
> -	int count;
> +	intptr_t count;
> } __attribute__((aligned(128)));
> 
> struct spinlock_test_data {
> @@ -234,7 +234,8 @@ void *test_percpu_spinlock_thread(void *arg)
> void test_percpu_spinlock(void)
> {
> 	const int num_threads = opt_threads;
> -	int i, sum, ret;
> +	int i, ret;
> +	intptr_t sum;
> 	pthread_t test_threads[num_threads];
> 	struct spinlock_test_data data;
> 	struct spinlock_thread_test_data thread_data[num_threads];
> @@ -308,7 +309,8 @@ void *test_percpu_inc_thread(void *arg)
> void test_percpu_inc(void)
> {
> 	const int num_threads = opt_threads;
> -	int i, sum, ret;
> +	int i, ret;
> +	intptr_t sum;
> 	pthread_t test_threads[num_threads];
> 	struct inc_test_data data;
> 	struct inc_thread_test_data thread_data[num_threads];
> --
> 2.9.0

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[toc] | [prev] | [next] | [standalone]

#1451681

FromMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date2016-07-28 05:20 +0200
Message-ID<rZK1z-kW-5@gated-at.bofh.it>
In reply to#1451335
----- On Jul 27, 2016, at 11:03 AM, Boqun Feng boqun.feng@gmail.com wrote:

> Hi Mathieu,
> 
> On Thu, Jul 21, 2016 at 05:14:16PM -0400, Mathieu Desnoyers wrote:
>> Expose a new system call allowing each thread to register one userspace
>> memory area to be used as an ABI between kernel and user-space for two
>> purposes: user-space restartable sequences and quick access to read the
>> current CPU number value from user-space.
>> 
>> * Restartable sequences (per-cpu atomics)
>> 
>> The restartable critical sections (percpu atomics) work has been started
>> by Paul Turner and Andrew Hunter. It lets the kernel handle restart of
>> critical sections. [1] [2] The re-implementation proposed here brings a
>> few simplifications to the ABI which facilitates porting to other
> 
> Agreed ;-)
> 
>> architectures and speeds up the user-space fast path. A locking-based
>> fall-back, purely implemented in user-space, is proposed here to deal
>> with debugger single-stepping. This fallback interacts with rseq_start()
>> and rseq_finish(), which force retries in response to concurrent
>> lock-based activity.
>> 
> 
> So I have enabled this on powerpc, thanks to your nice work to make
> things easy for porting ;-)
> 
> A patchset will follow in-reply-to this email, which includes patches
> enabling this on powerpc and a patch that improves the portability of
> the selftests, which I think it's not necessary to be a standalone
> patch, so it's OK to be merged into your patch #7.
> 
> I did some tests on 64bit little/big endian pSeries(guest) kernel with
> selftest cases(64bit LE selftest on 64bit LE kernel, 64/32bit BE
> selftest on 64bit BE kernel), things seemingly went well ;-)
> 
> Here are some benchmark results I got on a little endian guest with 64
> VCPUs:
> 
> Benchmarking various approaches for reading the current CPU number:
> 
> Power8 PSeries Guest(64 VCPUs, the host has 16 cores, 128 hardware
> threads):
>							
> - Baseline (empty loop):                                   1.56 ns
> - Read CPU from rseq cpu_id:                               1.56 ns
> - Read CPU from rseq cpu_id (lazy register):               2.08 ns
> - glibc 2.23-0ubuntu3 getcpu:                              7.72 ns
> - getcpu system call:                                     91.80 ns
> 
> 
> Benchmarking various approaches for counter increment:
> 
> Power8 PSeries KVM Guest(64 VCPUs, the host has 16 cores, 128 hardware
> threads):
> 
>                                 Counter increment speed (ns/increment)
>                              1 thread   2 threads   4 threads   8 threads   16 threads   32 threads
> global increment (baseline)     6.5          N/A         N/A         N/A
> N/A           N/A
> percpu rseq increment           6.9          6.9         7.2         7.3
> 15.4          35.5
> percpu rseq spinlock           19.0         18.9        19.4        19.4
> 35.5          71.8
> global atomic increment        25.8        111.0       261.0       905.2
> 2319.5        4170.5 (__sync_add_and_fetch_4)
> global atomic CAS              26.2        119.0       341.6      1183.0
> 3951.3        9312.5 (__sync_val_compare_and_swap_4)
> global pthread mutex           40.0        238.1       644.0      2052.2
> 4272.5        8612.2
> 
> 
> I surely need to run more tests for my patches in different
> environments, and will try to adjust the patchset according to whatever
> change you make(e.g. rseq_finish2) in the future.

I'm very glad to see it brings speedup on powerpc too! I plan
minor changes following the feedback I already got. I'll surely
grab your updated benchmark numbers into my changelog when I stop
hiding in RFC. ;)

Thanks,

Mathieu

> 
> (Add PPC maintainers in Cc)
> 
> Regards,
> Boqun
> 
>> Here are benchmarks of counter increment in various scenarios compared
>> to restartable sequences:
>> 
>> ARMv7 Processor rev 4 (v7l)
>> Machine model: Cubietruck
>> 
>>                       Counter increment speed (ns/increment)
>>                              1 thread    2 threads
>> global increment (baseline)      6           N/A
>> percpu rseq increment           50            52
>> percpu rseq spinlock            94            94
>> global atomic increment         48            74 (__sync_add_and_fetch_4)
>> global atomic CAS               50           172 (__sync_val_compare_and_swap_4)
>> global pthread mutex           148           862
>> 
>> ARMv7 Processor rev 10 (v7l)
>> Machine model: Wandboard
>> 
>>                       Counter increment speed (ns/increment)
>>                              1 thread    4 threads
>> global increment (baseline)      7           N/A
>> percpu rseq increment           50            50
>> percpu rseq spinlock            82            84
>> global atomic increment         44           262 (__sync_add_and_fetch_4)
>> global atomic CAS               46           316 (__sync_val_compare_and_swap_4)
>> global pthread mutex           146          1400
>> 
>> x86-64 Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz:
>> 
>>                       Counter increment speed (ns/increment)
>>                               1 thread           8 threads
>> global increment (baseline)      3.0                N/A
>> percpu rseq increment            3.6                3.8
>> percpu rseq spinlock             5.6                6.2
>> global LOCK; inc                 8.0              166.4
>> global LOCK; cmpxchg            13.4              435.2
>> global pthread mutex            25.2             1363.6
>> 
>> * Reading the current CPU number
>> 
>> Speeding up reading the current CPU number on which the caller thread is
>> running is done by keeping the current CPU number up do date within the
>> cpu_id field of the memory area registered by the thread. This is done
>> by making scheduler migration set the TIF_NOTIFY_RESUME flag on the
>> current thread. Upon return to user-space, a notify-resume handler
>> updates the current CPU value within the registered user-space memory
>> area. User-space can then read the current CPU number directly from
>> memory.
>> 
>> Keeping the current cpu id in a memory area shared between kernel and
>> user-space is an improvement over current mechanisms available to read
>> the current CPU number, which has the following benefits over
>> alternative approaches:
>> 
>> - 35x speedup on ARM vs system call through glibc
>> - 20x speedup on x86 compared to calling glibc, which calls vdso
>>   executing a "lsl" instruction,
>> - 14x speedup on x86 compared to inlined "lsl" instruction,
>> - Unlike vdso approaches, this cpu_id value can be read from an inline
>>   assembly, which makes it a useful building block for restartable
>>   sequences.
>> - The approach of reading the cpu id through memory mapping shared
>>   between kernel and user-space is portable (e.g. ARM), which is not the
>>   case for the lsl-based x86 vdso.
>> 
>> On x86, yet another possible approach would be to use the gs segment
>> selector to point to user-space per-cpu data. This approach performs
>> similarly to the cpu id cache, but it has two disadvantages: it is
>> not portable, and it is incompatible with existing applications already
>> using the gs segment selector for other purposes.
>> 
>> Benchmarking various approaches for reading the current CPU number:
>> 
>> ARMv7 Processor rev 4 (v7l)
>> Machine model: Cubietruck
>> - Baseline (empty loop):                                    8.4 ns
>> - Read CPU from rseq cpu_id:                               16.7 ns
>> - Read CPU from rseq cpu_id (lazy register):               19.8 ns
>> - glibc 2.19-0ubuntu6.6 getcpu:                           301.8 ns
>> - getcpu system call:                                     234.9 ns
>> 
>> x86-64 Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz:
>> - Baseline (empty loop):                                    0.8 ns
>> - Read CPU from rseq cpu_id:                                0.8 ns
>> - Read CPU from rseq cpu_id (lazy register):                0.8 ns
>> - Read using gs segment selector:                           0.8 ns
>> - "lsl" inline assembly:                                   13.0 ns
>> - glibc 2.19-0ubuntu6 getcpu:                              16.6 ns
>> - getcpu system call:                                      53.9 ns
>> 
>> - Speed
>> 
>> Running 10 runs of hackbench -l 100000 seems to indicate, contrary to
>> expectations, that enabling CONFIG_RSEQ slightly accelerates the
>> scheduler:
>> 
>> Configuration: 2 sockets * 8-core Intel(R) Xeon(R) CPU E5-2630 v3 @
>> 2.40GHz (directly on hardware, hyperthreading disabled in BIOS, energy
>> saving disabled in BIOS, turboboost disabled in BIOS, cpuidle.off=1
>> kernel parameter), with a Linux v4.6 defconfig+localyesconfig,
>> restartable sequences series applied.
>> 
> 
> [snip]

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web