Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1319061 > unrolled thread

[PATCH v4 5/5] x86: drop mfence in favor of lock+addl

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH v4 5/5] x86: drop mfence in favor of lock+addl "Michael S. Tsirkin" <mst@redhat.com> - 2016-01-27 16:20 +0100
    Re: [PATCH v4 5/5] x86: drop mfence in favor of lock+addl Linus Torvalds <torvalds@linux-foundation.org> - 2016-01-27 18:40 +0100
      Re: [PATCH v4 5/5] x86: drop mfence in favor of lock+addl "Michael S. Tsirkin" <mst@redhat.com> - 2016-01-27 19:00 +0100

#1319061 — [PATCH v4 5/5] x86: drop mfence in favor of lock+addl

mfence appears to be way slower than a locked instruction - let's use
lock+add unconditionally, as we always did on old 32-bit.

Just poking at SP would be the most natural, but if we
then read the value from SP, we get a false dependency
which will slow us down.

This was noted in this article:
http://shipilev.net/blog/2014/on-the-fence-with-dependencies/

And is easy to reproduce by sticking a barrier in a small non-inline
function.

So let's use a negative offset - which avoids this problem since we
build with the red zone disabled.

Unfortunately there's some code that wants to order clflush instructions
using mb(), so we can't replace that - but smp_mb should be safe
to replace.

Update mb/rmb/wmb on 32 bit to use the negative offset, too, for
consistency.

Suggested-by: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 arch/x86/include/asm/barrier.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h
index bfb28ca..7ab9581 100644
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -11,11 +11,11 @@
  */
 
 #ifdef CONFIG_X86_32
-#define mb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "mfence", \
+#define mb() asm volatile(ALTERNATIVE("lock; addl $0,-4(%%esp)", "mfence", \
 				      X86_FEATURE_XMM2) ::: "memory", "cc")
-#define rmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "lfence", \
+#define rmb() asm volatile(ALTERNATIVE("lock; addl $0,-4(%%esp)", "lfence", \
 				       X86_FEATURE_XMM2) ::: "memory", "cc")
-#define wmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "sfence", \
+#define wmb() asm volatile(ALTERNATIVE("lock; addl $0,-4(%%esp)", "sfence", \
 				       X86_FEATURE_XMM2) ::: "memory", "cc")
 #else
 #define mb() 	asm volatile("mfence":::"memory")
@@ -30,7 +30,7 @@
 #endif
 #define dma_wmb()	barrier()
 
-#define __smp_mb()	mb()
+#define __smp_mb()	asm volatile("lock; addl $0,-4(%%esp)" ::: "memory", "cc")
 #define __smp_rmb()	dma_rmb()
 #define __smp_wmb()	barrier()
 #define __smp_store_mb(var, value) do { (void)xchg(&var, value); } while (0)
-- 
MST

[toc] | [next] | [standalone]

#1319185

On Wed, Jan 27, 2016 at 7:10 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> -#define __smp_mb()     mb()
> +#define __smp_mb()     asm volatile("lock; addl $0,-4(%%esp)" ::: "memory", "cc")

So this doesn't look right for x86-64. Using %esp rather than %rsp.
How did that even work for you?

                Linus

[toc] | [prev] | [next] | [standalone]

#1319196

On Wed, Jan 27, 2016 at 09:37:45AM -0800, Linus Torvalds wrote:
> On Wed, Jan 27, 2016 at 7:10 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > -#define __smp_mb()     mb()
> > +#define __smp_mb()     asm volatile("lock; addl $0,-4(%%esp)" ::: "memory", "cc")
> 
> So this doesn't look right for x86-64. Using %esp rather than %rsp.
> How did that even work for you?
> 
>                 Linus

Oops, didn't test this version properly.
Pls disregard, I'll repost after some proper testing.

-- 
MST

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web