Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1330763 > unrolled thread

Re: [PATCH v10 3/4] x86, mce: Add __mcsafe_copy()

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v10 3/4] x86, mce: Add __mcsafe_copy() "Luck, Tony" <tony.luck@intel.com> - 2016-02-10 00:20 +0100
    Re: [PATCH v10 3/4] x86, mce: Add __mcsafe_copy() Borislav Petkov <bp@alien8.de> - 2016-02-10 12:00 +0100
      Re: [PATCH v10 3/4] x86, mce: Add __mcsafe_copy() "Luck, Tony" <tony.luck@intel.com> - 2016-02-10 20:40 +0100
        Re: [PATCH v10 3/4] x86, mce: Add __mcsafe_copy() Borislav Petkov <bp@alien8.de> - 2016-02-10 22:00 +0100

#1330763 — Re: [PATCH v10 3/4] x86, mce: Add __mcsafe_copy()

> You can save yourself this MOV here in what is, I'm assuming, the
> general likely case where @src is aligned and do:
> 
>         /* check for bad alignment of source */
>         testl $7, %esi
>         /* already aligned? */
>         jz 102f
> 
>         movl %esi,%ecx
>         subl $8,%ecx
>         negl %ecx
>         subl %ecx,%edx
> 0:      movb (%rsi),%al
>         movb %al,(%rdi)
>         incq %rsi
>         incq %rdi
>         decl %ecx
>         jnz 0b

The "testl $7, %esi" just checks the low three bits ... it doesn't
change %esi.  But the code from the "subl $8" on down assumes that
%ecx is a number in [1..7] as the count of bytes to copy until we
achieve alignment.

So your "movl %esi,%ecx" needs to be somthing that just copies the
low three bits and zeroes the high part of %ecx.  Is there a cute
way to do that in x86 assembler?

> Why aren't we pushing %r12-%r15 on the stack after the "jz 17f" above
> and using them too and thus copying a whole cacheline in one go?
> 
> We would need to restore them when we're done with the cacheline-wise
> shuffle, of course.

I copied that loop from arch/x86/lib/copy_user_64.S:__copy_user_nocache()
I guess the answer depends on whether you generally copy enough
cache lines to save enough time to cover the cost of saving and
restoring those registers.

-Tony

[toc] | [next] | [standalone]

#1331065

On Tue, Feb 09, 2016 at 03:15:57PM -0800, Luck, Tony wrote:
> > You can save yourself this MOV here in what is, I'm assuming, the
> > general likely case where @src is aligned and do:
> > 
> >         /* check for bad alignment of source */
> >         testl $7, %esi
> >         /* already aligned? */
> >         jz 102f
> > 
> >         movl %esi,%ecx
> >         subl $8,%ecx
> >         negl %ecx
> >         subl %ecx,%edx
> > 0:      movb (%rsi),%al
> >         movb %al,(%rdi)
> >         incq %rsi
> >         incq %rdi
> >         decl %ecx
> >         jnz 0b
> 
> The "testl $7, %esi" just checks the low three bits ... it doesn't
> change %esi.  But the code from the "subl $8" on down assumes that
> %ecx is a number in [1..7] as the count of bytes to copy until we
> achieve alignment.

Grr, sorry about that, I actually missed to copy-paste the AND:

        /* check for bad alignment of source */
        testl $7, %esi
        jz 102f                         /* already aligned */

        movl %esi,%ecx
        andl $7,%ecx
        subl $8,%ecx
        negl %ecx
        subl %ecx,%edx
0:      movb (%rsi),%al
        movb %al,(%rdi)
        incq %rsi
        incq %rdi
        decl %ecx
        jnz 0b

I basically am proposing to move the unlikely case out of line and
optimize the likely one.

> So your "movl %esi,%ecx" needs to be somthing that just copies the
> low three bits and zeroes the high part of %ecx.  Is there a cute
> way to do that in x86 assembler?

We could do some funky games with byte-sized moves but those are
generally slower anyway so doing the default operand size thing should
be ok.

> I copied that loop from arch/x86/lib/copy_user_64.S:__copy_user_nocache()
> I guess the answer depends on whether you generally copy enough
> cache lines to save enough time to cover the cost of saving and
> restoring those registers.

Well, that function will run on modern hw with a stack engine so I'd
assume those 4 pushes and pops would be paid for by the increased
registers count for the data shuffling.

But one could take out that function do some microbenchmarking with
different sizes and once with the current version and once with the
pushes and pops of r1[2-5] to see where the breakeven is.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

[toc] | [prev] | [next] | [standalone]

#1331452

On Wed, Feb 10, 2016 at 11:58:43AM +0100, Borislav Petkov wrote:
> But one could take out that function do some microbenchmarking with
> different sizes and once with the current version and once with the
> pushes and pops of r1[2-5] to see where the breakeven is.

On a 4K page copy from a source address that isn't in the
cache I see all sorts of answers.

On my desktop (i7-3960X) it is ~50 cycles slower to push and pop the four
registers.

On my latest Xeon - I can't post benchmarks ... but also a bit slower.

On an older Xeon it is a few cycles faster (but even though I'm
looking at the median of 10,000 runs I see more run-to-run variation
that I see difference between register choices.

Here's what I tested:

	push %r12
	push %r13
	push %r14
	push %r15

	/* Loop copying whole cache lines */
1:	movq (%rsi),%r8
2:	movq 1*8(%rsi),%r9
3:	movq 2*8(%rsi),%r10
4:	movq 3*8(%rsi),%r11
9:	movq 4*8(%rsi),%r12
10:	movq 5*8(%rsi),%r13
11:	movq 6*8(%rsi),%r14
12:	movq 7*8(%rsi),%r15
	movq %r8,(%rdi)
	movq %r9,1*8(%rdi)
	movq %r10,2*8(%rdi)
	movq %r11,3*8(%rdi)
	movq %r12,4*8(%rdi)
	movq %r13,5*8(%rdi)
	movq %r14,6*8(%rdi)
	movq %r15,7*8(%rdi)
	leaq 64(%rsi),%rsi
	leaq 64(%rdi),%rdi
	decl %ecx
	jnz 1b

	pop %r15
	pop %r14
	pop %r13
	pop %r12
-Tony

[toc] | [prev] | [next] | [standalone]

#1331481

On Wed, Feb 10, 2016 at 11:39:05AM -0800, Luck, Tony wrote:
> On Wed, Feb 10, 2016 at 11:58:43AM +0100, Borislav Petkov wrote:
> > But one could take out that function do some microbenchmarking with
> > different sizes and once with the current version and once with the
> > pushes and pops of r1[2-5] to see where the breakeven is.
> 
> On a 4K page copy from a source address that isn't in the
> cache I see all sorts of answers.
> 
> On my desktop (i7-3960X) it is ~50 cycles slower to push and pop the four
> registers.
> 
> On my latest Xeon - I can't post benchmarks ... but also a bit slower.
> 
> On an older Xeon it is a few cycles faster (but even though I'm
> looking at the median of 10,000 runs I see more run-to-run variation
> that I see difference between register choices.

Hmm, strange. Can you check whether perf doesn't show any significant
differences too. Something like:

perf stat --repeat 100 --sync --pre 'echo 3 > /proc/sys/vm/drop_caches' -- ./mcsafe_copy_1

and then

perf stat --repeat 100 --sync --pre 'echo 3 > /proc/sys/vm/drop_caches' -- ./mcsafe_copy_2

That'll be interesting...

Thanks.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web