Groups > linux.kernel > #1722662 > unrolled thread

[PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

Started by	Josh Poimboeuf <jpoimboe@redhat.com>
First post	2017-08-29 20:10 +0200
Last post	2017-09-08 20:00 +0200
Articles	17 — 4 participants

Back to article view | Back to linux.kernel

  [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
      Re: [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S Tim Chen <tim.c.chen@linux.intel.com> - 2017-09-06 18:20 +0200
    [PATCH 12/12] x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 11/12] x86/crypto: Fix RBP usage in sha512-avx2-asm.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 10/12] x86/crypto: Fix RBP usage in sha256-ssse3-asm.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 01/12] x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 05/12] x86/crypto: Fix RBP usage in des3_ede-asm_64.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 07/12] x86/crypto: Fix RBP usage in sha1_ssse3_asm.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 02/12] x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:10 +0200
    [PATCH 03/12] x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S Josh Poimboeuf <jpoimboe@redhat.com> - 2017-08-29 20:20 +0200
    Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S  files Eric Biggers <ebiggers3@gmail.com> - 2017-09-02 02:10 +0200
      Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S  files Josh Poimboeuf <jpoimboe@redhat.com> - 2017-09-07 02:20 +0200
      Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S  files Ingo Molnar <mingo@kernel.org> - 2017-09-07 09:20 +0200
        Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S  files Eric Biggers <ebiggers3@gmail.com> - 2017-09-07 20:00 +0200
          Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S  files Ingo Molnar <mingo@kernel.org> - 2017-09-07 23:30 +0200
            Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S  files Eric Biggers <ebiggers3@gmail.com> - 2017-09-08 20:00 +0200

#1722662 — [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<ujT7A-602-9@gated-at.bofh.it>

Many of the x86 crypto functions use RBP as a temporary register.  This
breaks frame pointer convention, and breaks stack traces when unwinding
from an interrupt in the crypto code.

Convert most* of them to leave RBP alone.

These pass the crypto boot tests for me.  Any further testing would be
appreciated!

[*] There are still a few crypto files left that need fixing, but the
    fixes weren't trivial and nobody reported unwinder warnings about
    them yet, so I'm skipping them for now.

Josh Poimboeuf (12):
  x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S
  x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S
  x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S
  x86/crypto: Fix RBP usage in cast6-avx-x86_64-asm_64.S
  x86/crypto: Fix RBP usage in des3_ede-asm_64.S
  x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S
  x86/crypto: Fix RBP usage in sha1_ssse3_asm.S
  x86/crypto: Fix RBP usage in sha256-avx-asm.S
  x86/crypto: Fix RBP usage in sha256-avx2-asm.S
  x86/crypto: Fix RBP usage in sha256-ssse3-asm.S
  x86/crypto: Fix RBP usage in sha512-avx2-asm.S
  x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S

 arch/x86/crypto/blowfish-x86_64-asm_64.S    | 48 ++++++++++++++-------------
 arch/x86/crypto/camellia-x86_64-asm_64.S    | 26 +++++++--------
 arch/x86/crypto/cast5-avx-x86_64-asm_64.S   | 47 +++++++++++++++++----------
 arch/x86/crypto/cast6-avx-x86_64-asm_64.S   | 50 ++++++++++++++++++++---------
 arch/x86/crypto/des3_ede-asm_64.S           | 15 +++++----
 arch/x86/crypto/sha1_avx2_x86_64_asm.S      |  4 +--
 arch/x86/crypto/sha1_ssse3_asm.S            | 11 +++----
 arch/x86/crypto/sha256-avx-asm.S            | 15 ++++-----
 arch/x86/crypto/sha256-avx2-asm.S           | 16 ++++-----
 arch/x86/crypto/sha256-ssse3-asm.S          | 15 ++++-----
 arch/x86/crypto/sha512-avx2-asm.S           | 21 ++++++++----
 arch/x86/crypto/twofish-avx-x86_64-asm_64.S | 12 +++----
 12 files changed, 160 insertions(+), 120 deletions(-)

-- 
2.13.5

[toc] | [next] | [standalone]

#1722664 — [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S
Message-ID	<ujT7A-602-27@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use R11 instead of RBP.  Since R11 isn't a callee-saved register, it
doesn't need to be saved and restored on the stack.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/sha1_avx2_x86_64_asm.S | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/arch/x86/crypto/sha1_avx2_x86_64_asm.S b/arch/x86/crypto/sha1_avx2_x86_64_asm.S
index 1eab79c9ac48..9f712a7dfd79 100644
--- a/arch/x86/crypto/sha1_avx2_x86_64_asm.S
+++ b/arch/x86/crypto/sha1_avx2_x86_64_asm.S
@@ -89,7 +89,7 @@
 #define	REG_RE	%rdx
 #define	REG_RTA	%r12
 #define	REG_RTB	%rbx
-#define	REG_T1	%ebp
+#define	REG_T1	%r11d
 #define	xmm_mov	vmovups
 #define	avx2_zeroupper	vzeroupper
 #define	RND_F1	1
@@ -637,7 +637,6 @@ _loop3:
 	ENTRY(\name)
 
 	push	%rbx
-	push	%rbp
 	push	%r12
 	push	%r13
 	push	%r14
@@ -673,7 +672,6 @@ _loop3:
 	pop	%r14
 	pop	%r13
 	pop	%r12
-	pop	%rbp
 	pop	%rbx
 
 	ret
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1727634 — Re: [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S

From	Tim Chen <tim.c.chen@linux.intel.com>
Date	2017-09-06 18:20 +0200
Subject	Re: [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S
Message-ID	<umLdw-4da-15@gated-at.bofh.it>
In reply to	#1722664

On 08/29/2017 11:05 AM, Josh Poimboeuf wrote:
> Using RBP as a temporary register breaks frame pointer convention and
> breaks stack traces when unwinding from an interrupt in the crypto code.
> 
> Use R11 instead of RBP.  Since R11 isn't a callee-saved register, it
> doesn't need to be saved and restored on the stack.

These changes seem okay.

Thanks.

Tim

> 
> Reported-by: Eric Biggers <ebiggers@google.com>
> Reported-by: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
> ---
>  arch/x86/crypto/sha1_avx2_x86_64_asm.S | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/arch/x86/crypto/sha1_avx2_x86_64_asm.S b/arch/x86/crypto/sha1_avx2_x86_64_asm.S
> index 1eab79c9ac48..9f712a7dfd79 100644
> --- a/arch/x86/crypto/sha1_avx2_x86_64_asm.S
> +++ b/arch/x86/crypto/sha1_avx2_x86_64_asm.S
> @@ -89,7 +89,7 @@
>  #define	REG_RE	%rdx
>  #define	REG_RTA	%r12
>  #define	REG_RTB	%rbx
> -#define	REG_T1	%ebp
> +#define	REG_T1	%r11d
>  #define	xmm_mov	vmovups
>  #define	avx2_zeroupper	vzeroupper
>  #define	RND_F1	1
> @@ -637,7 +637,6 @@ _loop3:
>  	ENTRY(\name)
>  
>  	push	%rbx
> -	push	%rbp
>  	push	%r12
>  	push	%r13
>  	push	%r14
> @@ -673,7 +672,6 @@ _loop3:
>  	pop	%r14
>  	pop	%r13
>  	pop	%r12
> -	pop	%rbp
>  	pop	%rbx
>  
>  	ret
>

[toc] | [prev] | [next] | [standalone]

#1722665 — [PATCH 12/12] x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 12/12] x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S
Message-ID	<ujT7A-602-29@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use R13 instead of RBP.  Both are callee-saved registers, so the
substitution is straightforward.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/twofish-avx-x86_64-asm_64.S | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/x86/crypto/twofish-avx-x86_64-asm_64.S b/arch/x86/crypto/twofish-avx-x86_64-asm_64.S
index b3f49d286348..73b471da3622 100644
--- a/arch/x86/crypto/twofish-avx-x86_64-asm_64.S
+++ b/arch/x86/crypto/twofish-avx-x86_64-asm_64.S
@@ -76,8 +76,8 @@
 #define RT %xmm14
 #define RR %xmm15
 
-#define RID1  %rbp
-#define RID1d %ebp
+#define RID1  %r13
+#define RID1d %r13d
 #define RID2  %rsi
 #define RID2d %esi
 
@@ -259,7 +259,7 @@ __twofish_enc_blk8:
 
 	vmovdqu w(CTX), RK1;
 
-	pushq %rbp;
+	pushq %r13;
 	pushq %rbx;
 	pushq %rcx;
 
@@ -282,7 +282,7 @@ __twofish_enc_blk8:
 
 	popq %rcx;
 	popq %rbx;
-	popq %rbp;
+	popq %r13;
 
 	outunpack_blocks(RC1, RD1, RA1, RB1, RK1, RX0, RY0, RK2);
 	outunpack_blocks(RC2, RD2, RA2, RB2, RK1, RX0, RY0, RK2);
@@ -301,7 +301,7 @@ __twofish_dec_blk8:
 
 	vmovdqu (w+4*4)(CTX), RK1;
 
-	pushq %rbp;
+	pushq %r13;
 	pushq %rbx;
 
 	inpack_blocks(RC1, RD1, RA1, RB1, RK1, RX0, RY0, RK2);
@@ -322,7 +322,7 @@ __twofish_dec_blk8:
 	vmovdqu (w)(CTX), RK1;
 
 	popq %rbx;
-	popq %rbp;
+	popq %r13;
 
 	outunpack_blocks(RA1, RB1, RC1, RD1, RK1, RX0, RY0, RK2);
 	outunpack_blocks(RA2, RB2, RC2, RD2, RK1, RX0, RY0, RK2);
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722667 — [PATCH 11/12] x86/crypto: Fix RBP usage in sha512-avx2-asm.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 11/12] x86/crypto: Fix RBP usage in sha512-avx2-asm.S
Message-ID	<ujT7A-602-23@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use R12 instead of RBP for the TBL register.  Since R12 is also used as
another temporary register (T1), it gets clobbered in each round of
computation.  So the TBL value needs to be freshly reloaded into R12
each time it's used.  Since the value of TBL can change, store its
permanent value on the stack at the frame_TBL offset.

Also remove the unused y4 variable.

Reported-by: Eric Biggers <ebiggers3@gmail.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/sha512-avx2-asm.S | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/arch/x86/crypto/sha512-avx2-asm.S b/arch/x86/crypto/sha512-avx2-asm.S
index 7f5f6c6ec72e..37cfc2004abd 100644
--- a/arch/x86/crypto/sha512-avx2-asm.S
+++ b/arch/x86/crypto/sha512-avx2-asm.S
@@ -81,7 +81,7 @@ d           = %r8
 e           = %rdx
 y3          = %rsi
 
-TBL   = %rbp
+TBL   = %r12 # clobbered by T1
 
 a     = %rax
 b     = %rbx
@@ -96,11 +96,10 @@ y0    = %r13
 y1    = %r14
 y2    = %r15
 
-y4    = %r12
-
 # Local variables (stack frame)
 XFER_SIZE = 4*8
 SRND_SIZE = 1*8
+TBL_SIZE = 1*8
 INP_SIZE = 1*8
 INPEND_SIZE = 1*8
 RSPSAVE_SIZE = 1*8
@@ -108,7 +107,8 @@ GPRSAVE_SIZE = 6*8
 
 frame_XFER = 0
 frame_SRND = frame_XFER + XFER_SIZE
-frame_INP = frame_SRND + SRND_SIZE
+frame_TBL = frame_SRND + SRND_SIZE
+frame_INP = frame_TBL + TBL_SIZE
 frame_INPEND = frame_INP + INP_SIZE
 frame_RSPSAVE = frame_INPEND + INPEND_SIZE
 frame_GPRSAVE = frame_RSPSAVE + RSPSAVE_SIZE
@@ -601,7 +601,7 @@ ENTRY(sha512_transform_rorx)
 	vmovdqa	PSHUFFLE_BYTE_FLIP_MASK(%rip), BYTE_FLIP_MASK
 
 loop0:
-	lea	K512(%rip), TBL
+	movq	$K512, frame_TBL(%rsp)
 
 	## byte swap first 16 dwords
 	COPY_YMM_AND_BSWAP	Y_0, (INP), BYTE_FLIP_MASK
@@ -616,39 +616,46 @@ loop0:
 
 .align 16
 loop1:
+	mov frame_TBL(%rsp), TBL
 	vpaddq	(TBL), Y_0, XFER
 	vmovdqa XFER, frame_XFER(%rsp)
 	FOUR_ROUNDS_AND_SCHED
 
+	mov frame_TBL(%rsp), TBL
 	vpaddq	1*32(TBL), Y_0, XFER
 	vmovdqa XFER, frame_XFER(%rsp)
 	FOUR_ROUNDS_AND_SCHED
 
+	mov frame_TBL(%rsp), TBL
 	vpaddq	2*32(TBL), Y_0, XFER
 	vmovdqa XFER, frame_XFER(%rsp)
 	FOUR_ROUNDS_AND_SCHED
 
+	mov frame_TBL(%rsp), TBL
 	vpaddq	3*32(TBL), Y_0, XFER
 	vmovdqa XFER, frame_XFER(%rsp)
-	add	$(4*32), TBL
 	FOUR_ROUNDS_AND_SCHED
 
+	addq	$(4*32), frame_TBL(%rsp)
 	subq	$1, frame_SRND(%rsp)
 	jne	loop1
 
 	movq	$2, frame_SRND(%rsp)
 loop2:
+	mov frame_TBL(%rsp), TBL
 	vpaddq	(TBL), Y_0, XFER
 	vmovdqa XFER, frame_XFER(%rsp)
 	DO_4ROUNDS
+
+	mov frame_TBL(%rsp), TBL
 	vpaddq	1*32(TBL), Y_1, XFER
 	vmovdqa XFER, frame_XFER(%rsp)
-	add	$(2*32), TBL
 	DO_4ROUNDS
 
 	vmovdqa	Y_2, Y_0
 	vmovdqa	Y_3, Y_1
 
+	add	$(2*32), frame_TBL(%rsp)
 	subq	$1, frame_SRND(%rsp)
 	jne	loop2
 
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722669 — [PATCH 10/12] x86/crypto: Fix RBP usage in sha256-ssse3-asm.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 10/12] x86/crypto: Fix RBP usage in sha256-ssse3-asm.S
Message-ID	<ujT7B-602-37@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Swap the usages of R12 and RBP.  Use R12 for the TBL register, and use
RBP to store the pre-aligned stack pointer.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/sha256-ssse3-asm.S | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/arch/x86/crypto/sha256-ssse3-asm.S b/arch/x86/crypto/sha256-ssse3-asm.S
index 39b83c93e7fd..c6c05ed2c16a 100644
--- a/arch/x86/crypto/sha256-ssse3-asm.S
+++ b/arch/x86/crypto/sha256-ssse3-asm.S
@@ -95,7 +95,7 @@ SRND = %rsi       # clobbers INP
 c = %ecx
 d = %r8d
 e = %edx
-TBL = %rbp
+TBL = %r12
 a = %eax
 b = %ebx
 
@@ -356,13 +356,13 @@ a = TMP_
 ENTRY(sha256_transform_ssse3)
 .align 32
 	pushq   %rbx
-	pushq   %rbp
+	pushq   %r12
 	pushq   %r13
 	pushq   %r14
 	pushq   %r15
-	pushq   %r12
+	pushq   %rbp
+	mov	%rsp, %rbp
 
-	mov	%rsp, %r12
 	subq    $STACK_SIZE, %rsp
 	and	$~15, %rsp
 
@@ -462,13 +462,12 @@ loop2:
 
 done_hash:
 
-	mov	%r12, %rsp
-
-	popq    %r12
+	mov	%rbp, %rsp
+	popq	%rbp
 	popq    %r15
 	popq    %r14
 	popq    %r13
-	popq    %rbp
+	popq    %r12
 	popq    %rbx
 
 	ret
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722671 — [PATCH 01/12] x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 01/12] x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S
Message-ID	<ujT7B-602-41@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use R12 instead of RBP.  R12 can't be used as the RT0 register because
of x86 instruction encoding limitations.  So use R12 for CTX and RDI for
CTX.  This means that CTX is no longer an implicit function argument.
Instead it needs to be explicitly copied from RDI.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/blowfish-x86_64-asm_64.S | 48 +++++++++++++++++---------------
 1 file changed, 26 insertions(+), 22 deletions(-)

diff --git a/arch/x86/crypto/blowfish-x86_64-asm_64.S b/arch/x86/crypto/blowfish-x86_64-asm_64.S
index 246c67006ed0..8c1fcb6bad21 100644
--- a/arch/x86/crypto/blowfish-x86_64-asm_64.S
+++ b/arch/x86/crypto/blowfish-x86_64-asm_64.S
@@ -33,7 +33,7 @@
 #define s3	((16 + 2 + (3 * 256)) * 4)
 
 /* register macros */
-#define CTX %rdi
+#define CTX %r12
 #define RIO %rsi
 
 #define RX0 %rax
@@ -56,12 +56,12 @@
 #define RX2bh %ch
 #define RX3bh %dh
 
-#define RT0 %rbp
+#define RT0 %rdi
 #define RT1 %rsi
 #define RT2 %r8
 #define RT3 %r9
 
-#define RT0d %ebp
+#define RT0d %edi
 #define RT1d %esi
 #define RT2d %r8d
 #define RT3d %r9d
@@ -120,13 +120,14 @@
 
 ENTRY(__blowfish_enc_blk)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 *	%rcx: bool, if true: xor output
 	 */
-	movq %rbp, %r11;
+	movq %r12, %r11;
 
+	movq %rdi, CTX;
 	movq %rsi, %r10;
 	movq %rdx, RIO;
 
@@ -142,7 +143,7 @@ ENTRY(__blowfish_enc_blk)
 	round_enc(14);
 	add_roundkey_enc(16);
 
-	movq %r11, %rbp;
+	movq %r11, %r12;
 
 	movq %r10, RIO;
 	test %cl, %cl;
@@ -157,12 +158,13 @@ ENDPROC(__blowfish_enc_blk)
 
 ENTRY(blowfish_dec_blk)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 */
-	movq %rbp, %r11;
+	movq %r12, %r11;
 
+	movq %rdi, CTX;
 	movq %rsi, %r10;
 	movq %rdx, RIO;
 
@@ -181,7 +183,7 @@ ENTRY(blowfish_dec_blk)
 	movq %r10, RIO;
 	write_block();
 
-	movq %r11, %rbp;
+	movq %r11, %r12;
 
 	ret;
 ENDPROC(blowfish_dec_blk)
@@ -298,20 +300,21 @@ ENDPROC(blowfish_dec_blk)
 
 ENTRY(__blowfish_enc_blk_4way)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 *	%rcx: bool, if true: xor output
 	 */
-	pushq %rbp;
+	pushq %r12;
 	pushq %rbx;
 	pushq %rcx;
 
-	preload_roundkey_enc(0);
-
+	movq %rdi, CTX
 	movq %rsi, %r11;
 	movq %rdx, RIO;
 
+	preload_roundkey_enc(0);
+
 	read_block4();
 
 	round_enc4(0);
@@ -324,39 +327,40 @@ ENTRY(__blowfish_enc_blk_4way)
 	round_enc4(14);
 	add_preloaded_roundkey4();
 
-	popq %rbp;
+	popq %r12;
 	movq %r11, RIO;
 
-	test %bpl, %bpl;
+	test %r12b, %r12b;
 	jnz .L__enc_xor4;
 
 	write_block4();
 
 	popq %rbx;
-	popq %rbp;
+	popq %r12;
 	ret;
 
 .L__enc_xor4:
 	xor_block4();
 
 	popq %rbx;
-	popq %rbp;
+	popq %r12;
 	ret;
 ENDPROC(__blowfish_enc_blk_4way)
 
 ENTRY(blowfish_dec_blk_4way)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 */
-	pushq %rbp;
+	pushq %r12;
 	pushq %rbx;
-	preload_roundkey_dec(17);
 
-	movq %rsi, %r11;
+	movq %rdi, CTX;
+	movq %rsi, %r11
 	movq %rdx, RIO;
 
+	preload_roundkey_dec(17);
 	read_block4();
 
 	round_dec4(17);
@@ -373,7 +377,7 @@ ENTRY(blowfish_dec_blk_4way)
 	write_block4();
 
 	popq %rbx;
-	popq %rbp;
+	popq %r12;
 
 	ret;
 ENDPROC(blowfish_dec_blk_4way)
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722672 — [PATCH 05/12] x86/crypto: Fix RBP usage in des3_ede-asm_64.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 05/12] x86/crypto: Fix RBP usage in des3_ede-asm_64.S
Message-ID	<ujT7C-602-45@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use RSI instead of RBP for RT1.  Since RSI is also used as a the 'dst'
function argument, it needs to be saved on the stack until the argument
is needed.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/des3_ede-asm_64.S | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/arch/x86/crypto/des3_ede-asm_64.S b/arch/x86/crypto/des3_ede-asm_64.S
index f3e91647ca27..8e49ce117494 100644
--- a/arch/x86/crypto/des3_ede-asm_64.S
+++ b/arch/x86/crypto/des3_ede-asm_64.S
@@ -64,12 +64,12 @@
 #define RW2bh %ch
 
 #define RT0 %r15
-#define RT1 %rbp
+#define RT1 %rsi
 #define RT2 %r14
 #define RT3 %rdx
 
 #define RT0d %r15d
-#define RT1d %ebp
+#define RT1d %esi
 #define RT2d %r14d
 #define RT3d %edx
 
@@ -177,13 +177,14 @@ ENTRY(des3_ede_x86_64_crypt_blk)
 	 *	%rsi: dst
 	 *	%rdx: src
 	 */
-	pushq %rbp;
 	pushq %rbx;
 	pushq %r12;
 	pushq %r13;
 	pushq %r14;
 	pushq %r15;
 
+	pushq %rsi; /* dst */
+
 	read_block(%rdx, RL0, RR0);
 	initial_permutation(RL0, RR0);
 
@@ -241,6 +242,8 @@ ENTRY(des3_ede_x86_64_crypt_blk)
 	round1(32+15, RL0, RR0, dummy2);
 
 	final_permutation(RR0, RL0);
+
+	popq %rsi /* dst */
 	write_block(%rsi, RR0, RL0);
 
 	popq %r15;
@@ -248,7 +251,6 @@ ENTRY(des3_ede_x86_64_crypt_blk)
 	popq %r13;
 	popq %r12;
 	popq %rbx;
-	popq %rbp;
 
 	ret;
 ENDPROC(des3_ede_x86_64_crypt_blk)
@@ -432,13 +434,14 @@ ENTRY(des3_ede_x86_64_crypt_blk_3way)
 	 *	%rdx: src (3 blocks)
 	 */
 
-	pushq %rbp;
 	pushq %rbx;
 	pushq %r12;
 	pushq %r13;
 	pushq %r14;
 	pushq %r15;
 
+	pushq %rsi /* dst */
+
 	/* load input */
 	movl 0 * 4(%rdx), RL0d;
 	movl 1 * 4(%rdx), RR0d;
@@ -520,6 +523,7 @@ ENTRY(des3_ede_x86_64_crypt_blk_3way)
 	bswapl RR2d;
 	bswapl RL2d;
 
+	popq %rsi /* dst */
 	movl RR0d, 0 * 4(%rsi);
 	movl RL0d, 1 * 4(%rsi);
 	movl RR1d, 2 * 4(%rsi);
@@ -532,7 +536,6 @@ ENTRY(des3_ede_x86_64_crypt_blk_3way)
 	popq %r13;
 	popq %r12;
 	popq %rbx;
-	popq %rbp;
 
 	ret;
 ENDPROC(des3_ede_x86_64_crypt_blk_3way)
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722673 — [PATCH 07/12] x86/crypto: Fix RBP usage in sha1_ssse3_asm.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 07/12] x86/crypto: Fix RBP usage in sha1_ssse3_asm.S
Message-ID	<ujT7C-602-47@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Swap the usages of R12 and RBP.  Use R12 for the REG_D register, and use
RBP to store the pre-aligned stack pointer.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/sha1_ssse3_asm.S | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/x86/crypto/sha1_ssse3_asm.S b/arch/x86/crypto/sha1_ssse3_asm.S
index a4109506a5e8..6204bd53528c 100644
--- a/arch/x86/crypto/sha1_ssse3_asm.S
+++ b/arch/x86/crypto/sha1_ssse3_asm.S
@@ -37,7 +37,7 @@
 #define REG_A	%ecx
 #define REG_B	%esi
 #define REG_C	%edi
-#define REG_D	%ebp
+#define REG_D	%r12d
 #define REG_E	%edx
 
 #define REG_T1	%eax
@@ -74,10 +74,10 @@
 	ENTRY(\name)
 
 	push	%rbx
-	push	%rbp
 	push	%r12
+	push	%rbp
+	mov	%rsp, %rbp
 
-	mov	%rsp, %r12
 	sub	$64, %rsp		# allocate workspace
 	and	$~15, %rsp		# align stack
 
@@ -99,10 +99,9 @@
 	xor	%rax, %rax
 	rep stosq
 
-	mov	%r12, %rsp		# deallocate workspace
-
-	pop	%r12
+	mov	%rbp, %rsp		# deallocate workspace
 	pop	%rbp
+	pop	%r12
 	pop	%rbx
 	ret
 
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722675 — [PATCH 02/12] x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:10 +0200
Subject	[PATCH 02/12] x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S
Message-ID	<ujT7C-602-55@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use R12 instead of RBP.  Both are callee-saved registers, so the
substitution is straightforward.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/camellia-x86_64-asm_64.S | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/x86/crypto/camellia-x86_64-asm_64.S b/arch/x86/crypto/camellia-x86_64-asm_64.S
index 310319c601ed..95ba6956a7f6 100644
--- a/arch/x86/crypto/camellia-x86_64-asm_64.S
+++ b/arch/x86/crypto/camellia-x86_64-asm_64.S
@@ -75,17 +75,17 @@
 #define RCD1bh %dh
 
 #define RT0 %rsi
-#define RT1 %rbp
+#define RT1 %r12
 #define RT2 %r8
 
 #define RT0d %esi
-#define RT1d %ebp
+#define RT1d %r12d
 #define RT2d %r8d
 
 #define RT2bl %r8b
 
 #define RXOR %r9
-#define RRBP %r10
+#define RR12 %r10
 #define RDST %r11
 
 #define RXORd %r9d
@@ -197,7 +197,7 @@ ENTRY(__camellia_enc_blk)
 	 *	%rdx: src
 	 *	%rcx: bool xor
 	 */
-	movq %rbp, RRBP;
+	movq %r12, RR12;
 
 	movq %rcx, RXOR;
 	movq %rsi, RDST;
@@ -227,13 +227,13 @@ ENTRY(__camellia_enc_blk)
 
 	enc_outunpack(mov, RT1);
 
-	movq RRBP, %rbp;
+	movq RR12, %r12;
 	ret;
 
 .L__enc_xor:
 	enc_outunpack(xor, RT1);
 
-	movq RRBP, %rbp;
+	movq RR12, %r12;
 	ret;
 ENDPROC(__camellia_enc_blk)
 
@@ -248,7 +248,7 @@ ENTRY(camellia_dec_blk)
 	movl $24, RXORd;
 	cmovel RXORd, RT2d; /* max */
 
-	movq %rbp, RRBP;
+	movq %r12, RR12;
 	movq %rsi, RDST;
 	movq %rdx, RIO;
 
@@ -271,7 +271,7 @@ ENTRY(camellia_dec_blk)
 
 	dec_outunpack();
 
-	movq RRBP, %rbp;
+	movq RR12, %r12;
 	ret;
 ENDPROC(camellia_dec_blk)
 
@@ -433,7 +433,7 @@ ENTRY(__camellia_enc_blk_2way)
 	 */
 	pushq %rbx;
 
-	movq %rbp, RRBP;
+	movq %r12, RR12;
 	movq %rcx, RXOR;
 	movq %rsi, RDST;
 	movq %rdx, RIO;
@@ -461,14 +461,14 @@ ENTRY(__camellia_enc_blk_2way)
 
 	enc_outunpack2(mov, RT2);
 
-	movq RRBP, %rbp;
+	movq RR12, %r12;
 	popq %rbx;
 	ret;
 
 .L__enc2_xor:
 	enc_outunpack2(xor, RT2);
 
-	movq RRBP, %rbp;
+	movq RR12, %r12;
 	popq %rbx;
 	ret;
 ENDPROC(__camellia_enc_blk_2way)
@@ -485,7 +485,7 @@ ENTRY(camellia_dec_blk_2way)
 	cmovel RXORd, RT2d; /* max */
 
 	movq %rbx, RXOR;
-	movq %rbp, RRBP;
+	movq %r12, RR12;
 	movq %rsi, RDST;
 	movq %rdx, RIO;
 
@@ -508,7 +508,7 @@ ENTRY(camellia_dec_blk_2way)
 
 	dec_outunpack2();
 
-	movq RRBP, %rbp;
+	movq RR12, %r12;
 	movq RXOR, %rbx;
 	ret;
 ENDPROC(camellia_dec_blk_2way)
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1722682 — [PATCH 03/12] x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-08-29 20:20 +0200
Subject	[PATCH 03/12] x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S
Message-ID	<ujThg-63v-21@gated-at.bofh.it>
In reply to	#1722662

Using RBP as a temporary register breaks frame pointer convention and
breaks stack traces when unwinding from an interrupt in the crypto code.

Use R15 instead of RBP.  R15 can't be used as the RID1 register because
of x86 instruction encoding limitations.  So use R15 for CTX and RDI for
CTX.  This means that CTX is no longer an implicit function argument.
Instead it needs to be explicitly copied from RDI.

Reported-by: Eric Biggers <ebiggers@google.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/crypto/cast5-avx-x86_64-asm_64.S | 47 ++++++++++++++++++++-----------
 1 file changed, 30 insertions(+), 17 deletions(-)

diff --git a/arch/x86/crypto/cast5-avx-x86_64-asm_64.S b/arch/x86/crypto/cast5-avx-x86_64-asm_64.S
index b4a8806234ea..86107c961bb4 100644
--- a/arch/x86/crypto/cast5-avx-x86_64-asm_64.S
+++ b/arch/x86/crypto/cast5-avx-x86_64-asm_64.S
@@ -47,7 +47,7 @@
 /**********************************************************************
   16-way AVX cast5
  **********************************************************************/
-#define CTX %rdi
+#define CTX %r15
 
 #define RL1 %xmm0
 #define RR1 %xmm1
@@ -70,8 +70,8 @@
 
 #define RTMP %xmm15
 
-#define RID1  %rbp
-#define RID1d %ebp
+#define RID1  %rdi
+#define RID1d %edi
 #define RID2  %rsi
 #define RID2d %esi
 
@@ -226,7 +226,7 @@
 .align 16
 __cast5_enc_blk16:
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	RL1: blocks 1 and 2
 	 *	RR1: blocks 3 and 4
 	 *	RL2: blocks 5 and 6
@@ -246,9 +246,11 @@ __cast5_enc_blk16:
 	 *	RR4: encrypted blocks 15 and 16
 	 */
 
-	pushq %rbp;
+	pushq %r15;
 	pushq %rbx;
 
+	movq %rdi, CTX;
+
 	vmovdqa .Lbswap_mask, RKM;
 	vmovd .Lfirst_mask, R1ST;
 	vmovd .L32_mask, R32;
@@ -283,7 +285,7 @@ __cast5_enc_blk16:
 
 .L__skip_enc:
 	popq %rbx;
-	popq %rbp;
+	popq %r15;
 
 	vmovdqa .Lbswap_mask, RKM;
 
@@ -298,7 +300,7 @@ ENDPROC(__cast5_enc_blk16)
 .align 16
 __cast5_dec_blk16:
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	RL1: encrypted blocks 1 and 2
 	 *	RR1: encrypted blocks 3 and 4
 	 *	RL2: encrypted blocks 5 and 6
@@ -318,9 +320,11 @@ __cast5_dec_blk16:
 	 *	RR4: decrypted blocks 15 and 16
 	 */
 
-	pushq %rbp;
+	pushq %r15;
 	pushq %rbx;
 
+	movq %rdi, CTX;
+
 	vmovdqa .Lbswap_mask, RKM;
 	vmovd .Lfirst_mask, R1ST;
 	vmovd .L32_mask, R32;
@@ -356,7 +360,7 @@ __cast5_dec_blk16:
 
 	vmovdqa .Lbswap_mask, RKM;
 	popq %rbx;
-	popq %rbp;
+	popq %r15;
 
 	outunpack_blocks(RR1, RL1, RTMP, RX, RKM);
 	outunpack_blocks(RR2, RL2, RTMP, RX, RKM);
@@ -372,12 +376,14 @@ ENDPROC(__cast5_dec_blk16)
 
 ENTRY(cast5_ecb_enc_16way)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 */
 	FRAME_BEGIN
+	pushq %r15;
 
+	movq %rdi, CTX;
 	movq %rsi, %r11;
 
 	vmovdqu (0*4*4)(%rdx), RL1;
@@ -400,18 +406,22 @@ ENTRY(cast5_ecb_enc_16way)
 	vmovdqu RR4, (6*4*4)(%r11);
 	vmovdqu RL4, (7*4*4)(%r11);
 
+	popq %r15;
 	FRAME_END
 	ret;
 ENDPROC(cast5_ecb_enc_16way)
 
 ENTRY(cast5_ecb_dec_16way)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 */
 
 	FRAME_BEGIN
+	pushq %r15;
+
+	movq %rdi, CTX;
 	movq %rsi, %r11;
 
 	vmovdqu (0*4*4)(%rdx), RL1;
@@ -434,20 +444,22 @@ ENTRY(cast5_ecb_dec_16way)
 	vmovdqu RR4, (6*4*4)(%r11);
 	vmovdqu RL4, (7*4*4)(%r11);
 
+	popq %r15;
 	FRAME_END
 	ret;
 ENDPROC(cast5_ecb_dec_16way)
 
 ENTRY(cast5_cbc_dec_16way)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 */
 	FRAME_BEGIN
-
 	pushq %r12;
+	pushq %r15;
 
+	movq %rdi, CTX;
 	movq %rsi, %r11;
 	movq %rdx, %r12;
 
@@ -483,23 +495,24 @@ ENTRY(cast5_cbc_dec_16way)
 	vmovdqu RR4, (6*16)(%r11);
 	vmovdqu RL4, (7*16)(%r11);
 
+	popq %r15;
 	popq %r12;
-
 	FRAME_END
 	ret;
 ENDPROC(cast5_cbc_dec_16way)
 
 ENTRY(cast5_ctr_16way)
 	/* input:
-	 *	%rdi: ctx, CTX
+	 *	%rdi: ctx
 	 *	%rsi: dst
 	 *	%rdx: src
 	 *	%rcx: iv (big endian, 64bit)
 	 */
 	FRAME_BEGIN
-
 	pushq %r12;
+	pushq %r15;
 
+	movq %rdi, CTX;
 	movq %rsi, %r11;
 	movq %rdx, %r12;
 
@@ -558,8 +571,8 @@ ENTRY(cast5_ctr_16way)
 	vmovdqu RR4, (6*16)(%r11);
 	vmovdqu RL4, (7*16)(%r11);
 
+	popq %r15;
 	popq %r12;
-
 	FRAME_END
 	ret;
 ENDPROC(cast5_ctr_16way)
-- 
2.13.5

[toc] | [prev] | [next] | [standalone]

#1725406 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Eric Biggers <ebiggers3@gmail.com>
Date	2017-09-02 02:10 +0200
Subject	Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<ul4aB-3Wi-3@gated-at.bofh.it>
In reply to	#1722662

Hi Josh,

On Tue, Aug 29, 2017 at 01:05:33PM -0500, Josh Poimboeuf wrote:
> Many of the x86 crypto functions use RBP as a temporary register.  This
> breaks frame pointer convention, and breaks stack traces when unwinding
> from an interrupt in the crypto code.
> 
> Convert most* of them to leave RBP alone.
> 
> These pass the crypto boot tests for me.  Any further testing would be
> appreciated!
> 
> [*] There are still a few crypto files left that need fixing, but the
>     fixes weren't trivial and nobody reported unwinder warnings about
>     them yet, so I'm skipping them for now.
> 
> Josh Poimboeuf (12):
>   x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S
>   x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S
>   x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S
>   x86/crypto: Fix RBP usage in cast6-avx-x86_64-asm_64.S
>   x86/crypto: Fix RBP usage in des3_ede-asm_64.S
>   x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S
>   x86/crypto: Fix RBP usage in sha1_ssse3_asm.S
>   x86/crypto: Fix RBP usage in sha256-avx-asm.S
>   x86/crypto: Fix RBP usage in sha256-avx2-asm.S
>   x86/crypto: Fix RBP usage in sha256-ssse3-asm.S
>   x86/crypto: Fix RBP usage in sha512-avx2-asm.S
>   x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S
> 

Thanks for fixing these!  I don't have time to review these in detail, but I ran
the crypto self-tests on the affected algorithms, and they all pass.  I also
benchmarked them before and after; the only noticable performance difference was
that sha256-avx2 and sha512-avx2 became a few percent slower.  I don't suppose
there is a way around that?  Otherwise it's probably not a big deal.

For reference, this was the list of affected algorithms I used:

shash: sha1-ssse3, sha1-avx, sha1-avx2, sha256-ssse3, sha256-avx, sha256-avx2,
	sha512-ssse3, sha512-avx, sha512-avx2

cipher: blowfish-asm, camellia-asm, des3_ede-asm

skcipher: ecb-blowfish-asm, cbc-blowfish-asm, ctr-blowfish-asm, ecb-cast5-avx,
	cbc-cast5-avx, ctr-cast5-avx, ecb-cast6-avx, cbc-cast6-avx, ctr-cast6-avx,
	lrw-cast6-avx, xts-cast6-avx, ecb-camellia-asm, cbc-camellia-asm,
	ctr-camellia-asm, lrw-camellia-asm, xts-camellia-asm, ecb-des3_ede-asm,
	cbc-des3_ede-asm, ctr-des3_ede-asm, ecb-twofish-avx, cbc-twofish-avx,
	ctr-twofish-avx, lrw-twofish-avx, xts-twofish-avx

Eric

[toc] | [prev] | [next] | [standalone]

#1727832 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Josh Poimboeuf <jpoimboe@redhat.com>
Date	2017-09-07 02:20 +0200
Subject	Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<umSI1-EN-1@gated-at.bofh.it>
In reply to	#1725406

On Fri, Sep 01, 2017 at 05:09:19PM -0700, Eric Biggers wrote:
> Hi Josh,
> 
> On Tue, Aug 29, 2017 at 01:05:33PM -0500, Josh Poimboeuf wrote:
> > Many of the x86 crypto functions use RBP as a temporary register.  This
> > breaks frame pointer convention, and breaks stack traces when unwinding
> > from an interrupt in the crypto code.
> > 
> > Convert most* of them to leave RBP alone.
> > 
> > These pass the crypto boot tests for me.  Any further testing would be
> > appreciated!
> > 
> > [*] There are still a few crypto files left that need fixing, but the
> >     fixes weren't trivial and nobody reported unwinder warnings about
> >     them yet, so I'm skipping them for now.
> > 
> > Josh Poimboeuf (12):
> >   x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S
> >   x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S
> >   x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S
> >   x86/crypto: Fix RBP usage in cast6-avx-x86_64-asm_64.S
> >   x86/crypto: Fix RBP usage in des3_ede-asm_64.S
> >   x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S
> >   x86/crypto: Fix RBP usage in sha1_ssse3_asm.S
> >   x86/crypto: Fix RBP usage in sha256-avx-asm.S
> >   x86/crypto: Fix RBP usage in sha256-avx2-asm.S
> >   x86/crypto: Fix RBP usage in sha256-ssse3-asm.S
> >   x86/crypto: Fix RBP usage in sha512-avx2-asm.S
> >   x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S
> > 
> 
> Thanks for fixing these!  I don't have time to review these in detail, but I ran
> the crypto self-tests on the affected algorithms, and they all pass.  I also
> benchmarked them before and after; the only noticable performance difference was
> that sha256-avx2 and sha512-avx2 became a few percent slower.  I don't suppose
> there is a way around that?  Otherwise it's probably not a big deal.

Thanks for testing.  I might have a way to make sha256-avx2 faster, but
not sha512-avx2.  I'll let you know when I have a patch.

-- 
Josh

[toc] | [prev] | [next] | [standalone]

#1727969 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Ingo Molnar <mingo@kernel.org>
Date	2017-09-07 09:20 +0200
Subject	Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<umZgv-5bg-19@gated-at.bofh.it>
In reply to	#1725406

* Eric Biggers <ebiggers3@gmail.com> wrote:

> Thanks for fixing these!  I don't have time to review these in detail, but I ran
> the crypto self-tests on the affected algorithms, and they all pass.  I also
> benchmarked them before and after; the only noticable performance difference was
> that sha256-avx2 and sha512-avx2 became a few percent slower.  I don't suppose
> there is a way around that?  Otherwise it's probably not a big deal.

Which CPU model did you use for the test?

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1728351 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Eric Biggers <ebiggers3@gmail.com>
Date	2017-09-07 20:00 +0200
Subject	Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<un9fS-3ll-43@gated-at.bofh.it>
In reply to	#1727969

On Thu, Sep 07, 2017 at 09:15:34AM +0200, Ingo Molnar wrote:
> 
> * Eric Biggers <ebiggers3@gmail.com> wrote:
> 
> > Thanks for fixing these!  I don't have time to review these in detail, but I ran
> > the crypto self-tests on the affected algorithms, and they all pass.  I also
> > benchmarked them before and after; the only noticable performance difference was
> > that sha256-avx2 and sha512-avx2 became a few percent slower.  I don't suppose
> > there is a way around that?  Otherwise it's probably not a big deal.
> 
> Which CPU model did you use for the test?
> 
> Thanks,
> 
> 	Ingo

This was on Haswell, "Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz".

Eric

[toc] | [prev] | [next] | [standalone]

#1728449 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Ingo Molnar <mingo@kernel.org>
Date	2017-09-07 23:30 +0200
Subject	Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<uncx4-5G0-13@gated-at.bofh.it>
In reply to	#1728351

* Eric Biggers <ebiggers3@gmail.com> wrote:

> On Thu, Sep 07, 2017 at 09:15:34AM +0200, Ingo Molnar wrote:
> > 
> > * Eric Biggers <ebiggers3@gmail.com> wrote:
> > 
> > > Thanks for fixing these!  I don't have time to review these in detail, but I ran
> > > the crypto self-tests on the affected algorithms, and they all pass.  I also
> > > benchmarked them before and after; the only noticable performance difference was
> > > that sha256-avx2 and sha512-avx2 became a few percent slower.  I don't suppose
> > > there is a way around that?  Otherwise it's probably not a big deal.
> > 
> > Which CPU model did you use for the test?
> > 
> > Thanks,
> > 
> > 	Ingo
> 
> This was on Haswell, "Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz".

Any chance to test this with the latest microarchitecture - any Skylake derivative 
Intel CPU you have access to?

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1729138 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

From	Eric Biggers <ebiggers3@gmail.com>
Date	2017-09-08 20:00 +0200
Subject	Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files
Message-ID	<unvJo-1M9-7@gated-at.bofh.it>
In reply to	#1728449

On Thu, Sep 07, 2017 at 11:26:47PM +0200, Ingo Molnar wrote:
> 
> * Eric Biggers <ebiggers3@gmail.com> wrote:
> 
> > On Thu, Sep 07, 2017 at 09:15:34AM +0200, Ingo Molnar wrote:
> > > 
> > > * Eric Biggers <ebiggers3@gmail.com> wrote:
> > > 
> > > > Thanks for fixing these!  I don't have time to review these in detail, but I ran
> > > > the crypto self-tests on the affected algorithms, and they all pass.  I also
> > > > benchmarked them before and after; the only noticable performance difference was
> > > > that sha256-avx2 and sha512-avx2 became a few percent slower.  I don't suppose
> > > > there is a way around that?  Otherwise it's probably not a big deal.
> > > 
> > > Which CPU model did you use for the test?
> > > 
> > > Thanks,
> > > 
> > > 	Ingo
> > 
> > This was on Haswell, "Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz".
> 
> Any chance to test this with the latest microarchitecture - any Skylake derivative 
> Intel CPU you have access to?
> 
> Thanks,
> 
> 	Ingo

Tested with Skylake, "Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz".  The results
were the following which seemed a bit worse than Haswell:

	sha256-avx2 became 3.5% slower
	sha512-avx2 became 7.5% slower

Note: it's tricky to benchmark this, especially with just a few percent
difference, so don't read too much into the exact numbers.

Eric

[toc] | [prev] | [standalone]

csiph-web

[PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

Contents

#1722662 — [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

#1722664 — [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S

#1727634 — Re: [PATCH 06/12] x86/crypto: Fix RBP usage in sha1_avx2_x86_64_asm.S

#1722665 — [PATCH 12/12] x86/crypto: Fix RBP usage in twofish-avx-x86_64-asm_64.S

#1722667 — [PATCH 11/12] x86/crypto: Fix RBP usage in sha512-avx2-asm.S

#1722669 — [PATCH 10/12] x86/crypto: Fix RBP usage in sha256-ssse3-asm.S

#1722671 — [PATCH 01/12] x86/crypto: Fix RBP usage in blowfish-x86_64-asm_64.S

#1722672 — [PATCH 05/12] x86/crypto: Fix RBP usage in des3_ede-asm_64.S

#1722673 — [PATCH 07/12] x86/crypto: Fix RBP usage in sha1_ssse3_asm.S

#1722675 — [PATCH 02/12] x86/crypto: Fix RBP usage in camellia-x86_64-asm_64.S

#1722682 — [PATCH 03/12] x86/crypto: Fix RBP usage in cast5-avx-x86_64-asm_64.S

#1725406 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

#1727832 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

#1727969 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

#1728351 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

#1728449 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files

#1729138 — Re: [PATCH 00/12] x86/crypto: Fix RBP usage in several crypto .S files