Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1727782
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [RFC 03/17] x86/asm/64: Move SWAPGS into the common iret-to-usermode path |
| Date | 2017-09-06 23:50 +0200 |
| Message-ID | <umQmS-7tQ-27@gated-at.bofh.it> (permalink) |
| References | <umQdc-7qi-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
All of the code paths that ended up doing IRET to usermode did SWAPGS immediately beforehand. Move the SWAPGS into the common code. Signed-off-by: Andy Lutomirski <luto@kernel.org> --- arch/x86/entry/entry_64.S | 26 ++++++++++---------------- arch/x86/entry/entry_64_compat.S | 3 +-- 2 files changed, 11 insertions(+), 18 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 34156d50c3d6..2cd01ed9cd59 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -249,7 +249,7 @@ return_from_SYSCALL_64: movq RCX(%rsp), %rcx movq RIP(%rsp), %r11 cmpq %rcx, %r11 /* RCX == RIP */ - jne opportunistic_sysret_failed + jne swapgs_restore_regs_and_return_to_usermode /* * On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP @@ -267,14 +267,14 @@ return_from_SYSCALL_64: /* If this changed %rcx, it was not canonical */ cmpq %rcx, %r11 - jne opportunistic_sysret_failed + jne swapgs_restore_regs_and_return_to_usermode cmpq $__USER_CS, CS(%rsp) /* CS must match SYSRET */ - jne opportunistic_sysret_failed + jne swapgs_restore_regs_and_return_to_usermode movq R11(%rsp), %r11 cmpq %r11, EFLAGS(%rsp) /* R11 == RFLAGS */ - jne opportunistic_sysret_failed + jne swapgs_restore_regs_and_return_to_usermode /* * SYSCALL clears RF when it saves RFLAGS in R11 and SYSRET cannot @@ -295,12 +295,12 @@ return_from_SYSCALL_64: * would never get past 'stuck_here'. */ testq $(X86_EFLAGS_RF|X86_EFLAGS_TF), %r11 - jnz opportunistic_sysret_failed + jnz swapgs_restore_regs_and_return_to_usermode /* nothing to check for RSP */ cmpq $__USER_DS, SS(%rsp) /* SS must match SYSRET */ - jne opportunistic_sysret_failed + jne swapgs_restore_regs_and_return_to_usermode /* * We win! This label is here just for ease of understanding @@ -312,10 +312,6 @@ syscall_return_via_sysret: RESTORE_C_REGS_EXCEPT_RCX_R11 movq RSP(%rsp), %rsp USERGS_SYSRET64 - -opportunistic_sysret_failed: - SWAPGS - jmp restore_regs_and_return_to_usermode END(entry_SYSCALL_64) ENTRY(stub_ptregs_64) @@ -411,8 +407,7 @@ ENTRY(ret_from_fork) movq %rsp, %rdi call syscall_return_slowpath /* returns with IRQs disabled */ TRACE_IRQS_ON /* user mode is traced as IRQS on */ - SWAPGS - jmp restore_regs_and_return_to_usermode + jmp swapgs_restore_regs_and_return_to_usermode 1: /* kernel thread */ @@ -523,9 +518,9 @@ GLOBAL(retint_user) mov %rsp,%rdi call prepare_exit_to_usermode TRACE_IRQS_IRETQ - SWAPGS -GLOBAL(restore_regs_and_return_to_usermode) +GLOBAL(swapgs_restore_regs_and_return_to_usermode) + SWAPGS RESTORE_EXTRA_REGS RESTORE_C_REGS REMOVE_PT_GPREGS_FROM_STACK 8 @@ -1265,8 +1260,7 @@ ENTRY(nmi) * Return back to user mode. We must *not* do the normal exit * work, because we don't want to enable interrupts. */ - SWAPGS - jmp restore_regs_and_return_to_usermode + jmp swapgs_restore_regs_and_return_to_usermode .Lnmi_from_kernel: /* diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S index e4ab8ffa94b1..72e143497ece 100644 --- a/arch/x86/entry/entry_64_compat.S +++ b/arch/x86/entry/entry_64_compat.S @@ -337,8 +337,7 @@ ENTRY(entry_INT80_compat) /* Go back to user mode. */ TRACE_IRQS_ON - SWAPGS - jmp restore_regs_and_return_to_usermode + jmp swapgs_restore_regs_and_return_to_usermode END(entry_INT80_compat) ALIGN -- 2.13.5
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[RFC 00/17] Pile o' entry stack changes Andy Lutomirski <luto@kernel.org> - 2017-09-06 23:50 +0200
[RFC 04/17] x86/asm/64: Simplify reg restore code in the standard IRET paths Andy Lutomirski <luto@kernel.org> - 2017-09-06 23:50 +0200
[RFC 03/17] x86/asm/64: Move SWAPGS into the common iret-to-usermode path Andy Lutomirski <luto@kernel.org> - 2017-09-06 23:50 +0200
[RFC 02/17] x86/asm/64: Split the iret-to-user and iret-to-kernel paths Andy Lutomirski <luto@kernel.org> - 2017-09-06 23:50 +0200
Re: [RFC 00/17] Pile o' entry stack changes Andi Kleen <andi@firstfloor.org> - 2017-09-07 00:20 +0200
Re: [RFC 00/17] Pile o' entry stack changes Andy Lutomirski <luto@kernel.org> - 2017-09-07 02:10 +0200
Re: [RFC 00/17] Pile o' entry stack changes Ingo Molnar <mingo@kernel.org> - 2017-09-07 09:10 +0200
csiph-web