Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1738003
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH 26/33] x86/fpu: Reinitialize FPU registers if restoring FPU state fails |
| Date | 2017-09-23 15:10 +0200 |
| Message-ID | <usSlX-3wU-7@gated-at.bofh.it> (permalink) |
| References | <usSlX-3wU-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
From: Eric Biggers <ebiggers@google.com>
Userspace can change the FPU state of a task using the ptrace() or
rt_sigreturn() system calls. Because reserved bits in the FPU state can
cause the XRSTOR instruction to fail, the kernel has to carefully
validate that no reserved bits or other invalid values are being set.
Unfortunately, there have been bugs in this validation code. For
example, we were not checking that the 'xcomp_bv' field in the
xstate_header was 0. As-is, such bugs are exploitable to read the FPU
registers of other processes on the system. To do so, an attacker can
create a task, assign to it an invalid FPU state, then spin in a loop
and monitor the values of the FPU registers. Because the task's FPU
registers are not being restored, sometimes the FPU registers will have
the values from another process.
This is likely to continue to be a problem in the future because the
validation done by the CPU instructions like XRSTOR is not immediately
visible to kernel developers. Nor will invalid FPU states ever be
encountered during ordinary use --- they will only be seen during
fuzzing or exploits. There can even be reserved bits outside the
xstate_header which are easy to forget about. For example, the MXCSR
register contains reserved bits, which were not validated by the
KVM_SET_XSAVE ioctl until commit a575813bfe4b ("KVM: x86: Fix load
damaged SSEx MXCSR register").
Therefore, mitigate this class of vulnerability by restoring the FPU
registers from init_fpstate if restoring from the task's state fails.
We actually used to do this, but it was (perhaps unwisely) removed by
commit 9ccc27a5d297 ("x86/fpu: Remove error return values from
copy_kernel_to_*regs() functions"). This new patch is also a bit
different. First, it only clears the registers, not also the bad
in-memory state; this is simpler and makes it easier to make the
mitigation cover all callers of __copy_kernel_to_fpregs(). Second, it
does the register clearing in an exception handler so that no extra
instructions are added to context switches. In fact, we *remove*
instructions, since previously we were always zeroing the register
containing 'err' even if CONFIG_X86_DEBUG_FPU was disabled.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Kevin Hao <haokexin@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael Halcrow <mhalcrow@google.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <wanpeng.li@hotmail.com>
Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
Cc: kernel-hardening@lists.openwall.com
Link: http://lkml.kernel.org/r/20170922174156.16780-4-ebiggers3@gmail.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/fpu/internal.h | 51 +++++++++++++++------------------------------------
arch/x86/mm/extable.c | 24 ++++++++++++++++++++++++
2 files changed, 39 insertions(+), 36 deletions(-)
diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h
index 2dca7c65319c..cf290d424e48 100644
--- a/arch/x86/include/asm/fpu/internal.h
+++ b/arch/x86/include/asm/fpu/internal.h
@@ -120,20 +120,11 @@ extern void fpstate_sanitize_xstate(struct fpu *fpu);
err; \
})
-#define check_insn(insn, output, input...) \
-({ \
- int err; \
+#define kernel_insn(insn, output, input...) \
asm volatile("1:" #insn "\n\t" \
"2:\n" \
- ".section .fixup,\"ax\"\n" \
- "3: movl $-1,%[err]\n" \
- " jmp 2b\n" \
- ".previous\n" \
- _ASM_EXTABLE(1b, 3b) \
- : [err] "=r" (err), output \
- : "0"(0), input); \
- err; \
-})
+ _ASM_EXTABLE_HANDLE(1b, 2b, ex_handler_fprestore) \
+ : output : input)
static inline int copy_fregs_to_user(struct fregs_state __user *fx)
{
@@ -153,20 +144,16 @@ static inline int copy_fxregs_to_user(struct fxregs_state __user *fx)
static inline void copy_kernel_to_fxregs(struct fxregs_state *fx)
{
- int err;
-
if (IS_ENABLED(CONFIG_X86_32)) {
- err = check_insn(fxrstor %[fx], "=m" (*fx), [fx] "m" (*fx));
+ kernel_insn(fxrstor %[fx], "=m" (*fx), [fx] "m" (*fx));
} else {
if (IS_ENABLED(CONFIG_AS_FXSAVEQ)) {
- err = check_insn(fxrstorq %[fx], "=m" (*fx), [fx] "m" (*fx));
+ kernel_insn(fxrstorq %[fx], "=m" (*fx), [fx] "m" (*fx));
} else {
/* See comment in copy_fxregs_to_kernel() below. */
- err = check_insn(rex64/fxrstor (%[fx]), "=m" (*fx), [fx] "R" (fx), "m" (*fx));
+ kernel_insn(rex64/fxrstor (%[fx]), "=m" (*fx), [fx] "R" (fx), "m" (*fx));
}
}
- /* Copying from a kernel buffer to FPU registers should never fail: */
- WARN_ON_FPU(err);
}
static inline int copy_user_to_fxregs(struct fxregs_state __user *fx)
@@ -183,9 +170,7 @@ static inline int copy_user_to_fxregs(struct fxregs_state __user *fx)
static inline void copy_kernel_to_fregs(struct fregs_state *fx)
{
- int err = check_insn(frstor %[fx], "=m" (*fx), [fx] "m" (*fx));
-
- WARN_ON_FPU(err);
+ kernel_insn(frstor %[fx], "=m" (*fx), [fx] "m" (*fx));
}
static inline int copy_user_to_fregs(struct fregs_state __user *fx)
@@ -281,18 +266,13 @@ static inline void copy_fxregs_to_kernel(struct fpu *fpu)
* Use XRSTORS to restore context if it is enabled. XRSTORS supports compact
* XSAVE area format.
*/
-#define XSTATE_XRESTORE(st, lmask, hmask, err) \
+#define XSTATE_XRESTORE(st, lmask, hmask) \
asm volatile(ALTERNATIVE(XRSTOR, \
XRSTORS, X86_FEATURE_XSAVES) \
"\n" \
- "xor %[err], %[err]\n" \
"3:\n" \
- ".pushsection .fixup,\"ax\"\n" \
- "4: movl $-2, %[err]\n" \
- "jmp 3b\n" \
- ".popsection\n" \
- _ASM_EXTABLE(661b, 4b) \
- : [err] "=r" (err) \
+ _ASM_EXTABLE_HANDLE(661b, 3b, ex_handler_fprestore)\
+ : \
: "D" (st), "m" (*st), "a" (lmask), "d" (hmask) \
: "memory")
@@ -336,7 +316,10 @@ static inline void copy_kernel_to_xregs_booting(struct xregs_state *xstate)
else
XSTATE_OP(XRSTOR, xstate, lmask, hmask, err);
- /* We should never fault when copying from a kernel buffer: */
+ /*
+ * We should never fault when copying from a kernel buffer, and the FPU
+ * state we set at boot time should be valid.
+ */
WARN_ON_FPU(err);
}
@@ -365,12 +348,8 @@ static inline void copy_kernel_to_xregs(struct xregs_state *xstate, u64 mask)
{
u32 lmask = mask;
u32 hmask = mask >> 32;
- int err;
-
- XSTATE_XRESTORE(xstate, lmask, hmask, err);
- /* We should never fault when copying from a kernel buffer: */
- WARN_ON_FPU(err);
+ XSTATE_XRESTORE(xstate, lmask, hmask);
}
/*
diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c
index c076f710de4c..c3521e2be396 100644
--- a/arch/x86/mm/extable.c
+++ b/arch/x86/mm/extable.c
@@ -2,6 +2,7 @@
#include <linux/uaccess.h>
#include <linux/sched/debug.h>
+#include <asm/fpu/internal.h>
#include <asm/traps.h>
#include <asm/kdebug.h>
@@ -78,6 +79,29 @@ bool ex_handler_refcount(const struct exception_table_entry *fixup,
}
EXPORT_SYMBOL_GPL(ex_handler_refcount);
+/*
+ * Handler for when we fail to restore a task's FPU state. We should never get
+ * here because the FPU state of a task using the FPU (task->thread.fpu.state)
+ * should always be valid. However, past bugs have allowed userspace to set
+ * reserved bits in the XSAVE area using PTRACE_SETREGSET or sys_rt_sigreturn().
+ * These caused XRSTOR to fail when switching to the task, leaking the FPU
+ * registers of the task previously executing on the CPU. Mitigate this class
+ * of vulnerability by restoring from the initial state (essentially, zeroing
+ * out all the FPU registers) if we can't restore from the task's FPU state.
+ */
+bool ex_handler_fprestore(const struct exception_table_entry *fixup,
+ struct pt_regs *regs, int trapnr)
+{
+ regs->ip = ex_fixup_addr(fixup);
+
+ WARN_ONCE(1, "Bad FPU state detected at %pB, reinitializing FPU registers.",
+ (void *)instruction_pointer(regs));
+
+ __copy_kernel_to_fpregs(&init_fpstate, -1);
+ return true;
+}
+EXPORT_SYMBOL_GPL(ex_handler_fprestore);
+
bool ex_handler_ext(const struct exception_table_entry *fixup,
struct pt_regs *regs, int trapnr)
{
--
2.11.0
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH 00/33] x86 FPU fixes and cleanups for v4.14 Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[PATCH 19/33] x86/fpu: Decouple fpregs_activate()/fpregs_deactivate() from fpu->fpregs_active Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Decouple fpregs_activate()/fpregs_deactivate() from fpu->fpregs_active tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 15/33] x86/fpu: Simplify fpu->fpregs_active use Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Simplify fpu->fpregs_active use tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 26/33] x86/fpu: Reinitialize FPU registers if restoring FPU state fails Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Reinitialize FPU registers if restoring FPU state fails tip-bot for Eric Biggers <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 32/33] x86/fpu: Rename fpu__activate_curr() to fpu__initialize() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Rename fpu__activate_curr() to fpu__initialize() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:50 +0200
[PATCH 31/33] x86/fpu: Simplify and speed up fpu__copy() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Simplify and speed up fpu__copy() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:50 +0200
[PATCH 21/33] x86/fpu: Add FPU state copying quirk to handle XRSTOR failure on Intel Skylake CPUs Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Add FPU state copying quirk to handle XRSTOR failure on Intel Skylake CPUs tip-bot for Rik van Riel <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 13/33] x86/fpu: Remove 'kbuf' parameter from the copy_user_to_xstate() API Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove 'kbuf' parameter from the copy_user_to_xstate() API tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 33/33] x86/fpu: Rename fpu__activate_fpstate_read/write() to fpu__read/write() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[PATCH 02/33] x86/fpu: Split copy_xstate_to_user() into copy_xstate_to_kernel() & copy_xstate_to_user() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Split copy_xstate_to_user() into copy_xstate_to_kernel() & copy_xstate_to_user() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:30 +0200
[PATCH 16/33] x86/fpu: Make the fpu state change in fpu__clear() scheduler-atomic Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Make the fpu state change in fpu__clear() scheduler-atomic tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 14/33] x86/fpu: Flip the parameter order in copy_*_to_xstate() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Flip the parameter order in copy_*_to_xstate() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 11/33] x86/fpu: Split copy_user_to_xstate() into copy_kernel_to_xstate() & copy_user_to_xstate() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Split copy_user_to_xstate() into copy_kernel_to_xstate() & copy_user_to_xstate() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 07/33] x86/fpu: Remove the 'start_pos' parameter from the __copy_xstate_to_*() functions Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove the 'start_pos' parameter from the __copy_xstate_to_*() functions tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 01/33] x86/fpu: Rename copyin_to_xsaves()/copyout_from_xsaves() to copy_user_to_xstate()/copy_xstate_to_user() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Rename copyin_to_xsaves()/copyout_from_xsaves() to copy_user_to_xstate()/copy_xstate_to_user() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:30 +0200
[PATCH 29/33] x86/fpu: Rename fpu::fpstate_active to fpu::initialized Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Rename fpu::fpstate_active to fpu::initialized tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:50 +0200
[PATCH 17/33] x86/fpu: Split the state handling in fpu__drop() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Split the state handling in fpu__drop() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 09/33] x86/fpu: Change 'size_total' parameter to unsigned and standardize the size checks in copy_xstate_to_*() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Change 'size_total' parameter to unsigned and standardize the size checks in copy_xstate_to_*() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 30/33] x86/fpu: Fix stale comments about lazy FPU logic Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Fix stale comments about lazy FPU logic tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:50 +0200
[PATCH 08/33] x86/fpu: Clarify parameter names in the copy_xstate_to_*() methods Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
Re: [PATCH 08/33] x86/fpu: Clarify parameter names in the copy_xstate_to_*() methods Thomas Gleixner <tglx@linutronix.de> - 2017-09-25 22:00 +0200
Re: [PATCH 08/33] x86/fpu: Clarify parameter names in the copy_xstate_to_*() methods Thomas Gleixner <tglx@linutronix.de> - 2017-09-25 22:10 +0200
[tip:x86/fpu] x86/fpu: Clarify parameter names in the copy_xstate_to_*() methods tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 20/33] x86/fpu: Remove struct fpu::fpregs_active Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove struct fpu::fpregs_active tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 03/33] x86/fpu: Remove 'ubuf' parameter from the copy_xstate_to_kernel() APIs Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove 'ubuf' parameter from the copy_xstate_to_kernel() APIs tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:30 +0200
[PATCH 23/33] x86/fpu: Turn WARN_ON() in context switch into WARN_ON_FPU() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Turn WARN_ON() in context switch into WARN_ON_FPU() tip-bot for Andi Kleen <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 22/33] x86/fpu: Fix boolreturn.cocci warnings Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Fix boolreturn.cocci warnings tip-bot for kbuild test robot <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 06/33] x86/fpu: Clean up the parameter definitions of copy_xstate_to_*() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Clean up the parameter definitions of copy_xstate_to_*() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 04/33] x86/fpu: Remove 'kbuf' parameter from the copy_xstate_to_user() APIs Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove 'kbuf' parameter from the copy_xstate_to_user() APIs tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 05/33] x86/fpu: Clean up parameter order in the copy_xstate_to_*() APIs Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Clean up parameter order in the copy_xstate_to_*() APIs tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 24/33] x86/fpu: Don't let userspace set bogus xcomp_bv Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Don't let userspace set bogus xcomp_bv tip-bot for Eric Biggers <tipbot@zytor.com> - 2017-09-26 10:50 +0200
[PATCH 18/33] x86/fpu: Change fpu->fpregs_active users to fpu->fpstate_active Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Change fpu->fpregs_active users to fpu->fpstate_active tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 27/33] x86/fpu: Simplify fpu__activate_fpstate_read() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Fix fpu__activate_fpstate_read() and update comments tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 12/33] x86/fpu: Remove 'ubuf' parameter from the copy_kernel_to_xstate() API Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove 'ubuf' parameter from the copy_kernel_to_xstate() API tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
[PATCH 25/33] x86/fpu: Tighten validation of user-supplied xstate_header Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[PATCH 28/33] x86/fpu: Remove fpu__current_fpstate_write_begin/end() Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
[tip:x86/fpu] x86/fpu: Remove fpu__current_fpstate_write_begin/end() tip-bot for Ingo Molnar <tipbot@zytor.com> - 2017-09-26 10:40 +0200
Re: [PATCH 00/33] x86 FPU fixes and cleanups for v4.14 Ingo Molnar <mingo@kernel.org> - 2017-09-23 15:10 +0200
Re: [PATCH 00/33] x86 FPU fixes and cleanups for v4.14 Juergen Gross <jgross@suse.com> - 2017-09-23 17:10 +0200
Re: [PATCH 00/33] x86 FPU fixes and cleanups for v4.14 Ingo Molnar <mingo@kernel.org> - 2017-09-24 01:30 +0200
csiph-web