Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1431600 > unrolled thread
| Started by | Andy Lutomirski <luto@kernel.org> |
|---|---|
| First post | 2016-06-27 00:00 +0200 |
| Last post | 2016-06-28 15:30 +0200 |
| Articles | 20 on this page of 65 — 10 participants |
Back to article view | Back to linux.kernel
[PATCH v4 00/29] virtually mapped stacks and thread_info cleanup Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 07/29] mm: Fix memcg stack accounting for sub-page stacks Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 24/29] x86/entry: Get rid of pt_regs_to_thread_info() Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 10/29] x86/die: Don't try to recover from an OOPS on a non-default stack Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
Re: [PATCH v4 10/29] x86/die: Don't try to recover from an OOPS on a non-default stack Borislav Petkov <bp@alien8.de> - 2016-07-02 19:30 +0200
Re: [PATCH v4 10/29] x86/die: Don't try to recover from an OOPS on a non-default stack Josh Poimboeuf <jpoimboe@redhat.com> - 2016-07-02 20:40 +0200
Re: [PATCH v4 10/29] x86/die: Don't try to recover from an OOPS on a non-default stack Borislav Petkov <bp@alien8.de> - 2016-07-03 11:50 +0200
Re: [PATCH v4 10/29] x86/die: Don't try to recover from an OOPS on a non-default stack Andy Lutomirski <luto@amacapital.net> - 2016-07-03 16:30 +0200
Re: [PATCH v4 10/29] x86/die: Don't try to recover from an OOPS on a non-default stack Borislav Petkov <bp@alien8.de> - 2016-07-03 20:50 +0200
[PATCH v4 12/29] x86/dumpstack: When dumping stack bytes due to OOPS, start with regs->sp Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 06/29] mm: Track NR_KERNEL_STACK in KiB instead of number of stacks Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 14/29] x86/dumpstack/64: Handle faults when printing the "Stack:" part of an OOPS Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 26/29] sched: Allow putting thread_info into task_struct Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
[PATCH v4 08/29] dma-api: Teach the "DMA-from-stack" check about vmapped stacks Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:00 +0200
Re: [PATCH v4 08/29] dma-api: Teach the "DMA-from-stack" check about vmapped stacks Borislav Petkov <bp@alien8.de> - 2016-06-30 21:40 +0200
Re: [PATCH v4 08/29] dma-api: Teach the "DMA-from-stack" check about vmapped stacks Andy Lutomirski <luto@amacapital.net> - 2016-07-06 15:30 +0200
[PATCH v4 25/29] um: Stop conflating task_struct::stack with thread_info Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 25/29] um: Stop conflating task_struct::stack with thread_info Andy Lutomirski <luto@amacapital.net> - 2016-06-27 01:50 +0200
Re: [PATCH v4 25/29] um: Stop conflating task_struct::stack with thread_info Brian Gerst <brgerst@gmail.com> - 2016-06-27 01:50 +0200
[PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct Brian Gerst <brgerst@gmail.com> - 2016-06-27 02:00 +0200
Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct Andy Lutomirski <luto@amacapital.net> - 2016-06-27 02:30 +0200
Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct Brian Gerst <brgerst@gmail.com> - 2016-06-27 02:40 +0200
Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct Andy Lutomirski <luto@amacapital.net> - 2016-06-27 02:50 +0200
[PATCH v4 19/29] signal: Consolidate {TS,TLF}_RESTORE_SIGMASK code Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Brian Gerst <brgerst@gmail.com> - 2016-06-27 17:10 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Brian Gerst <brgerst@gmail.com> - 2016-06-27 17:20 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Andy Lutomirski <luto@amacapital.net> - 2016-06-27 17:30 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Andy Lutomirski <luto@amacapital.net> - 2016-06-27 18:00 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Brian Gerst <brgerst@gmail.com> - 2016-06-27 18:20 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Andy Lutomirski <luto@amacapital.net> - 2016-06-27 18:40 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Brian Gerst <brgerst@gmail.com> - 2016-06-27 19:10 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Brian Gerst <brgerst@gmail.com> - 2016-06-27 19:30 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Linus Torvalds <torvalds@linux-foundation.org> - 2016-06-27 19:30 +0200
Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks Andy Lutomirski <luto@amacapital.net> - 2016-06-27 19:40 +0200
[PATCH v4 18/29] x86: Move addr_limit to thread_struct Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 17/29] x86: Move uaccess_err and sig_on_uaccess_err to thread_struct Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 03/29] x86/mm/hotplug: Don't remove PGD entries in remove_pagetable() Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 23/29] kdb: Use task_cpu() instead of task_thread_info()->cpu Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 16/29] x86/mm: Improve stack-overflow #PF handling Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 20/29] x86/smp: Remove stack_smp_processor_id() Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
[PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 09:40 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad Herbert Xu <herbert@gondor.apana.org.au> - 2016-06-28 09:40 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 11:10 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad Herbert Xu <herbert@gondor.apana.org.au> - 2016-06-28 11:50 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 09:50 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 10:00 +0200
[PATCH v4 04/29] x86/cpa: In populate_pgd, don't set the pgd entry until it's populated Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 04/29] x86/cpa: In populate_pgd, don't set the pgd entry until it's populated Borislav Petkov <bp@alien8.de> - 2016-06-28 21:10 +0200
Re: [PATCH v4 04/29] x86/cpa: In populate_pgd, don't set the pgd entry until it's populated Andy Lutomirski <luto@amacapital.net> - 2016-06-28 21:10 +0200
[PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Marcel Holtmann <marcel@holtmann.org> - 2016-06-27 08:00 +0200
Re: [PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Ingo Molnar <mingo@kernel.org> - 2016-06-27 11:00 +0200
Re: [PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Marcel Holtmann <marcel@holtmann.org> - 2016-06-28 00:40 +0200
Re: [PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Andy Lutomirski <luto@amacapital.net> - 2016-06-28 00:40 +0200
Re: [PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Marcel Holtmann <marcel@holtmann.org> - 2016-07-04 20:00 +0200
Re: [PATCH v4 01/29] bluetooth: Switch SMP to crypto_cipher_encrypt_one() Andy Lutomirski <luto@amacapital.net> - 2016-07-06 15:20 +0200
[PATCH v4 21/29] x86/smp: Remove unnecessary initialization of thread_info::cpu Andy Lutomirski <luto@kernel.org> - 2016-06-27 00:10 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad Herbert Xu <herbert@gondor.apana.org.au> - 2016-06-28 10:00 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 11:00 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad Herbert Xu <herbert@gondor.apana.org.au> - 2016-06-28 11:50 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 12:10 +0200
Re: [PATCH v4 02/29] rxrpc: Avoid using stack memory in SG lists in rxkad David Howells <dhowells@redhat.com> - 2016-06-28 15:30 +0200
Page 2 of 4 — ← Prev page 1 [2] 3 4 Next page →
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 02:00 +0200 |
| Subject | Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct |
| Message-ID | <rOs81-Zx-1@gated-at.bofh.it> |
| In reply to | #1431611 |
On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
> Becuase sched.h and thread_info.h are a tangled mess, I turned
> in_compat_syscall into a macro. If we had current_thread_struct()
> or similar and we could use it from thread_info.h, then this would
> be a bit cleaner.
>
> Signed-off-by: Andy Lutomirski <luto@kernel.org>
> ---
> arch/x86/entry/common.c | 4 ++--
> arch/x86/include/asm/processor.h | 12 ++++++++++++
> arch/x86/include/asm/syscall.h | 23 +++++------------------
> arch/x86/include/asm/thread_info.h | 23 ++++-------------------
> arch/x86/kernel/asm-offsets.c | 1 -
> arch/x86/kernel/fpu/init.c | 1 -
> arch/x86/kernel/process_64.c | 4 ++--
> arch/x86/kernel/ptrace.c | 2 +-
> 8 files changed, 26 insertions(+), 44 deletions(-)
>
> diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
> index ec138e538c44..c4150bec7982 100644
> --- a/arch/x86/entry/common.c
> +++ b/arch/x86/entry/common.c
> @@ -271,7 +271,7 @@ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs)
> * syscalls. The fixup is exercised by the ptrace_syscall_32
> * selftest.
> */
> - ti->status &= ~TS_COMPAT;
> + current->thread.status &= ~TS_COMPAT;
> #endif
>
> user_enter();
> @@ -369,7 +369,7 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs)
> unsigned int nr = (unsigned int)regs->orig_ax;
>
> #ifdef CONFIG_IA32_EMULATION
> - ti->status |= TS_COMPAT;
> + current->thread.status |= TS_COMPAT;
> #endif
>
> if (READ_ONCE(ti->flags) & _TIF_WORK_SYSCALL_ENTRY) {
> diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
> index a2e20d6d01fe..a75e720f6402 100644
> --- a/arch/x86/include/asm/processor.h
> +++ b/arch/x86/include/asm/processor.h
> @@ -388,6 +388,9 @@ struct thread_struct {
> unsigned short fsindex;
> unsigned short gsindex;
> #endif
> +
> + u32 status; /* thread synchronous flags */
> +
> #ifdef CONFIG_X86_32
> unsigned long ip;
> #endif
> @@ -437,6 +440,15 @@ struct thread_struct {
> };
>
> /*
> + * Thread-synchronous status.
> + *
> + * This is different from the flags in that nobody else
> + * ever touches our thread-synchronous status, so we don't
> + * have to worry about atomic accesses.
> + */
> +#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
> +
> +/*
> * Set IOPL bits in EFLAGS from given mask
> */
> static inline void native_set_iopl_mask(unsigned mask)
> diff --git a/arch/x86/include/asm/syscall.h b/arch/x86/include/asm/syscall.h
> index 999b7cd2e78c..17229e7e2a1c 100644
> --- a/arch/x86/include/asm/syscall.h
> +++ b/arch/x86/include/asm/syscall.h
> @@ -60,7 +60,7 @@ static inline long syscall_get_error(struct task_struct *task,
> * TS_COMPAT is set for 32-bit syscall entries and then
> * remains set until we return to user mode.
> */
> - if (task_thread_info(task)->status & TS_COMPAT)
> + if (task->thread.status & TS_COMPAT)
> /*
> * Sign-extend the value so (int)-EFOO becomes (long)-EFOO
> * and will match correctly in comparisons.
> @@ -116,7 +116,7 @@ static inline void syscall_get_arguments(struct task_struct *task,
> unsigned long *args)
> {
> # ifdef CONFIG_IA32_EMULATION
> - if (task_thread_info(task)->status & TS_COMPAT)
> + if (task->thread.status & TS_COMPAT)
> switch (i) {
> case 0:
> if (!n--) break;
> @@ -177,7 +177,7 @@ static inline void syscall_set_arguments(struct task_struct *task,
> const unsigned long *args)
> {
> # ifdef CONFIG_IA32_EMULATION
> - if (task_thread_info(task)->status & TS_COMPAT)
> + if (task->thread.status & TS_COMPAT)
> switch (i) {
> case 0:
> if (!n--) break;
> @@ -234,21 +234,8 @@ static inline void syscall_set_arguments(struct task_struct *task,
>
> static inline int syscall_get_arch(void)
> {
> -#ifdef CONFIG_IA32_EMULATION
> - /*
> - * TS_COMPAT is set for 32-bit syscall entry and then
> - * remains set until we return to user mode.
> - *
> - * TIF_IA32 tasks should always have TS_COMPAT set at
> - * system call time.
> - *
> - * x32 tasks should be considered AUDIT_ARCH_X86_64.
> - */
> - if (task_thread_info(current)->status & TS_COMPAT)
> - return AUDIT_ARCH_I386;
> -#endif
> - /* Both x32 and x86_64 are considered "64-bit". */
> - return AUDIT_ARCH_X86_64;
> + /* x32 tasks should be considered AUDIT_ARCH_X86_64. */
> + return in_ia32_syscall() ? AUDIT_ARCH_I386 : AUDIT_ARCH_X86_64;
> }
> #endif /* CONFIG_X86_32 */
>
> diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
> index b45ffdda3549..7b42c1e462ac 100644
> --- a/arch/x86/include/asm/thread_info.h
> +++ b/arch/x86/include/asm/thread_info.h
> @@ -55,7 +55,6 @@ struct task_struct;
> struct thread_info {
> struct task_struct *task; /* main task structure */
> __u32 flags; /* low level flags */
> - __u32 status; /* thread synchronous flags */
> __u32 cpu; /* current CPU */
> };
>
> @@ -211,28 +210,14 @@ static inline unsigned long current_stack_pointer(void)
>
> #endif
>
> -/*
> - * Thread-synchronous status.
> - *
> - * This is different from the flags in that nobody else
> - * ever touches our thread-synchronous status, so we don't
> - * have to worry about atomic accesses.
> - */
> -#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
> -
> #ifndef __ASSEMBLY__
>
> -static inline bool in_ia32_syscall(void)
> -{
> #ifdef CONFIG_X86_32
> - return true;
> -#endif
> -#ifdef CONFIG_IA32_EMULATION
> - if (current_thread_info()->status & TS_COMPAT)
> - return true;
> +#define in_ia32_syscall() true
> +#else
> +#define in_ia32_syscall() (IS_ENABLED(CONFIG_IA32_EMULATION) && \
> + current->thread.status & TS_COMPAT)
> #endif
> - return false;
> -}
>
> /*
> * Force syscall return via IRET by making it look as if there was
> diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c
> index 2bd5c6ff7ee7..a91a6ead24a2 100644
> --- a/arch/x86/kernel/asm-offsets.c
> +++ b/arch/x86/kernel/asm-offsets.c
> @@ -30,7 +30,6 @@
> void common(void) {
> BLANK();
> OFFSET(TI_flags, thread_info, flags);
> - OFFSET(TI_status, thread_info, status);
TI_status can be deleted. It's last users were removed in commit ee08c6bd.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2016-06-27 02:30 +0200 |
| Subject | Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct |
| Message-ID | <rOsB4-1om-5@gated-at.bofh.it> |
| In reply to | #1431646 |
On Sun, Jun 26, 2016 at 4:55 PM, Brian Gerst <brgerst@gmail.com> wrote:
> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>> Becuase sched.h and thread_info.h are a tangled mess, I turned
>> in_compat_syscall into a macro. If we had current_thread_struct()
>> or similar and we could use it from thread_info.h, then this would
>> be a bit cleaner.
>>
>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>> ---
>> arch/x86/entry/common.c | 4 ++--
>> arch/x86/include/asm/processor.h | 12 ++++++++++++
>> arch/x86/include/asm/syscall.h | 23 +++++------------------
>> arch/x86/include/asm/thread_info.h | 23 ++++-------------------
>> arch/x86/kernel/asm-offsets.c | 1 -
>> arch/x86/kernel/fpu/init.c | 1 -
>> arch/x86/kernel/process_64.c | 4 ++--
>> arch/x86/kernel/ptrace.c | 2 +-
>> 8 files changed, 26 insertions(+), 44 deletions(-)
>>
>> diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
>> index ec138e538c44..c4150bec7982 100644
>> --- a/arch/x86/entry/common.c
>> +++ b/arch/x86/entry/common.c
>> @@ -271,7 +271,7 @@ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs)
>> * syscalls. The fixup is exercised by the ptrace_syscall_32
>> * selftest.
>> */
>> - ti->status &= ~TS_COMPAT;
>> + current->thread.status &= ~TS_COMPAT;
>> #endif
>>
>> user_enter();
>> @@ -369,7 +369,7 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs)
>> unsigned int nr = (unsigned int)regs->orig_ax;
>>
>> #ifdef CONFIG_IA32_EMULATION
>> - ti->status |= TS_COMPAT;
>> + current->thread.status |= TS_COMPAT;
>> #endif
>>
>> if (READ_ONCE(ti->flags) & _TIF_WORK_SYSCALL_ENTRY) {
>> diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
>> index a2e20d6d01fe..a75e720f6402 100644
>> --- a/arch/x86/include/asm/processor.h
>> +++ b/arch/x86/include/asm/processor.h
>> @@ -388,6 +388,9 @@ struct thread_struct {
>> unsigned short fsindex;
>> unsigned short gsindex;
>> #endif
>> +
>> + u32 status; /* thread synchronous flags */
>> +
>> #ifdef CONFIG_X86_32
>> unsigned long ip;
>> #endif
>> @@ -437,6 +440,15 @@ struct thread_struct {
>> };
>>
>> /*
>> + * Thread-synchronous status.
>> + *
>> + * This is different from the flags in that nobody else
>> + * ever touches our thread-synchronous status, so we don't
>> + * have to worry about atomic accesses.
>> + */
>> +#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
>> +
>> +/*
>> * Set IOPL bits in EFLAGS from given mask
>> */
>> static inline void native_set_iopl_mask(unsigned mask)
>> diff --git a/arch/x86/include/asm/syscall.h b/arch/x86/include/asm/syscall.h
>> index 999b7cd2e78c..17229e7e2a1c 100644
>> --- a/arch/x86/include/asm/syscall.h
>> +++ b/arch/x86/include/asm/syscall.h
>> @@ -60,7 +60,7 @@ static inline long syscall_get_error(struct task_struct *task,
>> * TS_COMPAT is set for 32-bit syscall entries and then
>> * remains set until we return to user mode.
>> */
>> - if (task_thread_info(task)->status & TS_COMPAT)
>> + if (task->thread.status & TS_COMPAT)
>> /*
>> * Sign-extend the value so (int)-EFOO becomes (long)-EFOO
>> * and will match correctly in comparisons.
>> @@ -116,7 +116,7 @@ static inline void syscall_get_arguments(struct task_struct *task,
>> unsigned long *args)
>> {
>> # ifdef CONFIG_IA32_EMULATION
>> - if (task_thread_info(task)->status & TS_COMPAT)
>> + if (task->thread.status & TS_COMPAT)
>> switch (i) {
>> case 0:
>> if (!n--) break;
>> @@ -177,7 +177,7 @@ static inline void syscall_set_arguments(struct task_struct *task,
>> const unsigned long *args)
>> {
>> # ifdef CONFIG_IA32_EMULATION
>> - if (task_thread_info(task)->status & TS_COMPAT)
>> + if (task->thread.status & TS_COMPAT)
>> switch (i) {
>> case 0:
>> if (!n--) break;
>> @@ -234,21 +234,8 @@ static inline void syscall_set_arguments(struct task_struct *task,
>>
>> static inline int syscall_get_arch(void)
>> {
>> -#ifdef CONFIG_IA32_EMULATION
>> - /*
>> - * TS_COMPAT is set for 32-bit syscall entry and then
>> - * remains set until we return to user mode.
>> - *
>> - * TIF_IA32 tasks should always have TS_COMPAT set at
>> - * system call time.
>> - *
>> - * x32 tasks should be considered AUDIT_ARCH_X86_64.
>> - */
>> - if (task_thread_info(current)->status & TS_COMPAT)
>> - return AUDIT_ARCH_I386;
>> -#endif
>> - /* Both x32 and x86_64 are considered "64-bit". */
>> - return AUDIT_ARCH_X86_64;
>> + /* x32 tasks should be considered AUDIT_ARCH_X86_64. */
>> + return in_ia32_syscall() ? AUDIT_ARCH_I386 : AUDIT_ARCH_X86_64;
>> }
>> #endif /* CONFIG_X86_32 */
>>
>> diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
>> index b45ffdda3549..7b42c1e462ac 100644
>> --- a/arch/x86/include/asm/thread_info.h
>> +++ b/arch/x86/include/asm/thread_info.h
>> @@ -55,7 +55,6 @@ struct task_struct;
>> struct thread_info {
>> struct task_struct *task; /* main task structure */
>> __u32 flags; /* low level flags */
>> - __u32 status; /* thread synchronous flags */
>> __u32 cpu; /* current CPU */
>> };
>>
>> @@ -211,28 +210,14 @@ static inline unsigned long current_stack_pointer(void)
>>
>> #endif
>>
>> -/*
>> - * Thread-synchronous status.
>> - *
>> - * This is different from the flags in that nobody else
>> - * ever touches our thread-synchronous status, so we don't
>> - * have to worry about atomic accesses.
>> - */
>> -#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
>> -
>> #ifndef __ASSEMBLY__
>>
>> -static inline bool in_ia32_syscall(void)
>> -{
>> #ifdef CONFIG_X86_32
>> - return true;
>> -#endif
>> -#ifdef CONFIG_IA32_EMULATION
>> - if (current_thread_info()->status & TS_COMPAT)
>> - return true;
>> +#define in_ia32_syscall() true
>> +#else
>> +#define in_ia32_syscall() (IS_ENABLED(CONFIG_IA32_EMULATION) && \
>> + current->thread.status & TS_COMPAT)
>> #endif
>> - return false;
>> -}
>>
>> /*
>> * Force syscall return via IRET by making it look as if there was
>> diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c
>> index 2bd5c6ff7ee7..a91a6ead24a2 100644
>> --- a/arch/x86/kernel/asm-offsets.c
>> +++ b/arch/x86/kernel/asm-offsets.c
>> @@ -30,7 +30,6 @@
>> void common(void) {
>> BLANK();
>> OFFSET(TI_flags, thread_info, flags);
>> - OFFSET(TI_status, thread_info, status);
>
> TI_status can be deleted. It's last users were removed in commit ee08c6bd.
Indeed.
Just to double-check: are you saying that this patch is okay?
--Andy
[toc] | [prev] | [next] | [standalone]
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 02:40 +0200 |
| Subject | Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct |
| Message-ID | <rOsKJ-1rw-7@gated-at.bofh.it> |
| In reply to | #1431651 |
On Sun, Jun 26, 2016 at 8:23 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Sun, Jun 26, 2016 at 4:55 PM, Brian Gerst <brgerst@gmail.com> wrote:
>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>> Becuase sched.h and thread_info.h are a tangled mess, I turned
>>> in_compat_syscall into a macro. If we had current_thread_struct()
>>> or similar and we could use it from thread_info.h, then this would
>>> be a bit cleaner.
>>>
>>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>>> ---
>>> arch/x86/entry/common.c | 4 ++--
>>> arch/x86/include/asm/processor.h | 12 ++++++++++++
>>> arch/x86/include/asm/syscall.h | 23 +++++------------------
>>> arch/x86/include/asm/thread_info.h | 23 ++++-------------------
>>> arch/x86/kernel/asm-offsets.c | 1 -
>>> arch/x86/kernel/fpu/init.c | 1 -
>>> arch/x86/kernel/process_64.c | 4 ++--
>>> arch/x86/kernel/ptrace.c | 2 +-
>>> 8 files changed, 26 insertions(+), 44 deletions(-)
>>>
>>> diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
>>> index ec138e538c44..c4150bec7982 100644
>>> --- a/arch/x86/entry/common.c
>>> +++ b/arch/x86/entry/common.c
>>> @@ -271,7 +271,7 @@ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs)
>>> * syscalls. The fixup is exercised by the ptrace_syscall_32
>>> * selftest.
>>> */
>>> - ti->status &= ~TS_COMPAT;
>>> + current->thread.status &= ~TS_COMPAT;
>>> #endif
>>>
>>> user_enter();
>>> @@ -369,7 +369,7 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs)
>>> unsigned int nr = (unsigned int)regs->orig_ax;
>>>
>>> #ifdef CONFIG_IA32_EMULATION
>>> - ti->status |= TS_COMPAT;
>>> + current->thread.status |= TS_COMPAT;
>>> #endif
>>>
>>> if (READ_ONCE(ti->flags) & _TIF_WORK_SYSCALL_ENTRY) {
>>> diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
>>> index a2e20d6d01fe..a75e720f6402 100644
>>> --- a/arch/x86/include/asm/processor.h
>>> +++ b/arch/x86/include/asm/processor.h
>>> @@ -388,6 +388,9 @@ struct thread_struct {
>>> unsigned short fsindex;
>>> unsigned short gsindex;
>>> #endif
>>> +
>>> + u32 status; /* thread synchronous flags */
>>> +
>>> #ifdef CONFIG_X86_32
>>> unsigned long ip;
>>> #endif
>>> @@ -437,6 +440,15 @@ struct thread_struct {
>>> };
>>>
>>> /*
>>> + * Thread-synchronous status.
>>> + *
>>> + * This is different from the flags in that nobody else
>>> + * ever touches our thread-synchronous status, so we don't
>>> + * have to worry about atomic accesses.
>>> + */
>>> +#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
>>> +
>>> +/*
>>> * Set IOPL bits in EFLAGS from given mask
>>> */
>>> static inline void native_set_iopl_mask(unsigned mask)
>>> diff --git a/arch/x86/include/asm/syscall.h b/arch/x86/include/asm/syscall.h
>>> index 999b7cd2e78c..17229e7e2a1c 100644
>>> --- a/arch/x86/include/asm/syscall.h
>>> +++ b/arch/x86/include/asm/syscall.h
>>> @@ -60,7 +60,7 @@ static inline long syscall_get_error(struct task_struct *task,
>>> * TS_COMPAT is set for 32-bit syscall entries and then
>>> * remains set until we return to user mode.
>>> */
>>> - if (task_thread_info(task)->status & TS_COMPAT)
>>> + if (task->thread.status & TS_COMPAT)
>>> /*
>>> * Sign-extend the value so (int)-EFOO becomes (long)-EFOO
>>> * and will match correctly in comparisons.
>>> @@ -116,7 +116,7 @@ static inline void syscall_get_arguments(struct task_struct *task,
>>> unsigned long *args)
>>> {
>>> # ifdef CONFIG_IA32_EMULATION
>>> - if (task_thread_info(task)->status & TS_COMPAT)
>>> + if (task->thread.status & TS_COMPAT)
>>> switch (i) {
>>> case 0:
>>> if (!n--) break;
>>> @@ -177,7 +177,7 @@ static inline void syscall_set_arguments(struct task_struct *task,
>>> const unsigned long *args)
>>> {
>>> # ifdef CONFIG_IA32_EMULATION
>>> - if (task_thread_info(task)->status & TS_COMPAT)
>>> + if (task->thread.status & TS_COMPAT)
>>> switch (i) {
>>> case 0:
>>> if (!n--) break;
>>> @@ -234,21 +234,8 @@ static inline void syscall_set_arguments(struct task_struct *task,
>>>
>>> static inline int syscall_get_arch(void)
>>> {
>>> -#ifdef CONFIG_IA32_EMULATION
>>> - /*
>>> - * TS_COMPAT is set for 32-bit syscall entry and then
>>> - * remains set until we return to user mode.
>>> - *
>>> - * TIF_IA32 tasks should always have TS_COMPAT set at
>>> - * system call time.
>>> - *
>>> - * x32 tasks should be considered AUDIT_ARCH_X86_64.
>>> - */
>>> - if (task_thread_info(current)->status & TS_COMPAT)
>>> - return AUDIT_ARCH_I386;
>>> -#endif
>>> - /* Both x32 and x86_64 are considered "64-bit". */
>>> - return AUDIT_ARCH_X86_64;
>>> + /* x32 tasks should be considered AUDIT_ARCH_X86_64. */
>>> + return in_ia32_syscall() ? AUDIT_ARCH_I386 : AUDIT_ARCH_X86_64;
>>> }
>>> #endif /* CONFIG_X86_32 */
>>>
>>> diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
>>> index b45ffdda3549..7b42c1e462ac 100644
>>> --- a/arch/x86/include/asm/thread_info.h
>>> +++ b/arch/x86/include/asm/thread_info.h
>>> @@ -55,7 +55,6 @@ struct task_struct;
>>> struct thread_info {
>>> struct task_struct *task; /* main task structure */
>>> __u32 flags; /* low level flags */
>>> - __u32 status; /* thread synchronous flags */
>>> __u32 cpu; /* current CPU */
>>> };
>>>
>>> @@ -211,28 +210,14 @@ static inline unsigned long current_stack_pointer(void)
>>>
>>> #endif
>>>
>>> -/*
>>> - * Thread-synchronous status.
>>> - *
>>> - * This is different from the flags in that nobody else
>>> - * ever touches our thread-synchronous status, so we don't
>>> - * have to worry about atomic accesses.
>>> - */
>>> -#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
>>> -
>>> #ifndef __ASSEMBLY__
>>>
>>> -static inline bool in_ia32_syscall(void)
>>> -{
>>> #ifdef CONFIG_X86_32
>>> - return true;
>>> -#endif
>>> -#ifdef CONFIG_IA32_EMULATION
>>> - if (current_thread_info()->status & TS_COMPAT)
>>> - return true;
>>> +#define in_ia32_syscall() true
>>> +#else
>>> +#define in_ia32_syscall() (IS_ENABLED(CONFIG_IA32_EMULATION) && \
>>> + current->thread.status & TS_COMPAT)
>>> #endif
>>> - return false;
>>> -}
>>>
>>> /*
>>> * Force syscall return via IRET by making it look as if there was
>>> diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c
>>> index 2bd5c6ff7ee7..a91a6ead24a2 100644
>>> --- a/arch/x86/kernel/asm-offsets.c
>>> +++ b/arch/x86/kernel/asm-offsets.c
>>> @@ -30,7 +30,6 @@
>>> void common(void) {
>>> BLANK();
>>> OFFSET(TI_flags, thread_info, flags);
>>> - OFFSET(TI_status, thread_info, status);
>>
>> TI_status can be deleted. It's last users were removed in commit ee08c6bd.
>
> Indeed.
>
> Just to double-check: are you saying that this patch is okay?
It looks OK to me, but I haven't tested it. Another suggestion is to
change the compat flag to a bitfield, since there is only one TS_*
flag now and it's not referenced from asm.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2016-06-27 02:50 +0200 |
| Subject | Re: [PATCH v4 22/29] x86/asm: Move 'status' from struct thread_info to struct thread_struct |
| Message-ID | <rOsUp-1uE-1@gated-at.bofh.it> |
| In reply to | #1431652 |
On Sun, Jun 26, 2016 at 5:36 PM, Brian Gerst <brgerst@gmail.com> wrote:
> On Sun, Jun 26, 2016 at 8:23 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Sun, Jun 26, 2016 at 4:55 PM, Brian Gerst <brgerst@gmail.com> wrote:
>>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>>> --- a/arch/x86/kernel/asm-offsets.c
>>>> +++ b/arch/x86/kernel/asm-offsets.c
>>>> @@ -30,7 +30,6 @@
>>>> void common(void) {
>>>> BLANK();
>>>> OFFSET(TI_flags, thread_info, flags);
>>>> - OFFSET(TI_status, thread_info, status);
>>>
>>> TI_status can be deleted. It's last users were removed in commit ee08c6bd.
>>
>> Indeed.
>>
>> Just to double-check: are you saying that this patch is okay?
>
> It looks OK to me, but I haven't tested it. Another suggestion is to
> change the compat flag to a bitfield, since there is only one TS_*
> flag now and it's not referenced from asm.
That could also work.
As a silly alternative thought: we just might be able to get away with
shoving the "is ia32" flag into one of the high bits of
pt_regs->orig_ax. It wouldn't break any 32-bit ptrace users because
they can't see the high bits. It wouldn't break most 64-bit ptrace
users because they use the silly PTRACE_GETREGSET API that doesn't
show the high bits if the tracee is "32-bit". It would change
behavior when a 64-bit tracer traces a 64-bit process that does int
$0x80, but at least strace already gets that case completely wrong.
Of course, this proposal has all kinds of problems.
--Andy
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2016-06-27 00:10 +0200 |
| Subject | [PATCH v4 19/29] signal: Consolidate {TS,TLF}_RESTORE_SIGMASK code |
| Message-ID | <rOqpA-5R-9@gated-at.bofh.it> |
| In reply to | #1431600 |
In general, there's no need for the "restore sigmask" flag to live in
ti->flags. alpha, ia64, microblaze, powerpc, sh, sparc (64-bit only),
tile, and x86 use essentially identical alternative implementations,
placing the flag in ti->status.
Replace those optimized implementations with an equally good common
implementation that stores it in a bitfield in struct task_struct
and drop the custom implementations.
Additional architectures can opt in by removing their
TIF_RESTORE_SIGMASK defines.
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: x86@kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Borislav Petkov <bp@suse.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dmitry Safonov <dsafonov@virtuozzo.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-alpha@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-sh@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/alpha/include/asm/thread_info.h | 27 -------------
arch/ia64/include/asm/thread_info.h | 28 --------------
arch/microblaze/include/asm/thread_info.h | 27 -------------
arch/powerpc/include/asm/thread_info.h | 25 ------------
arch/sh/include/asm/thread_info.h | 26 -------------
arch/sparc/include/asm/thread_info_64.h | 24 ------------
arch/tile/include/asm/thread_info.h | 27 -------------
arch/x86/include/asm/thread_info.h | 24 ------------
include/linux/sched.h | 63 +++++++++++++++++++++++++++++++
include/linux/thread_info.h | 41 --------------------
10 files changed, 63 insertions(+), 249 deletions(-)
diff --git a/arch/alpha/include/asm/thread_info.h b/arch/alpha/include/asm/thread_info.h
index 32e920a83ae5..e9e90bfa2b50 100644
--- a/arch/alpha/include/asm/thread_info.h
+++ b/arch/alpha/include/asm/thread_info.h
@@ -86,33 +86,6 @@ register struct thread_info *__current_thread_info __asm__("$8");
#define TS_UAC_NOPRINT 0x0001 /* ! Preserve the following three */
#define TS_UAC_NOFIX 0x0002 /* ! flags as they match */
#define TS_UAC_SIGBUS 0x0004 /* ! userspace part of 'osf_sysinfo' */
-#define TS_RESTORE_SIGMASK 0x0008 /* restore signal mask in do_signal() */
-
-#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, (unsigned long *)&ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
-#endif
#define SET_UNALIGN_CTL(task,value) ({ \
__u32 status = task_thread_info(task)->status & ~UAC_BITMASK; \
diff --git a/arch/ia64/include/asm/thread_info.h b/arch/ia64/include/asm/thread_info.h
index f0a72e98e5a4..c7026429816b 100644
--- a/arch/ia64/include/asm/thread_info.h
+++ b/arch/ia64/include/asm/thread_info.h
@@ -121,32 +121,4 @@ struct thread_info {
/* like TIF_ALLWORK_BITS but sans TIF_SYSCALL_TRACE or TIF_SYSCALL_AUDIT */
#define TIF_WORK_MASK (TIF_ALLWORK_MASK&~(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT))
-#define TS_RESTORE_SIGMASK 2 /* restore signal mask in do_signal() */
-
-#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, &ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
-#endif /* !__ASSEMBLY__ */
-
#endif /* _ASM_IA64_THREAD_INFO_H */
diff --git a/arch/microblaze/include/asm/thread_info.h b/arch/microblaze/include/asm/thread_info.h
index 383f387b4eee..e7e8954e9815 100644
--- a/arch/microblaze/include/asm/thread_info.h
+++ b/arch/microblaze/include/asm/thread_info.h
@@ -148,33 +148,6 @@ static inline struct thread_info *current_thread_info(void)
*/
/* FPU was used by this task this quantum (SMP) */
#define TS_USEDFPU 0x0001
-#define TS_RESTORE_SIGMASK 0x0002
-
-#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, (unsigned long *)&ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
-#endif
#endif /* __KERNEL__ */
#endif /* _ASM_MICROBLAZE_THREAD_INFO_H */
diff --git a/arch/powerpc/include/asm/thread_info.h b/arch/powerpc/include/asm/thread_info.h
index 8febc3f66d53..cfc35195f95e 100644
--- a/arch/powerpc/include/asm/thread_info.h
+++ b/arch/powerpc/include/asm/thread_info.h
@@ -134,40 +134,15 @@ static inline struct thread_info *current_thread_info(void)
/* Don't move TLF_NAPPING without adjusting the code in entry_32.S */
#define TLF_NAPPING 0 /* idle thread enabled NAP mode */
#define TLF_SLEEPING 1 /* suspend code enabled SLEEP mode */
-#define TLF_RESTORE_SIGMASK 2 /* Restore signal mask in do_signal */
#define TLF_LAZY_MMU 3 /* tlb_batch is active */
#define TLF_RUNLATCH 4 /* Is the runlatch enabled? */
#define _TLF_NAPPING (1 << TLF_NAPPING)
#define _TLF_SLEEPING (1 << TLF_SLEEPING)
-#define _TLF_RESTORE_SIGMASK (1 << TLF_RESTORE_SIGMASK)
#define _TLF_LAZY_MMU (1 << TLF_LAZY_MMU)
#define _TLF_RUNLATCH (1 << TLF_RUNLATCH)
#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->local_flags |= _TLF_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, &ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->local_flags &= ~_TLF_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->local_flags & _TLF_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->local_flags & _TLF_RESTORE_SIGMASK))
- return false;
- ti->local_flags &= ~_TLF_RESTORE_SIGMASK;
- return true;
-}
static inline bool test_thread_local_flags(unsigned int flags)
{
diff --git a/arch/sh/include/asm/thread_info.h b/arch/sh/include/asm/thread_info.h
index 2afa321157be..6c65dcd470ab 100644
--- a/arch/sh/include/asm/thread_info.h
+++ b/arch/sh/include/asm/thread_info.h
@@ -151,19 +151,10 @@ extern void init_thread_xstate(void);
* ever touches our thread-synchronous status, so we don't
* have to worry about atomic accesses.
*/
-#define TS_RESTORE_SIGMASK 0x0001 /* restore signal mask in do_signal() */
#define TS_USEDFPU 0x0002 /* FPU used by this task this quantum */
#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, (unsigned long *)&ti->flags));
-}
-
#define TI_FLAG_FAULT_CODE_SHIFT 24
/*
@@ -182,23 +173,6 @@ static inline unsigned int get_thread_fault_code(void)
return ti->flags >> TI_FLAG_FAULT_CODE_SHIFT;
}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
-
#endif /* !__ASSEMBLY__ */
#endif /* __KERNEL__ */
diff --git a/arch/sparc/include/asm/thread_info_64.h b/arch/sparc/include/asm/thread_info_64.h
index bde59825d06c..3d7b925f6516 100644
--- a/arch/sparc/include/asm/thread_info_64.h
+++ b/arch/sparc/include/asm/thread_info_64.h
@@ -222,32 +222,8 @@ register struct thread_info *current_thread_info_reg asm("g6");
*
* Note that there are only 8 bits available.
*/
-#define TS_RESTORE_SIGMASK 0x0001 /* restore signal mask in do_signal() */
#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, &ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
#define thread32_stack_is_64bit(__SP) (((__SP) & 0x1) != 0)
#define test_thread_64bit_stack(__SP) \
diff --git a/arch/tile/include/asm/thread_info.h b/arch/tile/include/asm/thread_info.h
index c1467ac59ce6..b7659b8f1117 100644
--- a/arch/tile/include/asm/thread_info.h
+++ b/arch/tile/include/asm/thread_info.h
@@ -166,32 +166,5 @@ extern void _cpu_idle(void);
#ifdef __tilegx__
#define TS_COMPAT 0x0001 /* 32-bit compatibility mode */
#endif
-#define TS_RESTORE_SIGMASK 0x0008 /* restore signal mask in do_signal */
-
-#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, &ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
-#endif /* !__ASSEMBLY__ */
#endif /* _ASM_TILE_THREAD_INFO_H */
diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index 89bff044a6f5..b45ffdda3549 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -219,32 +219,8 @@ static inline unsigned long current_stack_pointer(void)
* have to worry about atomic accesses.
*/
#define TS_COMPAT 0x0002 /* 32bit syscall active (64BIT)*/
-#define TS_RESTORE_SIGMASK 0x0008 /* restore signal mask in do_signal() */
#ifndef __ASSEMBLY__
-#define HAVE_SET_RESTORE_SIGMASK 1
-static inline void set_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- ti->status |= TS_RESTORE_SIGMASK;
- WARN_ON(!test_bit(TIF_SIGPENDING, (unsigned long *)&ti->flags));
-}
-static inline void clear_restore_sigmask(void)
-{
- current_thread_info()->status &= ~TS_RESTORE_SIGMASK;
-}
-static inline bool test_restore_sigmask(void)
-{
- return current_thread_info()->status & TS_RESTORE_SIGMASK;
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- struct thread_info *ti = current_thread_info();
- if (!(ti->status & TS_RESTORE_SIGMASK))
- return false;
- ti->status &= ~TS_RESTORE_SIGMASK;
- return true;
-}
static inline bool in_ia32_syscall(void)
{
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 26869dba21f1..569df670407a 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1545,6 +1545,9 @@ struct task_struct {
/* unserialized, strictly 'current' */
unsigned in_execve:1; /* bit to tell LSMs we're in execve */
unsigned in_iowait:1;
+#if !defined(TIF_RESTORE_SIGMASK)
+ unsigned restore_sigmask:1;
+#endif
#ifdef CONFIG_MEMCG
unsigned memcg_may_oom:1;
#ifndef CONFIG_SLOB
@@ -2664,6 +2667,66 @@ extern void sigqueue_free(struct sigqueue *);
extern int send_sigqueue(struct sigqueue *, struct task_struct *, int group);
extern int do_sigaction(int, struct k_sigaction *, struct k_sigaction *);
+#ifdef TIF_RESTORE_SIGMASK
+/*
+ * Legacy restore_sigmask accessors. These are inefficient on
+ * SMP architectures because they require atomic operations.
+ */
+
+/**
+ * set_restore_sigmask() - make sure saved_sigmask processing gets done
+ *
+ * This sets TIF_RESTORE_SIGMASK and ensures that the arch signal code
+ * will run before returning to user mode, to process the flag. For
+ * all callers, TIF_SIGPENDING is already set or it's no harm to set
+ * it. TIF_RESTORE_SIGMASK need not be in the set of bits that the
+ * arch code will notice on return to user mode, in case those bits
+ * are scarce. We set TIF_SIGPENDING here to ensure that the arch
+ * signal code always gets run when TIF_RESTORE_SIGMASK is set.
+ */
+static inline void set_restore_sigmask(void)
+{
+ set_thread_flag(TIF_RESTORE_SIGMASK);
+ WARN_ON(!test_thread_flag(TIF_SIGPENDING));
+}
+static inline void clear_restore_sigmask(void)
+{
+ clear_thread_flag(TIF_RESTORE_SIGMASK);
+}
+static inline bool test_restore_sigmask(void)
+{
+ return test_thread_flag(TIF_RESTORE_SIGMASK);
+}
+static inline bool test_and_clear_restore_sigmask(void)
+{
+ return test_and_clear_thread_flag(TIF_RESTORE_SIGMASK);
+}
+
+#else /* TIF_RESTORE_SIGMASK */
+
+/* Higher-quality implementation, used if TIF_RESTORE_SIGMASK doesn't exist. */
+static inline void set_restore_sigmask(void)
+{
+ current->restore_sigmask = true;
+ WARN_ON(!test_thread_flag(TIF_SIGPENDING));
+}
+static inline void clear_restore_sigmask(void)
+{
+ current->restore_sigmask = false;
+}
+static inline bool test_restore_sigmask(void)
+{
+ return current->restore_sigmask;
+}
+static inline bool test_and_clear_restore_sigmask(void)
+{
+ if (!current->restore_sigmask)
+ return false;
+ current->restore_sigmask = false;
+ return true;
+}
+#endif
+
static inline void restore_saved_sigmask(void)
{
if (test_and_clear_restore_sigmask())
diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h
index b4c2a485b28a..352b1542f5cc 100644
--- a/include/linux/thread_info.h
+++ b/include/linux/thread_info.h
@@ -105,47 +105,6 @@ static inline int test_ti_thread_flag(struct thread_info *ti, int flag)
#define tif_need_resched() test_thread_flag(TIF_NEED_RESCHED)
-#if defined TIF_RESTORE_SIGMASK && !defined HAVE_SET_RESTORE_SIGMASK
-/*
- * An arch can define its own version of set_restore_sigmask() to get the
- * job done however works, with or without TIF_RESTORE_SIGMASK.
- */
-#define HAVE_SET_RESTORE_SIGMASK 1
-
-/**
- * set_restore_sigmask() - make sure saved_sigmask processing gets done
- *
- * This sets TIF_RESTORE_SIGMASK and ensures that the arch signal code
- * will run before returning to user mode, to process the flag. For
- * all callers, TIF_SIGPENDING is already set or it's no harm to set
- * it. TIF_RESTORE_SIGMASK need not be in the set of bits that the
- * arch code will notice on return to user mode, in case those bits
- * are scarce. We set TIF_SIGPENDING here to ensure that the arch
- * signal code always gets run when TIF_RESTORE_SIGMASK is set.
- */
-static inline void set_restore_sigmask(void)
-{
- set_thread_flag(TIF_RESTORE_SIGMASK);
- WARN_ON(!test_thread_flag(TIF_SIGPENDING));
-}
-static inline void clear_restore_sigmask(void)
-{
- clear_thread_flag(TIF_RESTORE_SIGMASK);
-}
-static inline bool test_restore_sigmask(void)
-{
- return test_thread_flag(TIF_RESTORE_SIGMASK);
-}
-static inline bool test_and_clear_restore_sigmask(void)
-{
- return test_and_clear_thread_flag(TIF_RESTORE_SIGMASK);
-}
-#endif /* TIF_RESTORE_SIGMASK && !HAVE_SET_RESTORE_SIGMASK */
-
-#ifndef HAVE_SET_RESTORE_SIGMASK
-#error "no set_restore_sigmask() provided and default one won't work"
-#endif
-
#endif /* __KERNEL__ */
#endif /* _LINUX_THREAD_INFO_H */
--
2.7.4
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2016-06-27 00:10 +0200 |
| Subject | [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOqpA-5R-13@gated-at.bofh.it> |
| In reply to | #1431600 |
This allows x86_64 kernels to enable vmapped stacks. There are a
couple of interesting bits.
First, x86 lazily faults in top-level paging entries for the vmalloc
area. This won't work if we get a page fault while trying to access
the stack: the CPU will promote it to a double-fault and we'll die.
To avoid this problem, probe the new stack when switching stacks and
forcibly populate the pgd entry for the stack when switching mms.
Second, once we have guard pages around the stack, we'll want to
detect and handle stack overflow.
I didn't enable it on x86_32. We'd need to rework the double-fault
code a bit and I'm concerned about running out of vmalloc virtual
addresses under some workloads.
This patch, by itself, will behave somewhat erratically when the
stack overflows while RSP is still more than a few tens of bytes
above the bottom of the stack. Specifically, we'll get #PF and make
it to no_context and an oops without triggering a double-fault, and
no_context doesn't know about stack overflows. The next patch will
improve that case.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/Kconfig | 1 +
arch/x86/include/asm/switch_to.h | 28 +++++++++++++++++++++++++++-
arch/x86/kernel/traps.c | 32 ++++++++++++++++++++++++++++++++
arch/x86/mm/tlb.c | 15 +++++++++++++++
4 files changed, 75 insertions(+), 1 deletion(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index d9a94da0c29f..afdcf96ef109 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -92,6 +92,7 @@ config X86
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
select HAVE_EBPF_JIT if X86_64
+ select HAVE_ARCH_VMAP_STACK if X86_64
select HAVE_CC_STACKPROTECTOR
select HAVE_CMPXCHG_DOUBLE
select HAVE_CMPXCHG_LOCAL
diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h
index 8f321a1b03a1..14e4b20f0aaf 100644
--- a/arch/x86/include/asm/switch_to.h
+++ b/arch/x86/include/asm/switch_to.h
@@ -8,6 +8,28 @@ struct tss_struct;
void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p,
struct tss_struct *tss);
+/* This runs runs on the previous thread's stack. */
+static inline void prepare_switch_to(struct task_struct *prev,
+ struct task_struct *next)
+{
+#ifdef CONFIG_VMAP_STACK
+ /*
+ * If we switch to a stack that has a top-level paging entry
+ * that is not present in the current mm, the resulting #PF will
+ * will be promoted to a double-fault and we'll panic. Probe
+ * the new stack now so that vmalloc_fault can fix up the page
+ * tables if needed. This can only happen if we use a stack
+ * in vmap space.
+ *
+ * We assume that the stack is aligned so that it never spans
+ * more than one top-level paging entry.
+ *
+ * To minimize cache pollution, just follow the stack pointer.
+ */
+ READ_ONCE(*(unsigned char *)next->thread.sp);
+#endif
+}
+
#ifdef CONFIG_X86_32
#ifdef CONFIG_CC_STACKPROTECTOR
@@ -39,6 +61,8 @@ do { \
*/ \
unsigned long ebx, ecx, edx, esi, edi; \
\
+ prepare_switch_to(prev, next); \
+ \
asm volatile("pushl %%ebp\n\t" /* save EBP */ \
"movl %%esp,%[prev_sp]\n\t" /* save ESP */ \
"movl %[next_sp],%%esp\n\t" /* restore ESP */ \
@@ -103,7 +127,9 @@ do { \
* clean in kernel mode, with the possible exception of IOPL. Kernel IOPL
* has no effect.
*/
-#define switch_to(prev, next, last) \
+#define switch_to(prev, next, last) \
+ prepare_switch_to(prev, next); \
+ \
asm volatile(SAVE_CONTEXT \
"movq %%rsp,%P[threadrsp](%[prev])\n\t" /* save RSP */ \
"movq %P[threadrsp](%[next]),%%rsp\n\t" /* restore RSP */ \
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 00f03d82e69a..9cb7ea781176 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -292,12 +292,30 @@ DO_ERROR(X86_TRAP_NP, SIGBUS, "segment not present", segment_not_present)
DO_ERROR(X86_TRAP_SS, SIGBUS, "stack segment", stack_segment)
DO_ERROR(X86_TRAP_AC, SIGBUS, "alignment check", alignment_check)
+#ifdef CONFIG_VMAP_STACK
+static void __noreturn handle_stack_overflow(const char *message,
+ struct pt_regs *regs,
+ unsigned long fault_address)
+{
+ printk(KERN_EMERG "BUG: stack guard page was hit at %p (stack is %p..%p)\n",
+ (void *)fault_address, current->stack,
+ (char *)current->stack + THREAD_SIZE - 1);
+ die(message, regs, 0);
+
+ /* Be absolutely certain we don't return. */
+ panic(message);
+}
+#endif
+
#ifdef CONFIG_X86_64
/* Runs on IST stack */
dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
{
static const char str[] = "double fault";
struct task_struct *tsk = current;
+#ifdef CONFIG_VMAP_STACK
+ unsigned long cr2;
+#endif
#ifdef CONFIG_X86_ESPFIX64
extern unsigned char native_irq_return_iret[];
@@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
tsk->thread.error_code = error_code;
tsk->thread.trap_nr = X86_TRAP_DF;
+#ifdef CONFIG_VMAP_STACK
+ /*
+ * If we overflow the stack into a guard page, the CPU will fail
+ * to deliver #PF and will send #DF instead. CR2 will contain
+ * the linear address of the second fault, which will be in the
+ * guard page below the bottom of the stack.
+ */
+ cr2 = read_cr2();
+ if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
+ handle_stack_overflow(
+ "kernel stack overflow (double-fault)",
+ regs, cr2);
+#endif
+
#ifdef CONFIG_DOUBLEFAULT
df_debug(regs, error_code);
#endif
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 5643fd0b1a7d..fbf036ae72ac 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -77,10 +77,25 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
unsigned cpu = smp_processor_id();
if (likely(prev != next)) {
+ if (IS_ENABLED(CONFIG_VMAP_STACK)) {
+ /*
+ * If our current stack is in vmalloc space and isn't
+ * mapped in the new pgd, we'll double-fault. Forcibly
+ * map it.
+ */
+ unsigned int stack_pgd_index =
+ pgd_index(current_stack_pointer());
+ pgd_t *pgd = next->pgd + stack_pgd_index;
+
+ if (unlikely(pgd_none(*pgd)))
+ set_pgd(pgd, init_mm.pgd[stack_pgd_index]);
+ }
+
#ifdef CONFIG_SMP
this_cpu_write(cpu_tlbstate.state, TLBSTATE_OK);
this_cpu_write(cpu_tlbstate.active_mm, next);
#endif
+
cpumask_set_cpu(cpu, mm_cpumask(next));
/*
--
2.7.4
[toc] | [prev] | [next] | [standalone]
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 17:10 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOGkF-1Xk-17@gated-at.bofh.it> |
| In reply to | #1431613 |
On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
> This allows x86_64 kernels to enable vmapped stacks. There are a
> couple of interesting bits.
>
> First, x86 lazily faults in top-level paging entries for the vmalloc
> area. This won't work if we get a page fault while trying to access
> the stack: the CPU will promote it to a double-fault and we'll die.
> To avoid this problem, probe the new stack when switching stacks and
> forcibly populate the pgd entry for the stack when switching mms.
>
> Second, once we have guard pages around the stack, we'll want to
> detect and handle stack overflow.
>
> I didn't enable it on x86_32. We'd need to rework the double-fault
> code a bit and I'm concerned about running out of vmalloc virtual
> addresses under some workloads.
>
> This patch, by itself, will behave somewhat erratically when the
> stack overflows while RSP is still more than a few tens of bytes
> above the bottom of the stack. Specifically, we'll get #PF and make
> it to no_context and an oops without triggering a double-fault, and
> no_context doesn't know about stack overflows. The next patch will
> improve that case.
>
> Signed-off-by: Andy Lutomirski <luto@kernel.org>
> ---
> arch/x86/Kconfig | 1 +
> arch/x86/include/asm/switch_to.h | 28 +++++++++++++++++++++++++++-
> arch/x86/kernel/traps.c | 32 ++++++++++++++++++++++++++++++++
> arch/x86/mm/tlb.c | 15 +++++++++++++++
> 4 files changed, 75 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index d9a94da0c29f..afdcf96ef109 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -92,6 +92,7 @@ config X86
> select HAVE_ARCH_TRACEHOOK
> select HAVE_ARCH_TRANSPARENT_HUGEPAGE
> select HAVE_EBPF_JIT if X86_64
> + select HAVE_ARCH_VMAP_STACK if X86_64
> select HAVE_CC_STACKPROTECTOR
> select HAVE_CMPXCHG_DOUBLE
> select HAVE_CMPXCHG_LOCAL
> diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h
> index 8f321a1b03a1..14e4b20f0aaf 100644
> --- a/arch/x86/include/asm/switch_to.h
> +++ b/arch/x86/include/asm/switch_to.h
> @@ -8,6 +8,28 @@ struct tss_struct;
> void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p,
> struct tss_struct *tss);
>
> +/* This runs runs on the previous thread's stack. */
> +static inline void prepare_switch_to(struct task_struct *prev,
> + struct task_struct *next)
> +{
> +#ifdef CONFIG_VMAP_STACK
> + /*
> + * If we switch to a stack that has a top-level paging entry
> + * that is not present in the current mm, the resulting #PF will
> + * will be promoted to a double-fault and we'll panic. Probe
> + * the new stack now so that vmalloc_fault can fix up the page
> + * tables if needed. This can only happen if we use a stack
> + * in vmap space.
> + *
> + * We assume that the stack is aligned so that it never spans
> + * more than one top-level paging entry.
> + *
> + * To minimize cache pollution, just follow the stack pointer.
> + */
> + READ_ONCE(*(unsigned char *)next->thread.sp);
> +#endif
> +}
> +
> #ifdef CONFIG_X86_32
>
> #ifdef CONFIG_CC_STACKPROTECTOR
> @@ -39,6 +61,8 @@ do { \
> */ \
> unsigned long ebx, ecx, edx, esi, edi; \
> \
> + prepare_switch_to(prev, next); \
> + \
> asm volatile("pushl %%ebp\n\t" /* save EBP */ \
> "movl %%esp,%[prev_sp]\n\t" /* save ESP */ \
> "movl %[next_sp],%%esp\n\t" /* restore ESP */ \
> @@ -103,7 +127,9 @@ do { \
> * clean in kernel mode, with the possible exception of IOPL. Kernel IOPL
> * has no effect.
> */
> -#define switch_to(prev, next, last) \
> +#define switch_to(prev, next, last) \
> + prepare_switch_to(prev, next); \
> + \
> asm volatile(SAVE_CONTEXT \
> "movq %%rsp,%P[threadrsp](%[prev])\n\t" /* save RSP */ \
> "movq %P[threadrsp](%[next]),%%rsp\n\t" /* restore RSP */ \
> diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
> index 00f03d82e69a..9cb7ea781176 100644
> --- a/arch/x86/kernel/traps.c
> +++ b/arch/x86/kernel/traps.c
> @@ -292,12 +292,30 @@ DO_ERROR(X86_TRAP_NP, SIGBUS, "segment not present", segment_not_present)
> DO_ERROR(X86_TRAP_SS, SIGBUS, "stack segment", stack_segment)
> DO_ERROR(X86_TRAP_AC, SIGBUS, "alignment check", alignment_check)
>
> +#ifdef CONFIG_VMAP_STACK
> +static void __noreturn handle_stack_overflow(const char *message,
> + struct pt_regs *regs,
> + unsigned long fault_address)
> +{
> + printk(KERN_EMERG "BUG: stack guard page was hit at %p (stack is %p..%p)\n",
> + (void *)fault_address, current->stack,
> + (char *)current->stack + THREAD_SIZE - 1);
> + die(message, regs, 0);
> +
> + /* Be absolutely certain we don't return. */
> + panic(message);
> +}
> +#endif
> +
> #ifdef CONFIG_X86_64
> /* Runs on IST stack */
> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
> {
> static const char str[] = "double fault";
> struct task_struct *tsk = current;
> +#ifdef CONFIG_VMAP_STACK
> + unsigned long cr2;
> +#endif
>
> #ifdef CONFIG_X86_ESPFIX64
> extern unsigned char native_irq_return_iret[];
> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
> tsk->thread.error_code = error_code;
> tsk->thread.trap_nr = X86_TRAP_DF;
>
> +#ifdef CONFIG_VMAP_STACK
> + /*
> + * If we overflow the stack into a guard page, the CPU will fail
> + * to deliver #PF and will send #DF instead. CR2 will contain
> + * the linear address of the second fault, which will be in the
> + * guard page below the bottom of the stack.
> + */
> + cr2 = read_cr2();
> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
> + handle_stack_overflow(
> + "kernel stack overflow (double-fault)",
> + regs, cr2);
> +#endif
Is there any other way to tell if this was from a page fault? If it
wasn't a page fault then CR2 is undefined.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 17:20 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOGul-20Y-9@gated-at.bofh.it> |
| In reply to | #1432108 |
On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>> This allows x86_64 kernels to enable vmapped stacks. There are a
>> couple of interesting bits.
>>
>> First, x86 lazily faults in top-level paging entries for the vmalloc
>> area. This won't work if we get a page fault while trying to access
>> the stack: the CPU will promote it to a double-fault and we'll die.
>> To avoid this problem, probe the new stack when switching stacks and
>> forcibly populate the pgd entry for the stack when switching mms.
>>
>> Second, once we have guard pages around the stack, we'll want to
>> detect and handle stack overflow.
>>
>> I didn't enable it on x86_32. We'd need to rework the double-fault
>> code a bit and I'm concerned about running out of vmalloc virtual
>> addresses under some workloads.
>>
>> This patch, by itself, will behave somewhat erratically when the
>> stack overflows while RSP is still more than a few tens of bytes
>> above the bottom of the stack. Specifically, we'll get #PF and make
>> it to no_context and an oops without triggering a double-fault, and
>> no_context doesn't know about stack overflows. The next patch will
>> improve that case.
>>
>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>> ---
>> arch/x86/Kconfig | 1 +
>> arch/x86/include/asm/switch_to.h | 28 +++++++++++++++++++++++++++-
>> arch/x86/kernel/traps.c | 32 ++++++++++++++++++++++++++++++++
>> arch/x86/mm/tlb.c | 15 +++++++++++++++
>> 4 files changed, 75 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>> index d9a94da0c29f..afdcf96ef109 100644
>> --- a/arch/x86/Kconfig
>> +++ b/arch/x86/Kconfig
>> @@ -92,6 +92,7 @@ config X86
>> select HAVE_ARCH_TRACEHOOK
>> select HAVE_ARCH_TRANSPARENT_HUGEPAGE
>> select HAVE_EBPF_JIT if X86_64
>> + select HAVE_ARCH_VMAP_STACK if X86_64
>> select HAVE_CC_STACKPROTECTOR
>> select HAVE_CMPXCHG_DOUBLE
>> select HAVE_CMPXCHG_LOCAL
>> diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h
>> index 8f321a1b03a1..14e4b20f0aaf 100644
>> --- a/arch/x86/include/asm/switch_to.h
>> +++ b/arch/x86/include/asm/switch_to.h
>> @@ -8,6 +8,28 @@ struct tss_struct;
>> void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p,
>> struct tss_struct *tss);
>>
>> +/* This runs runs on the previous thread's stack. */
>> +static inline void prepare_switch_to(struct task_struct *prev,
>> + struct task_struct *next)
>> +{
>> +#ifdef CONFIG_VMAP_STACK
>> + /*
>> + * If we switch to a stack that has a top-level paging entry
>> + * that is not present in the current mm, the resulting #PF will
>> + * will be promoted to a double-fault and we'll panic. Probe
>> + * the new stack now so that vmalloc_fault can fix up the page
>> + * tables if needed. This can only happen if we use a stack
>> + * in vmap space.
>> + *
>> + * We assume that the stack is aligned so that it never spans
>> + * more than one top-level paging entry.
>> + *
>> + * To minimize cache pollution, just follow the stack pointer.
>> + */
>> + READ_ONCE(*(unsigned char *)next->thread.sp);
>> +#endif
>> +}
>> +
>> #ifdef CONFIG_X86_32
>>
>> #ifdef CONFIG_CC_STACKPROTECTOR
>> @@ -39,6 +61,8 @@ do { \
>> */ \
>> unsigned long ebx, ecx, edx, esi, edi; \
>> \
>> + prepare_switch_to(prev, next); \
>> + \
>> asm volatile("pushl %%ebp\n\t" /* save EBP */ \
>> "movl %%esp,%[prev_sp]\n\t" /* save ESP */ \
>> "movl %[next_sp],%%esp\n\t" /* restore ESP */ \
>> @@ -103,7 +127,9 @@ do { \
>> * clean in kernel mode, with the possible exception of IOPL. Kernel IOPL
>> * has no effect.
>> */
>> -#define switch_to(prev, next, last) \
>> +#define switch_to(prev, next, last) \
>> + prepare_switch_to(prev, next); \
>> + \
>> asm volatile(SAVE_CONTEXT \
>> "movq %%rsp,%P[threadrsp](%[prev])\n\t" /* save RSP */ \
>> "movq %P[threadrsp](%[next]),%%rsp\n\t" /* restore RSP */ \
>> diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
>> index 00f03d82e69a..9cb7ea781176 100644
>> --- a/arch/x86/kernel/traps.c
>> +++ b/arch/x86/kernel/traps.c
>> @@ -292,12 +292,30 @@ DO_ERROR(X86_TRAP_NP, SIGBUS, "segment not present", segment_not_present)
>> DO_ERROR(X86_TRAP_SS, SIGBUS, "stack segment", stack_segment)
>> DO_ERROR(X86_TRAP_AC, SIGBUS, "alignment check", alignment_check)
>>
>> +#ifdef CONFIG_VMAP_STACK
>> +static void __noreturn handle_stack_overflow(const char *message,
>> + struct pt_regs *regs,
>> + unsigned long fault_address)
>> +{
>> + printk(KERN_EMERG "BUG: stack guard page was hit at %p (stack is %p..%p)\n",
>> + (void *)fault_address, current->stack,
>> + (char *)current->stack + THREAD_SIZE - 1);
>> + die(message, regs, 0);
>> +
>> + /* Be absolutely certain we don't return. */
>> + panic(message);
>> +}
>> +#endif
>> +
>> #ifdef CONFIG_X86_64
>> /* Runs on IST stack */
>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>> {
>> static const char str[] = "double fault";
>> struct task_struct *tsk = current;
>> +#ifdef CONFIG_VMAP_STACK
>> + unsigned long cr2;
>> +#endif
>>
>> #ifdef CONFIG_X86_ESPFIX64
>> extern unsigned char native_irq_return_iret[];
>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>> tsk->thread.error_code = error_code;
>> tsk->thread.trap_nr = X86_TRAP_DF;
>>
>> +#ifdef CONFIG_VMAP_STACK
>> + /*
>> + * If we overflow the stack into a guard page, the CPU will fail
>> + * to deliver #PF and will send #DF instead. CR2 will contain
>> + * the linear address of the second fault, which will be in the
>> + * guard page below the bottom of the stack.
>> + */
>> + cr2 = read_cr2();
>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>> + handle_stack_overflow(
>> + "kernel stack overflow (double-fault)",
>> + regs, cr2);
>> +#endif
>
> Is there any other way to tell if this was from a page fault? If it
> wasn't a page fault then CR2 is undefined.
I guess it doesn't really matter, since the fault is fatal either way.
The error message might be incorrect though.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2016-06-27 17:30 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOGE1-24t-15@gated-at.bofh.it> |
| In reply to | #1432117 |
On Mon, Jun 27, 2016 at 8:12 AM, Brian Gerst <brgerst@gmail.com> wrote:
> On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>> This allows x86_64 kernels to enable vmapped stacks. There are a
>>> couple of interesting bits.
>>>
>>> First, x86 lazily faults in top-level paging entries for the vmalloc
>>> area. This won't work if we get a page fault while trying to access
>>> the stack: the CPU will promote it to a double-fault and we'll die.
>>> To avoid this problem, probe the new stack when switching stacks and
>>> forcibly populate the pgd entry for the stack when switching mms.
>>>
>>> Second, once we have guard pages around the stack, we'll want to
>>> detect and handle stack overflow.
>>>
>>> I didn't enable it on x86_32. We'd need to rework the double-fault
>>> code a bit and I'm concerned about running out of vmalloc virtual
>>> addresses under some workloads.
>>>
>>> This patch, by itself, will behave somewhat erratically when the
>>> stack overflows while RSP is still more than a few tens of bytes
>>> above the bottom of the stack. Specifically, we'll get #PF and make
>>> it to no_context and an oops without triggering a double-fault, and
>>> no_context doesn't know about stack overflows. The next patch will
>>> improve that case.
>>>
>>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>>> ---
>>> arch/x86/Kconfig | 1 +
>>> arch/x86/include/asm/switch_to.h | 28 +++++++++++++++++++++++++++-
>>> arch/x86/kernel/traps.c | 32 ++++++++++++++++++++++++++++++++
>>> arch/x86/mm/tlb.c | 15 +++++++++++++++
>>> 4 files changed, 75 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>>> index d9a94da0c29f..afdcf96ef109 100644
>>> --- a/arch/x86/Kconfig
>>> +++ b/arch/x86/Kconfig
>>> @@ -92,6 +92,7 @@ config X86
>>> select HAVE_ARCH_TRACEHOOK
>>> select HAVE_ARCH_TRANSPARENT_HUGEPAGE
>>> select HAVE_EBPF_JIT if X86_64
>>> + select HAVE_ARCH_VMAP_STACK if X86_64
>>> select HAVE_CC_STACKPROTECTOR
>>> select HAVE_CMPXCHG_DOUBLE
>>> select HAVE_CMPXCHG_LOCAL
>>> diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h
>>> index 8f321a1b03a1..14e4b20f0aaf 100644
>>> --- a/arch/x86/include/asm/switch_to.h
>>> +++ b/arch/x86/include/asm/switch_to.h
>>> @@ -8,6 +8,28 @@ struct tss_struct;
>>> void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p,
>>> struct tss_struct *tss);
>>>
>>> +/* This runs runs on the previous thread's stack. */
>>> +static inline void prepare_switch_to(struct task_struct *prev,
>>> + struct task_struct *next)
>>> +{
>>> +#ifdef CONFIG_VMAP_STACK
>>> + /*
>>> + * If we switch to a stack that has a top-level paging entry
>>> + * that is not present in the current mm, the resulting #PF will
>>> + * will be promoted to a double-fault and we'll panic. Probe
>>> + * the new stack now so that vmalloc_fault can fix up the page
>>> + * tables if needed. This can only happen if we use a stack
>>> + * in vmap space.
>>> + *
>>> + * We assume that the stack is aligned so that it never spans
>>> + * more than one top-level paging entry.
>>> + *
>>> + * To minimize cache pollution, just follow the stack pointer.
>>> + */
>>> + READ_ONCE(*(unsigned char *)next->thread.sp);
>>> +#endif
>>> +}
>>> +
>>> #ifdef CONFIG_X86_32
>>>
>>> #ifdef CONFIG_CC_STACKPROTECTOR
>>> @@ -39,6 +61,8 @@ do { \
>>> */ \
>>> unsigned long ebx, ecx, edx, esi, edi; \
>>> \
>>> + prepare_switch_to(prev, next); \
>>> + \
>>> asm volatile("pushl %%ebp\n\t" /* save EBP */ \
>>> "movl %%esp,%[prev_sp]\n\t" /* save ESP */ \
>>> "movl %[next_sp],%%esp\n\t" /* restore ESP */ \
>>> @@ -103,7 +127,9 @@ do { \
>>> * clean in kernel mode, with the possible exception of IOPL. Kernel IOPL
>>> * has no effect.
>>> */
>>> -#define switch_to(prev, next, last) \
>>> +#define switch_to(prev, next, last) \
>>> + prepare_switch_to(prev, next); \
>>> + \
>>> asm volatile(SAVE_CONTEXT \
>>> "movq %%rsp,%P[threadrsp](%[prev])\n\t" /* save RSP */ \
>>> "movq %P[threadrsp](%[next]),%%rsp\n\t" /* restore RSP */ \
>>> diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
>>> index 00f03d82e69a..9cb7ea781176 100644
>>> --- a/arch/x86/kernel/traps.c
>>> +++ b/arch/x86/kernel/traps.c
>>> @@ -292,12 +292,30 @@ DO_ERROR(X86_TRAP_NP, SIGBUS, "segment not present", segment_not_present)
>>> DO_ERROR(X86_TRAP_SS, SIGBUS, "stack segment", stack_segment)
>>> DO_ERROR(X86_TRAP_AC, SIGBUS, "alignment check", alignment_check)
>>>
>>> +#ifdef CONFIG_VMAP_STACK
>>> +static void __noreturn handle_stack_overflow(const char *message,
>>> + struct pt_regs *regs,
>>> + unsigned long fault_address)
>>> +{
>>> + printk(KERN_EMERG "BUG: stack guard page was hit at %p (stack is %p..%p)\n",
>>> + (void *)fault_address, current->stack,
>>> + (char *)current->stack + THREAD_SIZE - 1);
>>> + die(message, regs, 0);
>>> +
>>> + /* Be absolutely certain we don't return. */
>>> + panic(message);
>>> +}
>>> +#endif
>>> +
>>> #ifdef CONFIG_X86_64
>>> /* Runs on IST stack */
>>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>> {
>>> static const char str[] = "double fault";
>>> struct task_struct *tsk = current;
>>> +#ifdef CONFIG_VMAP_STACK
>>> + unsigned long cr2;
>>> +#endif
>>>
>>> #ifdef CONFIG_X86_ESPFIX64
>>> extern unsigned char native_irq_return_iret[];
>>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>> tsk->thread.error_code = error_code;
>>> tsk->thread.trap_nr = X86_TRAP_DF;
>>>
>>> +#ifdef CONFIG_VMAP_STACK
>>> + /*
>>> + * If we overflow the stack into a guard page, the CPU will fail
>>> + * to deliver #PF and will send #DF instead. CR2 will contain
>>> + * the linear address of the second fault, which will be in the
>>> + * guard page below the bottom of the stack.
>>> + */
>>> + cr2 = read_cr2();
>>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>>> + handle_stack_overflow(
>>> + "kernel stack overflow (double-fault)",
>>> + regs, cr2);
>>> +#endif
>>
>> Is there any other way to tell if this was from a page fault? If it
>> wasn't a page fault then CR2 is undefined.
>
> I guess it doesn't really matter, since the fault is fatal either way.
> The error message might be incorrect though.
>
It's at least worth a comment, though. Maybe I should check if
regs->rsp is within 40 bytes of the bottom of the stack, too, such
that delivery of an inner fault would have double-faulted assuming the
inner fault didn't use an IST vector.
--Andy
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2016-06-27 18:00 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOH73-2eF-1@gated-at.bofh.it> |
| In reply to | #1432126 |
On Mon, Jun 27, 2016 at 8:22 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Mon, Jun 27, 2016 at 8:12 AM, Brian Gerst <brgerst@gmail.com> wrote:
>> On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>>> #ifdef CONFIG_X86_64
>>>> /* Runs on IST stack */
>>>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>> {
>>>> static const char str[] = "double fault";
>>>> struct task_struct *tsk = current;
>>>> +#ifdef CONFIG_VMAP_STACK
>>>> + unsigned long cr2;
>>>> +#endif
>>>>
>>>> #ifdef CONFIG_X86_ESPFIX64
>>>> extern unsigned char native_irq_return_iret[];
>>>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>> tsk->thread.error_code = error_code;
>>>> tsk->thread.trap_nr = X86_TRAP_DF;
>>>>
>>>> +#ifdef CONFIG_VMAP_STACK
>>>> + /*
>>>> + * If we overflow the stack into a guard page, the CPU will fail
>>>> + * to deliver #PF and will send #DF instead. CR2 will contain
>>>> + * the linear address of the second fault, which will be in the
>>>> + * guard page below the bottom of the stack.
>>>> + */
>>>> + cr2 = read_cr2();
>>>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>>>> + handle_stack_overflow(
>>>> + "kernel stack overflow (double-fault)",
>>>> + regs, cr2);
>>>> +#endif
>>>
>>> Is there any other way to tell if this was from a page fault? If it
>>> wasn't a page fault then CR2 is undefined.
>>
>> I guess it doesn't really matter, since the fault is fatal either way.
>> The error message might be incorrect though.
>>
>
> It's at least worth a comment, though. Maybe I should check if
> regs->rsp is within 40 bytes of the bottom of the stack, too, such
> that delivery of an inner fault would have double-faulted assuming the
> inner fault didn't use an IST vector.
>
How about:
/*
* If we overflow the stack into a guard page, the CPU will fail
* to deliver #PF and will send #DF instead. CR2 will contain
* the linear address of the second fault, which will be in the
* guard page below the bottom of the stack.
*
* We're limited to using heuristics here, since the CPU does
* not tell us what type of fault failed and, if the first fault
* wasn't a page fault, CR2 may contain stale garbage. To mostly
* rule out garbage, we check if the saved RSP is close enough to
* the bottom of the stack to cause exception delivery to fail.
* The worst case is 7 stack slots: one for alignment, five for
* SS..RIP, and one for the error code.
*/
tsk_stack = (unsigned long)task_stack_page(tsk);
if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
/* A double-fault due to #PF delivery failure is plausible. */
cr2 = read_cr2();
if (tsk_stack - 1 - cr2 < PAGE_SIZE)
handle_stack_overflow(
"kernel stack overflow (double-fault)",
regs, cr2);
}
--Andy
--
Andy Lutomirski
AMA Capital Management, LLC
[toc] | [prev] | [next] | [standalone]
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 18:20 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOHqp-2AK-11@gated-at.bofh.it> |
| In reply to | #1432134 |
On Mon, Jun 27, 2016 at 11:54 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Mon, Jun 27, 2016 at 8:22 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Mon, Jun 27, 2016 at 8:12 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>> On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>>>> #ifdef CONFIG_X86_64
>>>>> /* Runs on IST stack */
>>>>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>> {
>>>>> static const char str[] = "double fault";
>>>>> struct task_struct *tsk = current;
>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>> + unsigned long cr2;
>>>>> +#endif
>>>>>
>>>>> #ifdef CONFIG_X86_ESPFIX64
>>>>> extern unsigned char native_irq_return_iret[];
>>>>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>> tsk->thread.error_code = error_code;
>>>>> tsk->thread.trap_nr = X86_TRAP_DF;
>>>>>
>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>> + /*
>>>>> + * If we overflow the stack into a guard page, the CPU will fail
>>>>> + * to deliver #PF and will send #DF instead. CR2 will contain
>>>>> + * the linear address of the second fault, which will be in the
>>>>> + * guard page below the bottom of the stack.
>>>>> + */
>>>>> + cr2 = read_cr2();
>>>>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>>>>> + handle_stack_overflow(
>>>>> + "kernel stack overflow (double-fault)",
>>>>> + regs, cr2);
>>>>> +#endif
>>>>
>>>> Is there any other way to tell if this was from a page fault? If it
>>>> wasn't a page fault then CR2 is undefined.
>>>
>>> I guess it doesn't really matter, since the fault is fatal either way.
>>> The error message might be incorrect though.
>>>
>>
>> It's at least worth a comment, though. Maybe I should check if
>> regs->rsp is within 40 bytes of the bottom of the stack, too, such
>> that delivery of an inner fault would have double-faulted assuming the
>> inner fault didn't use an IST vector.
>>
>
> How about:
>
> /*
> * If we overflow the stack into a guard page, the CPU will fail
> * to deliver #PF and will send #DF instead. CR2 will contain
> * the linear address of the second fault, which will be in the
> * guard page below the bottom of the stack.
> *
> * We're limited to using heuristics here, since the CPU does
> * not tell us what type of fault failed and, if the first fault
> * wasn't a page fault, CR2 may contain stale garbage. To mostly
> * rule out garbage, we check if the saved RSP is close enough to
> * the bottom of the stack to cause exception delivery to fail.
> * The worst case is 7 stack slots: one for alignment, five for
> * SS..RIP, and one for the error code.
> */
> tsk_stack = (unsigned long)task_stack_page(tsk);
> if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
> /* A double-fault due to #PF delivery failure is plausible. */
> cr2 = read_cr2();
> if (tsk_stack - 1 - cr2 < PAGE_SIZE)
> handle_stack_overflow(
> "kernel stack overflow (double-fault)",
> regs, cr2);
> }
I think RSP anywhere in the guard page would be best, since it could
have been decremented by a function prologue into the guard page
before an access that triggers the page fault.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2016-06-27 18:40 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOHJM-2Ik-17@gated-at.bofh.it> |
| In reply to | #1432144 |
On Mon, Jun 27, 2016 at 9:17 AM, Brian Gerst <brgerst@gmail.com> wrote:
> On Mon, Jun 27, 2016 at 11:54 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Mon, Jun 27, 2016 at 8:22 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>> On Mon, Jun 27, 2016 at 8:12 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>> On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>>>>> #ifdef CONFIG_X86_64
>>>>>> /* Runs on IST stack */
>>>>>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>>> {
>>>>>> static const char str[] = "double fault";
>>>>>> struct task_struct *tsk = current;
>>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>>> + unsigned long cr2;
>>>>>> +#endif
>>>>>>
>>>>>> #ifdef CONFIG_X86_ESPFIX64
>>>>>> extern unsigned char native_irq_return_iret[];
>>>>>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>>> tsk->thread.error_code = error_code;
>>>>>> tsk->thread.trap_nr = X86_TRAP_DF;
>>>>>>
>>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>>> + /*
>>>>>> + * If we overflow the stack into a guard page, the CPU will fail
>>>>>> + * to deliver #PF and will send #DF instead. CR2 will contain
>>>>>> + * the linear address of the second fault, which will be in the
>>>>>> + * guard page below the bottom of the stack.
>>>>>> + */
>>>>>> + cr2 = read_cr2();
>>>>>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>>>>>> + handle_stack_overflow(
>>>>>> + "kernel stack overflow (double-fault)",
>>>>>> + regs, cr2);
>>>>>> +#endif
>>>>>
>>>>> Is there any other way to tell if this was from a page fault? If it
>>>>> wasn't a page fault then CR2 is undefined.
>>>>
>>>> I guess it doesn't really matter, since the fault is fatal either way.
>>>> The error message might be incorrect though.
>>>>
>>>
>>> It's at least worth a comment, though. Maybe I should check if
>>> regs->rsp is within 40 bytes of the bottom of the stack, too, such
>>> that delivery of an inner fault would have double-faulted assuming the
>>> inner fault didn't use an IST vector.
>>>
>>
>> How about:
>>
>> /*
>> * If we overflow the stack into a guard page, the CPU will fail
>> * to deliver #PF and will send #DF instead. CR2 will contain
>> * the linear address of the second fault, which will be in the
>> * guard page below the bottom of the stack.
>> *
>> * We're limited to using heuristics here, since the CPU does
>> * not tell us what type of fault failed and, if the first fault
>> * wasn't a page fault, CR2 may contain stale garbage. To mostly
>> * rule out garbage, we check if the saved RSP is close enough to
>> * the bottom of the stack to cause exception delivery to fail.
>> * The worst case is 7 stack slots: one for alignment, five for
>> * SS..RIP, and one for the error code.
>> */
>> tsk_stack = (unsigned long)task_stack_page(tsk);
>> if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
>> /* A double-fault due to #PF delivery failure is plausible. */
>> cr2 = read_cr2();
>> if (tsk_stack - 1 - cr2 < PAGE_SIZE)
>> handle_stack_overflow(
>> "kernel stack overflow (double-fault)",
>> regs, cr2);
>> }
>
> I think RSP anywhere in the guard page would be best, since it could
> have been decremented by a function prologue into the guard page
> before an access that triggers the page fault.
>
I think that can miss some stack overflows. Suppose that RSP points
very close to the bottom of the stack and we take an unrelated fault.
The CPU can fail to deliver that fault and we get a double fault
instead. But I counted wrong, too. Do you like this version and its
explanation?
/*
* If we overflow the stack into a guard page, the CPU will fail
* to deliver #PF and will send #DF instead. Similarly, if we
* take any non-IST exception while too close to the bottom of
* the stack, the processor will get a page fault while
* delivering the exception and will generate a double fault.
*
* According to the SDM (footnote in 6.15 under "Interrupt 14 -
* Page-Fault Exception (#PF):
*
* Processors update CR2 whenever a page fault is detected. If a
* second page fault occurs while an earlier page fault is being
* deliv- ered, the faulting linear address of the second fault will
* overwrite the contents of CR2 (replacing the previous
* address). These updates to CR2 occur even if the page fault
* results in a double fault or occurs during the delivery of a
* double fault.
*
* However, if we got here due to a non-page-fault exception while
* delivering a non-page-fault exception, CR2 may contain a
* stale value.
*
* As a heuristic: we consider this double fault to be a stack
* overflow if CR2 points to the guard page and RSP is either
* in the guard page or close enough to the bottom of the stack
*
* We're limited to using heuristics here, since the CPU does
* not tell us what type of fault failed and, if the first fault
* wasn't a page fault, CR2 may contain stale garbage. To
* mostly rule out garbage, we check if the saved RSP is close
* enough to the bottom of the stack to cause exception delivery
* to fail. If RSP == tsk_stack + 48 and we take an exception,
* the stack is already aligned and there will be enough room
* SS, RSP, RFLAGS, CS, RIP, and a possible error code. With
* any less space left, exception delivery could fail.
*/
tsk_stack = (unsigned long)task_stack_page(tsk);
if (regs->rsp < tsk_stack + 48 && regs->rsp > tsk_stack - PAGE_SIZE) {
/* A double-fault due to #PF delivery failure is plausible. */
cr2 = read_cr2();
if (tsk_stack - 1 - cr2 < PAGE_SIZE)
handle_stack_overflow(
"kernel stack overflow (double-fault)",
regs, cr2);
}
[toc] | [prev] | [next] | [standalone]
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 19:10 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOIcN-39i-15@gated-at.bofh.it> |
| In reply to | #1432152 |
On Mon, Jun 27, 2016 at 12:35 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Mon, Jun 27, 2016 at 9:17 AM, Brian Gerst <brgerst@gmail.com> wrote:
>> On Mon, Jun 27, 2016 at 11:54 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>> On Mon, Jun 27, 2016 at 8:22 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>>> On Mon, Jun 27, 2016 at 8:12 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>>> On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>>>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>>>>>> #ifdef CONFIG_X86_64
>>>>>>> /* Runs on IST stack */
>>>>>>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>>>> {
>>>>>>> static const char str[] = "double fault";
>>>>>>> struct task_struct *tsk = current;
>>>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>>>> + unsigned long cr2;
>>>>>>> +#endif
>>>>>>>
>>>>>>> #ifdef CONFIG_X86_ESPFIX64
>>>>>>> extern unsigned char native_irq_return_iret[];
>>>>>>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>>>> tsk->thread.error_code = error_code;
>>>>>>> tsk->thread.trap_nr = X86_TRAP_DF;
>>>>>>>
>>>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>>>> + /*
>>>>>>> + * If we overflow the stack into a guard page, the CPU will fail
>>>>>>> + * to deliver #PF and will send #DF instead. CR2 will contain
>>>>>>> + * the linear address of the second fault, which will be in the
>>>>>>> + * guard page below the bottom of the stack.
>>>>>>> + */
>>>>>>> + cr2 = read_cr2();
>>>>>>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>>>>>>> + handle_stack_overflow(
>>>>>>> + "kernel stack overflow (double-fault)",
>>>>>>> + regs, cr2);
>>>>>>> +#endif
>>>>>>
>>>>>> Is there any other way to tell if this was from a page fault? If it
>>>>>> wasn't a page fault then CR2 is undefined.
>>>>>
>>>>> I guess it doesn't really matter, since the fault is fatal either way.
>>>>> The error message might be incorrect though.
>>>>>
>>>>
>>>> It's at least worth a comment, though. Maybe I should check if
>>>> regs->rsp is within 40 bytes of the bottom of the stack, too, such
>>>> that delivery of an inner fault would have double-faulted assuming the
>>>> inner fault didn't use an IST vector.
>>>>
>>>
>>> How about:
>>>
>>> /*
>>> * If we overflow the stack into a guard page, the CPU will fail
>>> * to deliver #PF and will send #DF instead. CR2 will contain
>>> * the linear address of the second fault, which will be in the
>>> * guard page below the bottom of the stack.
>>> *
>>> * We're limited to using heuristics here, since the CPU does
>>> * not tell us what type of fault failed and, if the first fault
>>> * wasn't a page fault, CR2 may contain stale garbage. To mostly
>>> * rule out garbage, we check if the saved RSP is close enough to
>>> * the bottom of the stack to cause exception delivery to fail.
>>> * The worst case is 7 stack slots: one for alignment, five for
>>> * SS..RIP, and one for the error code.
>>> */
>>> tsk_stack = (unsigned long)task_stack_page(tsk);
>>> if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
>>> /* A double-fault due to #PF delivery failure is plausible. */
>>> cr2 = read_cr2();
>>> if (tsk_stack - 1 - cr2 < PAGE_SIZE)
>>> handle_stack_overflow(
>>> "kernel stack overflow (double-fault)",
>>> regs, cr2);
>>> }
>>
>> I think RSP anywhere in the guard page would be best, since it could
>> have been decremented by a function prologue into the guard page
>> before an access that triggers the page fault.
>>
>
> I think that can miss some stack overflows. Suppose that RSP points
> very close to the bottom of the stack and we take an unrelated fault.
> The CPU can fail to deliver that fault and we get a double fault
> instead. But I counted wrong, too. Do you like this version and its
> explanation?
>
> /*
> * If we overflow the stack into a guard page, the CPU will fail
> * to deliver #PF and will send #DF instead. Similarly, if we
> * take any non-IST exception while too close to the bottom of
> * the stack, the processor will get a page fault while
> * delivering the exception and will generate a double fault.
> *
> * According to the SDM (footnote in 6.15 under "Interrupt 14 -
> * Page-Fault Exception (#PF):
> *
> * Processors update CR2 whenever a page fault is detected. If a
> * second page fault occurs while an earlier page fault is being
> * deliv- ered, the faulting linear address of the second fault will
> * overwrite the contents of CR2 (replacing the previous
> * address). These updates to CR2 occur even if the page fault
> * results in a double fault or occurs during the delivery of a
> * double fault.
> *
> * However, if we got here due to a non-page-fault exception while
> * delivering a non-page-fault exception, CR2 may contain a
> * stale value.
> *
> * As a heuristic: we consider this double fault to be a stack
> * overflow if CR2 points to the guard page and RSP is either
> * in the guard page or close enough to the bottom of the stack
> *
> * We're limited to using heuristics here, since the CPU does
> * not tell us what type of fault failed and, if the first fault
> * wasn't a page fault, CR2 may contain stale garbage. To
> * mostly rule out garbage, we check if the saved RSP is close
> * enough to the bottom of the stack to cause exception delivery
> * to fail. If RSP == tsk_stack + 48 and we take an exception,
> * the stack is already aligned and there will be enough room
> * SS, RSP, RFLAGS, CS, RIP, and a possible error code. With
> * any less space left, exception delivery could fail.
> */
> tsk_stack = (unsigned long)task_stack_page(tsk);
> if (regs->rsp < tsk_stack + 48 && regs->rsp > tsk_stack - PAGE_SIZE) {
> /* A double-fault due to #PF delivery failure is plausible. */
> cr2 = read_cr2();
> if (tsk_stack - 1 - cr2 < PAGE_SIZE)
> handle_stack_overflow(
> "kernel stack overflow (double-fault)",
> regs, cr2);
> }
Actually I think your quote from the SDM contradicts this. The second
#PF (when trying to invoke the page fault handler) would update CR2
with an address in the guard page.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Brian Gerst <brgerst@gmail.com> |
|---|---|
| Date | 2016-06-27 19:30 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOIw9-3gD-7@gated-at.bofh.it> |
| In reply to | #1432168 |
On Mon, Jun 27, 2016 at 1:09 PM, Brian Gerst <brgerst@gmail.com> wrote:
> On Mon, Jun 27, 2016 at 12:35 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Mon, Jun 27, 2016 at 9:17 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>> On Mon, Jun 27, 2016 at 11:54 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>>> On Mon, Jun 27, 2016 at 8:22 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>>>> On Mon, Jun 27, 2016 at 8:12 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>>>> On Mon, Jun 27, 2016 at 11:01 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>>>>> On Sun, Jun 26, 2016 at 5:55 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>>>>>>> #ifdef CONFIG_X86_64
>>>>>>>> /* Runs on IST stack */
>>>>>>>> dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>>>>> {
>>>>>>>> static const char str[] = "double fault";
>>>>>>>> struct task_struct *tsk = current;
>>>>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>>>>> + unsigned long cr2;
>>>>>>>> +#endif
>>>>>>>>
>>>>>>>> #ifdef CONFIG_X86_ESPFIX64
>>>>>>>> extern unsigned char native_irq_return_iret[];
>>>>>>>> @@ -332,6 +350,20 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
>>>>>>>> tsk->thread.error_code = error_code;
>>>>>>>> tsk->thread.trap_nr = X86_TRAP_DF;
>>>>>>>>
>>>>>>>> +#ifdef CONFIG_VMAP_STACK
>>>>>>>> + /*
>>>>>>>> + * If we overflow the stack into a guard page, the CPU will fail
>>>>>>>> + * to deliver #PF and will send #DF instead. CR2 will contain
>>>>>>>> + * the linear address of the second fault, which will be in the
>>>>>>>> + * guard page below the bottom of the stack.
>>>>>>>> + */
>>>>>>>> + cr2 = read_cr2();
>>>>>>>> + if ((unsigned long)tsk->stack - 1 - cr2 < PAGE_SIZE)
>>>>>>>> + handle_stack_overflow(
>>>>>>>> + "kernel stack overflow (double-fault)",
>>>>>>>> + regs, cr2);
>>>>>>>> +#endif
>>>>>>>
>>>>>>> Is there any other way to tell if this was from a page fault? If it
>>>>>>> wasn't a page fault then CR2 is undefined.
>>>>>>
>>>>>> I guess it doesn't really matter, since the fault is fatal either way.
>>>>>> The error message might be incorrect though.
>>>>>>
>>>>>
>>>>> It's at least worth a comment, though. Maybe I should check if
>>>>> regs->rsp is within 40 bytes of the bottom of the stack, too, such
>>>>> that delivery of an inner fault would have double-faulted assuming the
>>>>> inner fault didn't use an IST vector.
>>>>>
>>>>
>>>> How about:
>>>>
>>>> /*
>>>> * If we overflow the stack into a guard page, the CPU will fail
>>>> * to deliver #PF and will send #DF instead. CR2 will contain
>>>> * the linear address of the second fault, which will be in the
>>>> * guard page below the bottom of the stack.
>>>> *
>>>> * We're limited to using heuristics here, since the CPU does
>>>> * not tell us what type of fault failed and, if the first fault
>>>> * wasn't a page fault, CR2 may contain stale garbage. To mostly
>>>> * rule out garbage, we check if the saved RSP is close enough to
>>>> * the bottom of the stack to cause exception delivery to fail.
>>>> * The worst case is 7 stack slots: one for alignment, five for
>>>> * SS..RIP, and one for the error code.
>>>> */
>>>> tsk_stack = (unsigned long)task_stack_page(tsk);
>>>> if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
>>>> /* A double-fault due to #PF delivery failure is plausible. */
>>>> cr2 = read_cr2();
>>>> if (tsk_stack - 1 - cr2 < PAGE_SIZE)
>>>> handle_stack_overflow(
>>>> "kernel stack overflow (double-fault)",
>>>> regs, cr2);
>>>> }
>>>
>>> I think RSP anywhere in the guard page would be best, since it could
>>> have been decremented by a function prologue into the guard page
>>> before an access that triggers the page fault.
>>>
>>
>> I think that can miss some stack overflows. Suppose that RSP points
>> very close to the bottom of the stack and we take an unrelated fault.
>> The CPU can fail to deliver that fault and we get a double fault
>> instead. But I counted wrong, too. Do you like this version and its
>> explanation?
>>
>> /*
>> * If we overflow the stack into a guard page, the CPU will fail
>> * to deliver #PF and will send #DF instead. Similarly, if we
>> * take any non-IST exception while too close to the bottom of
>> * the stack, the processor will get a page fault while
>> * delivering the exception and will generate a double fault.
>> *
>> * According to the SDM (footnote in 6.15 under "Interrupt 14 -
>> * Page-Fault Exception (#PF):
>> *
>> * Processors update CR2 whenever a page fault is detected. If a
>> * second page fault occurs while an earlier page fault is being
>> * deliv- ered, the faulting linear address of the second fault will
>> * overwrite the contents of CR2 (replacing the previous
>> * address). These updates to CR2 occur even if the page fault
>> * results in a double fault or occurs during the delivery of a
>> * double fault.
>> *
>> * However, if we got here due to a non-page-fault exception while
>> * delivering a non-page-fault exception, CR2 may contain a
>> * stale value.
>> *
>> * As a heuristic: we consider this double fault to be a stack
>> * overflow if CR2 points to the guard page and RSP is either
>> * in the guard page or close enough to the bottom of the stack
>> *
>> * We're limited to using heuristics here, since the CPU does
>> * not tell us what type of fault failed and, if the first fault
>> * wasn't a page fault, CR2 may contain stale garbage. To
>> * mostly rule out garbage, we check if the saved RSP is close
>> * enough to the bottom of the stack to cause exception delivery
>> * to fail. If RSP == tsk_stack + 48 and we take an exception,
>> * the stack is already aligned and there will be enough room
>> * SS, RSP, RFLAGS, CS, RIP, and a possible error code. With
>> * any less space left, exception delivery could fail.
>> */
>> tsk_stack = (unsigned long)task_stack_page(tsk);
>> if (regs->rsp < tsk_stack + 48 && regs->rsp > tsk_stack - PAGE_SIZE) {
>> /* A double-fault due to #PF delivery failure is plausible. */
>> cr2 = read_cr2();
>> if (tsk_stack - 1 - cr2 < PAGE_SIZE)
>> handle_stack_overflow(
>> "kernel stack overflow (double-fault)",
>> regs, cr2);
>> }
>
> Actually I think your quote from the SDM contradicts this. The second
> #PF (when trying to invoke the page fault handler) would update CR2
> with an address in the guard page.
Nevermind, I'm totally misunderstanding this. I guess the real
question is when is RSP updated? During each push or once before or
after the frame is pushed. Checking for the bottom of the valid stack
covers the case when RSP isn't updated until after the whole frame is
pushed successfully.
--
Brian Gerst
[toc] | [prev] | [next] | [standalone]
| From | Linus Torvalds <torvalds@linux-foundation.org> |
|---|---|
| Date | 2016-06-27 19:30 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOIw9-3gD-13@gated-at.bofh.it> |
| In reply to | #1432134 |
On Mon, Jun 27, 2016 at 8:54 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>
> How about:
>
> tsk_stack = (unsigned long)task_stack_page(tsk);
> if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
I'm not at all convinced that regs->rsp will be all that reliable
under a double-fault scenario either. I'd be more inclined to trusr
cr2 than the register state.
It's true that double faults can happen for *other* reasons entirely,
and as such it's not clear that %cr2 is reliable either, but since
this is all just about a printout, I'd rather go that way anyway.
Linus
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2016-06-27 19:40 +0200 |
| Subject | Re: [PATCH v4 15/29] x86/mm/64: Enable vmapped stacks |
| Message-ID | <rOIFP-3km-3@gated-at.bofh.it> |
| In reply to | #1432186 |
On Mon, Jun 27, 2016 at 10:28 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Jun 27, 2016 at 8:54 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>
>> How about:
>>
>> tsk_stack = (unsigned long)task_stack_page(tsk);
>> if (regs->rsp <= tsk_stack + 7*8 && regs->rsp > tsk_stack - PAGE_SIZE) {
>
> I'm not at all convinced that regs->rsp will be all that reliable
> under a double-fault scenario either. I'd be more inclined to trusr
> cr2 than the register state.
>
> It's true that double faults can happen for *other* reasons entirely,
> and as such it's not clear that %cr2 is reliable either, but since
> this is all just about a printout, I'd rather go that way anyway.
Fair enough. The chance that we get #GP-in-#GP or similar while CR2
coincidentally points to the guard page is quite low. I'll add all
the details to the comment but I'll leave the code alone.
FWIW, the manual only says that CS and RIP are untrustworthy, not that
RSP is untrustworthy, but it doesn't specify *what* RSP would contain
anywhere I can find. I don't think this is important enough to start
harassing the Intel and AMD folks over.
--Andy
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2016-06-27 00:10 +0200 |
| Subject | [PATCH v4 18/29] x86: Move addr_limit to thread_struct |
| Message-ID | <rOqpA-5R-15@gated-at.bofh.it> |
| In reply to | #1431600 |
thread_info is a legacy mess. To prepare for its partial removal,
move addr_limit out.
As an added benefit, this way is simpler.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/include/asm/checksum_32.h | 3 +--
arch/x86/include/asm/processor.h | 17 ++++++++++-------
arch/x86/include/asm/thread_info.h | 7 -------
arch/x86/include/asm/uaccess.h | 6 +++---
arch/x86/kernel/asm-offsets.c | 4 +++-
arch/x86/lib/copy_user_64.S | 8 ++++----
arch/x86/lib/csum-wrappers_64.c | 1 +
arch/x86/lib/getuser.S | 20 ++++++++++----------
arch/x86/lib/putuser.S | 10 +++++-----
arch/x86/lib/usercopy_64.c | 2 +-
drivers/pnp/isapnp/proc.c | 2 +-
lib/bitmap.c | 2 +-
12 files changed, 40 insertions(+), 42 deletions(-)
diff --git a/arch/x86/include/asm/checksum_32.h b/arch/x86/include/asm/checksum_32.h
index 532f85e6651f..7b53743ed267 100644
--- a/arch/x86/include/asm/checksum_32.h
+++ b/arch/x86/include/asm/checksum_32.h
@@ -2,8 +2,7 @@
#define _ASM_X86_CHECKSUM_32_H
#include <linux/in6.h>
-
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
/*
* computes the checksum of a memory block at buff, length len,
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index f53ae57bd985..a2e20d6d01fe 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -371,6 +371,10 @@ extern unsigned int xstate_size;
struct perf_event;
+typedef struct {
+ unsigned long seg;
+} mm_segment_t;
+
struct thread_struct {
/* Cached TLS descriptors: */
struct desc_struct tls_array[GDT_ENTRY_TLS_ENTRIES];
@@ -419,6 +423,8 @@ struct thread_struct {
/* Max allowed port in the bitmap, in bytes: */
unsigned io_bitmap_max;
+ mm_segment_t addr_limit;
+
unsigned int sig_on_uaccess_error:1;
unsigned int uaccess_err:1; /* uaccess failed */
@@ -493,11 +499,6 @@ static inline void load_sp0(struct tss_struct *tss,
#define set_iopl_mask native_set_iopl_mask
#endif /* CONFIG_PARAVIRT */
-typedef struct {
- unsigned long seg;
-} mm_segment_t;
-
-
/* Free all resources held by a thread. */
extern void release_thread(struct task_struct *);
@@ -719,6 +720,7 @@ static inline void spin_lock_prefetch(const void *x)
.sp0 = TOP_OF_INIT_STACK, \
.sysenter_cs = __KERNEL_CS, \
.io_bitmap_ptr = NULL, \
+ .addr_limit = KERNEL_DS, \
}
extern unsigned long thread_saved_pc(struct task_struct *tsk);
@@ -768,8 +770,9 @@ extern unsigned long thread_saved_pc(struct task_struct *tsk);
#define STACK_TOP TASK_SIZE
#define STACK_TOP_MAX TASK_SIZE_MAX
-#define INIT_THREAD { \
- .sp0 = TOP_OF_INIT_STACK \
+#define INIT_THREAD { \
+ .sp0 = TOP_OF_INIT_STACK, \
+ .addr_limit = KERNEL_DS, \
}
/*
diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index 7c47bb659ecd..89bff044a6f5 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -57,7 +57,6 @@ struct thread_info {
__u32 flags; /* low level flags */
__u32 status; /* thread synchronous flags */
__u32 cpu; /* current CPU */
- mm_segment_t addr_limit;
};
#define INIT_THREAD_INFO(tsk) \
@@ -65,7 +64,6 @@ struct thread_info {
.task = &tsk, \
.flags = 0, \
.cpu = 0, \
- .addr_limit = KERNEL_DS, \
}
#define init_thread_info (init_thread_union.thread_info)
@@ -184,11 +182,6 @@ static inline unsigned long current_stack_pointer(void)
# define cpu_current_top_of_stack (cpu_tss + TSS_sp0)
#endif
-/* Load thread_info address into "reg" */
-#define GET_THREAD_INFO(reg) \
- _ASM_MOV PER_CPU_VAR(cpu_current_top_of_stack),reg ; \
- _ASM_SUB $(THREAD_SIZE),reg ;
-
/*
* ASM operand which evaluates to a 'thread_info' address of
* the current task, if it is known that "reg" is exactly "off"
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 4d2a726e8e6d..0eb18b4ac492 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -29,12 +29,12 @@
#define USER_DS MAKE_MM_SEG(TASK_SIZE_MAX)
#define get_ds() (KERNEL_DS)
-#define get_fs() (current_thread_info()->addr_limit)
-#define set_fs(x) (current_thread_info()->addr_limit = (x))
+#define get_fs() (current->thread.addr_limit)
+#define set_fs(x) (current->thread.addr_limit = (x))
#define segment_eq(a, b) ((a).seg == (b).seg)
-#define user_addr_max() (current_thread_info()->addr_limit.seg)
+#define user_addr_max() (current->thread.addr_limit.seg)
#define __addr_ok(addr) \
((unsigned long __force)(addr) < user_addr_max())
diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c
index 674134e9f5e5..2bd5c6ff7ee7 100644
--- a/arch/x86/kernel/asm-offsets.c
+++ b/arch/x86/kernel/asm-offsets.c
@@ -31,7 +31,9 @@ void common(void) {
BLANK();
OFFSET(TI_flags, thread_info, flags);
OFFSET(TI_status, thread_info, status);
- OFFSET(TI_addr_limit, thread_info, addr_limit);
+
+ BLANK();
+ OFFSET(TASK_addr_limit, task_struct, thread.addr_limit);
BLANK();
OFFSET(crypto_tfm_ctx_offset, crypto_tfm, __crt_ctx);
diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S
index 2b0ef26da0bd..bf603ebbfd8e 100644
--- a/arch/x86/lib/copy_user_64.S
+++ b/arch/x86/lib/copy_user_64.S
@@ -17,11 +17,11 @@
/* Standard copy_to_user with segment limit checking */
ENTRY(_copy_to_user)
- GET_THREAD_INFO(%rax)
+ mov PER_CPU_VAR(current_task), %rax
movq %rdi,%rcx
addq %rdx,%rcx
jc bad_to_user
- cmpq TI_addr_limit(%rax),%rcx
+ cmpq TASK_addr_limit(%rax),%rcx
ja bad_to_user
ALTERNATIVE_2 "jmp copy_user_generic_unrolled", \
"jmp copy_user_generic_string", \
@@ -32,11 +32,11 @@ ENDPROC(_copy_to_user)
/* Standard copy_from_user with segment limit checking */
ENTRY(_copy_from_user)
- GET_THREAD_INFO(%rax)
+ mov PER_CPU_VAR(current_task), %rax
movq %rsi,%rcx
addq %rdx,%rcx
jc bad_from_user
- cmpq TI_addr_limit(%rax),%rcx
+ cmpq TASK_addr_limit(%rax),%rcx
ja bad_from_user
ALTERNATIVE_2 "jmp copy_user_generic_unrolled", \
"jmp copy_user_generic_string", \
diff --git a/arch/x86/lib/csum-wrappers_64.c b/arch/x86/lib/csum-wrappers_64.c
index 28a6654f0d08..b6fcb9a9ddbc 100644
--- a/arch/x86/lib/csum-wrappers_64.c
+++ b/arch/x86/lib/csum-wrappers_64.c
@@ -6,6 +6,7 @@
*/
#include <asm/checksum.h>
#include <linux/module.h>
+#include <linux/uaccess.h>
#include <asm/smap.h>
/**
diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S
index 46668cda4ffd..0ef5128c2de8 100644
--- a/arch/x86/lib/getuser.S
+++ b/arch/x86/lib/getuser.S
@@ -35,8 +35,8 @@
.text
ENTRY(__get_user_1)
- GET_THREAD_INFO(%_ASM_DX)
- cmp TI_addr_limit(%_ASM_DX),%_ASM_AX
+ mov PER_CPU_VAR(current_task), %_ASM_DX
+ cmp TASK_addr_limit(%_ASM_DX),%_ASM_AX
jae bad_get_user
ASM_STAC
1: movzbl (%_ASM_AX),%edx
@@ -48,8 +48,8 @@ ENDPROC(__get_user_1)
ENTRY(__get_user_2)
add $1,%_ASM_AX
jc bad_get_user
- GET_THREAD_INFO(%_ASM_DX)
- cmp TI_addr_limit(%_ASM_DX),%_ASM_AX
+ mov PER_CPU_VAR(current_task), %_ASM_DX
+ cmp TASK_addr_limit(%_ASM_DX),%_ASM_AX
jae bad_get_user
ASM_STAC
2: movzwl -1(%_ASM_AX),%edx
@@ -61,8 +61,8 @@ ENDPROC(__get_user_2)
ENTRY(__get_user_4)
add $3,%_ASM_AX
jc bad_get_user
- GET_THREAD_INFO(%_ASM_DX)
- cmp TI_addr_limit(%_ASM_DX),%_ASM_AX
+ mov PER_CPU_VAR(current_task), %_ASM_DX
+ cmp TASK_addr_limit(%_ASM_DX),%_ASM_AX
jae bad_get_user
ASM_STAC
3: movl -3(%_ASM_AX),%edx
@@ -75,8 +75,8 @@ ENTRY(__get_user_8)
#ifdef CONFIG_X86_64
add $7,%_ASM_AX
jc bad_get_user
- GET_THREAD_INFO(%_ASM_DX)
- cmp TI_addr_limit(%_ASM_DX),%_ASM_AX
+ mov PER_CPU_VAR(current_task), %_ASM_DX
+ cmp TASK_addr_limit(%_ASM_DX),%_ASM_AX
jae bad_get_user
ASM_STAC
4: movq -7(%_ASM_AX),%rdx
@@ -86,8 +86,8 @@ ENTRY(__get_user_8)
#else
add $7,%_ASM_AX
jc bad_get_user_8
- GET_THREAD_INFO(%_ASM_DX)
- cmp TI_addr_limit(%_ASM_DX),%_ASM_AX
+ mov PER_CPU_VAR(current_task), %_ASM_DX
+ cmp TASK_addr_limit(%_ASM_DX),%_ASM_AX
jae bad_get_user_8
ASM_STAC
4: movl -7(%_ASM_AX),%edx
diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S
index e0817a12d323..c891ece81e5b 100644
--- a/arch/x86/lib/putuser.S
+++ b/arch/x86/lib/putuser.S
@@ -29,14 +29,14 @@
* as they get called from within inline assembly.
*/
-#define ENTER GET_THREAD_INFO(%_ASM_BX)
+#define ENTER mov PER_CPU_VAR(current_task), %_ASM_BX
#define EXIT ASM_CLAC ; \
ret
.text
ENTRY(__put_user_1)
ENTER
- cmp TI_addr_limit(%_ASM_BX),%_ASM_CX
+ cmp TASK_addr_limit(%_ASM_BX),%_ASM_CX
jae bad_put_user
ASM_STAC
1: movb %al,(%_ASM_CX)
@@ -46,7 +46,7 @@ ENDPROC(__put_user_1)
ENTRY(__put_user_2)
ENTER
- mov TI_addr_limit(%_ASM_BX),%_ASM_BX
+ mov TASK_addr_limit(%_ASM_BX),%_ASM_BX
sub $1,%_ASM_BX
cmp %_ASM_BX,%_ASM_CX
jae bad_put_user
@@ -58,7 +58,7 @@ ENDPROC(__put_user_2)
ENTRY(__put_user_4)
ENTER
- mov TI_addr_limit(%_ASM_BX),%_ASM_BX
+ mov TASK_addr_limit(%_ASM_BX),%_ASM_BX
sub $3,%_ASM_BX
cmp %_ASM_BX,%_ASM_CX
jae bad_put_user
@@ -70,7 +70,7 @@ ENDPROC(__put_user_4)
ENTRY(__put_user_8)
ENTER
- mov TI_addr_limit(%_ASM_BX),%_ASM_BX
+ mov TASK_addr_limit(%_ASM_BX),%_ASM_BX
sub $7,%_ASM_BX
cmp %_ASM_BX,%_ASM_CX
jae bad_put_user
diff --git a/arch/x86/lib/usercopy_64.c b/arch/x86/lib/usercopy_64.c
index 0a42327a59d7..9f760cdcaf40 100644
--- a/arch/x86/lib/usercopy_64.c
+++ b/arch/x86/lib/usercopy_64.c
@@ -6,7 +6,7 @@
* Copyright 2002 Andi Kleen <ak@suse.de>
*/
#include <linux/module.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
/*
* Zero Userspace
diff --git a/drivers/pnp/isapnp/proc.c b/drivers/pnp/isapnp/proc.c
index 5edee645d890..262285e48a09 100644
--- a/drivers/pnp/isapnp/proc.c
+++ b/drivers/pnp/isapnp/proc.c
@@ -21,7 +21,7 @@
#include <linux/isapnp.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
extern struct pnp_protocol isapnp_protocol;
diff --git a/lib/bitmap.c b/lib/bitmap.c
index c66da508cbf7..eca88087fa8a 100644
--- a/lib/bitmap.c
+++ b/lib/bitmap.c
@@ -14,9 +14,9 @@
#include <linux/bug.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/uaccess.h>
#include <asm/page.h>
-#include <asm/uaccess.h>
/*
* bitmaps provide an array of bits, implemented using an an
--
2.7.4
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2016-06-27 00:10 +0200 |
| Subject | [PATCH v4 17/29] x86: Move uaccess_err and sig_on_uaccess_err to thread_struct |
| Message-ID | <rOqpA-5R-21@gated-at.bofh.it> |
| In reply to | #1431600 |
thread_info is a legacy mess. To prepare for its partial removal,
move the uaccess control fields out -- they're straightforward.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/entry/vsyscall/vsyscall_64.c | 6 +++---
arch/x86/include/asm/processor.h | 3 +++
arch/x86/include/asm/thread_info.h | 2 --
arch/x86/include/asm/uaccess.h | 4 ++--
arch/x86/mm/extable.c | 2 +-
arch/x86/mm/fault.c | 2 +-
6 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c
index 174c2549939d..3aba2b043050 100644
--- a/arch/x86/entry/vsyscall/vsyscall_64.c
+++ b/arch/x86/entry/vsyscall/vsyscall_64.c
@@ -221,8 +221,8 @@ bool emulate_vsyscall(struct pt_regs *regs, unsigned long address)
* With a real vsyscall, page faults cause SIGSEGV. We want to
* preserve that behavior to make writing exploits harder.
*/
- prev_sig_on_uaccess_error = current_thread_info()->sig_on_uaccess_error;
- current_thread_info()->sig_on_uaccess_error = 1;
+ prev_sig_on_uaccess_error = current->thread.sig_on_uaccess_error;
+ current->thread.sig_on_uaccess_error = 1;
ret = -EFAULT;
switch (vsyscall_nr) {
@@ -243,7 +243,7 @@ bool emulate_vsyscall(struct pt_regs *regs, unsigned long address)
break;
}
- current_thread_info()->sig_on_uaccess_error = prev_sig_on_uaccess_error;
+ current->thread.sig_on_uaccess_error = prev_sig_on_uaccess_error;
check_fault:
if (ret == -EFAULT) {
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 62c6cc3cc5d3..f53ae57bd985 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -419,6 +419,9 @@ struct thread_struct {
/* Max allowed port in the bitmap, in bytes: */
unsigned io_bitmap_max;
+ unsigned int sig_on_uaccess_error:1;
+ unsigned int uaccess_err:1; /* uaccess failed */
+
/* Floating point and extended processor state */
struct fpu fpu;
/*
diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index 30c133ac05cd..7c47bb659ecd 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -58,8 +58,6 @@ struct thread_info {
__u32 status; /* thread synchronous flags */
__u32 cpu; /* current CPU */
mm_segment_t addr_limit;
- unsigned int sig_on_uaccess_error:1;
- unsigned int uaccess_err:1; /* uaccess failed */
};
#define INIT_THREAD_INFO(tsk) \
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 2982387ba817..4d2a726e8e6d 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -468,13 +468,13 @@ struct __large_struct { unsigned long buf[100]; };
* uaccess_try and catch
*/
#define uaccess_try do { \
- current_thread_info()->uaccess_err = 0; \
+ current->thread.uaccess_err = 0; \
__uaccess_begin(); \
barrier();
#define uaccess_catch(err) \
__uaccess_end(); \
- (err) |= (current_thread_info()->uaccess_err ? -EFAULT : 0); \
+ (err) |= (current->thread.uaccess_err ? -EFAULT : 0); \
} while (0)
/**
diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c
index 4bb53b89f3c5..0f90cc218d04 100644
--- a/arch/x86/mm/extable.c
+++ b/arch/x86/mm/extable.c
@@ -37,7 +37,7 @@ bool ex_handler_ext(const struct exception_table_entry *fixup,
struct pt_regs *regs, int trapnr)
{
/* Special hack for uaccess_err */
- current_thread_info()->uaccess_err = 1;
+ current->thread.uaccess_err = 1;
regs->ip = ex_fixup_addr(fixup);
return true;
}
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index c68b81f5659f..d34e21e81b07 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -737,7 +737,7 @@ no_context(struct pt_regs *regs, unsigned long error_code,
* In this case we need to make sure we're not recursively
* faulting through the emulate_vsyscall() logic.
*/
- if (current_thread_info()->sig_on_uaccess_error && signal) {
+ if (current->thread.sig_on_uaccess_error && signal) {
tsk->thread.trap_nr = X86_TRAP_PF;
tsk->thread.error_code = error_code | PF_USER;
tsk->thread.cr2 = address;
--
2.7.4
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2016-06-27 00:10 +0200 |
| Subject | [PATCH v4 03/29] x86/mm/hotplug: Don't remove PGD entries in remove_pagetable() |
| Message-ID | <rOqpA-5R-23@gated-at.bofh.it> |
| In reply to | #1431600 |
From: Ingo Molnar <mingo@kernel.org>
So when memory hotplug removes a piece of physical memory from pagetable
mappings, it also frees the underlying PGD entry.
This complicates PGD management, so don't do this. We can keep the
PGD mapped and the PUD table all clear - it's only a single 4K page
per 512 GB of memory hotplugged.
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Waiman Long <Waiman.Long@hp.com>
Cc: linux-mm@kvack.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Message-Id: <1442903021-3893-4-git-send-email-mingo@kernel.org>
---
arch/x86/mm/init_64.c | 27 ---------------------------
1 file changed, 27 deletions(-)
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index bce2e5d9edd4..c7465453d64e 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -702,27 +702,6 @@ static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud)
spin_unlock(&init_mm.page_table_lock);
}
-/* Return true if pgd is changed, otherwise return false. */
-static bool __meminit free_pud_table(pud_t *pud_start, pgd_t *pgd)
-{
- pud_t *pud;
- int i;
-
- for (i = 0; i < PTRS_PER_PUD; i++) {
- pud = pud_start + i;
- if (pud_val(*pud))
- return false;
- }
-
- /* free a pud table */
- free_pagetable(pgd_page(*pgd), 0);
- spin_lock(&init_mm.page_table_lock);
- pgd_clear(pgd);
- spin_unlock(&init_mm.page_table_lock);
-
- return true;
-}
-
static void __meminit
remove_pte_table(pte_t *pte_start, unsigned long addr, unsigned long end,
bool direct)
@@ -913,7 +892,6 @@ remove_pagetable(unsigned long start, unsigned long end, bool direct)
unsigned long addr;
pgd_t *pgd;
pud_t *pud;
- bool pgd_changed = false;
for (addr = start; addr < end; addr = next) {
next = pgd_addr_end(addr, end);
@@ -924,13 +902,8 @@ remove_pagetable(unsigned long start, unsigned long end, bool direct)
pud = (pud_t *)pgd_page_vaddr(*pgd);
remove_pud_table(pud, addr, next, direct);
- if (free_pud_table(pud, pgd))
- pgd_changed = true;
}
- if (pgd_changed)
- sync_global_pgds(start, end - 1, 1);
-
flush_tlb_all();
}
--
2.7.4
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Date | 2016-06-27 00:10 +0200 |
| Subject | [PATCH v4 23/29] kdb: Use task_cpu() instead of task_thread_info()->cpu |
| Message-ID | <rOqpA-5R-29@gated-at.bofh.it> |
| In reply to | #1431600 |
We'll need this cleanup to make the cpu field in thread_info be
optional.
Cc: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
include/linux/kdb.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/kdb.h b/include/linux/kdb.h
index a19bcf9e762e..410decacff8f 100644
--- a/include/linux/kdb.h
+++ b/include/linux/kdb.h
@@ -177,7 +177,7 @@ extern int kdb_get_kbd_char(void);
static inline
int kdb_process_cpu(const struct task_struct *p)
{
- unsigned int cpu = task_thread_info(p)->cpu;
+ unsigned int cpu = task_cpu(p);
if (cpu > num_possible_cpus())
cpu = 0;
return cpu;
--
2.7.4
[toc] | [prev] | [next] | [standalone]
Page 2 of 4 — ← Prev page 1 [2] 3 4 Next page →
Back to top | Article view | linux.kernel
csiph-web