Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1686080 > unrolled thread

[RFC PATCH 0/6] arm64: alternative VMAP_STACK implementation

Started byMark Rutland <mark.rutland@arm.com>
First post2017-07-13 00:40 +0200
Last post2017-07-13 13:30 +0200
Articles 9 on this page of 29 — 5 participants

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH 0/6] arm64: alternative VMAP_STACK implementation Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
    [RFC PATCH 5/6] arm64: keep track of current stack Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
    [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
      Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-13 09:00 +0200
        Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-13 13:00 +0200
          Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-13 13:50 +0200
            Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-13 18:20 +0200
              Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-13 20:00 +0200
                Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-13 20:30 +0200
                  Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-14 12:40 +0200
                    Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-14 12:50 +0200
                      Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-14 14:30 +0200
                        Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-14 16:10 +0200
                          Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-14 16:20 +0200
                          Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Robin Murphy <robin.murphy@arm.com> - 2017-07-14 16:40 +0200
                            Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Robin Murphy <robin.murphy@arm.com> - 2017-07-14 17:10 +0200
                              Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-14 17:20 +0200
                              Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-14 17:30 +0200
                          Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-14 23:30 +0200
                            Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-16 02:10 +0200
                      Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Mark Rutland <mark.rutland@arm.com> - 2017-07-14 15:00 +0200
                        Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and  detect out-of-bounds SP Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2017-07-14 15:00 +0200
    [RFC PATCH 4/6] arm64: pass stack base to secondary_start_kernel Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
    [RFC PATCH 1/6] arm64: use tpidr_el1 for current, free sp_el0 Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
      Re: [RFC PATCH 1/6] arm64: use tpidr_el1 for current, free sp_el0 Will Deacon <will.deacon@arm.com> - 2017-07-14 03:40 +0200
    [RFC PATCH 3/6] arm64: pad stacks to PAGE_SIZE for VMAP_STACK Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
    [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER} Mark Rutland <mark.rutland@arm.com> - 2017-07-13 00:40 +0200
      Re: [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER} James Morse <james.morse@arm.com> - 2017-07-13 12:20 +0200
        Re: [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER} Mark Rutland <mark.rutland@arm.com> - 2017-07-13 13:30 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1687370 — Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP

FromMark Rutland <mark.rutland@arm.com>
Date2017-07-14 15:00 +0200
SubjectRe: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP
Message-ID<u38mm-3Ag-11@gated-at.bofh.it>
In reply to#1687243
On Fri, Jul 14, 2017 at 11:48:20AM +0100, Ard Biesheuvel wrote:
> On 14 July 2017 at 11:32, Mark Rutland <mark.rutland@arm.com> wrote:
> > On Thu, Jul 13, 2017 at 07:28:48PM +0100, Ard Biesheuvel wrote:
> >> On 13 July 2017 at 18:55, Mark Rutland <mark.rutland@arm.com> wrote:
> >> > On Thu, Jul 13, 2017 at 05:10:50PM +0100, Mark Rutland wrote:
> >> >> On Thu, Jul 13, 2017 at 12:49:48PM +0100, Ard Biesheuvel wrote:
> >> >> > On 13 July 2017 at 11:49, Mark Rutland <mark.rutland@arm.com> wrote:
> >> >> > > On Thu, Jul 13, 2017 at 07:58:50AM +0100, Ard Biesheuvel wrote:
> >> >> > >> On 12 July 2017 at 23:33, Mark Rutland <mark.rutland@arm.com> wrote:
> > This means that we have to align the initial task, so the kernel Image
> > will grow by THREAD_SIZE. Likewise for IRQ stacks, unless we can rework
> > things such that we can dynamically allocate all of those.
> >
> 
> We can't currently do that for 64k pages, since the segment alignment
> is only 64k. But we should be able to patch that up I think

I was assuming that the linked would bump up the segment alignment if a
more-aligned object were placed inside. I guess that doesn't happen in
all cases?

... or do you mean when the EFI stub relocates the kernel, assuming
relaxed alignment constraints?

> >> >> I believe that determining whether the exception was caused by a stack
> >> >> overflow is not something we can do robustly or efficiently.
> >>
> >> Actually, if the stack pointer is within S_FRAME_SIZE of the base, and
> >> the faulting address points into the guard page, that is a pretty
> >> strong indicator that the stack overflowed. That shouldn't be too
> >> costly?
> >
> > Sure, but that's still a a heuristic. For example, that also catches an
> > unrelated vmalloc address gone wrong, while SP was close to the end of
> > the stack.
> 
> Yes, but the likelihood that an unrelated stray vmalloc access is
> within 16 KB of a stack pointer that is close ot its limit is
> extremely low, so we should be able to live with the risk of
> misidentifying it.

I guess, but at that point, why bother?

That gives us a fuzzy check for one specific "stack overflow", while not
catching the general case.

So long as we have a reliable stack trace, we can figure out that was
the case, and we don't set the expectation that we're trying to
categorize the general case (minefield and all).

Thanks,
Mark.

[toc] | [prev] | [next] | [standalone]

#1687380 — Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP

FromArd Biesheuvel <ard.biesheuvel@linaro.org>
Date2017-07-14 15:00 +0200
SubjectRe: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP
Message-ID<u38mn-3Ag-45@gated-at.bofh.it>
In reply to#1687370
On 14 July 2017 at 13:52, Mark Rutland <mark.rutland@arm.com> wrote:
> On Fri, Jul 14, 2017 at 11:48:20AM +0100, Ard Biesheuvel wrote:
>> On 14 July 2017 at 11:32, Mark Rutland <mark.rutland@arm.com> wrote:
>> > On Thu, Jul 13, 2017 at 07:28:48PM +0100, Ard Biesheuvel wrote:
>> >> On 13 July 2017 at 18:55, Mark Rutland <mark.rutland@arm.com> wrote:
>> >> > On Thu, Jul 13, 2017 at 05:10:50PM +0100, Mark Rutland wrote:
>> >> >> On Thu, Jul 13, 2017 at 12:49:48PM +0100, Ard Biesheuvel wrote:
>> >> >> > On 13 July 2017 at 11:49, Mark Rutland <mark.rutland@arm.com> wrote:
>> >> >> > > On Thu, Jul 13, 2017 at 07:58:50AM +0100, Ard Biesheuvel wrote:
>> >> >> > >> On 12 July 2017 at 23:33, Mark Rutland <mark.rutland@arm.com> wrote:
>> > This means that we have to align the initial task, so the kernel Image
>> > will grow by THREAD_SIZE. Likewise for IRQ stacks, unless we can rework
>> > things such that we can dynamically allocate all of those.
>> >
>>
>> We can't currently do that for 64k pages, since the segment alignment
>> is only 64k. But we should be able to patch that up I think
>
> I was assuming that the linked would bump up the segment alignment if a
> more-aligned object were placed inside. I guess that doesn't happen in
> all cases?
>
> ... or do you mean when the EFI stub relocates the kernel, assuming
> relaxed alignment constraints?
>

No, I mean under KASLR, which randomizes at SEGMENT_ALIGN granularity.

>> >> >> I believe that determining whether the exception was caused by a stack
>> >> >> overflow is not something we can do robustly or efficiently.
>> >>
>> >> Actually, if the stack pointer is within S_FRAME_SIZE of the base, and
>> >> the faulting address points into the guard page, that is a pretty
>> >> strong indicator that the stack overflowed. That shouldn't be too
>> >> costly?
>> >
>> > Sure, but that's still a a heuristic. For example, that also catches an
>> > unrelated vmalloc address gone wrong, while SP was close to the end of
>> > the stack.
>>
>> Yes, but the likelihood that an unrelated stray vmalloc access is
>> within 16 KB of a stack pointer that is close ot its limit is
>> extremely low, so we should be able to live with the risk of
>> misidentifying it.
>
> I guess, but at that point, why bother?
>
> That gives us a fuzzy check for one specific "stack overflow", while not
> catching the general case.
>
> So long as we have a reliable stack trace, we can figure out that was
> the case, and we don't set the expectation that we're trying to
> categorize the general case (minefield and all).
>

Yes. As long as the context is described accurately, there is no need
to make any inferences on behalf of the user.

[toc] | [prev] | [next] | [standalone]

#1686086 — [RFC PATCH 4/6] arm64: pass stack base to secondary_start_kernel

FromMark Rutland <mark.rutland@arm.com>
Date2017-07-13 00:40 +0200
Subject[RFC PATCH 4/6] arm64: pass stack base to secondary_start_kernel
Message-ID<u2ysy-5Ke-19@gated-at.bofh.it>
In reply to#1686080
In subsequent patches, we'll want the base of the secondary stack in
secondary_start_kernel.

Pass the stack base down, as we do in the primary path, and add the
offset in secondary_start_kernel. Unfortunately, we can't encode
STACK_START_SP in an add immediate, so use a mov immedaite, which has
greater range.

This is far from a hot path, so the overhead shouldn't matter.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
---
 arch/arm64/kernel/head.S | 3 ++-
 arch/arm64/kernel/smp.c  | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index a58ecda..db77cac 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -613,7 +613,8 @@ __secondary_switched:
 
 	adr_l	x0, secondary_data
 	ldr	x1, [x0, #CPU_BOOT_STACK]	// get secondary_data.stack
-	mov	sp, x1
+	mov	x3, #THREAD_START_SP
+	add	sp, x1, x3
 	ldr	x2, [x0, #CPU_BOOT_TASK]
 	msr	tpidr_el1, x2
 	mov	x29, #0
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 6e0e16a..269c957 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -154,7 +154,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
 	 * page tables.
 	 */
 	secondary_data.task = idle;
-	secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
+	secondary_data.stack = task_stack_page(idle);
 	update_cpu_boot_status(CPU_MMU_OFF);
 	__flush_dcache_area(&secondary_data, sizeof(secondary_data));
 
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1686087 — [RFC PATCH 1/6] arm64: use tpidr_el1 for current, free sp_el0

FromMark Rutland <mark.rutland@arm.com>
Date2017-07-13 00:40 +0200
Subject[RFC PATCH 1/6] arm64: use tpidr_el1 for current, free sp_el0
Message-ID<u2ysy-5Ke-17@gated-at.bofh.it>
In reply to#1686080
Today we use TPIDR_EL1 for our percpu offset, and SP_EL0 for current
(and current::thread_info, which is at offset 0).

Using SP_EL0 in this way prevents us from using EL1 thread mode, where
SP_EL0 is not addressable (since it's used as the active SP). It also
means we can't use SP_EL0 for other purposes (e.g. as a
scratch-register).

This patch frees up SP_EL0 for such usage, by storing the percpu offset
in current::thread_info, and using TPIDR_EL1 to store current. As we no
longer need to update SP_EL0 at EL0 exception boundaries, this allows us
to delete some code.

This new organisation means that we need to perform an additional load
to acquire the prcpu offset. However, our assembly constraints allow
current to be cached, and therefore allow the offset to be cached.
Additionally, in most cases where we need the percpu offset, we also
need to fiddle with the preempt count or other data stored in
current::thread_info, so this data should already be hot in the caches.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
---
 arch/arm64/include/asm/assembler.h   | 11 ++++++++---
 arch/arm64/include/asm/current.h     |  6 +++---
 arch/arm64/include/asm/percpu.h      | 15 ++++-----------
 arch/arm64/include/asm/thread_info.h |  1 +
 arch/arm64/kernel/asm-offsets.c      |  1 +
 arch/arm64/kernel/entry.S            | 11 ++---------
 arch/arm64/kernel/head.S             |  4 ++--
 arch/arm64/kernel/process.c          | 16 ++++------------
 8 files changed, 25 insertions(+), 40 deletions(-)

diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 1b67c37..f7da6b5 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -229,6 +229,11 @@
 #endif
 	.endm
 
+	.macro	get_this_cpu_offset dst
+	mrs	\dst, tpidr_el1
+	ldr	\dst, [\dst, #TSK_TI_PCP]
+	.endm
+
 	/*
 	 * @dst: Result of per_cpu(sym, smp_processor_id())
 	 * @sym: The name of the per-cpu variable
@@ -236,7 +241,7 @@
 	 */
 	.macro adr_this_cpu, dst, sym, tmp
 	adr_l	\dst, \sym
-	mrs	\tmp, tpidr_el1
+	get_this_cpu_offset \tmp
 	add	\dst, \dst, \tmp
 	.endm
 
@@ -247,7 +252,7 @@
 	 */
 	.macro ldr_this_cpu dst, sym, tmp
 	adr_l	\dst, \sym
-	mrs	\tmp, tpidr_el1
+	get_this_cpu_offset \tmp
 	ldr	\dst, [\dst, \tmp]
 	.endm
 
@@ -438,7 +443,7 @@
  * Return the current thread_info.
  */
 	.macro	get_thread_info, rd
-	mrs	\rd, sp_el0
+	mrs	\rd, tpidr_el1
 	.endm
 
 /*
diff --git a/arch/arm64/include/asm/current.h b/arch/arm64/include/asm/current.h
index f6580d4..54b271a 100644
--- a/arch/arm64/include/asm/current.h
+++ b/arch/arm64/include/asm/current.h
@@ -13,11 +13,11 @@
  */
 static __always_inline struct task_struct *get_current(void)
 {
-	unsigned long sp_el0;
+	unsigned long cur;
 
-	asm ("mrs %0, sp_el0" : "=r" (sp_el0));
+	asm ("mrs %0, tpidr_el1" : "=r" (cur));
 
-	return (struct task_struct *)sp_el0;
+	return (struct task_struct *)cur;
 }
 
 #define current get_current()
diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h
index 3bd498e..05cf0f8 100644
--- a/arch/arm64/include/asm/percpu.h
+++ b/arch/arm64/include/asm/percpu.h
@@ -18,23 +18,16 @@
 
 #include <asm/stack_pointer.h>
 
+#include <linux/thread_info.h>
+
 static inline void set_my_cpu_offset(unsigned long off)
 {
-	asm volatile("msr tpidr_el1, %0" :: "r" (off) : "memory");
+	current_thread_info()->pcp_offset = off;
 }
 
 static inline unsigned long __my_cpu_offset(void)
 {
-	unsigned long off;
-
-	/*
-	 * We want to allow caching the value, so avoid using volatile and
-	 * instead use a fake stack read to hazard against barrier().
-	 */
-	asm("mrs %0, tpidr_el1" : "=r" (off) :
-		"Q" (*(const unsigned long *)current_stack_pointer));
-
-	return off;
+	return current_thread_info()->pcp_offset;
 }
 #define __my_cpu_offset __my_cpu_offset()
 
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 46c3b93..141f13e9 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -50,6 +50,7 @@ struct thread_info {
 #ifdef CONFIG_ARM64_SW_TTBR0_PAN
 	u64			ttbr0;		/* saved TTBR0_EL1 */
 #endif
+	unsigned long		pcp_offset;
 	int			preempt_count;	/* 0 => preemptable, <0 => bug */
 };
 
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index b3bb7ef..17001be 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -38,6 +38,7 @@ int main(void)
   BLANK();
   DEFINE(TSK_TI_FLAGS,		offsetof(struct task_struct, thread_info.flags));
   DEFINE(TSK_TI_PREEMPT,	offsetof(struct task_struct, thread_info.preempt_count));
+  DEFINE(TSK_TI_PCP,		offsetof(struct task_struct, thread_info.pcp_offset));
   DEFINE(TSK_TI_ADDR_LIMIT,	offsetof(struct task_struct, thread_info.addr_limit));
 #ifdef CONFIG_ARM64_SW_TTBR0_PAN
   DEFINE(TSK_TI_TTBR0,		offsetof(struct task_struct, thread_info.ttbr0));
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index b738880..773b3fea 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -92,7 +92,7 @@
 
 	.if	\el == 0
 	mrs	x21, sp_el0
-	ldr_this_cpu	tsk, __entry_task, x20	// Ensure MDSCR_EL1.SS is clear,
+	get_thread_info tsk			// Ensure MDSCR_EL1.SS is clear,
 	ldr	x19, [tsk, #TSK_TI_FLAGS]	// since we can unmask debug
 	disable_step_tsk x19, x20		// exceptions when scheduling.
 
@@ -147,13 +147,6 @@ alternative_else_nop_endif
 	.endif
 
 	/*
-	 * Set sp_el0 to current thread_info.
-	 */
-	.if	\el == 0
-	msr	sp_el0, tsk
-	.endif
-
-	/*
 	 * Registers that may be useful after this macro is invoked:
 	 *
 	 * x21 - aborted SP
@@ -734,7 +727,7 @@ ENTRY(cpu_switch_to)
 	ldp	x29, x9, [x8], #16
 	ldr	lr, [x8]
 	mov	sp, x9
-	msr	sp_el0, x1
+	msr	tpidr_el1, x1
 	ret
 ENDPROC(cpu_switch_to)
 
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 973df7d..a58ecda 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -324,7 +324,7 @@ __primary_switched:
 	adrp	x4, init_thread_union
 	add	sp, x4, #THREAD_SIZE
 	adr_l	x5, init_task
-	msr	sp_el0, x5			// Save thread_info
+	msr	tpidr_el1, x5			// Save thread_info
 
 	adr_l	x8, vectors			// load VBAR_EL1 with virtual
 	msr	vbar_el1, x8			// vector table address
@@ -615,7 +615,7 @@ __secondary_switched:
 	ldr	x1, [x0, #CPU_BOOT_STACK]	// get secondary_data.stack
 	mov	sp, x1
 	ldr	x2, [x0, #CPU_BOOT_TASK]
-	msr	sp_el0, x2
+	msr	tpidr_el1, x2
 	mov	x29, #0
 	b	secondary_start_kernel
 ENDPROC(__secondary_switched)
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index ae2a835..4212da3 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -323,18 +323,10 @@ void uao_thread_switch(struct task_struct *next)
 	}
 }
 
-/*
- * We store our current task in sp_el0, which is clobbered by userspace. Keep a
- * shadow copy so that we can restore this upon entry from userspace.
- *
- * This is *only* for exception entry from EL0, and is not valid until we
- * __switch_to() a user task.
- */
-DEFINE_PER_CPU(struct task_struct *, __entry_task);
-
-static void entry_task_switch(struct task_struct *next)
+/* Ensure the new task has this CPU's offset */
+void pcp_thread_switch(struct task_struct *next)
 {
-	__this_cpu_write(__entry_task, next);
+	next->thread_info.pcp_offset = current_thread_info()->pcp_offset;
 }
 
 /*
@@ -349,8 +341,8 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev,
 	tls_thread_switch(next);
 	hw_breakpoint_thread_switch(next);
 	contextidr_thread_switch(next);
-	entry_task_switch(next);
 	uao_thread_switch(next);
+	pcp_thread_switch(next);
 
 	/*
 	 * Complete any pending TLB or cache maintenance on this CPU in case
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1687007 — Re: [RFC PATCH 1/6] arm64: use tpidr_el1 for current, free sp_el0

FromWill Deacon <will.deacon@arm.com>
Date2017-07-14 03:40 +0200
SubjectRe: [RFC PATCH 1/6] arm64: use tpidr_el1 for current, free sp_el0
Message-ID<u2XKi-4Ps-3@gated-at.bofh.it>
In reply to#1686087
On Wed, Jul 12, 2017 at 11:32:58PM +0100, Mark Rutland wrote:
> Today we use TPIDR_EL1 for our percpu offset, and SP_EL0 for current
> (and current::thread_info, which is at offset 0).
> 
> Using SP_EL0 in this way prevents us from using EL1 thread mode, where
> SP_EL0 is not addressable (since it's used as the active SP). It also
> means we can't use SP_EL0 for other purposes (e.g. as a
> scratch-register).
> 
> This patch frees up SP_EL0 for such usage, by storing the percpu offset
> in current::thread_info, and using TPIDR_EL1 to store current. As we no
> longer need to update SP_EL0 at EL0 exception boundaries, this allows us
> to delete some code.

Does this mean we can just use asm-generic/percpu.h?

Will

[toc] | [prev] | [next] | [standalone]

#1686089 — [RFC PATCH 3/6] arm64: pad stacks to PAGE_SIZE for VMAP_STACK

FromMark Rutland <mark.rutland@arm.com>
Date2017-07-13 00:40 +0200
Subject[RFC PATCH 3/6] arm64: pad stacks to PAGE_SIZE for VMAP_STACK
Message-ID<u2ysz-5Ke-23@gated-at.bofh.it>
In reply to#1686080
Our THREAD_SIZE may be smaller than PAGE_SIZE. With VMAP_STACK, we can't
allow stacks to share a page with anything else, so may as well pad
up-to PAGE_SIZE, and have 64K stacks when we have 64K pages.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
---
 arch/arm64/include/asm/thread_info.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 6d0c59a..3684f86 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -25,7 +25,13 @@
 
 #include <asm/page.h>
 
-#define THREAD_SHIFT		14
+#define __THREAD_SHIFT		14
+
+#if defined(CONFIG_VMAP_STACK) && (__THREAD_SHIFT < PAGE_SHIFT)
+#define THREAD_SHIFT		PAGE_SHIFT
+#else
+#define THREAD_SHIFT		__THREAD_SHIFT
+#endif
 
 #if THREAD_SHIFT >= PAGE_SHIFT
 #define THREAD_SIZE_ORDER	(THREAD_SHIFT - PAGE_SHIFT)
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1686091 — [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER}

FromMark Rutland <mark.rutland@arm.com>
Date2017-07-13 00:40 +0200
Subject[RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER}
Message-ID<u2ysz-5Ke-31@gated-at.bofh.it>
In reply to#1686080
Currently we define THREAD_SIZE_ORDER dependent on which arm64-specific
page size kconfig symbol was selected. This is unfortunate, as it hides
the relationship between THREAD_SIZE_ORDER and THREAD_SIZE, and makes it
painful more painful than necessary to modify the thread size as we will
need to do for some debug configurations.

This patch follows arch/metag's approach of consistently defining
THREAD_SIZE in terms of THREAD_SIZE_ORDER. This avoids having ifdefs for
particular page size configurations, and allows us to change a single
definition to change the thread size.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/include/asm/thread_info.h | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 141f13e9..6d0c59a 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -23,13 +23,17 @@
 
 #include <linux/compiler.h>
 
-#ifdef CONFIG_ARM64_4K_PAGES
-#define THREAD_SIZE_ORDER	2
-#elif defined(CONFIG_ARM64_16K_PAGES)
+#include <asm/page.h>
+
+#define THREAD_SHIFT		14
+
+#if THREAD_SHIFT >= PAGE_SHIFT
+#define THREAD_SIZE_ORDER	(THREAD_SHIFT - PAGE_SHIFT)
+#else
 #define THREAD_SIZE_ORDER	0
 #endif
 
-#define THREAD_SIZE		16384
+#define THREAD_SIZE		(PAGE_SIZE << THREAD_SIZE_ORDER)
 #define THREAD_START_SP		(THREAD_SIZE - 16)
 
 #ifndef __ASSEMBLY__
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1686433 — Re: [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER}

FromJames Morse <james.morse@arm.com>
Date2017-07-13 12:20 +0200
SubjectRe: [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER}
Message-ID<u2JnY-4lu-17@gated-at.bofh.it>
In reply to#1686091
Hi Mark,

On 12/07/17 23:32, Mark Rutland wrote:
> Currently we define THREAD_SIZE_ORDER dependent on which arm64-specific
> page size kconfig symbol was selected. This is unfortunate, as it hides
> the relationship between THREAD_SIZE_ORDER and THREAD_SIZE, and makes it
> painful more painful than necessary to modify the thread size as we will
> need to do for some debug configurations.
> 
> This patch follows arch/metag's approach of consistently defining
> THREAD_SIZE in terms of THREAD_SIZE_ORDER. This avoids having ifdefs for
> particular page size configurations, and allows us to change a single
> definition to change the thread size.

I think this has unintended side effects for 64K page systems.  (or at least not
yet intended)

Today:
> #ifdef CONFIG_ARM64_4K_PAGES
> #define THREAD_SIZE_ORDER	2
> #elif defined(CONFIG_ARM64_16K_PAGES)
> #define THREAD_SIZE_ORDER	0
> #endif

Means THREAD_SIZE_ORDER is unset on 64K, and THREAD_SIZE is always:
> #define THREAD_SIZE		16384

/kernel/fork.c matches this with its:
> # if THREAD_SIZE >= PAGE_SIZE || defined(CONFIG_VMAP_STACK)
[...]
> #else
[...]
> void thread_stack_cache_init(void)
> {
>	thread_stack_cache = kmem_cache_create("thread_stack", THREAD_SIZE,
> 					      THREAD_SIZE, 0, NULL);
> 	BUG_ON(thread_stack_cache == NULL);
> }
> #endif

To create a kmemcache to share 64K pages as 16K stacks.


After this patch:
> #define THREAD_SHIFT		14
>
> #if THREAD_SHIFT >= PAGE_SHIFT
> #define THREAD_SIZE_ORDER	(THREAD_SHIFT - PAGE_SHIFT)
> #else
> #define THREAD_SIZE_ORDER	0
> #endif

Means THREAD_SIZE_ORDER is 0, and:
> #define THREAD_SIZE		(PAGE_SIZE << THREAD_SIZE_ORDER)

gives us a 64K THREAD_SIZE.



Thanks,

James





> diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
> index 141f13e9..6d0c59a 100644
> --- a/arch/arm64/include/asm/thread_info.h
> +++ b/arch/arm64/include/asm/thread_info.h
> @@ -23,13 +23,17 @@
>  
>  #include <linux/compiler.h>
>  
> -#ifdef CONFIG_ARM64_4K_PAGES
> -#define THREAD_SIZE_ORDER	2
> -#elif defined(CONFIG_ARM64_16K_PAGES)
> +#include <asm/page.h>
> +
> +#define THREAD_SHIFT		14
> +
> +#if THREAD_SHIFT >= PAGE_SHIFT
> +#define THREAD_SIZE_ORDER	(THREAD_SHIFT - PAGE_SHIFT)
> +#else
>  #define THREAD_SIZE_ORDER	0
>  #endif
>  
> -#define THREAD_SIZE		16384
> +#define THREAD_SIZE		(PAGE_SIZE << THREAD_SIZE_ORDER)
>  #define THREAD_START_SP		(THREAD_SIZE - 16)
>  
>  #ifndef __ASSEMBLY__
>

[toc] | [prev] | [next] | [standalone]

#1686459 — Re: [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER}

FromMark Rutland <mark.rutland@arm.com>
Date2017-07-13 13:30 +0200
SubjectRe: [RFC PATCH 2/6] arm64: avoid open-coding THREAD_SIZE{,_ORDER}
Message-ID<u2KtH-4Y8-3@gated-at.bofh.it>
In reply to#1686433
On Thu, Jul 13, 2017 at 11:18:35AM +0100, James Morse wrote:
> Hi Mark,
> 
> On 12/07/17 23:32, Mark Rutland wrote:
> > Currently we define THREAD_SIZE_ORDER dependent on which arm64-specific
> > page size kconfig symbol was selected. This is unfortunate, as it hides
> > the relationship between THREAD_SIZE_ORDER and THREAD_SIZE, and makes it
> > painful more painful than necessary to modify the thread size as we will
> > need to do for some debug configurations.
> > 
> > This patch follows arch/metag's approach of consistently defining
> > THREAD_SIZE in terms of THREAD_SIZE_ORDER. This avoids having ifdefs for
> > particular page size configurations, and allows us to change a single
> > definition to change the thread size.
> 
> I think this has unintended side effects for 64K page systems.  (or at least not
> yet intended)
> 
> Today:
> > #ifdef CONFIG_ARM64_4K_PAGES
> > #define THREAD_SIZE_ORDER	2
> > #elif defined(CONFIG_ARM64_16K_PAGES)
> > #define THREAD_SIZE_ORDER	0
> > #endif
> 
> Means THREAD_SIZE_ORDER is unset on 64K, and THREAD_SIZE is always:
> > #define THREAD_SIZE		16384
> 
> /kernel/fork.c matches this with its:
> > # if THREAD_SIZE >= PAGE_SIZE || defined(CONFIG_VMAP_STACK)
> [...]
> > #else
> [...]
> > void thread_stack_cache_init(void)
> > {
> >	thread_stack_cache = kmem_cache_create("thread_stack", THREAD_SIZE,
> > 					      THREAD_SIZE, 0, NULL);
> > 	BUG_ON(thread_stack_cache == NULL);
> > }
> > #endif
> 
> To create a kmemcache to share 64K pages as 16K stacks.
> 
> 
> After this patch:
> > #define THREAD_SHIFT		14
> >
> > #if THREAD_SHIFT >= PAGE_SHIFT
> > #define THREAD_SIZE_ORDER	(THREAD_SHIFT - PAGE_SHIFT)
> > #else
> > #define THREAD_SIZE_ORDER	0
> > #endif
> 
> Means THREAD_SIZE_ORDER is 0, and:
> > #define THREAD_SIZE		(PAGE_SIZE << THREAD_SIZE_ORDER)
> 
> gives us a 64K THREAD_SIZE.

Yes; I'd gotten confused as to what I was doing here. Thanks for
spotting that.

I've folded this and the next patch, with the resultant logic being as
below, which I think fixes this.

Thanks,
Mark.

---->8----
#define MIN_THREAD_SHIFT	14

/*
 * Each VMAP stack is a separate VMALLOC allocation, which is at least
 * PAGE_SIZE.
 */
#if defined(CONFIG_VMAP_STACK) && (MIN_THREAD_SHIFT < PAGE_SHIFT)
#define THREAD_SHIFT		PAGE_SHIFT
#else
#define THREAD_SHIFT		MIN_THREAD_SHIFT
#endif

#if THREAD_SHIFT >= PAGE_SHIFT
#define THREAD_SIZE_ORDER	(THREAD_SHIFT - PAGE_SHIFT)
#endif

#define THREAD_SIZE		(1UL << THREAD_SHIFT)
#define THREAD_START_SP		(THREAD_SIZE - 16)

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web