Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1240078 > unrolled thread

[PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit

Back to article view | Back to linux.kernel

Contents

  [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
    [PATCH v2 04/36] selftests/x86: Add a test for syscall restart and arg modification Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] selftests/x86:   Add a test for ptrace syscall restart and arg modification tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-07 18:20 +0200
    [PATCH v2 20/36] x86/entry: Add do_syscall_32, a C function to do 32-bit syscalls Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/entry: Add do_syscall_32(),   a C function to do 32-bit syscalls tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
    [PATCH v2 13/36] x86/vdso/32: Save extra registers in the INT80 vsyscall path Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/vdso/32:   Save extra registers in the INT80 vsyscall path tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:10 +0200
    [PATCH v2 11/36] x86/vdso: Replace hex int80 CFI annotations with gas directives Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/vdso:   Replace hex int80 CFI annotations with GAS directives tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:10 +0200
    [PATCH v2 02/36] x86/uaccess: __chk_range_not_ok is unlikely to return true Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      Re: [PATCH v2 02/36] x86/uaccess: __chk_range_not_ok is unlikely to  return true Borislav Petkov <bp@alien8.de> - 2015-10-07 13:00 +0200
        Re: [PATCH v2 02/36] x86/uaccess: __chk_range_not_ok is unlikely to  return true Ingo Molnar <mingo@kernel.org> - 2015-10-07 18:30 +0200
      [tip:x86/asm] x86/uaccess: Add unlikely() to __chk_range_not_ok()   failure paths tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-07 18:20 +0200
    [PATCH v2 06/36] x86/entry: Move lockdep_sys_exit to prepare_exit_to_usermode Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/entry, locking/lockdep: Move lockdep_sys_exit()   to prepare_exit_to_usermode() tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-07 18:20 +0200
        Re: [tip:x86/asm] x86/entry, locking/lockdep: Move  lockdep_sys_exit() to prepare_exit_to_usermode() Peter Zijlstra <peterz@infradead.org> - 2015-10-08 11:00 +0200
          Re: [tip:x86/asm] x86/entry, locking/lockdep: Move lockdep_sys_exit()  to prepare_exit_to_usermode() Andy Lutomirski <luto@amacapital.net> - 2015-10-09 21:40 +0200
    [PATCH v2 16/36] x86/entry/64/compat: Remove most of the fast system call machinery Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/entry/64/compat:   Remove most of the fast system call machinery tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
    [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Linus Torvalds <torvalds@linux-foundation.org> - 2015-10-06 10:30 +0200
        Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Andy Lutomirski <luto@amacapital.net> - 2015-10-06 20:30 +0200
      Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Linus Torvalds <torvalds@linux-foundation.org> - 2015-10-06 10:30 +0200
      [tip:x86/asm] x86/entry: Add C code for fast system call entries tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
    [PATCH v2 30/36] x86/entry: Make irqs_disabled checks in exit code depend on lockdep Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/entry:   Make irqs_disabled checks in exit code depend on lockdep tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
    [PATCH v2 15/36] x86/entry/64/compat: Remove audit optimizations Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      [tip:x86/asm] x86/entry/64/compat: Remove audit optimizations tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
    [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
      Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall  arg fetch Ingo Molnar <mingo@kernel.org> - 2015-10-09 09:40 +0200
        Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall  arg fetch Andy Lutomirski <luto@amacapital.net> - 2015-10-09 21:30 +0200
          Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall  arg fetch Ingo Molnar <mingo@kernel.org> - 2015-10-10 11:10 +0200
      [tip:x86/asm] x86/entry:   Micro-optimize compat fast syscall arg fetch tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
    Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Linus Torvalds <torvalds@linux-foundation.org> - 2015-10-06 10:40 +0200
      Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native  64-bit Ingo Molnar <mingo@kernel.org> - 2015-10-06 11:00 +0200
      Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Andy Lutomirski <luto@amacapital.net> - 2015-10-06 20:30 +0200
    Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native  64-bit Ingo Molnar <mingo@kernel.org> - 2015-10-09 15:10 +0200
      Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Richard Weinberger <richard.weinberger@gmail.com> - 2015-10-12 20:40 +0200
        Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Andy Lutomirski <luto@amacapital.net> - 2015-10-12 20:50 +0200
          Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native  64-bit Richard Weinberger <richard@nod.at> - 2015-10-12 23:10 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1240242 — Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries

On Tue, Oct 6, 2015 at 9:25 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>> +       local_irq_enable();
>> +       if (get_user(*(u32 *)&regs->cx,
>> +                    (u32 __user __force *)(unsigned long)(u32)regs->sp)) {
> ...
>> +       local_irq_disable();
>
> this is expensive. Since we now do it in C code and can easily do
> this, why does the code not do this all with interrupts disabled,
> which is valid for user accesses but disables page faults, and then in
> the unlikely situation where that fails, we do it the slow and careful
> way?

Ok. I notice that then a later patch removes the local_irq_disable()
and calls do_syscall_32_irqs_on().

So I guess that "just run get_user with interrupts disabled"
optimization is pointless, because we'll just end up enabling
interrupts at some point anyway, and it can just be done before the
get_user().

So never mind.

            Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240843 — Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries

On Tue, Oct 6, 2015 at 1:29 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Oct 6, 2015 at 9:25 AM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>>> +       local_irq_enable();
>>> +       if (get_user(*(u32 *)&regs->cx,
>>> +                    (u32 __user __force *)(unsigned long)(u32)regs->sp)) {
>> ...
>>> +       local_irq_disable();
>>
>> this is expensive. Since we now do it in C code and can easily do
>> this, why does the code not do this all with interrupts disabled,
>> which is valid for user accesses but disables page faults, and then in
>> the unlikely situation where that fails, we do it the slow and careful
>> way?
>
> Ok. I notice that then a later patch removes the local_irq_disable()
> and calls do_syscall_32_irqs_on().
>
> So I guess that "just run get_user with interrupts disabled"
> optimization is pointless, because we'll just end up enabling
> interrupts at some point anyway, and it can just be done before the
> get_user().
>
> So never mind.
>

I'll improve the changelog.  This is all clear in my head, but I could
certainly describe it better.

I tried to structure this as much as possible as simple unoptimized
changes that would be easyish to understand from a correctness
perspective and then to optimize at the end.

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240243 — Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries

On Tue, Oct 6, 2015 at 1:48 AM, Andy Lutomirski <luto@kernel.org> wrote:
> This handles both SYSENTER and SYSCALL.  The asm glue will take care
> of the differences.

This seems non-optimal.

> +__visible void do_fast_syscall_32(struct pt_regs *regs)

.. so this is the fast case on 32-bit, but:

> +       local_irq_enable();
> +       if (get_user(*(u32 *)&regs->cx,
> +                    (u32 __user __force *)(unsigned long)(u32)regs->sp)) {
...
> +       local_irq_disable();

this is expensive. Since we now do it in C code and can easily do
this, why does the code not do this all with interrupts disabled,
which is valid for user accesses but disables page faults, and then in
the unlikely situation where that fails, we do it the slow and careful
way?

It would be nice to see this patch series not just get rid of asm
code, but also do things to optimize cases that used to be annoyingly
hard to do in user space. No?

         Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243338 — [tip:x86/asm] x86/entry: Add C code for fast system call entries

Commit-ID:  710246df58041106b7de645f4b45770f8a59a269
Gitweb:     http://git.kernel.org/tip/710246df58041106b7de645f4b45770f8a59a269
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Mon, 5 Oct 2015 17:48:10 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 9 Oct 2015 09:41:09 +0200

x86/entry: Add C code for fast system call entries

This handles both SYSENTER and SYSCALL.  The asm glue will take
care of the differences.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Link: http://lkml.kernel.org/r/6041a58a9b8ef6d2522ab4350deb1a1945eb563f.1444091585.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/entry/common.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 41d1750..1b2606e 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -24,6 +24,8 @@
 
 #include <asm/desc.h>
 #include <asm/traps.h>
+#include <asm/vdso.h>
+#include <asm/uaccess.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/syscalls.h>
@@ -360,4 +362,45 @@ __visible void do_int80_syscall_32(struct pt_regs *regs)
 
 	syscall_return_slowpath(regs);
 }
+
+__visible void do_fast_syscall_32(struct pt_regs *regs)
+{
+	/*
+	 * Called using the internal vDSO SYSENTER/SYSCALL32 calling
+	 * convention.  Adjust regs so it looks like we entered using int80.
+	 */
+
+	unsigned long landing_pad = (unsigned long)current->mm->context.vdso +
+		vdso_image_32.sym_int80_landing_pad;
+
+	/*
+	 * SYSENTER loses EIP, and even SYSCALL32 needs us to skip forward
+	 * so that 'regs->ip -= 2' lands back on an int $0x80 instruction.
+	 * Fix it up.
+	 */
+	regs->ip = landing_pad;
+
+	/*
+	 * Fetch ECX from where the vDSO stashed it.
+	 *
+	 * WARNING: We are in CONTEXT_USER and RCU isn't paying attention!
+	 */
+	local_irq_enable();
+	if (get_user(*(u32 *)&regs->cx,
+		     (u32 __user __force *)(unsigned long)(u32)regs->sp)) {
+		/* User code screwed up. */
+		local_irq_disable();
+		regs->ax = -EFAULT;
+#ifdef CONFIG_CONTEXT_TRACKING
+		enter_from_user_mode();
+#endif
+		prepare_exit_to_usermode(regs);
+		return;
+	}
+	local_irq_disable();
+
+	/* Now this is just like a normal syscall. */
+	do_int80_syscall_32(regs);
+	return;
+}
 #endif
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240088 — [PATCH v2 30/36] x86/entry: Make irqs_disabled checks in exit code depend on lockdep

These checks are quite slow.  Disable them in non-lockdep kernels to
reduce the performance hit.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/entry/common.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 7d740e16038e..1d95f8835322 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -219,7 +219,7 @@ static struct thread_info *pt_regs_to_thread_info(struct pt_regs *regs)
 /* Called with IRQs disabled. */
 __visible void prepare_exit_to_usermode(struct pt_regs *regs)
 {
-	if (WARN_ON(!irqs_disabled()))
+	if (IS_ENABLED(CONFIG_PROVE_LOCKING) && WARN_ON(!irqs_disabled()))
 		local_irq_disable();
 
 	lockdep_sys_exit();
@@ -281,8 +281,8 @@ __visible void syscall_return_slowpath(struct pt_regs *regs)
 
 	CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
 
-	if (WARN(irqs_disabled(), "syscall %ld left IRQs disabled",
-		 regs->orig_ax))
+	if (IS_ENABLED(CONFIG_PROVE_LOCKING) &&
+	    WARN(irqs_disabled(), "syscall %ld left IRQs disabled", regs->orig_ax))
 		local_irq_enable();
 
 	/*
-- 
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243347 — [tip:x86/asm] x86/entry: Make irqs_disabled checks in exit code depend on lockdep

Commit-ID:  460d12453e1afe20416ce9536cfecb31d17a9abd
Gitweb:     http://git.kernel.org/tip/460d12453e1afe20416ce9536cfecb31d17a9abd
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Mon, 5 Oct 2015 17:48:18 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 9 Oct 2015 09:41:11 +0200

x86/entry: Make irqs_disabled checks in exit code depend on lockdep

These checks are quite slow.  Disable them in non-lockdep
kernels to reduce the performance hit.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Link: http://lkml.kernel.org/r/eccff2a154ae6fb50f40228901003a6e9c24f3d0.1444091585.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/entry/common.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 0d1c842..03aacd1 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -219,7 +219,7 @@ static struct thread_info *pt_regs_to_thread_info(struct pt_regs *regs)
 /* Called with IRQs disabled. */
 __visible void prepare_exit_to_usermode(struct pt_regs *regs)
 {
-	if (WARN_ON(!irqs_disabled()))
+	if (IS_ENABLED(CONFIG_PROVE_LOCKING) && WARN_ON(!irqs_disabled()))
 		local_irq_disable();
 
 	lockdep_sys_exit();
@@ -281,8 +281,8 @@ __visible void syscall_return_slowpath(struct pt_regs *regs)
 
 	CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
 
-	if (WARN(irqs_disabled(), "syscall %ld left IRQs disabled",
-		 regs->orig_ax))
+	if (IS_ENABLED(CONFIG_PROVE_LOCKING) &&
+	    WARN(irqs_disabled(), "syscall %ld left IRQs disabled", regs->orig_ax))
 		local_irq_enable();
 
 	/*
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240089 — [PATCH v2 15/36] x86/entry/64/compat: Remove audit optimizations

These audit optimizations are messy and hard to maintain.  We'll
get a similar effect from opportunistic sysret when fast compat
system calls are re-implemented.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/entry/entry_64_compat.S | 98 +---------------------------------------
 1 file changed, 2 insertions(+), 96 deletions(-)

diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S
index e11aa38696fe..8f847722ce72 100644
--- a/arch/x86/entry/entry_64_compat.S
+++ b/arch/x86/entry/entry_64_compat.S
@@ -16,16 +16,6 @@
 #include <linux/linkage.h>
 #include <linux/err.h>
 
-/* Avoid __ASSEMBLER__'ifying <linux/audit.h> just for this.  */
-#include <linux/elf-em.h>
-#define AUDIT_ARCH_I386		(EM_386|__AUDIT_ARCH_LE)
-#define __AUDIT_ARCH_LE		0x40000000
-
-#ifndef CONFIG_AUDITSYSCALL
-# define sysexit_audit		ia32_ret_from_sys_call_irqs_off
-# define sysretl_audit		ia32_ret_from_sys_call_irqs_off
-#endif
-
 	.section .entry.text, "ax"
 
 #ifdef CONFIG_PARAVIRT
@@ -148,7 +138,7 @@ sysenter_dispatch:
 	DISABLE_INTERRUPTS(CLBR_NONE)
 	TRACE_IRQS_OFF
 	testl	$_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jnz	sysexit_audit
+	jnz	ia32_ret_from_sys_call_irqs_off
 sysexit_from_sys_call:
 	/*
 	 * NB: SYSEXIT is not obviously safe for 64-bit kernels -- an
@@ -200,81 +190,12 @@ sysexit_from_sys_call:
 	 */
 	USERGS_SYSRET32
 
-#ifdef CONFIG_AUDITSYSCALL
-	.macro auditsys_entry_common
-	/*
-	 * At this point, registers hold syscall args in the 32-bit syscall ABI:
-	 * EAX is syscall number, the 6 args are in EBX,ECX,EDX,ESI,EDI,EBP.
-	 *
-	 * We want to pass them to __audit_syscall_entry(), which is a 64-bit
-	 * C function with 5 parameters, so shuffle them to match what
-	 * the function expects: RDI,RSI,RDX,RCX,R8.
-	 */
-	movl	%esi, %r8d		/* arg5 (R8 ) <= 4th syscall arg (ESI) */
-	xchg	%ecx, %edx		/* arg4 (RCX) <= 3rd syscall arg (EDX) */
-					/* arg3 (RDX) <= 2nd syscall arg (ECX) */
-	movl	%ebx, %esi		/* arg2 (RSI) <= 1st syscall arg (EBX) */
-	movl	%eax, %edi		/* arg1 (RDI) <= syscall number  (EAX) */
-	call	__audit_syscall_entry
-
-	/*
-	 * We are going to jump back to the syscall dispatch code.
-	 * Prepare syscall args as required by the 64-bit C ABI.
-	 * Registers clobbered by __audit_syscall_entry() are
-	 * loaded from pt_regs on stack:
-	 */
-	movl	ORIG_RAX(%rsp), %eax	/* syscall number */
-	movl	%ebx, %edi		/* arg1 */
-	movl	RCX(%rsp), %esi		/* arg2 */
-	movl	RDX(%rsp), %edx		/* arg3 */
-	movl	RSI(%rsp), %ecx		/* arg4 */
-	movl	RDI(%rsp), %r8d		/* arg5 */
-	.endm
-
-	.macro auditsys_exit exit
-	TRACE_IRQS_ON
-	ENABLE_INTERRUPTS(CLBR_NONE)
-	testl	$(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jnz	ia32_ret_from_sys_call
-	movl	%eax, %esi		/* second arg, syscall return value */
-	cmpl	$-MAX_ERRNO, %eax	/* is it an error ? */
-	jbe	1f
-	movslq	%eax, %rsi		/* if error sign extend to 64 bits */
-1:	setbe	%al			/* 1 if error, 0 if not */
-	movzbl	%al, %edi		/* zero-extend that into %edi */
-	call	__audit_syscall_exit
-	movl	$(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), %edi
-	DISABLE_INTERRUPTS(CLBR_NONE)
-	TRACE_IRQS_OFF
-	testl	%edi, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jz	\exit
-	xorl	%eax, %eax		/* Do not leak kernel information */
-	movq	%rax, R11(%rsp)
-	movq	%rax, R10(%rsp)
-	movq	%rax, R9(%rsp)
-	movq	%rax, R8(%rsp)
-	jmp	int_ret_from_sys_call_irqs_off
-	.endm
-
-sysenter_auditsys:
-	auditsys_entry_common
-	movl	%ebp, %r9d		/* reload 6th syscall arg */
-	jmp	sysenter_dispatch
-
-sysexit_audit:
-	auditsys_exit sysexit_from_sys_call
-#endif
-
 sysenter_fix_flags:
 	pushq	$X86_EFLAGS_FIXED
 	popfq
 	jmp	sysenter_flags_fixed
 
 sysenter_tracesys:
-#ifdef CONFIG_AUDITSYSCALL
-	testl	$(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jz	sysenter_auditsys
-#endif
 	SAVE_EXTRA_REGS
 	xorl	%eax, %eax		/* Do not leak kernel information */
 	movq	%rax, R11(%rsp)
@@ -389,7 +310,7 @@ cstar_dispatch:
 	DISABLE_INTERRUPTS(CLBR_NONE)
 	TRACE_IRQS_OFF
 	testl	$_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jnz	sysretl_audit
+	jnz	ia32_ret_from_sys_call_irqs_off
 
 sysretl_from_sys_call:
 	andl	$~TS_COMPAT, ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
@@ -420,22 +341,7 @@ sysretl_from_sys_call:
 	 */
 	USERGS_SYSRET32
 
-#ifdef CONFIG_AUDITSYSCALL
-cstar_auditsys:
-	movl	%r9d, R9(%rsp)		/* register to be clobbered by call */
-	auditsys_entry_common
-	movl	R9(%rsp), %r9d		/* reload 6th syscall arg */
-	jmp	cstar_dispatch
-
-sysretl_audit:
-	auditsys_exit sysretl_from_sys_call
-#endif
-
 cstar_tracesys:
-#ifdef CONFIG_AUDITSYSCALL
-	testl	$(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jz	cstar_auditsys
-#endif
 	xchgl	%r9d, %ebp
 	SAVE_EXTRA_REGS
 	xorl	%eax, %eax		/* Do not leak kernel information */
-- 
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243343 — [tip:x86/asm] x86/entry/64/compat: Remove audit optimizations

Commit-ID:  c5f638ac90d514202155c87aa58730e86d484d9e
Gitweb:     http://git.kernel.org/tip/c5f638ac90d514202155c87aa58730e86d484d9e
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Mon, 5 Oct 2015 17:48:03 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 9 Oct 2015 09:41:07 +0200

x86/entry/64/compat: Remove audit optimizations

These audit optimizations are messy and hard to maintain.  We'll
get a similar effect from opportunistic sysret when fast compat
system calls are re-implemented.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Link: http://lkml.kernel.org/r/0bcca79ac7ff835d0e5a38725298865b01347a82.1444091584.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/entry/entry_64_compat.S | 98 +---------------------------------------
 1 file changed, 2 insertions(+), 96 deletions(-)

diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S
index 06a8966..75f5fcf 100644
--- a/arch/x86/entry/entry_64_compat.S
+++ b/arch/x86/entry/entry_64_compat.S
@@ -16,16 +16,6 @@
 #include <linux/linkage.h>
 #include <linux/err.h>
 
-/* Avoid __ASSEMBLER__'ifying <linux/audit.h> just for this.  */
-#include <linux/elf-em.h>
-#define AUDIT_ARCH_I386		(EM_386|__AUDIT_ARCH_LE)
-#define __AUDIT_ARCH_LE		0x40000000
-
-#ifndef CONFIG_AUDITSYSCALL
-# define sysexit_audit		ia32_ret_from_sys_call_irqs_off
-# define sysretl_audit		ia32_ret_from_sys_call_irqs_off
-#endif
-
 	.section .entry.text, "ax"
 
 #ifdef CONFIG_PARAVIRT
@@ -148,7 +138,7 @@ sysenter_dispatch:
 	DISABLE_INTERRUPTS(CLBR_NONE)
 	TRACE_IRQS_OFF
 	testl	$_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jnz	sysexit_audit
+	jnz	ia32_ret_from_sys_call_irqs_off
 sysexit_from_sys_call:
 	/*
 	 * NB: SYSEXIT is not obviously safe for 64-bit kernels -- an
@@ -200,81 +190,12 @@ sysexit_from_sys_call:
 	 */
 	USERGS_SYSRET32
 
-#ifdef CONFIG_AUDITSYSCALL
-	.macro auditsys_entry_common
-	/*
-	 * At this point, registers hold syscall args in the 32-bit syscall ABI:
-	 * EAX is syscall number, the 6 args are in EBX,ECX,EDX,ESI,EDI,EBP.
-	 *
-	 * We want to pass them to __audit_syscall_entry(), which is a 64-bit
-	 * C function with 5 parameters, so shuffle them to match what
-	 * the function expects: RDI,RSI,RDX,RCX,R8.
-	 */
-	movl	%esi, %r8d		/* arg5 (R8 ) <= 4th syscall arg (ESI) */
-	xchg	%ecx, %edx		/* arg4 (RCX) <= 3rd syscall arg (EDX) */
-					/* arg3 (RDX) <= 2nd syscall arg (ECX) */
-	movl	%ebx, %esi		/* arg2 (RSI) <= 1st syscall arg (EBX) */
-	movl	%eax, %edi		/* arg1 (RDI) <= syscall number  (EAX) */
-	call	__audit_syscall_entry
-
-	/*
-	 * We are going to jump back to the syscall dispatch code.
-	 * Prepare syscall args as required by the 64-bit C ABI.
-	 * Registers clobbered by __audit_syscall_entry() are
-	 * loaded from pt_regs on stack:
-	 */
-	movl	ORIG_RAX(%rsp), %eax	/* syscall number */
-	movl	%ebx, %edi		/* arg1 */
-	movl	RCX(%rsp), %esi		/* arg2 */
-	movl	RDX(%rsp), %edx		/* arg3 */
-	movl	RSI(%rsp), %ecx		/* arg4 */
-	movl	RDI(%rsp), %r8d		/* arg5 */
-	.endm
-
-	.macro auditsys_exit exit
-	TRACE_IRQS_ON
-	ENABLE_INTERRUPTS(CLBR_NONE)
-	testl	$(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jnz	ia32_ret_from_sys_call
-	movl	%eax, %esi		/* second arg, syscall return value */
-	cmpl	$-MAX_ERRNO, %eax	/* is it an error ? */
-	jbe	1f
-	movslq	%eax, %rsi		/* if error sign extend to 64 bits */
-1:	setbe	%al			/* 1 if error, 0 if not */
-	movzbl	%al, %edi		/* zero-extend that into %edi */
-	call	__audit_syscall_exit
-	movl	$(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), %edi
-	DISABLE_INTERRUPTS(CLBR_NONE)
-	TRACE_IRQS_OFF
-	testl	%edi, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jz	\exit
-	xorl	%eax, %eax		/* Do not leak kernel information */
-	movq	%rax, R11(%rsp)
-	movq	%rax, R10(%rsp)
-	movq	%rax, R9(%rsp)
-	movq	%rax, R8(%rsp)
-	jmp	int_ret_from_sys_call_irqs_off
-	.endm
-
-sysenter_auditsys:
-	auditsys_entry_common
-	movl	%ebp, %r9d		/* reload 6th syscall arg */
-	jmp	sysenter_dispatch
-
-sysexit_audit:
-	auditsys_exit sysexit_from_sys_call
-#endif
-
 sysenter_fix_flags:
 	pushq	$X86_EFLAGS_FIXED
 	popfq
 	jmp	sysenter_flags_fixed
 
 sysenter_tracesys:
-#ifdef CONFIG_AUDITSYSCALL
-	testl	$(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jz	sysenter_auditsys
-#endif
 	SAVE_EXTRA_REGS
 	xorl	%eax, %eax		/* Do not leak kernel information */
 	movq	%rax, R11(%rsp)
@@ -389,7 +310,7 @@ cstar_dispatch:
 	DISABLE_INTERRUPTS(CLBR_NONE)
 	TRACE_IRQS_OFF
 	testl	$_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jnz	sysretl_audit
+	jnz	ia32_ret_from_sys_call_irqs_off
 
 sysretl_from_sys_call:
 	andl	$~TS_COMPAT, ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
@@ -420,22 +341,7 @@ sysretl_from_sys_call:
 	 */
 	USERGS_SYSRET32
 
-#ifdef CONFIG_AUDITSYSCALL
-cstar_auditsys:
-	movl	%r9d, R9(%rsp)		/* register to be clobbered by call */
-	auditsys_entry_common
-	movl	R9(%rsp), %r9d		/* reload 6th syscall arg */
-	jmp	cstar_dispatch
-
-sysretl_audit:
-	auditsys_exit sysretl_from_sys_call
-#endif
-
 cstar_tracesys:
-#ifdef CONFIG_AUDITSYSCALL
-	testl	$(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
-	jz	cstar_auditsys
-#endif
 	xchgl	%r9d, %ebp
 	SAVE_EXTRA_REGS
 	xorl	%eax, %eax		/* Do not leak kernel information */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240090 — [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch

we're following a 32-bit pointer, and the uaccess code isn't smart
enough to figure out that the access_ok check isn't needed.

This saves about three cycles on a cache-hot fast syscall.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/entry/common.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 6e1ea6a401f2..3ae6191ad1a2 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -394,8 +394,20 @@ __visible long do_fast_syscall_32(struct pt_regs *regs)
 	 * WARNING: We are in CONTEXT_USER and RCU isn't paying attention!
 	 */
 	local_irq_enable();
-	if (get_user(*(u32 *)&regs->cx,
-		     (u32 __user __force *)(unsigned long)(u32)regs->sp)) {
+	if (
+#ifdef CONFIG_X86_64
+		/*
+		 * Micro-optimization: the pointer we're following is explicitly
+		 * 32 bits, so it can't be out of range.
+		 */
+		__get_user(*(u32 *)&regs->cx,
+			    (u32 __user __force *)(unsigned long)(u32)regs->sp)
+#else
+		get_user(*(u32 *)&regs->cx,
+			 (u32 __user __force *)(unsigned long)(u32)regs->sp)
+#endif
+		) {
+
 		/* User code screwed up. */
 		local_irq_disable();
 		regs->ax = -EFAULT;
-- 
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243092 — Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch

* Andy Lutomirski <luto@kernel.org> wrote:

> we're following a 32-bit pointer, and the uaccess code isn't smart
> enough to figure out that the access_ok check isn't needed.
> 
> This saves about three cycles on a cache-hot fast syscall.

Another request: could you please stick the benchmarking code of the various x86 
system call variants into 'perf bench' - under tools/perf/bench/, so that 
measurements can be done on more hardware and can be reproduced easily?

I'd suggest we dedicate an entirely new benchmark family to it: 'perf bench x86' 
and then have:

   perf bench x86 syscall vdso
   perf bench x86 syscall int80
   perf bench x86 syscall vdso-compat

or so?

( I have some perf bench cleanups in -tip, so if you do this please base it on top 
  of that. )

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243618 — Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch

On Fri, Oct 9, 2015 at 12:32 AM, Ingo Molnar <mingo@kernel.org> wrote:
>
> * Andy Lutomirski <luto@kernel.org> wrote:
>
>> we're following a 32-bit pointer, and the uaccess code isn't smart
>> enough to figure out that the access_ok check isn't needed.
>>
>> This saves about three cycles on a cache-hot fast syscall.
>
> Another request: could you please stick the benchmarking code of the various x86
> system call variants into 'perf bench' - under tools/perf/bench/, so that
> measurements can be done on more hardware and can be reproduced easily?
>
> I'd suggest we dedicate an entirely new benchmark family to it: 'perf bench x86'
> and then have:
>
>    perf bench x86 syscall vdso
>    perf bench x86 syscall int80
>    perf bench x86 syscall vdso-compat

I'll play with this.  I'm not too familiar with the perf bench stuff.

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243858 — Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch

* Andy Lutomirski <luto@amacapital.net> wrote:

> On Fri, Oct 9, 2015 at 12:32 AM, Ingo Molnar <mingo@kernel.org> wrote:
> >
> > * Andy Lutomirski <luto@kernel.org> wrote:
> >
> >> we're following a 32-bit pointer, and the uaccess code isn't smart
> >> enough to figure out that the access_ok check isn't needed.
> >>
> >> This saves about three cycles on a cache-hot fast syscall.
> >
> > Another request: could you please stick the benchmarking code of the various x86
> > system call variants into 'perf bench' - under tools/perf/bench/, so that
> > measurements can be done on more hardware and can be reproduced easily?
> >
> > I'd suggest we dedicate an entirely new benchmark family to it: 'perf bench x86'
> > and then have:
> >
> >    perf bench x86 syscall vdso
> >    perf bench x86 syscall int80
> >    perf bench x86 syscall vdso-compat
> 
> I'll play with this.  I'm not too familiar with the perf bench stuff.

So the perf bench stuff is meant to be a familiar home to kernel developers we'd 
like to slap a micro (or macro) benchmark into an easy to modify place.

Over the years it has gathered a number of benchmarks - but more are always 
welcome.

Just copy one of the existing benchmark modules (the tools/perf/bench/numa.c one 
is the most advanced one, tools/perf/bench/sched-pipe.c is the simplest one) and 
off you go.

Here's a commit that adds a new benchmark suite:

  a043971141f1 ("perf bench: Add futex-hash microbenchmark")

There are no big restrictions on the benchmarks: just put your existing code in 
that produces stdout output and it will be likely very close to upstream 
acceptable.

Can help should you get stuck anywhere.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243359 — [tip:x86/asm] x86/entry: Micro-optimize compat fast syscall arg fetch

Commit-ID:  c68ca6787bdd6d2df37cf950135aa11e71af358a
Gitweb:     http://git.kernel.org/tip/c68ca6787bdd6d2df37cf950135aa11e71af358a
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Mon, 5 Oct 2015 17:48:20 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 9 Oct 2015 09:41:12 +0200

x86/entry: Micro-optimize compat fast syscall arg fetch

We're following a 32-bit pointer, and the uaccess code isn't
smart enough to figure out that the access_ok() check isn't
needed.

This saves about three cycles on a cache-hot fast syscall.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Link: http://lkml.kernel.org/r/bdff034e2f23c5eb974c760cf494cb5bddce8f29.1444091585.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/entry/common.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index d5eee85..08a945d 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -394,8 +394,20 @@ __visible long do_fast_syscall_32(struct pt_regs *regs)
 	 * WARNING: We are in CONTEXT_USER and RCU isn't paying attention!
 	 */
 	local_irq_enable();
-	if (get_user(*(u32 *)&regs->cx,
-		     (u32 __user __force *)(unsigned long)(u32)regs->sp)) {
+	if (
+#ifdef CONFIG_X86_64
+		/*
+		 * Micro-optimization: the pointer we're following is explicitly
+		 * 32 bits, so it can't be out of range.
+		 */
+		__get_user(*(u32 *)&regs->cx,
+			    (u32 __user __force *)(unsigned long)(u32)regs->sp)
+#else
+		get_user(*(u32 *)&regs->cx,
+			 (u32 __user __force *)(unsigned long)(u32)regs->sp)
+#endif
+		) {
+
 		/* User code screwed up. */
 		local_irq_disable();
 		regs->ax = -EFAULT;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240259

On Tue, Oct 6, 2015 at 1:47 AM, Andy Lutomirski <luto@kernel.org> wrote:
>
> The patchset is structured as a removal of the old fast syscall
> code, then the change that makes syscalls into real functions, then
> a clean re-implementation of fast syscalls.
>
> If we want some of the 25 cycles back, we could consider open-coding
> a new C fast path.

Ok, the 25 cycles are apparently just 12 cycles. Did you forget to
update that after doing some of the optimizations - or is it 25 cycles
on 64-bit and 12 cycles on 32-bit?

But regardless, I've read through the series twice, and I don't see
anything wrong. The one thing I thought I noticed ended up being
mooted by a patch later in the series.

Of course, this is the kind of code where reading the patch doesn't
necessarily catch any of the really subtle problems at all, but you
seem to have tested it fairly heavily too.

So ACK on the whole series. I don't mind a few cycles for the fast
case if it means it's more maintainable, and sadly I suspect that with
so many distributions using audit functionality, the fast case is
almost unheard of anyway, and the fact that you sped up the slow case
is what matters more. Snif.

I'd like to see this as a git tree, with your longish explanation as
the pull request for the x86 merge, so that that gets saved for
posterity too in the kernel commit logs. But that's between you and
Ingo or whoever ends up being the person who picks this up.

                   Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240272 — Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit

* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> [...]
> 
> I'd like to see this as a git tree, with your longish explanation as the pull 
> request for the x86 merge, so that that gets saved for posterity too in the 
> kernel commit logs. But that's between you and Ingo or whoever ends up being the 
> person who picks this up.

Yeah, will make sure that description gets preserved.

Will wait a days or so for reviews to trickle in before applying the patches, but 
this looks really good so far, kudos Andy!

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240848

On Oct 6, 2015 1:39 AM, "Linus Torvalds" <torvalds@linux-foundation.org> wrote:
>
> On Tue, Oct 6, 2015 at 1:47 AM, Andy Lutomirski <luto@kernel.org> wrote:
> >
> > The patchset is structured as a removal of the old fast syscall
> > code, then the change that makes syscalls into real functions, then
> > a clean re-implementation of fast syscalls.
> >
> > If we want some of the 25 cycles back, we could consider open-coding
> > a new C fast path.
>
> Ok, the 25 cycles are apparently just 12 cycles. Did you forget to
> update that after doing some of the optimizations - or is it 25 cycles
> on 64-bit and 12 cycles on 32-bit?

Forgot to update.  I'll fix this in v3.  I'll also re-benchmark on
native 32-bit, which is a bit easier now that I figured out why my
Debian VM couldn't do fast syscalls.  (Hint: if you have a 32-bit
Debian or Debian-derived setup without libc6-i686 installed, install
it and everything works better.)

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243328 — Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit

* Andy Lutomirski <luto@kernel.org> wrote:

> The first two patches are optimizations that I'm surprised we didn't
> already have.  I noticed them when I was looking at the generated
> asm.
> 
> The next two patches are tests and some old stuff.  There's a test
> that validates the vDSO AT_SYSINFO annotations.  There's also a test
> that exercises some assumptions that signal handling and ptracers
> make about syscalls that currently do *not* hold on 64-bit AMD using
> 32-bit AT_SYSINFO.
> 
> The next three patches are NT cleanups and a lockdep cleanup.
> 
> It may pay to apply the beginning of the series (at most through
> "x86/entry/64/compat: After SYSENTER, move STI after the NT fixup")
> without waiting for everyone to wrap their heads around the rest.
> 
> The rest is basically a rewrite of syscalls for all cases except
> 64-bit native.  With these patches applied, there is a single 32-bit
> vDSO and it uses SYSCALL, SYSENTER, and INT80 almost interchangeably
> via alternatives.  The semantics of SYSENTER and SYSCALL are defined
> as:
> 
>  1. If SYSCALL, ESP = ECX
>  2. ECX = *ESP
>  3. IP = INT80 landing pad
>  4. Opportunistic SYSRET/SYSEXIT is enabled on return
> 
> The vDSO is rearranged so that these semantics work.  Anything that
> backs IP up by 2 ends up pointing at a bona fide int $0x80
> instruction with the expected regs.
> 
> In the process, the vDSO CFI annotations (which are actually used)
> get rewritten using normal CFI directives.
> 
> Opportunistic SYSRET/SYSEXIT only happens on return when CS and SS
> are as expected, IP points to the INT80 landing pad, and flags are
> in good shape.  (There is no longer any assumption that full
> fast-path 32-bit syscalls don't muck with the registers that matter
> for fast exits -- I played with maintaining an optimization like
> that with poor results.  I may try again if it saves a few cycles.)
> 
> Other than that, the system call entries are simplified to the bare
> minimum prologue and a call to a C function.  Amusingly, SYSENTER
> and SYSCALL32 use the same C function.
> 
> To make that work, I had to remove all the 32-bit syscall stubs
> except the clone argument hack.  This is because, for C code to call
> through the system call table, the system call table entries need to
> be real function pointers with C-compatible ABIs.
> 
> There is nothing at all anymore that requires that x86_32 syscalls
> be asmlinkage.  That could be removed in a subsequent patch.
> 
> The upshot appears to be a ~16 cycle performance hit on 32-bit fast
> path syscalls.  (On my system, my little prctl test takes 172 cycles
> before and 188 cycles with these patches applied.)
> 
> The slow path is probably faster under most circumstances and, if
> the exit slow path gets hit, it'll be much faster because (as we
> already do in the 64-bit native case) we can still use
> SYSEXIT/SYSRET.
> 
> The patchset is structured as a removal of the old fast syscall
> code, then the change that makes syscalls into real functions, then
> a clean re-implementation of fast syscalls.
> 
> If we want some of the 25 cycles back, we could consider open-coding
> a new C fast path.
> 
> Changes from v1:
>  - The unwind_vdso_32 test now warns on broken Debian installations
>    instead of failing.  The problem is now fully understood, will
>    be fixed by Debian and possibly also fixed by upstream glibc.
>  - execve was rather broken in v1.
>  - It's quite a bit faster now (the optimizations at the end are mostly new).
>  - int80 on 64-bit no longer clobbers extra regs (thanks Denys!).
>  - The uaccess stuff is new.
>  - Lots of other things that I forgot, I'm sure.
> 
> Andy Lutomirski (36):
>   x86/uaccess: Tell the compiler that uaccess is unlikely to fault
>   x86/uaccess: __chk_range_not_ok is unlikely to return true
>   selftests/x86: Add a test for vDSO unwinding
>   selftests/x86: Add a test for syscall restart and arg modification
>   x86/entry/64/compat: Fix SYSENTER's NT flag before user memory access
>   x86/entry: Move lockdep_sys_exit to prepare_exit_to_usermode
>   x86/entry/64/compat: After SYSENTER, move STI after the NT fixup
>   x86/vdso: Remove runtime 32-bit vDSO selection
>   x86/asm: Re-add manual CFI infrastructure
>   x86/vdso: Define BUILD_VDSO while building and emit .eh_frame in asm
>   x86/vdso: Replace hex int80 CFI annotations with gas directives
>   x86/elf/64: Clear more registers in elf_common_init
>   x86/vdso/32: Save extra registers in the INT80 vsyscall path
>   x86/entry/64/compat: Disable SYSENTER and SYSCALL32 entries
>   x86/entry/64/compat: Remove audit optimizations
>   x86/entry/64/compat: Remove most of the fast system call machinery
>   x86/entry/64/compat: Set up full pt_regs for all compat syscalls
>   x86/entry/syscalls: Move syscall table declarations into
>     asm/syscalls.h
>   x86/syscalls: Give sys_call_ptr_t a useful type
>   x86/entry: Add do_syscall_32, a C function to do 32-bit syscalls
>   x86/entry/64/compat: Migrate the body of the syscall entry to C
>   x86/entry: Add C code for fast system call entries
>   x86/vdso/compat: Wire up SYSENTER and SYSCSALL for compat userspace
>   x86/entry/compat: Implement opportunistic SYSRETL for compat syscalls
>   x86/entry/32: Open-code return tracking from fork and kthreads
>   x86/entry/32: Switch INT80 to the new C syscall path
>   x86/entry/32: Re-implement SYSENTER using the new C path
>   x86/asm: Remove thread_info.sysenter_return
>   x86/entry: Remove unnecessary IRQ twiddling in fast 32-bit syscalls
>   x86/entry: Make irqs_disabled checks in exit code depend on lockdep
>   x86/entry: Force inlining of 32-bit syscall code
>   x86/entry: Micro-optimize compat fast syscall arg fetch
>   x86/entry: Hide two syscall entry assertions behind CONFIG_DEBUG_ENTRY
>   x86/entry: Use pt_regs_to_thread_info() in syscall entry tracing
>   x86/entry: Split and inline prepare_exit_to_usermode
>   x86/entry: Split and inline syscall_return_slowpath
> 
>  arch/x86/Makefile                                  |  10 +-
>  arch/x86/entry/common.c                            | 255 ++++++++--
>  arch/x86/entry/entry_32.S                          | 184 +++----
>  arch/x86/entry/entry_64.S                          |   9 +-
>  arch/x86/entry/entry_64_compat.S                   | 541 +++++----------------
>  arch/x86/entry/syscall_32.c                        |   9 +-
>  arch/x86/entry/syscall_64.c                        |   4 +-
>  arch/x86/entry/syscalls/syscall_32.tbl             |  12 +-
>  arch/x86/entry/vdso/Makefile                       |  39 +-
>  arch/x86/entry/vdso/vdso2c.c                       |   2 +-
>  arch/x86/entry/vdso/vdso32-setup.c                 |  28 +-
>  arch/x86/entry/vdso/vdso32/int80.S                 |  56 ---
>  arch/x86/entry/vdso/vdso32/syscall.S               |  75 ---
>  arch/x86/entry/vdso/vdso32/sysenter.S              | 116 -----
>  arch/x86/entry/vdso/vdso32/system_call.S           |  57 +++
>  arch/x86/entry/vdso/vma.c                          |  13 +-
>  arch/x86/ia32/ia32_signal.c                        |   4 +-
>  arch/x86/include/asm/dwarf2.h                      | 177 +++++++
>  arch/x86/include/asm/elf.h                         |  10 +-
>  arch/x86/include/asm/syscall.h                     |  14 +-
>  arch/x86/include/asm/thread_info.h                 |   1 -
>  arch/x86/include/asm/uaccess.h                     |  14 +-
>  arch/x86/include/asm/vdso.h                        |  10 +-
>  arch/x86/kernel/asm-offsets.c                      |   3 -
>  arch/x86/kernel/signal.c                           |   4 +-
>  arch/x86/um/sys_call_table_32.c                    |   7 +-
>  arch/x86/um/sys_call_table_64.c                    |   7 +-
>  arch/x86/xen/setup.c                               |  13 +-
>  tools/testing/selftests/x86/Makefile               |   5 +-
>  tools/testing/selftests/x86/ptrace_syscall.c       | 294 +++++++++++
>  .../testing/selftests/x86/raw_syscall_helper_32.S  |  46 ++
>  tools/testing/selftests/x86/unwind_vdso.c          | 209 ++++++++
>  32 files changed, 1258 insertions(+), 970 deletions(-)
>  delete mode 100644 arch/x86/entry/vdso/vdso32/int80.S
>  delete mode 100644 arch/x86/entry/vdso/vdso32/syscall.S
>  delete mode 100644 arch/x86/entry/vdso/vdso32/sysenter.S
>  create mode 100644 arch/x86/entry/vdso/vdso32/system_call.S
>  create mode 100644 arch/x86/include/asm/dwarf2.h
>  create mode 100644 tools/testing/selftests/x86/ptrace_syscall.c
>  create mode 100644 tools/testing/selftests/x86/raw_syscall_helper_32.S
>  create mode 100644 tools/testing/selftests/x86/unwind_vdso.c

Ok, so I applied all of them to tip:x86/asm, in two phases, with small (stylistic) 
edits - it all seems to work fine for me so far, so I pushed it all out to -tip 
and linux-next.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1244987

On Fri, Oct 9, 2015 at 3:06 PM, Ingo Molnar <mingo@kernel.org> wrote:
> Ok, so I applied all of them to tip:x86/asm, in two phases, with small (stylistic)
> edits - it all seems to work fine for me so far, so I pushed it all out to -tip
> and linux-next.
>

FYI, this breaks the UML build:

  CC      arch/x86/um/sys_call_table_64.o
arch/x86/um/sys_call_table_64.c:49:1: error: unknown type name ‘sys_call_ptr_t’
 const sys_call_ptr_t sys_call_table[] ____cacheline_aligned = {
 ^
arch/x86/um/sys_call_table_64.c:54:2: warning: initialization makes
integer from pointer without a cast [enabled by default]
  [0 ... __NR_syscall_max] = &sys_ni_syscall,
  ^
arch/x86/um/sys_call_table_64.c:54:2: warning: (near initialization
for ‘sys_call_table[0]’) [enabled by default]
arch/x86/um/sys_call_table_64.c:54:2: error: initializer element is
not computable at load time
arch/x86/um/sys_call_table_64.c:54:2: error: (near initialization for
‘sys_call_table[0]’)

Andy, I suspect UML's syscall.h needs the sys_call_ptr_t type too?

-- 
Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1245023

On Mon, Oct 12, 2015 at 11:30 AM, Richard Weinberger
<richard.weinberger@gmail.com> wrote:
> On Fri, Oct 9, 2015 at 3:06 PM, Ingo Molnar <mingo@kernel.org> wrote:
>> Ok, so I applied all of them to tip:x86/asm, in two phases, with small (stylistic)
>> edits - it all seems to work fine for me so far, so I pushed it all out to -tip
>> and linux-next.
>>
>
> FYI, this breaks the UML build:
>
>   CC      arch/x86/um/sys_call_table_64.o
> arch/x86/um/sys_call_table_64.c:49:1: error: unknown type name ‘sys_call_ptr_t’
>  const sys_call_ptr_t sys_call_table[] ____cacheline_aligned = {
>  ^
> arch/x86/um/sys_call_table_64.c:54:2: warning: initialization makes
> integer from pointer without a cast [enabled by default]
>   [0 ... __NR_syscall_max] = &sys_ni_syscall,
>   ^
> arch/x86/um/sys_call_table_64.c:54:2: warning: (near initialization
> for ‘sys_call_table[0]’) [enabled by default]
> arch/x86/um/sys_call_table_64.c:54:2: error: initializer element is
> not computable at load time
> arch/x86/um/sys_call_table_64.c:54:2: error: (near initialization for
> ‘sys_call_table[0]’)
>
> Andy, I suspect UML's syscall.h needs the sys_call_ptr_t type too?
>

Whoops, yes.  UML has some weird casts in the syscall code that might
be removable with that change, too.  Want to fix it or should I?

--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1245134 — Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit

Am 12.10.2015 um 20:41 schrieb Andy Lutomirski:
> On Mon, Oct 12, 2015 at 11:30 AM, Richard Weinberger
> <richard.weinberger@gmail.com> wrote:
>> On Fri, Oct 9, 2015 at 3:06 PM, Ingo Molnar <mingo@kernel.org> wrote:
>>> Ok, so I applied all of them to tip:x86/asm, in two phases, with small (stylistic)
>>> edits - it all seems to work fine for me so far, so I pushed it all out to -tip
>>> and linux-next.
>>>
>>
>> FYI, this breaks the UML build:
>>
>>   CC      arch/x86/um/sys_call_table_64.o
>> arch/x86/um/sys_call_table_64.c:49:1: error: unknown type name ‘sys_call_ptr_t’
>>  const sys_call_ptr_t sys_call_table[] ____cacheline_aligned = {
>>  ^
>> arch/x86/um/sys_call_table_64.c:54:2: warning: initialization makes
>> integer from pointer without a cast [enabled by default]
>>   [0 ... __NR_syscall_max] = &sys_ni_syscall,
>>   ^
>> arch/x86/um/sys_call_table_64.c:54:2: warning: (near initialization
>> for ‘sys_call_table[0]’) [enabled by default]
>> arch/x86/um/sys_call_table_64.c:54:2: error: initializer element is
>> not computable at load time
>> arch/x86/um/sys_call_table_64.c:54:2: error: (near initialization for
>> ‘sys_call_table[0]’)
>>
>> Andy, I suspect UML's syscall.h needs the sys_call_ptr_t type too?
>>
> 
> Whoops, yes.  UML has some weird casts in the syscall code that might
> be removable with that change, too.  Want to fix it or should I?

Please send a fix. I'll happily review/test it.

Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web