Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1243328
| From | Ingo Molnar <mingo@kernel.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit |
| Date | 2015-10-09 15:10 +0200 |
| Message-ID | <qhFAS-30e-7@gated-at.bofh.it> (permalink) |
| References | <qgoC6-715-5@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
* Andy Lutomirski <luto@kernel.org> wrote: > The first two patches are optimizations that I'm surprised we didn't > already have. I noticed them when I was looking at the generated > asm. > > The next two patches are tests and some old stuff. There's a test > that validates the vDSO AT_SYSINFO annotations. There's also a test > that exercises some assumptions that signal handling and ptracers > make about syscalls that currently do *not* hold on 64-bit AMD using > 32-bit AT_SYSINFO. > > The next three patches are NT cleanups and a lockdep cleanup. > > It may pay to apply the beginning of the series (at most through > "x86/entry/64/compat: After SYSENTER, move STI after the NT fixup") > without waiting for everyone to wrap their heads around the rest. > > The rest is basically a rewrite of syscalls for all cases except > 64-bit native. With these patches applied, there is a single 32-bit > vDSO and it uses SYSCALL, SYSENTER, and INT80 almost interchangeably > via alternatives. The semantics of SYSENTER and SYSCALL are defined > as: > > 1. If SYSCALL, ESP = ECX > 2. ECX = *ESP > 3. IP = INT80 landing pad > 4. Opportunistic SYSRET/SYSEXIT is enabled on return > > The vDSO is rearranged so that these semantics work. Anything that > backs IP up by 2 ends up pointing at a bona fide int $0x80 > instruction with the expected regs. > > In the process, the vDSO CFI annotations (which are actually used) > get rewritten using normal CFI directives. > > Opportunistic SYSRET/SYSEXIT only happens on return when CS and SS > are as expected, IP points to the INT80 landing pad, and flags are > in good shape. (There is no longer any assumption that full > fast-path 32-bit syscalls don't muck with the registers that matter > for fast exits -- I played with maintaining an optimization like > that with poor results. I may try again if it saves a few cycles.) > > Other than that, the system call entries are simplified to the bare > minimum prologue and a call to a C function. Amusingly, SYSENTER > and SYSCALL32 use the same C function. > > To make that work, I had to remove all the 32-bit syscall stubs > except the clone argument hack. This is because, for C code to call > through the system call table, the system call table entries need to > be real function pointers with C-compatible ABIs. > > There is nothing at all anymore that requires that x86_32 syscalls > be asmlinkage. That could be removed in a subsequent patch. > > The upshot appears to be a ~16 cycle performance hit on 32-bit fast > path syscalls. (On my system, my little prctl test takes 172 cycles > before and 188 cycles with these patches applied.) > > The slow path is probably faster under most circumstances and, if > the exit slow path gets hit, it'll be much faster because (as we > already do in the 64-bit native case) we can still use > SYSEXIT/SYSRET. > > The patchset is structured as a removal of the old fast syscall > code, then the change that makes syscalls into real functions, then > a clean re-implementation of fast syscalls. > > If we want some of the 25 cycles back, we could consider open-coding > a new C fast path. > > Changes from v1: > - The unwind_vdso_32 test now warns on broken Debian installations > instead of failing. The problem is now fully understood, will > be fixed by Debian and possibly also fixed by upstream glibc. > - execve was rather broken in v1. > - It's quite a bit faster now (the optimizations at the end are mostly new). > - int80 on 64-bit no longer clobbers extra regs (thanks Denys!). > - The uaccess stuff is new. > - Lots of other things that I forgot, I'm sure. > > Andy Lutomirski (36): > x86/uaccess: Tell the compiler that uaccess is unlikely to fault > x86/uaccess: __chk_range_not_ok is unlikely to return true > selftests/x86: Add a test for vDSO unwinding > selftests/x86: Add a test for syscall restart and arg modification > x86/entry/64/compat: Fix SYSENTER's NT flag before user memory access > x86/entry: Move lockdep_sys_exit to prepare_exit_to_usermode > x86/entry/64/compat: After SYSENTER, move STI after the NT fixup > x86/vdso: Remove runtime 32-bit vDSO selection > x86/asm: Re-add manual CFI infrastructure > x86/vdso: Define BUILD_VDSO while building and emit .eh_frame in asm > x86/vdso: Replace hex int80 CFI annotations with gas directives > x86/elf/64: Clear more registers in elf_common_init > x86/vdso/32: Save extra registers in the INT80 vsyscall path > x86/entry/64/compat: Disable SYSENTER and SYSCALL32 entries > x86/entry/64/compat: Remove audit optimizations > x86/entry/64/compat: Remove most of the fast system call machinery > x86/entry/64/compat: Set up full pt_regs for all compat syscalls > x86/entry/syscalls: Move syscall table declarations into > asm/syscalls.h > x86/syscalls: Give sys_call_ptr_t a useful type > x86/entry: Add do_syscall_32, a C function to do 32-bit syscalls > x86/entry/64/compat: Migrate the body of the syscall entry to C > x86/entry: Add C code for fast system call entries > x86/vdso/compat: Wire up SYSENTER and SYSCSALL for compat userspace > x86/entry/compat: Implement opportunistic SYSRETL for compat syscalls > x86/entry/32: Open-code return tracking from fork and kthreads > x86/entry/32: Switch INT80 to the new C syscall path > x86/entry/32: Re-implement SYSENTER using the new C path > x86/asm: Remove thread_info.sysenter_return > x86/entry: Remove unnecessary IRQ twiddling in fast 32-bit syscalls > x86/entry: Make irqs_disabled checks in exit code depend on lockdep > x86/entry: Force inlining of 32-bit syscall code > x86/entry: Micro-optimize compat fast syscall arg fetch > x86/entry: Hide two syscall entry assertions behind CONFIG_DEBUG_ENTRY > x86/entry: Use pt_regs_to_thread_info() in syscall entry tracing > x86/entry: Split and inline prepare_exit_to_usermode > x86/entry: Split and inline syscall_return_slowpath > > arch/x86/Makefile | 10 +- > arch/x86/entry/common.c | 255 ++++++++-- > arch/x86/entry/entry_32.S | 184 +++---- > arch/x86/entry/entry_64.S | 9 +- > arch/x86/entry/entry_64_compat.S | 541 +++++---------------- > arch/x86/entry/syscall_32.c | 9 +- > arch/x86/entry/syscall_64.c | 4 +- > arch/x86/entry/syscalls/syscall_32.tbl | 12 +- > arch/x86/entry/vdso/Makefile | 39 +- > arch/x86/entry/vdso/vdso2c.c | 2 +- > arch/x86/entry/vdso/vdso32-setup.c | 28 +- > arch/x86/entry/vdso/vdso32/int80.S | 56 --- > arch/x86/entry/vdso/vdso32/syscall.S | 75 --- > arch/x86/entry/vdso/vdso32/sysenter.S | 116 ----- > arch/x86/entry/vdso/vdso32/system_call.S | 57 +++ > arch/x86/entry/vdso/vma.c | 13 +- > arch/x86/ia32/ia32_signal.c | 4 +- > arch/x86/include/asm/dwarf2.h | 177 +++++++ > arch/x86/include/asm/elf.h | 10 +- > arch/x86/include/asm/syscall.h | 14 +- > arch/x86/include/asm/thread_info.h | 1 - > arch/x86/include/asm/uaccess.h | 14 +- > arch/x86/include/asm/vdso.h | 10 +- > arch/x86/kernel/asm-offsets.c | 3 - > arch/x86/kernel/signal.c | 4 +- > arch/x86/um/sys_call_table_32.c | 7 +- > arch/x86/um/sys_call_table_64.c | 7 +- > arch/x86/xen/setup.c | 13 +- > tools/testing/selftests/x86/Makefile | 5 +- > tools/testing/selftests/x86/ptrace_syscall.c | 294 +++++++++++ > .../testing/selftests/x86/raw_syscall_helper_32.S | 46 ++ > tools/testing/selftests/x86/unwind_vdso.c | 209 ++++++++ > 32 files changed, 1258 insertions(+), 970 deletions(-) > delete mode 100644 arch/x86/entry/vdso/vdso32/int80.S > delete mode 100644 arch/x86/entry/vdso/vdso32/syscall.S > delete mode 100644 arch/x86/entry/vdso/vdso32/sysenter.S > create mode 100644 arch/x86/entry/vdso/vdso32/system_call.S > create mode 100644 arch/x86/include/asm/dwarf2.h > create mode 100644 tools/testing/selftests/x86/ptrace_syscall.c > create mode 100644 tools/testing/selftests/x86/raw_syscall_helper_32.S > create mode 100644 tools/testing/selftests/x86/unwind_vdso.c Ok, so I applied all of them to tip:x86/asm, in two phases, with small (stylistic) edits - it all seems to work fine for me so far, so I pushed it all out to -tip and linux-next. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[PATCH v2 04/36] selftests/x86: Add a test for syscall restart and arg modification Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] selftests/x86: Add a test for ptrace syscall restart and arg modification tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-07 18:20 +0200
[PATCH v2 20/36] x86/entry: Add do_syscall_32, a C function to do 32-bit syscalls Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/entry: Add do_syscall_32(), a C function to do 32-bit syscalls tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
[PATCH v2 13/36] x86/vdso/32: Save extra registers in the INT80 vsyscall path Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/vdso/32: Save extra registers in the INT80 vsyscall path tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:10 +0200
[PATCH v2 11/36] x86/vdso: Replace hex int80 CFI annotations with gas directives Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/vdso: Replace hex int80 CFI annotations with GAS directives tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:10 +0200
[PATCH v2 02/36] x86/uaccess: __chk_range_not_ok is unlikely to return true Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
Re: [PATCH v2 02/36] x86/uaccess: __chk_range_not_ok is unlikely to return true Borislav Petkov <bp@alien8.de> - 2015-10-07 13:00 +0200
Re: [PATCH v2 02/36] x86/uaccess: __chk_range_not_ok is unlikely to return true Ingo Molnar <mingo@kernel.org> - 2015-10-07 18:30 +0200
[tip:x86/asm] x86/uaccess: Add unlikely() to __chk_range_not_ok() failure paths tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-07 18:20 +0200
[PATCH v2 06/36] x86/entry: Move lockdep_sys_exit to prepare_exit_to_usermode Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/entry, locking/lockdep: Move lockdep_sys_exit() to prepare_exit_to_usermode() tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-07 18:20 +0200
Re: [tip:x86/asm] x86/entry, locking/lockdep: Move lockdep_sys_exit() to prepare_exit_to_usermode() Peter Zijlstra <peterz@infradead.org> - 2015-10-08 11:00 +0200
Re: [tip:x86/asm] x86/entry, locking/lockdep: Move lockdep_sys_exit() to prepare_exit_to_usermode() Andy Lutomirski <luto@amacapital.net> - 2015-10-09 21:40 +0200
[PATCH v2 16/36] x86/entry/64/compat: Remove most of the fast system call machinery Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/entry/64/compat: Remove most of the fast system call machinery tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
[PATCH v2 22/36] x86/entry: Add C code for fast system call entries Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Linus Torvalds <torvalds@linux-foundation.org> - 2015-10-06 10:30 +0200
Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Andy Lutomirski <luto@amacapital.net> - 2015-10-06 20:30 +0200
Re: [PATCH v2 22/36] x86/entry: Add C code for fast system call entries Linus Torvalds <torvalds@linux-foundation.org> - 2015-10-06 10:30 +0200
[tip:x86/asm] x86/entry: Add C code for fast system call entries tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
[PATCH v2 30/36] x86/entry: Make irqs_disabled checks in exit code depend on lockdep Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/entry: Make irqs_disabled checks in exit code depend on lockdep tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
[PATCH v2 15/36] x86/entry/64/compat: Remove audit optimizations Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
[tip:x86/asm] x86/entry/64/compat: Remove audit optimizations tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
[PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch Andy Lutomirski <luto@kernel.org> - 2015-10-06 03:00 +0200
Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch Ingo Molnar <mingo@kernel.org> - 2015-10-09 09:40 +0200
Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch Andy Lutomirski <luto@amacapital.net> - 2015-10-09 21:30 +0200
Re: [PATCH v2 32/36] x86/entry: Micro-optimize compat fast syscall arg fetch Ingo Molnar <mingo@kernel.org> - 2015-10-10 11:10 +0200
[tip:x86/asm] x86/entry: Micro-optimize compat fast syscall arg fetch tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2015-10-09 15:20 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Linus Torvalds <torvalds@linux-foundation.org> - 2015-10-06 10:40 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Ingo Molnar <mingo@kernel.org> - 2015-10-06 11:00 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Andy Lutomirski <luto@amacapital.net> - 2015-10-06 20:30 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Ingo Molnar <mingo@kernel.org> - 2015-10-09 15:10 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Richard Weinberger <richard.weinberger@gmail.com> - 2015-10-12 20:40 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Andy Lutomirski <luto@amacapital.net> - 2015-10-12 20:50 +0200
Re: [PATCH v2 00/36] x86: Rewrite all syscall entries except native 64-bit Richard Weinberger <richard@nod.at> - 2015-10-12 23:10 +0200
csiph-web