Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1583444 > unrolled thread
| Started by | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| First post | 2017-02-17 15:20 +0100 |
| Last post | 2017-02-17 15:30 +0100 |
| Articles | 20 on this page of 47 — 10 participants |
Back to article view | Back to linux.kernel
[PATCHv3 00/33] 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 30/33] x86/mm: make kernel_physical_mapping_init() support 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 14/33] x86/kexec: support p4d_t "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 25/33] x86/dump_pagetables: support 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 12/33] x86/mm: add support of p4d_t in vmalloc_fault() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 10/33] x86/gup: add 5-level paging support "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 27/33] x86/espfix: support 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 05/33] asm-generic: introduce <asm-generic/pgtable-nop4d.h> "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 15/33] x86/efi: handle p4d in EFI pagetables "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
Re: [PATCHv3 15/33] x86/efi: handle p4d in EFI pagetables Matt Fleming <matt@codeblueprint.co.uk> - 2017-02-28 13:40 +0100
[PATCHv3 32/33] x86: enable 5-level paging support "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 26/33] x86/kasan: extend to support 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
[PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:20 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Andy Lutomirski <luto@amacapital.net> - 2017-02-17 18:00 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Dmitry Safonov <0x7f454c46@gmail.com> - 2017-02-21 13:00 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-02-21 13:50 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Dave Hansen <dave.hansen@intel.com> - 2017-02-17 18:30 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Andy Lutomirski <luto@amacapital.net> - 2017-02-17 18:30 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Linus Torvalds <torvalds@linux-foundation.org> - 2017-02-17 21:10 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Andy Lutomirski <luto@amacapital.net> - 2017-02-17 21:30 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Linus Torvalds <torvalds@linux-foundation.org> - 2017-02-17 22:10 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Andy Lutomirski <luto@amacapital.net> - 2017-02-18 00:10 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR hpa@zytor.com - 2017-02-18 00:20 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Michael Pratt <mpratt@google.com> - 2017-02-21 07:20 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Catalin Marinas <catalin.marinas@arm.com> - 2017-02-21 11:40 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-02-21 11:50 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Catalin Marinas <catalin.marinas@arm.com> - 2017-02-21 12:00 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Dave Hansen <dave.hansen@intel.com> - 2017-02-17 22:10 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Linus Torvalds <torvalds@linux-foundation.org> - 2017-02-17 22:20 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR hpa@zytor.com - 2017-02-17 23:00 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-02-18 10:30 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-02-20 14:20 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR Dave Hansen <dave.hansen@intel.com> - 2017-02-21 21:50 +0100
Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-02-22 14:10 +0100
[PATCHv3 23/33] x86/paravirt: make paravirt code support 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 13/33] x86/power: support p4d_t in hibernate code "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 28/33] x86/mm: add support of additional page table level during early boot "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 21/33] x86/asm: remove __VIRTUAL_MASK_SHIFT==47 assert "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 04/33] arch, mm: convert all architectures to use 5level-fixup.h "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 01/33] x86/cpufeature: Add 5-level paging detection "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 08/33] x86: basic changes into headers for 5-level paging "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 03/33] asm-generic: introduce __ARCH_USE_5LEVEL_HACK "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 07/33] mm: introduce __p4d_alloc() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 16/33] x86/mm/pat: handle additional page table "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 19/33] x86: convert the rest of the code to support p4d_t "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 11/33] x86/ident_map: add 5-level paging support "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
[PATCHv3 18/33] x86/xen: convert __xen_pgd_walk() and xen_cleanmfnmap() to support p4d "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-02-17 15:30 +0100
Page 1 of 3 [1] 2 3 Next page →
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 00/33] 5-level paging |
| Message-ID | <tbRy9-5mq-3@gated-at.bofh.it> |
Here is v3 of 5-level paging patchset. Please review and consider applying.
== Overview ==
x86-64 is currently limited to 256 TiB of virtual address space and 64 TiB
of physical address space. We are already bumping into this limit: some
vendors offers servers with 64 TiB of memory today.
To overcome the limitation upcoming hardware will introduce support for
5-level paging[1]. It is a straight-forward extension of the current page
table structure adding one more layer of translation.
It bumps the limits to 128 PiB of virtual address space and 4 PiB of
physical address space. This "ought to be enough for anybody" ©.
== Patches ==
The patchset is build on top of v4.10-rc8.
Current QEMU upstream git supports 5-level paging. Use "-cpu qemu64,+la57"
to enable it.
Patch 1:
Detect la57 feature for /proc/cpuinfo.
Patches 2-7:
Brings 5-level paging to generic code and convert all
architectures to it using <asm-generic/5level-fixup.h>
Patches 8-19:
Convert x86 to properly folded p4d layer using
<asm-generic/pgtable-nop4d.h>.
Patches 20-32:
Enabling of real 5-level paging.
CONFIG_X86_5LEVEL=y will enable new paging mode.
Patch 33:
Introduce new prctl(2) handles -- PR_SET_MAX_VADDR and PR_GET_MAX_VADDR.
This aims to address compatibility issue. Only supports x86 for
now, but should be useful for other archtectures.
Git:
git://git.kernel.org/pub/scm/linux/kernel/git/kas/linux.git la57/v3
== TODO ==
There is still work to do:
- CONFIG_XEN is broken for 5-level paging.
Xen for 5-level paging requires more work to get functional.
Xen on 4-level paging works, so it's not a regression.
- Boot-time switch between 4- and 5-level paging.
We assume that distributions will be keen to avoid returning to the
i386 days where we shipped one kernel binary for each page table
layout.
As page table format is the same for 4- and 5-level paging it should
be possible to have single kernel binary and switch between them at
boot-time without too much hassle.
For now I only implemented compile-time switch.
This will implemented with separate patchset.
== Changelong ==
v3:
- Rebased to v4.10-rc5;
- prctl() handles for large address space opt-in;
- Xen works for 4-level paging;
- EFI boot fixed for both 4- and 5-level paging;
- Hibernation fixed for 4-level paging;
- kexec() fixed;
- Couple of build fixes;
v2:
- Rebased to v4.10-rc1;
- RLIMIT_VADDR proposal;
- Fix virtual map and update documentation;
- Fix few build errors;
- Rework cpuid helpers in boot code;
- Fix espfix code to work with 5-level pages;
[1] https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf
Kirill A. Shutemov (33):
x86/cpufeature: Add 5-level paging detection
asm-generic: introduce 5level-fixup.h
asm-generic: introduce __ARCH_USE_5LEVEL_HACK
arch, mm: convert all architectures to use 5level-fixup.h
asm-generic: introduce <asm-generic/pgtable-nop4d.h>
mm: convert generic code to 5-level paging
mm: introduce __p4d_alloc()
x86: basic changes into headers for 5-level paging
x86: trivial portion of 5-level paging conversion
x86/gup: add 5-level paging support
x86/ident_map: add 5-level paging support
x86/mm: add support of p4d_t in vmalloc_fault()
x86/power: support p4d_t in hibernate code
x86/kexec: support p4d_t
x86/efi: handle p4d in EFI pagetables
x86/mm/pat: handle additional page table
x86/kasan: prepare clear_pgds() to switch to
<asm-generic/pgtable-nop4d.h>
x86/xen: convert __xen_pgd_walk() and xen_cleanmfnmap() to support p4d
x86: convert the rest of the code to support p4d_t
x86: detect 5-level paging support
x86/asm: remove __VIRTUAL_MASK_SHIFT==47 assert
x86/mm: define virtual memory map for 5-level paging
x86/paravirt: make paravirt code support 5-level paging
x86/mm: basic defines/helpers for CONFIG_X86_5LEVEL
x86/dump_pagetables: support 5-level paging
x86/kasan: extend to support 5-level paging
x86/espfix: support 5-level paging
x86/mm: add support of additional page table level during early boot
x86/mm: add sync_global_pgds() for configuration with 5-level paging
x86/mm: make kernel_physical_mapping_init() support 5-level paging
x86/mm: add support for 5-level paging for KASLR
x86: enable 5-level paging support
mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR
Documentation/x86/x86_64/mm.txt | 33 +-
arch/arc/include/asm/hugepage.h | 1 +
arch/arc/include/asm/pgtable.h | 1 +
arch/arm/include/asm/pgtable.h | 1 +
arch/arm64/include/asm/pgtable-types.h | 4 +
arch/avr32/include/asm/pgtable-2level.h | 1 +
arch/cris/include/asm/pgtable.h | 1 +
arch/frv/include/asm/pgtable.h | 1 +
arch/h8300/include/asm/pgtable.h | 1 +
arch/hexagon/include/asm/pgtable.h | 1 +
arch/ia64/include/asm/pgtable.h | 2 +
arch/metag/include/asm/pgtable.h | 1 +
arch/mips/include/asm/pgtable-32.h | 1 +
arch/mips/include/asm/pgtable-64.h | 1 +
arch/mn10300/include/asm/page.h | 1 +
arch/nios2/include/asm/pgtable.h | 1 +
arch/openrisc/include/asm/pgtable.h | 1 +
arch/powerpc/include/asm/book3s/32/pgtable.h | 1 +
arch/powerpc/include/asm/book3s/64/pgtable.h | 2 +
arch/powerpc/include/asm/nohash/32/pgtable.h | 1 +
arch/powerpc/include/asm/nohash/64/pgtable-4k.h | 3 +
arch/powerpc/include/asm/nohash/64/pgtable-64k.h | 1 +
arch/s390/include/asm/pgtable.h | 1 +
arch/score/include/asm/pgtable.h | 1 +
arch/sh/include/asm/pgtable-2level.h | 1 +
arch/sh/include/asm/pgtable-3level.h | 1 +
arch/sparc/include/asm/pgtable_64.h | 1 +
arch/tile/include/asm/pgtable_32.h | 1 +
arch/tile/include/asm/pgtable_64.h | 1 +
arch/um/include/asm/pgtable-2level.h | 1 +
arch/um/include/asm/pgtable-3level.h | 1 +
arch/unicore32/include/asm/pgtable.h | 1 +
arch/x86/Kconfig | 6 +
arch/x86/boot/compressed/head_64.S | 23 +-
arch/x86/boot/cpucheck.c | 9 +
arch/x86/boot/cpuflags.c | 12 +-
arch/x86/entry/entry_64.S | 7 +-
arch/x86/include/asm/cpufeatures.h | 3 +-
arch/x86/include/asm/disabled-features.h | 8 +-
arch/x86/include/asm/elf.h | 2 +-
arch/x86/include/asm/kasan.h | 9 +-
arch/x86/include/asm/kexec.h | 1 +
arch/x86/include/asm/mmu.h | 2 +
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/page_64_types.h | 10 +
arch/x86/include/asm/paravirt.h | 65 +++-
arch/x86/include/asm/paravirt_types.h | 17 +-
arch/x86/include/asm/pgalloc.h | 37 +-
arch/x86/include/asm/pgtable-2level_types.h | 1 +
arch/x86/include/asm/pgtable-3level_types.h | 1 +
arch/x86/include/asm/pgtable.h | 85 ++++-
arch/x86/include/asm/pgtable_64.h | 29 +-
arch/x86/include/asm/pgtable_64_types.h | 27 ++
arch/x86/include/asm/pgtable_types.h | 42 ++-
arch/x86/include/asm/processor.h | 25 +-
arch/x86/include/asm/required-features.h | 8 +-
arch/x86/include/asm/sparsemem.h | 9 +-
arch/x86/include/asm/xen/page.h | 8 +-
arch/x86/include/uapi/asm/processor-flags.h | 2 +
arch/x86/kernel/espfix_64.c | 12 +-
arch/x86/kernel/head64.c | 40 ++-
arch/x86/kernel/head_64.S | 63 +++-
arch/x86/kernel/machine_kexec_32.c | 4 +-
arch/x86/kernel/machine_kexec_64.c | 16 +-
arch/x86/kernel/paravirt.c | 13 +-
arch/x86/kernel/process.c | 18 +
arch/x86/kernel/sys_x86_64.c | 6 +-
arch/x86/kernel/tboot.c | 6 +-
arch/x86/kernel/vm86_32.c | 6 +-
arch/x86/mm/dump_pagetables.c | 51 ++-
arch/x86/mm/fault.c | 57 ++-
arch/x86/mm/gup.c | 33 +-
arch/x86/mm/hugetlbpage.c | 8 +-
arch/x86/mm/ident_map.c | 47 ++-
arch/x86/mm/init_32.c | 22 +-
arch/x86/mm/init_64.c | 269 ++++++++++++--
arch/x86/mm/ioremap.c | 3 +-
arch/x86/mm/kasan_init_64.c | 41 ++-
arch/x86/mm/kaslr.c | 82 ++++-
arch/x86/mm/mmap.c | 4 +-
arch/x86/mm/mpx.c | 17 +-
arch/x86/mm/pageattr.c | 56 ++-
arch/x86/mm/pgtable.c | 38 +-
arch/x86/mm/pgtable_32.c | 8 +-
arch/x86/platform/efi/efi_64.c | 38 +-
arch/x86/power/hibernate_32.c | 7 +-
arch/x86/power/hibernate_64.c | 49 ++-
arch/x86/realmode/init.c | 2 +-
arch/x86/xen/Kconfig | 1 +
arch/x86/xen/mmu.c | 433 ++++++++++++++---------
arch/x86/xen/mmu.h | 1 +
arch/xtensa/include/asm/pgtable.h | 1 +
drivers/misc/sgi-gru/grufault.c | 9 +-
fs/binfmt_aout.c | 2 -
fs/binfmt_elf.c | 10 +-
fs/hugetlbfs/inode.c | 6 +-
fs/userfaultfd.c | 6 +-
include/asm-generic/4level-fixup.h | 3 +-
include/asm-generic/5level-fixup.h | 41 +++
include/asm-generic/pgtable-nop4d-hack.h | 62 ++++
include/asm-generic/pgtable-nop4d.h | 56 +++
include/asm-generic/pgtable-nopud.h | 48 +--
include/asm-generic/pgtable.h | 48 ++-
include/asm-generic/tlb.h | 14 +-
include/linux/hugetlb.h | 5 +-
include/linux/kasan.h | 1 +
include/linux/mm.h | 34 +-
include/linux/sched.h | 8 +
include/trace/events/xen.h | 28 +-
include/uapi/linux/prctl.h | 3 +
kernel/events/uprobes.c | 5 +-
kernel/sys.c | 23 +-
lib/ioremap.c | 39 +-
mm/gup.c | 46 ++-
mm/huge_memory.c | 7 +-
mm/hugetlb.c | 29 +-
mm/kasan/kasan_init.c | 35 +-
mm/memory.c | 230 ++++++++++--
mm/mlock.c | 1 +
mm/mmap.c | 20 +-
mm/mprotect.c | 26 +-
mm/mremap.c | 16 +-
mm/nommu.c | 2 +-
mm/pagewalk.c | 32 +-
mm/pgtable-generic.c | 6 +
mm/rmap.c | 13 +-
mm/shmem.c | 8 +-
mm/sparse-vmemmap.c | 22 +-
mm/swapfile.c | 26 +-
mm/userfaultfd.c | 23 +-
mm/vmalloc.c | 81 +++--
131 files changed, 2435 insertions(+), 611 deletions(-)
create mode 100644 include/asm-generic/5level-fixup.h
create mode 100644 include/asm-generic/pgtable-nop4d-hack.h
create mode 100644 include/asm-generic/pgtable-nop4d.h
--
2.11.0
[toc] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 30/33] x86/mm: make kernel_physical_mapping_init() support 5-level paging |
| Message-ID | <tbRya-5mq-29@gated-at.bofh.it> |
| In reply to | #1583444 |
Properly populate addition pagetable level if CONFIG_X86_5LEVEL is
enabled.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/x86/mm/init_64.c | 71 ++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 62 insertions(+), 9 deletions(-)
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 58be5713ec09..d53eda53419d 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -622,6 +622,58 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_end,
return paddr_last;
}
+static unsigned long __meminit
+phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, unsigned long paddr_end,
+ unsigned long page_size_mask)
+{
+ unsigned long paddr_next, paddr_last = paddr_end;
+ unsigned long vaddr = (unsigned long)__va(paddr);
+ int i = p4d_index(vaddr);
+
+ if (!IS_ENABLED(CONFIG_X86_5LEVEL))
+ return phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, page_size_mask);
+
+ for (; i < PTRS_PER_P4D; i++, paddr = paddr_next) {
+ p4d_t *p4d;
+ pud_t *pud;
+
+ vaddr = (unsigned long)__va(paddr);
+ p4d = p4d_page + p4d_index(vaddr);
+ paddr_next = (paddr & P4D_MASK) + P4D_SIZE;
+
+ if (paddr >= paddr_end) {
+ if (!after_bootmem &&
+ !e820_any_mapped(paddr & P4D_MASK, paddr_next,
+ E820_RAM) &&
+ !e820_any_mapped(paddr & P4D_MASK, paddr_next,
+ E820_RESERVED_KERN)) {
+ set_p4d(p4d, __p4d(0));
+ }
+ continue;
+ }
+
+ if (!p4d_none(*p4d)) {
+ pud = pud_offset(p4d, 0);
+ paddr_last = phys_pud_init(pud, paddr,
+ paddr_end,
+ page_size_mask);
+ __flush_tlb_all();
+ continue;
+ }
+
+ pud = alloc_low_page();
+ paddr_last = phys_pud_init(pud, paddr, paddr_end,
+ page_size_mask);
+
+ spin_lock(&init_mm.page_table_lock);
+ p4d_populate(&init_mm, p4d, pud);
+ spin_unlock(&init_mm.page_table_lock);
+ }
+ __flush_tlb_all();
+
+ return paddr_last;
+}
+
/*
* Create page table mapping for the physical memory for specific physical
* addresses. The virtual and physical addresses have to be aligned on PMD level
@@ -643,26 +695,27 @@ kernel_physical_mapping_init(unsigned long paddr_start,
for (; vaddr < vaddr_end; vaddr = vaddr_next) {
pgd_t *pgd = pgd_offset_k(vaddr);
p4d_t *p4d;
- pud_t *pud;
vaddr_next = (vaddr & PGDIR_MASK) + PGDIR_SIZE;
- BUILD_BUG_ON(pgd_none(*pgd));
- p4d = p4d_offset(pgd, vaddr);
- if (p4d_val(*p4d)) {
- pud = (pud_t *)p4d_page_vaddr(*p4d);
- paddr_last = phys_pud_init(pud, __pa(vaddr),
+ if (pgd_val(*pgd)) {
+ p4d = (p4d_t *)pgd_page_vaddr(*pgd);
+ paddr_last = phys_p4d_init(p4d, __pa(vaddr),
__pa(vaddr_end),
page_size_mask);
continue;
}
- pud = alloc_low_page();
- paddr_last = phys_pud_init(pud, __pa(vaddr), __pa(vaddr_end),
+ p4d = alloc_low_page();
+ paddr_last = phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end),
page_size_mask);
spin_lock(&init_mm.page_table_lock);
- p4d_populate(&init_mm, p4d, pud);
+ if (IS_ENABLED(CONFIG_X86_5LEVEL))
+ pgd_populate(&init_mm, pgd, p4d);
+ else
+ p4d_populate(&init_mm, p4d_offset(pgd, vaddr),
+ (pud_t *) p4d);
spin_unlock(&init_mm.page_table_lock);
pgd_changed = true;
}
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 14/33] x86/kexec: support p4d_t |
| Message-ID | <tbRya-5mq-35@gated-at.bofh.it> |
| In reply to | #1583444 |
Handle additional page table level in kexec code.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/x86/include/asm/kexec.h | 1 +
arch/x86/kernel/machine_kexec_32.c | 4 +++-
arch/x86/kernel/machine_kexec_64.c | 14 ++++++++++++--
3 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 282630e4c6ea..70ef205489f0 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -164,6 +164,7 @@ struct kimage_arch {
};
#else
struct kimage_arch {
+ p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
pte_t *pte;
diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c
index 469b23d6acc2..5f43cec296c5 100644
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -103,6 +103,7 @@ static void machine_kexec_page_table_set_one(
pgd_t *pgd, pmd_t *pmd, pte_t *pte,
unsigned long vaddr, unsigned long paddr)
{
+ p4d_t *p4d;
pud_t *pud;
pgd += pgd_index(vaddr);
@@ -110,7 +111,8 @@ static void machine_kexec_page_table_set_one(
if (!(pgd_val(*pgd) & _PAGE_PRESENT))
set_pgd(pgd, __pgd(__pa(pmd) | _PAGE_PRESENT));
#endif
- pud = pud_offset(pgd, vaddr);
+ p4d = p4d_offset(pgd, vaddr);
+ pud = pud_offset(p4d, vaddr);
pmd = pmd_offset(pud, vaddr);
if (!(pmd_val(*pmd) & _PAGE_PRESENT))
set_pmd(pmd, __pmd(__pa(pte) | _PAGE_TABLE));
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 307b1f4543de..42eae96c8450 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -36,6 +36,7 @@ static struct kexec_file_ops *kexec_file_loaders[] = {
static void free_transition_pgtable(struct kimage *image)
{
+ free_page((unsigned long)image->arch.p4d);
free_page((unsigned long)image->arch.pud);
free_page((unsigned long)image->arch.pmd);
free_page((unsigned long)image->arch.pte);
@@ -43,6 +44,7 @@ static void free_transition_pgtable(struct kimage *image)
static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
{
+ p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
pte_t *pte;
@@ -53,13 +55,21 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE);
pgd += pgd_index(vaddr);
if (!pgd_present(*pgd)) {
+ p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);
+ if (!p4d)
+ goto err;
+ image->arch.p4d = p4d;
+ set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
+ }
+ p4d = p4d_offset(pgd, vaddr);
+ if (!p4d_present(*p4d)) {
pud = (pud_t *)get_zeroed_page(GFP_KERNEL);
if (!pud)
goto err;
image->arch.pud = pud;
- set_pgd(pgd, __pgd(__pa(pud) | _KERNPG_TABLE));
+ set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
}
- pud = pud_offset(pgd, vaddr);
+ pud = pud_offset(p4d, vaddr);
if (!pud_present(*pud)) {
pmd = (pmd_t *)get_zeroed_page(GFP_KERNEL);
if (!pmd)
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 25/33] x86/dump_pagetables: support 5-level paging |
| Message-ID | <tbRya-5mq-33@gated-at.bofh.it> |
| In reply to | #1583444 |
Simple extension to support one more page table level.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/x86/mm/dump_pagetables.c | 49 ++++++++++++++++++++++++++++++++++++-------
1 file changed, 42 insertions(+), 7 deletions(-)
diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c
index 8aa6bea1cd6c..874afcc6af9e 100644
--- a/arch/x86/mm/dump_pagetables.c
+++ b/arch/x86/mm/dump_pagetables.c
@@ -101,7 +101,8 @@ static struct addr_marker address_markers[] = {
#define PTE_LEVEL_MULT (PAGE_SIZE)
#define PMD_LEVEL_MULT (PTRS_PER_PTE * PTE_LEVEL_MULT)
#define PUD_LEVEL_MULT (PTRS_PER_PMD * PMD_LEVEL_MULT)
-#define PGD_LEVEL_MULT (PTRS_PER_PUD * PUD_LEVEL_MULT)
+#define P4D_LEVEL_MULT (PTRS_PER_PUD * PUD_LEVEL_MULT)
+#define PGD_LEVEL_MULT (PTRS_PER_PUD * P4D_LEVEL_MULT)
#define pt_dump_seq_printf(m, to_dmesg, fmt, args...) \
({ \
@@ -327,14 +328,14 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr,
#if PTRS_PER_PUD > 1
-static void walk_pud_level(struct seq_file *m, struct pg_state *st, pgd_t addr,
+static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr,
unsigned long P)
{
int i;
pud_t *start;
pgprotval_t prot;
- start = (pud_t *) pgd_page_vaddr(addr);
+ start = (pud_t *) p4d_page_vaddr(addr);
for (i = 0; i < PTRS_PER_PUD; i++) {
st->current_address = normalize_addr(P + i * PUD_LEVEL_MULT);
@@ -354,9 +355,43 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, pgd_t addr,
}
#else
-#define walk_pud_level(m,s,a,p) walk_pmd_level(m,s,__pud(pgd_val(a)),p)
-#define pgd_large(a) pud_large(__pud(pgd_val(a)))
-#define pgd_none(a) pud_none(__pud(pgd_val(a)))
+#define walk_pud_level(m,s,a,p) walk_pmd_level(m,s,__pud(p4d_val(a)),p)
+#define p4d_large(a) pud_large(__pud(p4d_val(a)))
+#define p4d_none(a) pud_none(__pud(p4d_val(a)))
+#endif
+
+#if PTRS_PER_P4D > 1
+
+static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr,
+ unsigned long P)
+{
+ int i;
+ p4d_t *start;
+ pgprotval_t prot;
+
+ start = (p4d_t *) pgd_page_vaddr(addr);
+
+ for (i = 0; i < PTRS_PER_P4D; i++) {
+ st->current_address = normalize_addr(P + i * P4D_LEVEL_MULT);
+ if (!p4d_none(*start)) {
+ if (p4d_large(*start) || !p4d_present(*start)) {
+ prot = p4d_flags(*start);
+ note_page(m, st, __pgprot(prot), 2);
+ } else {
+ walk_pud_level(m, st, *start,
+ P + i * P4D_LEVEL_MULT);
+ }
+ } else
+ note_page(m, st, __pgprot(0), 2);
+
+ start++;
+ }
+}
+
+#else
+#define walk_p4d_level(m,s,a,p) walk_pud_level(m,s,__p4d(pgd_val(a)),p)
+#define pgd_large(a) p4d_large(__p4d(pgd_val(a)))
+#define pgd_none(a) p4d_none(__p4d(pgd_val(a)))
#endif
static inline bool is_hypervisor_range(int idx)
@@ -401,7 +436,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd,
prot = pgd_flags(*start);
note_page(m, &st, __pgprot(prot), 1);
} else {
- walk_pud_level(m, &st, *start,
+ walk_p4d_level(m, &st, *start,
i * PGD_LEVEL_MULT);
}
} else
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 12/33] x86/mm: add support of p4d_t in vmalloc_fault() |
| Message-ID | <tbRyb-5mq-41@gated-at.bofh.it> |
| In reply to | #1583444 |
With 4-level paging copying happens on p4d level, as we have pgd_none()
always false when p4d_t folded.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/x86/mm/fault.c | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 59104b78e8a7..b4bcf2dac4a9 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -434,6 +434,7 @@ void vmalloc_sync_all(void)
static noinline int vmalloc_fault(unsigned long address)
{
pgd_t *pgd, *pgd_ref;
+ p4d_t *p4d, *p4d_ref;
pud_t *pud, *pud_ref;
pmd_t *pmd, *pmd_ref;
pte_t *pte, *pte_ref;
@@ -461,13 +462,26 @@ static noinline int vmalloc_fault(unsigned long address)
BUG_ON(pgd_page_vaddr(*pgd) != pgd_page_vaddr(*pgd_ref));
}
+ /* With 4-level paging copying happens on p4d level. */
+ p4d = p4d_offset(pgd, address);
+ p4d_ref = p4d_offset(pgd_ref, address);
+ if (p4d_none(*p4d_ref))
+ return -1;
+
+ if (p4d_none(*p4d)) {
+ set_p4d(p4d, *p4d_ref);
+ arch_flush_lazy_mmu_mode();
+ } else {
+ BUG_ON(p4d_pfn(*p4d) != p4d_pfn(*p4d_ref));
+ }
+
/*
* Below here mismatches are bugs because these lower tables
* are shared:
*/
- pud = pud_offset(pgd, address);
- pud_ref = pud_offset(pgd_ref, address);
+ pud = pud_offset(p4d, address);
+ pud_ref = pud_offset(p4d_ref, address);
if (pud_none(*pud_ref))
return -1;
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 10/33] x86/gup: add 5-level paging support |
| Message-ID | <tbRyb-5mq-57@gated-at.bofh.it> |
| In reply to | #1583444 |
It's simply extension for one more page table level.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/x86/mm/gup.c | 33 +++++++++++++++++++++++++++------
1 file changed, 27 insertions(+), 6 deletions(-)
diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c
index 0d4fb3ebbbac..27b92430a8cd 100644
--- a/arch/x86/mm/gup.c
+++ b/arch/x86/mm/gup.c
@@ -76,9 +76,9 @@ static void undo_dev_pagemap(int *nr, int nr_start, struct page **pages)
}
/*
- * 'pteval' can come from a pte, pmd or pud. We only check
+ * 'pteval' can come from a pte, pmd, pud or p4d. We only check
* _PAGE_PRESENT, _PAGE_USER, and _PAGE_RW in here which are the
- * same value on all 3 types.
+ * same value on all 4 types.
*/
static inline int pte_allows_gup(unsigned long pteval, int write)
{
@@ -270,13 +270,13 @@ static noinline int gup_huge_pud(pud_t pud, unsigned long addr,
return 1;
}
-static int gup_pud_range(pgd_t pgd, unsigned long addr, unsigned long end,
+static int gup_pud_range(p4d_t p4d, unsigned long addr, unsigned long end,
int write, struct page **pages, int *nr)
{
unsigned long next;
pud_t *pudp;
- pudp = pud_offset(&pgd, addr);
+ pudp = pud_offset(&p4d, addr);
do {
pud_t pud = *pudp;
@@ -295,6 +295,27 @@ static int gup_pud_range(pgd_t pgd, unsigned long addr, unsigned long end,
return 1;
}
+static int gup_p4d_range(pgd_t pgd, unsigned long addr, unsigned long end,
+ int write, struct page **pages, int *nr)
+{
+ unsigned long next;
+ p4d_t *p4dp;
+
+ p4dp = p4d_offset(&pgd, addr);
+ do {
+ p4d_t p4d = *p4dp;
+
+ next = p4d_addr_end(addr, end);
+ if (p4d_none(p4d))
+ return 0;
+ BUILD_BUG_ON(p4d_large(p4d));
+ if (!gup_pud_range(p4d, addr, next, write, pages, nr))
+ return 0;
+ } while (p4dp++, addr = next, addr != end);
+
+ return 1;
+}
+
/*
* Like get_user_pages_fast() except its IRQ-safe in that it won't fall
* back to the regular GUP.
@@ -343,7 +364,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
next = pgd_addr_end(addr, end);
if (pgd_none(pgd))
break;
- if (!gup_pud_range(pgd, addr, next, write, pages, &nr))
+ if (!gup_p4d_range(pgd, addr, next, write, pages, &nr))
break;
} while (pgdp++, addr = next, addr != end);
local_irq_restore(flags);
@@ -415,7 +436,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,
next = pgd_addr_end(addr, end);
if (pgd_none(pgd))
goto slow;
- if (!gup_pud_range(pgd, addr, next, write, pages, &nr))
+ if (!gup_p4d_range(pgd, addr, next, write, pages, &nr))
goto slow;
} while (pgdp++, addr = next, addr != end);
local_irq_enable();
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 27/33] x86/espfix: support 5-level paging |
| Message-ID | <tbRyb-5mq-45@gated-at.bofh.it> |
| In reply to | #1583444 |
We don't need extra virtual address space for ESPFIX, so it stays within
one PUD page table for both 4- and 5-level paging.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/x86/kernel/espfix_64.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c
index 04f89caef9c4..8e598a1ad986 100644
--- a/arch/x86/kernel/espfix_64.c
+++ b/arch/x86/kernel/espfix_64.c
@@ -50,11 +50,11 @@
#define ESPFIX_STACKS_PER_PAGE (PAGE_SIZE/ESPFIX_STACK_SIZE)
/* There is address space for how many espfix pages? */
-#define ESPFIX_PAGE_SPACE (1UL << (PGDIR_SHIFT-PAGE_SHIFT-16))
+#define ESPFIX_PAGE_SPACE (1UL << (P4D_SHIFT-PAGE_SHIFT-16))
#define ESPFIX_MAX_CPUS (ESPFIX_STACKS_PER_PAGE * ESPFIX_PAGE_SPACE)
#if CONFIG_NR_CPUS > ESPFIX_MAX_CPUS
-# error "Need more than one PGD for the ESPFIX hack"
+# error "Need more virtual address space for the ESPFIX hack"
#endif
#define PGALLOC_GFP (GFP_KERNEL | __GFP_NOTRACK | __GFP_ZERO)
@@ -121,11 +121,13 @@ static void init_espfix_random(void)
void __init init_espfix_bsp(void)
{
- pgd_t *pgd_p;
+ pgd_t *pgd;
+ p4d_t *p4d;
/* Install the espfix pud into the kernel page directory */
- pgd_p = &init_level4_pgt[pgd_index(ESPFIX_BASE_ADDR)];
- pgd_populate(&init_mm, pgd_p, (pud_t *)espfix_pud_page);
+ pgd = &init_level4_pgt[pgd_index(ESPFIX_BASE_ADDR)];
+ p4d = p4d_alloc(&init_mm, pgd, ESPFIX_BASE_ADDR);
+ p4d_populate(&init_mm, p4d, espfix_pud_page);
/* Randomize the locations */
init_espfix_random();
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 05/33] asm-generic: introduce <asm-generic/pgtable-nop4d.h> |
| Message-ID | <tbRyb-5mq-51@gated-at.bofh.it> |
| In reply to | #1583444 |
Like with pgtable-nopud.h for 4-level paging, this new header is base
for converting an architectures to properly folded p4d_t level.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
include/asm-generic/pgtable-nop4d.h | 56 +++++++++++++++++++++++++++++++++++++
include/asm-generic/pgtable-nopud.h | 43 ++++++++++++++--------------
include/asm-generic/tlb.h | 14 ++++++++--
3 files changed, 89 insertions(+), 24 deletions(-)
create mode 100644 include/asm-generic/pgtable-nop4d.h
diff --git a/include/asm-generic/pgtable-nop4d.h b/include/asm-generic/pgtable-nop4d.h
new file mode 100644
index 000000000000..de364ecb8df6
--- /dev/null
+++ b/include/asm-generic/pgtable-nop4d.h
@@ -0,0 +1,56 @@
+#ifndef _PGTABLE_NOP4D_H
+#define _PGTABLE_NOP4D_H
+
+#ifndef __ASSEMBLY__
+
+#define __PAGETABLE_P4D_FOLDED
+
+typedef struct { pgd_t pgd; } p4d_t;
+
+#define P4D_SHIFT PGDIR_SHIFT
+#define PTRS_PER_P4D 1
+#define P4D_SIZE (1UL << P4D_SHIFT)
+#define P4D_MASK (~(P4D_SIZE-1))
+
+/*
+ * The "pgd_xxx()" functions here are trivial for a folded two-level
+ * setup: the p4d is never bad, and a p4d always exists (as it's folded
+ * into the pgd entry)
+ */
+static inline int pgd_none(pgd_t pgd) { return 0; }
+static inline int pgd_bad(pgd_t pgd) { return 0; }
+static inline int pgd_present(pgd_t pgd) { return 1; }
+static inline void pgd_clear(pgd_t *pgd) { }
+#define p4d_ERROR(p4d) (pgd_ERROR((p4d).pgd))
+
+#define pgd_populate(mm, pgd, p4d) do { } while (0)
+/*
+ * (p4ds are folded into pgds so this doesn't get actually called,
+ * but the define is needed for a generic inline function.)
+ */
+#define set_pgd(pgdptr, pgdval) set_p4d((p4d_t *)(pgdptr), (p4d_t) { pgdval })
+
+static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)
+{
+ return (p4d_t *)pgd;
+}
+
+#define p4d_val(x) (pgd_val((x).pgd))
+#define __p4d(x) ((p4d_t) { __pgd(x) })
+
+#define pgd_page(pgd) (p4d_page((p4d_t){ pgd }))
+#define pgd_page_vaddr(pgd) (p4d_page_vaddr((p4d_t){ pgd }))
+
+/*
+ * allocating and freeing a p4d is trivial: the 1-entry p4d is
+ * inside the pgd, so has no extra memory associated with it.
+ */
+#define p4d_alloc_one(mm, address) NULL
+#define p4d_free(mm, x) do { } while (0)
+#define __p4d_free_tlb(tlb, x, a) do { } while (0)
+
+#undef p4d_addr_end
+#define p4d_addr_end(addr, end) (end)
+
+#endif /* __ASSEMBLY__ */
+#endif /* _PGTABLE_NOP4D_H */
diff --git a/include/asm-generic/pgtable-nopud.h b/include/asm-generic/pgtable-nopud.h
index 5e49430a30a4..c2b9b96d6268 100644
--- a/include/asm-generic/pgtable-nopud.h
+++ b/include/asm-generic/pgtable-nopud.h
@@ -6,53 +6,54 @@
#ifdef __ARCH_USE_5LEVEL_HACK
#include <asm-generic/pgtable-nop4d-hack.h>
#else
+#include <asm-generic/pgtable-nop4d.h>
#define __PAGETABLE_PUD_FOLDED
/*
- * Having the pud type consist of a pgd gets the size right, and allows
- * us to conceptually access the pgd entry that this pud is folded into
+ * Having the pud type consist of a p4d gets the size right, and allows
+ * us to conceptually access the p4d entry that this pud is folded into
* without casting.
*/
-typedef struct { pgd_t pgd; } pud_t;
+typedef struct { p4d_t p4d; } pud_t;
-#define PUD_SHIFT PGDIR_SHIFT
+#define PUD_SHIFT P4D_SHIFT
#define PTRS_PER_PUD 1
#define PUD_SIZE (1UL << PUD_SHIFT)
#define PUD_MASK (~(PUD_SIZE-1))
/*
- * The "pgd_xxx()" functions here are trivial for a folded two-level
+ * The "p4d_xxx()" functions here are trivial for a folded two-level
* setup: the pud is never bad, and a pud always exists (as it's folded
- * into the pgd entry)
+ * into the p4d entry)
*/
-static inline int pgd_none(pgd_t pgd) { return 0; }
-static inline int pgd_bad(pgd_t pgd) { return 0; }
-static inline int pgd_present(pgd_t pgd) { return 1; }
-static inline void pgd_clear(pgd_t *pgd) { }
-#define pud_ERROR(pud) (pgd_ERROR((pud).pgd))
+static inline int p4d_none(p4d_t p4d) { return 0; }
+static inline int p4d_bad(p4d_t p4d) { return 0; }
+static inline int p4d_present(p4d_t p4d) { return 1; }
+static inline void p4d_clear(p4d_t *p4d) { }
+#define pud_ERROR(pud) (p4d_ERROR((pud).p4d))
-#define pgd_populate(mm, pgd, pud) do { } while (0)
+#define p4d_populate(mm, p4d, pud) do { } while (0)
/*
- * (puds are folded into pgds so this doesn't get actually called,
+ * (puds are folded into p4ds so this doesn't get actually called,
* but the define is needed for a generic inline function.)
*/
-#define set_pgd(pgdptr, pgdval) set_pud((pud_t *)(pgdptr), (pud_t) { pgdval })
+#define set_p4d(p4dptr, p4dval) set_pud((pud_t *)(p4dptr), (pud_t) { p4dval })
-static inline pud_t * pud_offset(pgd_t * pgd, unsigned long address)
+static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
{
- return (pud_t *)pgd;
+ return (pud_t *)p4d;
}
-#define pud_val(x) (pgd_val((x).pgd))
-#define __pud(x) ((pud_t) { __pgd(x) } )
+#define pud_val(x) (p4d_val((x).p4d))
+#define __pud(x) ((pud_t) { __p4d(x) })
-#define pgd_page(pgd) (pud_page((pud_t){ pgd }))
-#define pgd_page_vaddr(pgd) (pud_page_vaddr((pud_t){ pgd }))
+#define p4d_page(p4d) (pud_page((pud_t){ p4d }))
+#define p4d_page_vaddr(p4d) (pud_page_vaddr((pud_t){ p4d }))
/*
* allocating and freeing a pud is trivial: the 1-entry pud is
- * inside the pgd, so has no extra memory associated with it.
+ * inside the p4d, so has no extra memory associated with it.
*/
#define pud_alloc_one(mm, address) NULL
#define pud_free(mm, x) do { } while (0)
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 7eed8cf3130a..a6b51b1a7b7f 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -256,6 +256,12 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb,
__pte_free_tlb(tlb, ptep, address); \
} while (0)
+#define pmd_free_tlb(tlb, pmdp, address) \
+ do { \
+ __tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ __pmd_free_tlb(tlb, pmdp, address); \
+ } while (0)
+
#ifndef __ARCH_HAS_4LEVEL_HACK
#define pud_free_tlb(tlb, pudp, address) \
do { \
@@ -264,11 +270,13 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb,
} while (0)
#endif
-#define pmd_free_tlb(tlb, pmdp, address) \
+#ifndef __ARCH_HAS_5LEVEL_HACK
+#define p4d_free_tlb(tlb, pudp, address) \
do { \
- __tlb_adjust_range(tlb, address, PAGE_SIZE); \
- __pmd_free_tlb(tlb, pmdp, address); \
+ __tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ __p4d_free_tlb(tlb, pudp, address); \
} while (0)
+#endif
#define tlb_migrate_finish(mm) do {} while (0)
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 15/33] x86/efi: handle p4d in EFI pagetables |
| Message-ID | <tbRyb-5mq-55@gated-at.bofh.it> |
| In reply to | #1583444 |
Allocate additional page table level and change efi_sync_low_kernel_mappings()
to make syncing logic work with additional page table level.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Matt Fleming <matt@codeblueprint.co.uk>
---
arch/x86/platform/efi/efi_64.c | 33 +++++++++++++++++++++++----------
1 file changed, 23 insertions(+), 10 deletions(-)
diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index 8f26e707e39c..7d80fc737f58 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -135,6 +135,7 @@ static pgd_t *efi_pgd;
int __init efi_alloc_page_tables(void)
{
pgd_t *pgd;
+ p4d_t *p4d;
pud_t *pud;
gfp_t gfp_mask;
@@ -147,15 +148,20 @@ int __init efi_alloc_page_tables(void)
return -ENOMEM;
pgd = efi_pgd + pgd_index(EFI_VA_END);
+ p4d = p4d_alloc(&init_mm, pgd, EFI_VA_END);
+ if (!p4d) {
+ free_page((unsigned long)efi_pgd);
+ return -ENOMEM;
+ }
- pud = pud_alloc_one(NULL, 0);
+ pud = pud_alloc(&init_mm, p4d, EFI_VA_END);
if (!pud) {
+ if (CONFIG_PGTABLE_LEVELS > 4)
+ free_page((unsigned long) pgd_page_vaddr(*pgd));
free_page((unsigned long)efi_pgd);
return -ENOMEM;
}
- pgd_populate(NULL, pgd, pud);
-
return 0;
}
@@ -190,6 +196,18 @@ void efi_sync_low_kernel_mappings(void)
num_entries = pgd_index(EFI_VA_END) - pgd_index(PAGE_OFFSET);
memcpy(pgd_efi, pgd_k, sizeof(pgd_t) * num_entries);
+ /* The same story as with PGD entries */
+ BUILD_BUG_ON(p4d_index(EFI_VA_END) != p4d_index(MODULES_END));
+ BUILD_BUG_ON((EFI_VA_START & P4D_MASK) != (EFI_VA_END & P4D_MASK));
+
+ pgd_efi = efi_pgd + pgd_index(EFI_VA_END);
+ pgd_k = pgd_offset_k(EFI_VA_END);
+ p4d_efi = p4d_offset(pgd_efi, 0);
+ p4d_k = p4d_offset(pgd_k, 0);
+
+ num_entries = p4d_index(EFI_VA_END);
+ memcpy(p4d_efi, p4d_k, sizeof(p4d_t) * num_entries);
+
/*
* We share all the PUD entries apart from those that map the
* EFI regions. Copy around them.
@@ -197,20 +215,15 @@ void efi_sync_low_kernel_mappings(void)
BUILD_BUG_ON((EFI_VA_START & ~PUD_MASK) != 0);
BUILD_BUG_ON((EFI_VA_END & ~PUD_MASK) != 0);
- pgd_efi = efi_pgd + pgd_index(EFI_VA_END);
- p4d_efi = p4d_offset(pgd_efi, 0);
+ p4d_efi = p4d_offset(pgd_efi, EFI_VA_END);
+ p4d_k = p4d_offset(pgd_k, EFI_VA_END);
pud_efi = pud_offset(p4d_efi, 0);
-
- pgd_k = pgd_offset_k(EFI_VA_END);
- p4d_k = p4d_offset(pgd_k, 0);
pud_k = pud_offset(p4d_k, 0);
num_entries = pud_index(EFI_VA_END);
memcpy(pud_efi, pud_k, sizeof(pud_t) * num_entries);
- p4d_efi = p4d_offset(pgd_efi, EFI_VA_START);
pud_efi = pud_offset(p4d_efi, EFI_VA_START);
- p4d_k = p4d_offset(pgd_k, EFI_VA_START);
pud_k = pud_offset(p4d_k, EFI_VA_START);
num_entries = PTRS_PER_PUD - pud_index(EFI_VA_START);
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Matt Fleming <matt@codeblueprint.co.uk> |
|---|---|
| Date | 2017-02-28 13:40 +0100 |
| Subject | Re: [PATCHv3 15/33] x86/efi: handle p4d in EFI pagetables |
| Message-ID | <tfPer-4pW-27@gated-at.bofh.it> |
| In reply to | #1583457 |
On Fri, 17 Feb, at 05:13:10PM, Kirill A. Shutemov wrote: > Allocate additional page table level and change efi_sync_low_kernel_mappings() > to make syncing logic work with additional page table level. > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> > Cc: Matt Fleming <matt@codeblueprint.co.uk> > --- > arch/x86/platform/efi/efi_64.c | 33 +++++++++++++++++++++++---------- > 1 file changed, 23 insertions(+), 10 deletions(-) Looks fine to me, but I haven't tested it. Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk>
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 32/33] x86: enable 5-level paging support |
| Message-ID | <tbRyb-5mq-53@gated-at.bofh.it> |
| In reply to | #1583444 |
Most of things are in place and we can enable support of 5-level paging. Enabling XEN with 5-level paging requires more work. The patch makes XEN dependent on !X86_5LEVEL. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> --- arch/x86/Kconfig | 5 +++++ arch/x86/xen/Kconfig | 1 + 2 files changed, 6 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 3d51256a9e61..f97b149145f8 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -315,6 +315,7 @@ config DEBUG_RODATA config PGTABLE_LEVELS int + default 5 if X86_5LEVEL default 4 if X86_64 default 3 if X86_PAE default 2 @@ -1379,6 +1380,10 @@ config X86_PAE has the cost of more pagetable lookup overhead, and also consumes more pagetable space per process. +config X86_5LEVEL + bool "Enable 5-level page tables support" + depends on X86_64 + config ARCH_PHYS_ADDR_T_64BIT def_bool y depends on X86_64 || X86_PAE diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig index c7b15f3e2cf3..e69eee1a5bad 100644 --- a/arch/x86/xen/Kconfig +++ b/arch/x86/xen/Kconfig @@ -5,6 +5,7 @@ config XEN bool "Xen guest support" depends on PARAVIRT + depends on !X86_5LEVEL select PARAVIRT_CLOCK select XEN_HAVE_PVMMU select XEN_HAVE_VPMU -- 2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 26/33] x86/kasan: extend to support 5-level paging |
| Message-ID | <tbRyb-5mq-59@gated-at.bofh.it> |
| In reply to | #1583444 |
This patch bring support for non-folded additional page table level.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dmitry Vyukov <dvyukov@google.com
---
arch/x86/mm/kasan_init_64.c | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index 72ffe4c55c2d..11ec306f7913 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -49,8 +49,18 @@ static void __init kasan_map_early_shadow(pgd_t *pgd)
unsigned long end = KASAN_SHADOW_END;
for (i = pgd_index(start); start < end; i++) {
- pgd[i] = __pgd(__pa_nodebug(kasan_zero_pud)
- | _KERNPG_TABLE);
+ switch (CONFIG_PGTABLE_LEVELS) {
+ case 4:
+ pgd[i] = __pgd(__pa_nodebug(kasan_zero_pud) |
+ _KERNPG_TABLE);
+ break;
+ case 5:
+ pgd[i] = __pgd(__pa_nodebug(kasan_zero_p4d) |
+ _KERNPG_TABLE);
+ break;
+ default:
+ BUILD_BUG();
+ }
start += PGDIR_SIZE;
}
}
@@ -78,6 +88,7 @@ void __init kasan_early_init(void)
pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL;
pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE;
pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE;
+ p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE;
for (i = 0; i < PTRS_PER_PTE; i++)
kasan_zero_pte[i] = __pte(pte_val);
@@ -88,6 +99,9 @@ void __init kasan_early_init(void)
for (i = 0; i < PTRS_PER_PUD; i++)
kasan_zero_pud[i] = __pud(pud_val);
+ for (i = 0; CONFIG_PGTABLE_LEVELS >= 5 && i < PTRS_PER_P4D; i++)
+ kasan_zero_p4d[i] = __p4d(p4d_val);
+
kasan_map_early_shadow(early_level4_pgt);
kasan_map_early_shadow(init_level4_pgt);
}
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Date | 2017-02-17 15:20 +0100 |
| Subject | [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tbRyb-5mq-61@gated-at.bofh.it> |
| In reply to | #1583444 |
This patch introduces two new prctl(2) handles to manage maximum virtual
address available to userspace to map.
On x86, 5-level paging enables 56-bit userspace virtual address space.
Not all user space is ready to handle wide addresses. It's known that
at least some JIT compilers use higher bits in pointers to encode their
information. It collides with valid pointers with 5-level paging and
leads to crashes.
The patch aims to address this compatibility issue.
MM would use the address as upper limit of virtual address available to
map by userspace, instead of TASK_SIZE.
The limit will be equal to TASK_SIZE everywhere, but the machine
with 5-level paging enabled. In this case, the default limit would be
(1UL << 47) - PAGE_SIZE. It’s current x86-64 TASK_SIZE_MAX with 4-level
paging which known to be safe.
Changing the limit would affect only future virtual address space
allocations. Currently existing VMAs are intact.
MPX can't at the moment handle addresses above 47-bits, so we refuse to
increase the limit above 47-bits. We also refuse to enable MPX if the
limit is already above 47-bits or if there is a VMA above the 47-bit
boundary.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-api@vger.kernel.org
---
arch/x86/include/asm/elf.h | 2 +-
arch/x86/include/asm/mmu.h | 2 ++
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/processor.h | 25 ++++++++++++++++++++-----
arch/x86/kernel/process.c | 18 ++++++++++++++++++
arch/x86/kernel/sys_x86_64.c | 6 +++---
arch/x86/mm/hugetlbpage.c | 8 ++++----
arch/x86/mm/mmap.c | 4 ++--
arch/x86/mm/mpx.c | 17 ++++++++++++++++-
fs/binfmt_aout.c | 2 --
fs/binfmt_elf.c | 10 +++++-----
fs/hugetlbfs/inode.c | 6 +++---
include/linux/sched.h | 8 ++++++++
include/uapi/linux/prctl.h | 3 +++
kernel/events/uprobes.c | 5 +++--
kernel/sys.c | 23 ++++++++++++++++++++---
mm/mmap.c | 20 +++++++++++---------
mm/mremap.c | 3 ++-
mm/nommu.c | 2 +-
mm/shmem.c | 8 ++++----
20 files changed, 127 insertions(+), 46 deletions(-)
diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h
index e7f155c3045e..5ce6f2b2b105 100644
--- a/arch/x86/include/asm/elf.h
+++ b/arch/x86/include/asm/elf.h
@@ -250,7 +250,7 @@ extern int force_personality32;
the loader. We need to make sure that it is out of the way of the program
that it will "exec", and that there is sufficient room for the brk. */
-#define ELF_ET_DYN_BASE (TASK_SIZE / 3 * 2)
+#define ELF_ET_DYN_BASE (mmap_max_addr() / 3 * 2)
/* This yields a mask that user programs can use to figure out what
instruction set this CPU supports. This could be done in user space,
diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h
index f9813b6d8b80..174dc3b60165 100644
--- a/arch/x86/include/asm/mmu.h
+++ b/arch/x86/include/asm/mmu.h
@@ -35,6 +35,8 @@ typedef struct {
/* address of the bounds directory */
void __user *bd_addr;
#endif
+ /* maximum virtual address the process can create VMA at */
+ unsigned long max_vaddr;
} mm_context_t;
#ifdef CONFIG_SMP
diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index 306c7e12af55..50bdfd6ab866 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -117,6 +117,7 @@ static inline int init_new_context(struct task_struct *tsk,
}
#endif
init_new_context_ldt(tsk, mm);
+ mm->context.max_vaddr = MAX_VADDR_DEFAULT;
return 0;
}
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index e6cfe7ba2d65..173f9a6b3b6b 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -789,8 +789,9 @@ static inline void spin_lock_prefetch(const void *x)
*/
#define TASK_SIZE PAGE_OFFSET
#define TASK_SIZE_MAX TASK_SIZE
-#define STACK_TOP TASK_SIZE
-#define STACK_TOP_MAX STACK_TOP
+#define MAX_VADDR_DEFAULT TASK_SIZE
+#define STACK_TOP mmap_max_addr()
+#define STACK_TOP_MAX TASK_SIZE
#define INIT_THREAD { \
.sp0 = TOP_OF_INIT_STACK, \
@@ -828,7 +829,14 @@ static inline void spin_lock_prefetch(const void *x)
* particular problem by preventing anything from being mapped
* at the maximum canonical address.
*/
-#define TASK_SIZE_MAX ((1UL << 47) - PAGE_SIZE)
+#define TASK_SIZE_MAX ((1UL << __VIRTUAL_MASK_SHIFT) - PAGE_SIZE)
+
+/*
+ * Default maximum virtual address. This is required for
+ * compatibility with applications that assumes 47-bit VA.
+ * The limit can be changed with prctl(PR_SET_MAX_VADDR).
+ */
+#define MAX_VADDR_DEFAULT ((1UL << 47) - PAGE_SIZE)
/* This decides where the kernel will search for a free chunk of vm
* space during mmap's.
@@ -841,7 +849,7 @@ static inline void spin_lock_prefetch(const void *x)
#define TASK_SIZE_OF(child) ((test_tsk_thread_flag(child, TIF_ADDR32)) ? \
IA32_PAGE_OFFSET : TASK_SIZE_MAX)
-#define STACK_TOP TASK_SIZE
+#define STACK_TOP mmap_max_addr()
#define STACK_TOP_MAX TASK_SIZE_MAX
#define INIT_THREAD { \
@@ -863,7 +871,7 @@ extern void start_thread(struct pt_regs *regs, unsigned long new_ip,
* This decides where the kernel will search for a free chunk of vm
* space during mmap's.
*/
-#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 3))
+#define TASK_UNMAPPED_BASE (PAGE_ALIGN(mmap_max_addr() / 3))
#define KSTK_EIP(task) (task_pt_regs(task)->ip)
@@ -892,6 +900,13 @@ static inline int mpx_disable_management(void)
}
#endif /* CONFIG_X86_INTEL_MPX */
+extern unsigned long set_max_vaddr(unsigned long addr);
+
+#define SET_MAX_VADDR(addr) set_max_vaddr(addr)
+#define GET_MAX_VADDR() READ_ONCE(current->mm->context.max_vaddr)
+
+#define mmap_max_addr() min(TASK_SIZE, GET_MAX_VADDR())
+
extern u16 amd_get_nb_id(int cpu);
extern u32 amd_get_nodes_per_socket(void);
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index b615a1113f58..ddc5af35f146 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -32,6 +32,7 @@
#include <asm/mce.h>
#include <asm/vm86.h>
#include <asm/switch_to.h>
+#include <asm/mpx.h>
/*
* per-CPU TSS segments. Threads are completely 'soft' on Linux,
@@ -536,3 +537,20 @@ unsigned long get_wchan(struct task_struct *p)
put_task_stack(p);
return ret;
}
+
+unsigned long set_max_vaddr(unsigned long addr)
+{
+ down_write(¤t->mm->mmap_sem);
+ if (addr > TASK_SIZE_MAX)
+ goto out;
+ /*
+ * MPX cannot handle addresses above 47-bits. Refuse to increase
+ * max_vaddr above the limit if MPX is enabled.
+ */
+ if (addr > MAX_VADDR_DEFAULT && kernel_managing_mpx_tables(current->mm))
+ goto out;
+ current->mm->context.max_vaddr = addr;
+out:
+ up_write(¤t->mm->mmap_sem);
+ return 0;
+}
diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c
index a55ed63b9f91..e31f5b0c5468 100644
--- a/arch/x86/kernel/sys_x86_64.c
+++ b/arch/x86/kernel/sys_x86_64.c
@@ -115,7 +115,7 @@ static void find_start_end(unsigned long flags, unsigned long *begin,
}
} else {
*begin = current->mm->mmap_legacy_base;
- *end = TASK_SIZE;
+ *end = mmap_max_addr();
}
}
@@ -168,7 +168,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
struct vm_unmapped_area_info info;
/* requested length too big for entire address space */
- if (len > TASK_SIZE)
+ if (len > mmap_max_addr())
return -ENOMEM;
if (flags & MAP_FIXED)
@@ -182,7 +182,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
if (addr) {
addr = PAGE_ALIGN(addr);
vma = find_vma(mm, addr);
- if (TASK_SIZE - len >= addr &&
+ if (mmap_max_addr() - len >= addr &&
(!vma || addr + len <= vma->vm_start))
return addr;
}
diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
index 2ae8584b44c7..b55b04b82097 100644
--- a/arch/x86/mm/hugetlbpage.c
+++ b/arch/x86/mm/hugetlbpage.c
@@ -82,7 +82,7 @@ static unsigned long hugetlb_get_unmapped_area_bottomup(struct file *file,
info.flags = 0;
info.length = len;
info.low_limit = current->mm->mmap_legacy_base;
- info.high_limit = TASK_SIZE;
+ info.high_limit = mmap_max_addr();
info.align_mask = PAGE_MASK & ~huge_page_mask(h);
info.align_offset = 0;
return vm_unmapped_area(&info);
@@ -114,7 +114,7 @@ static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file,
VM_BUG_ON(addr != -ENOMEM);
info.flags = 0;
info.low_limit = TASK_UNMAPPED_BASE;
- info.high_limit = TASK_SIZE;
+ info.high_limit = mmap_max_addr();
addr = vm_unmapped_area(&info);
}
@@ -131,7 +131,7 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
if (len & ~huge_page_mask(h))
return -EINVAL;
- if (len > TASK_SIZE)
+ if (len > mmap_max_addr())
return -ENOMEM;
if (flags & MAP_FIXED) {
@@ -143,7 +143,7 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
if (addr) {
addr = ALIGN(addr, huge_page_size(h));
vma = find_vma(mm, addr);
- if (TASK_SIZE - len >= addr &&
+ if (mmap_max_addr() - len >= addr &&
(!vma || addr + len <= vma->vm_start))
return addr;
}
diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index d2dc0438d654..c22f0b802576 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -52,7 +52,7 @@ static unsigned long stack_maxrandom_size(void)
* Leave an at least ~128 MB hole with possible stack randomization.
*/
#define MIN_GAP (128*1024*1024UL + stack_maxrandom_size())
-#define MAX_GAP (TASK_SIZE/6*5)
+#define MAX_GAP (mmap_max_addr()/6*5)
static int mmap_is_legacy(void)
{
@@ -90,7 +90,7 @@ static unsigned long mmap_base(unsigned long rnd)
else if (gap > MAX_GAP)
gap = MAX_GAP;
- return PAGE_ALIGN(TASK_SIZE - gap - rnd);
+ return PAGE_ALIGN(mmap_max_addr() - gap - rnd);
}
/*
diff --git a/arch/x86/mm/mpx.c b/arch/x86/mm/mpx.c
index af59f808742f..c19707d3e104 100644
--- a/arch/x86/mm/mpx.c
+++ b/arch/x86/mm/mpx.c
@@ -354,10 +354,25 @@ int mpx_enable_management(void)
*/
bd_base = mpx_get_bounds_dir();
down_write(&mm->mmap_sem);
+
+ /*
+ * MPX doesn't support addresses above 47-bits yes.
+ * Make sure it's not allowed to map above the limit and nothing is
+ * mapped there before enabling.
+ */
+ if (mmap_max_addr() > MAX_VADDR_DEFAULT ||
+ find_vma(mm, MAX_VADDR_DEFAULT)) {
+ pr_warn_once("%s (%d): MPX cannot handle addresses "
+ "above 47-bits. Disabling.",
+ current->comm, current->pid);
+ ret = -ENXIO;
+ goto out;
+ }
+
mm->context.bd_addr = bd_base;
if (mm->context.bd_addr == MPX_INVALID_BOUNDS_DIR)
ret = -ENXIO;
-
+out:
up_write(&mm->mmap_sem);
return ret;
}
diff --git a/fs/binfmt_aout.c b/fs/binfmt_aout.c
index 2a59139f520b..7a7f6dba6b00 100644
--- a/fs/binfmt_aout.c
+++ b/fs/binfmt_aout.c
@@ -121,8 +121,6 @@ static struct linux_binfmt aout_format = {
.min_coredump = PAGE_SIZE
};
-#define BAD_ADDR(x) ((unsigned long)(x) >= TASK_SIZE)
-
static int set_brk(unsigned long start, unsigned long end)
{
start = PAGE_ALIGN(start);
diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index 422370293cfd..b5dbea735c6d 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -89,7 +89,7 @@ static struct linux_binfmt elf_format = {
.min_coredump = ELF_EXEC_PAGESIZE,
};
-#define BAD_ADDR(x) ((unsigned long)(x) >= TASK_SIZE)
+#define BAD_ADDR(x) ((unsigned long)(x) >= mmap_max_addr())
static int set_brk(unsigned long start, unsigned long end)
{
@@ -587,8 +587,8 @@ static unsigned long load_elf_interp(struct elfhdr *interp_elf_ex,
k = load_addr + eppnt->p_vaddr;
if (BAD_ADDR(k) ||
eppnt->p_filesz > eppnt->p_memsz ||
- eppnt->p_memsz > TASK_SIZE ||
- TASK_SIZE - eppnt->p_memsz < k) {
+ eppnt->p_memsz > mmap_max_addr() ||
+ mmap_max_addr() - eppnt->p_memsz < k) {
error = -ENOMEM;
goto out;
}
@@ -960,8 +960,8 @@ static int load_elf_binary(struct linux_binprm *bprm)
* <= p_memsz so it is only necessary to check p_memsz.
*/
if (BAD_ADDR(k) || elf_ppnt->p_filesz > elf_ppnt->p_memsz ||
- elf_ppnt->p_memsz > TASK_SIZE ||
- TASK_SIZE - elf_ppnt->p_memsz < k) {
+ elf_ppnt->p_memsz > mmap_max_addr() ||
+ mmap_max_addr() - elf_ppnt->p_memsz < k) {
/* set_brk can never work. Avoid overflows. */
retval = -EINVAL;
goto out_free_dentry;
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 54de77e78775..e132e93b85fb 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -178,7 +178,7 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
if (len & ~huge_page_mask(h))
return -EINVAL;
- if (len > TASK_SIZE)
+ if (len > mmap_max_addr())
return -ENOMEM;
if (flags & MAP_FIXED) {
@@ -190,7 +190,7 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
if (addr) {
addr = ALIGN(addr, huge_page_size(h));
vma = find_vma(mm, addr);
- if (TASK_SIZE - len >= addr &&
+ if (mmap_max_addr() - len >= addr &&
(!vma || addr + len <= vma->vm_start))
return addr;
}
@@ -198,7 +198,7 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
info.flags = 0;
info.length = len;
info.low_limit = TASK_UNMAPPED_BASE;
- info.high_limit = TASK_SIZE;
+ info.high_limit = mmap_max_addr();
info.align_mask = PAGE_MASK & ~huge_page_mask(h);
info.align_offset = 0;
return vm_unmapped_area(&info);
diff --git a/include/linux/sched.h b/include/linux/sched.h
index ad3ec9ec61f7..bf47a62fde5d 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -3671,4 +3671,12 @@ void cpufreq_add_update_util_hook(int cpu, struct update_util_data *data,
void cpufreq_remove_update_util_hook(int cpu);
#endif /* CONFIG_CPU_FREQ */
+#ifndef mmap_max_addr
+#define mmap_max_addr mmap_max_addr
+static inline unsigned long mmap_max_addr(void)
+{
+ return TASK_SIZE;
+}
+#endif
+
#endif
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index a8d0759a9e40..e9478ccd4386 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -197,4 +197,7 @@ struct prctl_mm_map {
# define PR_CAP_AMBIENT_LOWER 3
# define PR_CAP_AMBIENT_CLEAR_ALL 4
+#define PR_SET_MAX_VADDR 48
+#define PR_GET_MAX_VADDR 49
+
#endif /* _LINUX_PRCTL_H */
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index d416f3baf392..651f571a1a79 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1142,8 +1142,9 @@ static int xol_add_vma(struct mm_struct *mm, struct xol_area *area)
if (!area->vaddr) {
/* Try to map as high as possible, this is only a hint. */
- area->vaddr = get_unmapped_area(NULL, TASK_SIZE - PAGE_SIZE,
- PAGE_SIZE, 0, 0);
+ area->vaddr = get_unmapped_area(NULL,
+ mmap_max_addr() - PAGE_SIZE,
+ PAGE_SIZE, 0, 0);
if (area->vaddr & ~PAGE_MASK) {
ret = area->vaddr;
goto fail;
diff --git a/kernel/sys.c b/kernel/sys.c
index 842914ef7de4..366ba7be92a7 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -103,6 +103,12 @@
#ifndef SET_FP_MODE
# define SET_FP_MODE(a,b) (-EINVAL)
#endif
+#ifndef SET_MAX_VADDR
+# define SET_MAX_VADDR(a) (-EINVAL)
+#endif
+#ifndef GET_MAX_VADDR
+# define GET_MAX_VADDR() (-EINVAL)
+#endif
/*
* this is where the system-wide overflow UID and GID are defined, for
@@ -1718,7 +1724,7 @@ static int prctl_set_mm_exe_file(struct mm_struct *mm, unsigned int fd)
*/
static int validate_prctl_map(struct prctl_mm_map *prctl_map)
{
- unsigned long mmap_max_addr = TASK_SIZE;
+ unsigned long max_addr = mmap_max_addr();
struct mm_struct *mm = current->mm;
int error = -EINVAL, i;
@@ -1743,7 +1749,7 @@ static int validate_prctl_map(struct prctl_mm_map *prctl_map)
for (i = 0; i < ARRAY_SIZE(offsets); i++) {
u64 val = *(u64 *)((char *)prctl_map + offsets[i]);
- if ((unsigned long)val >= mmap_max_addr ||
+ if ((unsigned long)val >= max_addr ||
(unsigned long)val < mmap_min_addr)
goto out;
}
@@ -1949,7 +1955,7 @@ static int prctl_set_mm(int opt, unsigned long addr,
if (opt == PR_SET_MM_AUXV)
return prctl_set_auxv(mm, addr, arg4);
- if (addr >= TASK_SIZE || addr < mmap_min_addr)
+ if (addr >= mmap_max_addr() || addr < mmap_min_addr)
return -EINVAL;
error = -EINVAL;
@@ -2261,6 +2267,17 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
case PR_GET_FP_MODE:
error = GET_FP_MODE(me);
break;
+ case PR_SET_MAX_VADDR:
+ if (arg3 || arg4 || arg5)
+ return -EINVAL;
+ error = SET_MAX_VADDR(arg2);
+ break;
+ case PR_GET_MAX_VADDR:
+ if (arg3 || arg4 || arg5)
+ return -EINVAL;
+ error = put_user(GET_MAX_VADDR(),
+ (unsigned long __user *) arg2);
+ break;
default:
error = -EINVAL;
break;
diff --git a/mm/mmap.c b/mm/mmap.c
index dc4291dcc99b..a3384f23359e 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1966,7 +1966,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr,
struct vm_area_struct *vma;
struct vm_unmapped_area_info info;
- if (len > TASK_SIZE - mmap_min_addr)
+ if (len > mmap_max_addr() - mmap_min_addr)
return -ENOMEM;
if (flags & MAP_FIXED)
@@ -1975,15 +1975,16 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr,
if (addr) {
addr = PAGE_ALIGN(addr);
vma = find_vma(mm, addr);
- if (TASK_SIZE - len >= addr && addr >= mmap_min_addr &&
- (!vma || addr + len <= vma->vm_start))
+ if (mmap_max_addr() - len >= addr &&
+ addr >= mmap_min_addr &&
+ (!vma || addr + len <= vma->vm_start))
return addr;
}
info.flags = 0;
info.length = len;
info.low_limit = mm->mmap_base;
- info.high_limit = TASK_SIZE;
+ info.high_limit = mmap_max_addr();
info.align_mask = 0;
return vm_unmapped_area(&info);
}
@@ -2005,7 +2006,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
struct vm_unmapped_area_info info;
/* requested length too big for entire address space */
- if (len > TASK_SIZE - mmap_min_addr)
+ if (len > mmap_max_addr() - mmap_min_addr)
return -ENOMEM;
if (flags & MAP_FIXED)
@@ -2015,7 +2016,8 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
if (addr) {
addr = PAGE_ALIGN(addr);
vma = find_vma(mm, addr);
- if (TASK_SIZE - len >= addr && addr >= mmap_min_addr &&
+ if (mmap_max_addr() - len >= addr &&
+ addr >= mmap_min_addr &&
(!vma || addr + len <= vma->vm_start))
return addr;
}
@@ -2037,7 +2039,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
VM_BUG_ON(addr != -ENOMEM);
info.flags = 0;
info.low_limit = TASK_UNMAPPED_BASE;
- info.high_limit = TASK_SIZE;
+ info.high_limit = mmap_max_addr();
addr = vm_unmapped_area(&info);
}
@@ -2057,7 +2059,7 @@ get_unmapped_area(struct file *file, unsigned long addr, unsigned long len,
return error;
/* Careful about overflows.. */
- if (len > TASK_SIZE)
+ if (len > mmap_max_addr())
return -ENOMEM;
get_area = current->mm->get_unmapped_area;
@@ -2078,7 +2080,7 @@ get_unmapped_area(struct file *file, unsigned long addr, unsigned long len,
if (IS_ERR_VALUE(addr))
return addr;
- if (addr > TASK_SIZE - len)
+ if (addr > mmap_max_addr() - len)
return -ENOMEM;
if (offset_in_page(addr))
return -EINVAL;
diff --git a/mm/mremap.c b/mm/mremap.c
index 2b3bfcd51c75..a8b4fba3dce6 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -433,7 +433,8 @@ static unsigned long mremap_to(unsigned long addr, unsigned long old_len,
if (offset_in_page(new_addr))
goto out;
- if (new_len > TASK_SIZE || new_addr > TASK_SIZE - new_len)
+ if (new_len > mmap_max_addr() ||
+ new_addr > mmap_max_addr() - new_len)
goto out;
/* Ensure the old/new locations do not overlap */
diff --git a/mm/nommu.c b/mm/nommu.c
index 24f9f5f39145..6043b8b82083 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -905,7 +905,7 @@ static int validate_mmap_request(struct file *file,
/* Careful about overflows.. */
rlen = PAGE_ALIGN(len);
- if (!rlen || rlen > TASK_SIZE)
+ if (!rlen || rlen > mmap_max_addr())
return -ENOMEM;
/* offset overflow? */
diff --git a/mm/shmem.c b/mm/shmem.c
index 3a7587a0314d..54d1ebfb577d 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1983,7 +1983,7 @@ unsigned long shmem_get_unmapped_area(struct file *file,
unsigned long inflated_addr;
unsigned long inflated_offset;
- if (len > TASK_SIZE)
+ if (len > mmap_max_addr())
return -ENOMEM;
get_area = current->mm->get_unmapped_area;
@@ -1995,7 +1995,7 @@ unsigned long shmem_get_unmapped_area(struct file *file,
return addr;
if (addr & ~PAGE_MASK)
return addr;
- if (addr > TASK_SIZE - len)
+ if (addr > mmap_max_addr() - len)
return addr;
if (shmem_huge == SHMEM_HUGE_DENY)
@@ -2038,7 +2038,7 @@ unsigned long shmem_get_unmapped_area(struct file *file,
return addr;
inflated_len = len + HPAGE_PMD_SIZE - PAGE_SIZE;
- if (inflated_len > TASK_SIZE)
+ if (inflated_len > mmap_max_addr())
return addr;
if (inflated_len < len)
return addr;
@@ -2054,7 +2054,7 @@ unsigned long shmem_get_unmapped_area(struct file *file,
if (inflated_offset > offset)
inflated_addr += HPAGE_PMD_SIZE;
- if (inflated_addr > TASK_SIZE - len)
+ if (inflated_addr > mmap_max_addr() - len)
return addr;
return inflated_addr;
}
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2017-02-17 18:00 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tbU30-6Np-15@gated-at.bofh.it> |
| In reply to | #1583460 |
On Fri, Feb 17, 2017 at 6:13 AM, Kirill A. Shutemov
<kirill.shutemov@linux.intel.com> wrote:
> This patch introduces two new prctl(2) handles to manage maximum virtual
> address available to userspace to map.
>
> On x86, 5-level paging enables 56-bit userspace virtual address space.
> Not all user space is ready to handle wide addresses. It's known that
> at least some JIT compilers use higher bits in pointers to encode their
> information. It collides with valid pointers with 5-level paging and
> leads to crashes.
>
> The patch aims to address this compatibility issue.
>
> MM would use the address as upper limit of virtual address available to
> map by userspace, instead of TASK_SIZE.
>
> The limit will be equal to TASK_SIZE everywhere, but the machine
> with 5-level paging enabled. In this case, the default limit would be
> (1UL << 47) - PAGE_SIZE. It’s current x86-64 TASK_SIZE_MAX with 4-level
> paging which known to be safe.
I think this patch need to be split up. In particular, the addition
and use of mmap_max_addr() should be its own patch that doesn't change
any semantics.
> diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
> index 306c7e12af55..50bdfd6ab866 100644
> --- a/arch/x86/include/asm/mmu_context.h
> +++ b/arch/x86/include/asm/mmu_context.h
> @@ -117,6 +117,7 @@ static inline int init_new_context(struct task_struct *tsk,
> }
> #endif
> init_new_context_ldt(tsk, mm);
> + mm->context.max_vaddr = MAX_VADDR_DEFAULT;
Is this actually correct for 32-bit binaries? Although, given the
stuff Dmitry is working on, it might pay to separately track the
32-bit and 64-bit limits per mm. If you haven't been following it,
Dmitry is trying to fix a bug in which an explicit 32-bit syscall
(int80 or similar) in an otherwise 64-bit process can allocate a VMA
above 4GB that gets truncated.
Also, why the macro? Why not just put the number in here?
> -#define TASK_SIZE_MAX ((1UL << 47) - PAGE_SIZE)
> +#define TASK_SIZE_MAX ((1UL << __VIRTUAL_MASK_SHIFT) - PAGE_SIZE)
This should be in the
> -#define STACK_TOP TASK_SIZE
> +#define STACK_TOP mmap_max_addr()
Off the top of my head, this looks wrong. The 32-bit check got lost, I think.
> +unsigned long set_max_vaddr(unsigned long addr)
> +{
Perhaps this function could set a different field depending on
is_compat_syscall().
Anyway, can you and Dmitry try to reconcile your patches?
[toc] | [prev] | [next] | [standalone]
| From | Dmitry Safonov <0x7f454c46@gmail.com> |
|---|---|
| Date | 2017-02-21 13:00 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tdhgS-252-11@gated-at.bofh.it> |
| In reply to | #1583595 |
2017-02-17 19:50 GMT+03:00 Andy Lutomirski <luto@amacapital.net>:
> On Fri, Feb 17, 2017 at 6:13 AM, Kirill A. Shutemov
> <kirill.shutemov@linux.intel.com> wrote:
>> This patch introduces two new prctl(2) handles to manage maximum virtual
>> address available to userspace to map.
...
> Anyway, can you and Dmitry try to reconcile your patches?
So, how can I help that?
Is there the patch's version, on which I could rebase?
Here are BTW the last patches, which I will resend with trivial ifdef-fixup
after the merge window:
http://marc.info/?i=20170214183621.2537-1-dsafonov%20()%20virtuozzo%20!%20com
--
Dmitry
[toc] | [prev] | [next] | [standalone]
| From | "Kirill A. Shutemov" <kirill@shutemov.name> |
|---|---|
| Date | 2017-02-21 13:50 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tdi3f-2DN-9@gated-at.bofh.it> |
| In reply to | #1585251 |
On Tue, Feb 21, 2017 at 02:54:20PM +0300, Dmitry Safonov wrote: > 2017-02-17 19:50 GMT+03:00 Andy Lutomirski <luto@amacapital.net>: > > On Fri, Feb 17, 2017 at 6:13 AM, Kirill A. Shutemov > > <kirill.shutemov@linux.intel.com> wrote: > >> This patch introduces two new prctl(2) handles to manage maximum virtual > >> address available to userspace to map. > ... > > Anyway, can you and Dmitry try to reconcile your patches? > > So, how can I help that? > Is there the patch's version, on which I could rebase? > Here are BTW the last patches, which I will resend with trivial ifdef-fixup > after the merge window: > http://marc.info/?i=20170214183621.2537-1-dsafonov%20()%20virtuozzo%20!%20com Could you check if this patch collides with anything you do: http://lkml.kernel.org/r/20170220131515.GA9502@node.shutemov.name -- Kirill A. Shutemov
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave.hansen@intel.com> |
|---|---|
| Date | 2017-02-17 18:30 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tbUw1-7dc-13@gated-at.bofh.it> |
| In reply to | #1583460 |
On 02/17/2017 06:13 AM, Kirill A. Shutemov wrote: > +/* > + * Default maximum virtual address. This is required for > + * compatibility with applications that assumes 47-bit VA. > + * The limit can be changed with prctl(PR_SET_MAX_VADDR). > + */ > +#define MAX_VADDR_DEFAULT ((1UL << 47) - PAGE_SIZE) This is a bit goofy. It's not the largest virtual adddress that can be accessed, but the beginning of the last page. Isn't this easier to deal with in userspace if we make it a "limit", so we can do: if (addr >= limit) // error Now, we have to do: prctl(PR_GET_MAX_VADDR, &max_vaddr, 0, 0, 0); if (addr > (max_vaddr + PAGE_SIZE)) // error I don't care what you track in the kernel, but I think we need to provide a more usable number out to userspace.
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2017-02-17 18:30 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tbUw1-7dc-17@gated-at.bofh.it> |
| In reply to | #1583615 |
On Fri, Feb 17, 2017 at 9:19 AM, Dave Hansen <dave.hansen@intel.com> wrote: > On 02/17/2017 06:13 AM, Kirill A. Shutemov wrote: >> +/* >> + * Default maximum virtual address. This is required for >> + * compatibility with applications that assumes 47-bit VA. >> + * The limit can be changed with prctl(PR_SET_MAX_VADDR). >> + */ >> +#define MAX_VADDR_DEFAULT ((1UL << 47) - PAGE_SIZE) > > This is a bit goofy. It's not the largest virtual adddress that can be > accessed, but the beginning of the last page. No, it really is the limit. We don't allow user code to map the last page because ti would be a root hole due to SYSRET. Thanks, Intel. See the comment near TASK_SIZE_MAX IIRC. --Andy
[toc] | [prev] | [next] | [standalone]
| From | Linus Torvalds <torvalds@linux-foundation.org> |
|---|---|
| Date | 2017-02-17 21:10 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tbX0T-q0-47@gated-at.bofh.it> |
| In reply to | #1583460 |
On Fri, Feb 17, 2017 at 6:13 AM, Kirill A. Shutemov
<kirill.shutemov@linux.intel.com> wrote:
> This patch introduces two new prctl(2) handles to manage maximum virtual
> address available to userspace to map.
So this is my least favorite patch of the whole series, for a couple of reasons:
(a) adding new code, and mixing it with the mindless TASK_SIZE ->
get_max_addr() conversion.
(b) what's the point of that whole TASK_SIZE vs get_max_addr() thing?
When use one, when the other?
so I think this patch needs a lot more thought and/or explanation.
Honestly, (a) is a no-brainer, and can be fixed by just splitting the
patch up. But I think (b) is more fundamental.
In particular, I think that get_max_addr() thing is badly defined.
When should you use TASK_SIZE, when should you use TASK_SIZE_MAX, and
when should you use get_max_addr()? I don't find that clear at all,
and I think that needs to be a whole lot more explicit and documented.
I also get he feeling that the whole thing is unnecessary. I'm
wondering if we should just instead say that the whole 47 vs 56-bit
virtual address is _purely_ about "get_unmapped_area()", and nothing
else.
IOW, I'm wondering if we can't just say that
- if the processor and kernel support 56-bit user address space, then
you can *always* use the whole space
- but by default, get_unmapped_area() will only return mappings that
fit in the 47 bit address space.
So if you use MAP_FIXED and give an address in the high range, it will
just always work, and the MM will always consider the task size to be
the full address space.
But for the common case where a process does no use MAP_FIXED, the
kernel will never give a high address by default, and you have to do
the process control thing to say "I want those high addresses".
Hmm?
In other words, I'd like to at least start out trying to keep the
differences between the 47-bit and 56-bit models as simple and minimal
as possible. Not make such a big deal out of it.
We already have "arch_get_unmapped_area()" that controls the whole
"what will non-MAP_FIXED mmap allocations return", so I'd hope that
the above kind of semantics could be done without *any* actual
TASK_SIZE changes _anywhere_ in the VM code.
Comments?
Linus
[toc] | [prev] | [next] | [standalone]
| From | Andy Lutomirski <luto@amacapital.net> |
|---|---|
| Date | 2017-02-17 21:30 +0100 |
| Subject | Re: [PATCHv3 33/33] mm, x86: introduce PR_SET_MAX_VADDR and PR_GET_MAX_VADDR |
| Message-ID | <tbXkd-wK-7@gated-at.bofh.it> |
| In reply to | #1583703 |
On Fri, Feb 17, 2017 at 12:02 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > On Fri, Feb 17, 2017 at 6:13 AM, Kirill A. Shutemov > <kirill.shutemov@linux.intel.com> wrote: >> This patch introduces two new prctl(2) handles to manage maximum virtual >> address available to userspace to map. > > So this is my least favorite patch of the whole series, for a couple of reasons: > > (a) adding new code, and mixing it with the mindless TASK_SIZE -> > get_max_addr() conversion. > > (b) what's the point of that whole TASK_SIZE vs get_max_addr() thing? > When use one, when the other? > > so I think this patch needs a lot more thought and/or explanation. > > Honestly, (a) is a no-brainer, and can be fixed by just splitting the > patch up. But I think (b) is more fundamental. > > In particular, I think that get_max_addr() thing is badly defined. > When should you use TASK_SIZE, when should you use TASK_SIZE_MAX, and > when should you use get_max_addr()? I don't find that clear at all, > and I think that needs to be a whole lot more explicit and documented. > > I also get he feeling that the whole thing is unnecessary. I'm > wondering if we should just instead say that the whole 47 vs 56-bit > virtual address is _purely_ about "get_unmapped_area()", and nothing > else. > > IOW, I'm wondering if we can't just say that > > - if the processor and kernel support 56-bit user address space, then > you can *always* use the whole space > > - but by default, get_unmapped_area() will only return mappings that > fit in the 47 bit address space. > > So if you use MAP_FIXED and give an address in the high range, it will > just always work, and the MM will always consider the task size to be > the full address space. At the very least, I'd want to see MAP_FIXED_BUT_DONT_BLOODY_UNMAP_ANYTHING. I *hate* the current interface. > > But for the common case where a process does no use MAP_FIXED, the > kernel will never give a high address by default, and you have to do > the process control thing to say "I want those high addresses". > > Hmm? How about MAP_LIMIT where the address passed in is interpreted as an upper bound instead of a fixed address? --Andy
[toc] | [prev] | [next] | [standalone]
Page 1 of 3 [1] 2 3 Next page →
Back to top | Article view | linux.kernel
csiph-web