Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1333117 > unrolled thread
| Started by | Dave Hansen <dave@sr71.net> |
|---|---|
| First post | 2016-02-12 22:10 +0100 |
| Last post | 2016-02-16 10:30 +0100 |
| Articles | 20 on this page of 43 — 7 participants |
Back to article view | Back to linux.kernel
[PATCH 00/33] x86: Memory Protection Keys (v10) Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[PATCH 18/33] x86, mm: simplify get_user_pages() PTE bit handling Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/gup: Simplify get_user_pages() PTE bit handling tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 23/33] x86, pkeys: dump PKRU with other kernel registers Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/pkeys: Dump PKRU with other kernel registers tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 30/33] x86, fpu: allow setting of XSAVE state Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/fpu: Allow setting of XSAVE state tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 03/33] mm, gup: switch callers of get_user_pages() to not pass tsk/mm Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] mm/gup: Switch all callers of get_user_pages() to not pass tsk/mm tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 19/33] x86, pkeys: check VMAs and PTEs for protection keys Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] mm/gup, x86/mm/pkeys: Check VMAs and PTEs for protection keys tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 26/33] x86, pkeys: actually enable Memory Protection Keys in CPU Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/pkeys: Actually enable Memory Protection Keys in the CPU tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 07/33] x86, pkeys: define new CR4 bit Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/cpu, x86/mm/pkeys: Define new CR4 bit tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 13/33] x86, pkeys: pass VMA down in to fault signal generation code Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/pkeys: Pass VMA down in to fault signal generation code tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 11/33] x86, pkeys: store protection in high VMA flags Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] mm/core, x86/mm/pkeys: Store protection bits in high VMA flags tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 10/33] x86, pkeys: new page fault error code bit: PF_PK Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/mm/pkeys: Add new 'PF_PK' page fault error code bit tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 09/33] x86, pkeys: PTE bits for storing protection key Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/mm/pkeys: Add PTE bits for storing protection key tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 06/33] x86, pkeys: cpuid bit definition Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/cpufeature, x86/mm/pkeys: Add protection keys related CPUID definitions tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 02/33] mm: overload get_user_pages() functions Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
Re: [PATCH 02/33] mm: overload get_user_pages() functions Ingo Molnar <mingo@kernel.org> - 2016-02-16 09:40 +0100
Re: [PATCH 02/33] mm: overload get_user_pages() functions Dave Hansen <dave@sr71.net> - 2016-02-17 19:20 +0100
[tip:mm/pkeys] mm/gup: Overload get_user_pages() functions tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 01/33] mm: introduce get_user_pages_remote() Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
Re: [PATCH 01/33] mm: introduce get_user_pages_remote() Balbir Singh <bsingharora@gmail.com> - 2016-02-15 07:10 +0100
Re: [PATCH 01/33] mm: introduce get_user_pages_remote() Dave Hansen <dave@sr71.net> - 2016-02-15 17:30 +0100
Re: [PATCH 01/33] mm: introduce get_user_pages_remote() Srikar Dronamraju <srikar@linux.vnet.ibm.com> - 2016-02-15 07:20 +0100
[tip:x86/pkeys] mm/gup: Introduce get_user_pages_remote() tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-16 13:20 +0100
Re: [tip:x86/pkeys] mm/gup: Introduce get_user_pages_remote() Konstantin Khlebnikov <koct9i@gmail.com> - 2016-02-20 07:30 +0100
[PATCH 05/33] x86, pkeys: Add Kconfig option Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/mm/pkeys: Add Kconfig option tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH] x86/mm/pkeys: Do not enable them by default Borislav Petkov <bp@alien8.de> - 2016-02-19 12:30 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Dave Hansen <dave@sr71.net> - 2016-02-19 18:20 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Borislav Petkov <bp@alien8.de> - 2016-02-19 18:30 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Dave Hansen <dave@sr71.net> - 2016-02-19 18:50 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Borislav Petkov <bp@alien8.de> - 2016-02-19 19:40 +0100
Re: [PATCH 00/33] x86: Memory Protection Keys (v10) Ingo Molnar <mingo@kernel.org> - 2016-02-16 10:30 +0100
Page 1 of 3 [1] 2 3 Next page →
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 00/33] x86: Memory Protection Keys (v10) |
| Message-ID | <r1t8t-3TO-3@gated-at.bofh.it> |
Memory Protection Keys for User pages is a CPU feature which will
first appear on Skylake Servers, but will also be supported on
future non-server parts (there is also a QEMU implementation). It
provides a mechanism for enforcing page-based protections, but
without requiring modification of the page tables when an
application changes wishes to change permissions.
This set introduces supported limited to:
1. Allows "execute-only" memory
2. Enables KVM to run Protection-Key-enabled guests
This set contains the vast majority of of the code, with the
small but tricky explicit user interface parts left off. We can
have a more focused review on those at a later time in a (much
smaller) follow-on series.
Changes from v9:
* Added macros to allow some source-level backward compatability
between the old-style get_user_pages*() calls taking tsk/mm and
the new-style which assumes the current tsk/mm.
* renamed the new gup variant to get_user_pages_remote() instead
of get_user_pages_foreign()
* rebased against 4.5-rc3
Changes from v8:
* Reorganization of get_user_pages() patch with awesome feedback
from Vlastimil Babka and suggestions from Jan Kara.
* fix EXPORT_SYMBOL() and use get_current_user_page() in nommu.c
Changes from v7:
* Fixed merge issue with cpu feature bitmap definitions
* Fixed up some comments in get_user_pages() and smaps patches
(thanks Vlastimil!)
Changes from v6:
* fix up ??'s showing up in in smaps' VmFlags field
* added execute-only support
* removed all the new syscalls from this set. We can discuss
them in detail after this is merged.
Changes from v5:
* make types in read_pkru() u32's, not ints
* rework VM_* bits to avoid using __ffsl() and clean up
vma_pkey()
* rework pte_allows_gup() to use p??_val() instead of passing
around p{te,md,ud}_t types.
* Fix up some inconsistent bool vs. int usage
* corrected name of ARCH_VM_PKEY_FLAGS in patch description
* remove NR_PKEYS... config option. Just define it directly
Changes from v4:
* Made "allow setting of XSAVE state" safe if we got preempted
between when we saved our FPU state and when we restore it.
(I would appreciate a look from Ingo on this patch).
* Fixed up a few things from Thomas's latest comments: splt up
siginfo in to x86 and generic, removed extra 'eax' variable
in rdpkru function, reworked vm_flags assignment, reworded
a comment in pte_allows_gup()
* Add missing DISABLED/REQUIRED_MASK14 in cpufeature.h
* Added comment about compile optimization in fault path
* Left get_user_pages_locked() alone. Andrea thinks we need it.
Changes from RFCv3:
* Added 'current' and 'foreign' variants of get_user_pages() to
help indicate whether protection keys should be enforced.
Thanks to Jerome Glisse for pointing out this issue.
* Added "allocation" and set/get system calls so that we can do
management of proection keys in the kernel. This opens the
door to use of specific protection keys for kernel use in the
future, such as for execute-only memory.
* Removed the kselftest code for the moment. It will be
submitted separately.
Thanks Ingo and Thomas for most of these):
Changes from RFCv2 (Thanks Ingo and Thomas for most of these):
* few minor compile warnings
* changed 'nopku' interaction with cpuid bits. Now, we do not
clear the PKU cpuid bit, we just skip enabling it.
* changed __pkru_allows_write() to also check access disable bit
* removed the unused write_pkru()
* made si_pkey a u64 and added some patch description details.
Also made it share space in siginfo with MPX and clarified
comments.
* give some real text for the Processor Trace xsave state
* made vma_pkey() less ugly (and much more optimized actually)
* added SEGV_PKUERR to copy_siginfo_to_user()
* remove page table walk when filling in si_pkey, added some
big fat comments about it being inherently racy.
* added self test code
This code is not runnable to anyone outside of Intel unless they
have some special hardware or a fancy simulator. There is a qemu
model to emulate the feature, but it is not currently implemented
fully enough to be usable. If you are interested in running this
for real, please get in touch with me. Hardware is available to a
very small but nonzero number of people.
This set is also available here:
git://git.kernel.org/pub/scm/linux/kernel/git/daveh/x86-pkeys.git pkeys-v024
=== diffstat ===
Dave Hansen (33):
mm: introduce get_user_pages_remote()
mm: overload get_user_pages() functions
mm, gup: switch callers of get_user_pages() to not pass tsk/mm
x86, fpu: add placeholder for Processor Trace XSAVE state
x86, pkeys: Add Kconfig option
x86, pkeys: cpuid bit definition
x86, pkeys: define new CR4 bit
x86, pkeys: add PKRU xsave fields and data structure(s)
x86, pkeys: PTE bits for storing protection key
x86, pkeys: new page fault error code bit: PF_PK
x86, pkeys: store protection in high VMA flags
x86, pkeys: arch-specific protection bits
x86, pkeys: pass VMA down in to fault signal generation code
signals, pkeys: notify userspace about protection key faults
x86, pkeys: fill in pkey field in siginfo
x86, pkeys: add functions to fetch PKRU
mm: factor out VMA fault permission checking
x86, mm: simplify get_user_pages() PTE bit handling
x86, pkeys: check VMAs and PTEs for protection keys
mm: do not enforce PKEY permissions on "foreign" mm access
x86, pkeys: optimize fault handling in access_error()
x86, pkeys: differentiate instruction fetches
x86, pkeys: dump PKRU with other kernel registers
x86, pkeys: dump pkey from VMA in /proc/pid/smaps
x86, pkeys: add Kconfig prompt to existing config option
x86, pkeys: actually enable Memory Protection Keys in CPU
mm, multi-arch: pass a protection key in to calc_vm_flag_bits()
x86, pkeys: add arch_validate_pkey()
x86: separate out LDT init from context init
x86, fpu: allow setting of XSAVE state
x86, pkeys: allow kernel to modify user pkey rights register
x86, pkeys: create an x86 arch_calc_vm_prot_bits() for VMA flags
x86, pkeys: execute-only support
Documentation/kernel-parameters.txt | 3 +
arch/cris/arch-v32/drivers/cryptocop.c | 8 +-
arch/ia64/kernel/err_inject.c | 3 +-
arch/mips/mm/gup.c | 3 +-
arch/powerpc/include/asm/mman.h | 5 +-
arch/powerpc/include/asm/mmu_context.h | 12 ++
arch/s390/include/asm/mmu_context.h | 12 ++
arch/s390/mm/gup.c | 4 +-
arch/sh/mm/gup.c | 2 +-
arch/sparc/mm/gup.c | 2 +-
arch/unicore32/include/asm/mmu_context.h | 12 ++
arch/x86/Kconfig | 16 ++
arch/x86/include/asm/cpufeature.h | 61 ++++---
arch/x86/include/asm/disabled-features.h | 15 ++
arch/x86/include/asm/fpu/internal.h | 2 +
arch/x86/include/asm/fpu/types.h | 12 ++
arch/x86/include/asm/fpu/xstate.h | 3 +-
arch/x86/include/asm/mmu_context.h | 85 ++++++++-
arch/x86/include/asm/pgtable.h | 38 ++++
arch/x86/include/asm/pgtable_types.h | 39 ++++-
arch/x86/include/asm/pkeys.h | 34 ++++
arch/x86/include/asm/required-features.h | 7 +
arch/x86/include/asm/special_insns.h | 22 +++
arch/x86/include/uapi/asm/mman.h | 22 +++
arch/x86/include/uapi/asm/processor-flags.h | 2 +
arch/x86/kernel/cpu/common.c | 42 +++++
arch/x86/kernel/fpu/core.c | 63 +++++++
arch/x86/kernel/fpu/xstate.c | 185 +++++++++++++++++++-
arch/x86/kernel/ldt.c | 4 +-
arch/x86/kernel/process_64.c | 2 +
arch/x86/kernel/setup.c | 9 +
arch/x86/mm/Makefile | 2 +
arch/x86/mm/fault.c | 168 +++++++++++++++---
arch/x86/mm/gup.c | 45 +++--
arch/x86/mm/mpx.c | 4 +-
arch/x86/mm/pkeys.c | 101 +++++++++++
drivers/char/agp/frontend.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +-
drivers/gpu/drm/etnaviv/etnaviv_gem.c | 6 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 10 +-
drivers/gpu/drm/radeon/radeon_ttm.c | 3 +-
drivers/gpu/drm/via/via_dmablit.c | 3 +-
drivers/infiniband/core/umem.c | 2 +-
drivers/infiniband/core/umem_odp.c | 8 +-
drivers/infiniband/hw/mthca/mthca_memfree.c | 3 +-
drivers/infiniband/hw/qib/qib_user_pages.c | 3 +-
drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +-
drivers/iommu/amd_iommu_v2.c | 1 +
drivers/media/pci/ivtv/ivtv-udma.c | 4 +-
drivers/media/pci/ivtv/ivtv-yuv.c | 10 +-
drivers/media/v4l2-core/videobuf-dma-sg.c | 3 +-
drivers/misc/mic/scif/scif_rma.c | 2 -
drivers/misc/sgi-gru/grufault.c | 3 +-
drivers/scsi/st.c | 2 -
drivers/staging/android/ashmem.c | 4 +-
drivers/video/fbdev/pvr2fb.c | 4 +-
drivers/virt/fsl_hypervisor.c | 5 +-
fs/exec.c | 8 +-
fs/proc/task_mmu.c | 14 ++
include/asm-generic/mm_hooks.h | 12 ++
include/linux/mm.h | 102 +++++++++--
include/linux/mman.h | 6 +-
include/linux/pkeys.h | 33 ++++
include/uapi/asm-generic/siginfo.h | 17 +-
kernel/events/uprobes.c | 10 +-
kernel/signal.c | 4 +
mm/Kconfig | 5 +
mm/frame_vector.c | 2 +-
mm/gup.c | 127 +++++++++++---
mm/ksm.c | 12 +-
mm/memory.c | 8 +-
mm/mempolicy.c | 6 +-
mm/mmap.c | 10 +-
mm/mprotect.c | 8 +-
mm/nommu.c | 66 ++++---
mm/process_vm_access.c | 11 +-
mm/util.c | 4 +-
net/ceph/pagevec.c | 2 +-
security/tomoyo/domain.c | 9 +-
virt/kvm/async_pf.c | 8 +-
virt/kvm/kvm_main.c | 10 +-
81 files changed, 1393 insertions(+), 233 deletions(-)
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: aarcange@redhat.com
Cc: akpm@linux-foundation.org
Cc: jack@suse.cz
Cc: kirill.shutemov@linux.intel.com
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: n-horiguchi@ah.jp.nec.com
Cc: x86@kernel.org
Cc: torvalds@linux-foundation.org
Cc: vbabka@suse.cz
[toc] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 18/33] x86, mm: simplify get_user_pages() PTE bit handling |
| Message-ID | <r1t8w-3TO-51@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
The current get_user_pages() code is a wee bit more complicated
than it needs to be for pte bit checking. Currently, it establishes
a mask of required pte _PAGE_* bits and ensures that the pte it
goes after has all those bits.
This consolidates the three identical copies of this code.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/mm/gup.c | 38 ++++++++++++++++++++++----------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff -puN arch/x86/mm/gup.c~pkeys-12-gup-swizzle arch/x86/mm/gup.c
--- a/arch/x86/mm/gup.c~pkeys-12-gup-swizzle 2016-02-12 10:44:21.572490755 -0800
+++ b/arch/x86/mm/gup.c 2016-02-12 10:44:21.575490892 -0800
@@ -75,6 +75,24 @@ static void undo_dev_pagemap(int *nr, in
}
/*
+ * 'pteval' can come from a pte, pmd or pud. We only check
+ * _PAGE_PRESENT, _PAGE_USER, and _PAGE_RW in here which are the
+ * same value on all 3 types.
+ */
+static inline int pte_allows_gup(unsigned long pteval, int write)
+{
+ unsigned long need_pte_bits = _PAGE_PRESENT|_PAGE_USER;
+
+ if (write)
+ need_pte_bits |= _PAGE_RW;
+
+ if ((pteval & need_pte_bits) != need_pte_bits)
+ return 0;
+
+ return 1;
+}
+
+/*
* The performance critical leaf functions are made noinline otherwise gcc
* inlines everything into a single function which results in too much
* register pressure.
@@ -83,14 +101,9 @@ static noinline int gup_pte_range(pmd_t
unsigned long end, int write, struct page **pages, int *nr)
{
struct dev_pagemap *pgmap = NULL;
- unsigned long mask;
int nr_start = *nr;
pte_t *ptep;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
-
ptep = pte_offset_map(&pmd, addr);
do {
pte_t pte = gup_get_pte(ptep);
@@ -110,7 +123,8 @@ static noinline int gup_pte_range(pmd_t
pte_unmap(ptep);
return 0;
}
- } else if ((pte_flags(pte) & (mask | _PAGE_SPECIAL)) != mask) {
+ } else if (!pte_allows_gup(pte_val(pte), write) ||
+ pte_special(pte)) {
pte_unmap(ptep);
return 0;
}
@@ -164,14 +178,10 @@ static int __gup_device_huge_pmd(pmd_t p
static noinline int gup_huge_pmd(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pmd_flags(pmd) & mask) != mask)
+ if (!pte_allows_gup(pmd_val(pmd), write))
return 0;
VM_BUG_ON(!pfn_valid(pmd_pfn(pmd)));
@@ -231,14 +241,10 @@ static int gup_pmd_range(pud_t pud, unsi
static noinline int gup_huge_pud(pud_t pud, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pud_flags(pud) & mask) != mask)
+ if (!pte_allows_gup(pud_val(pud), write))
return 0;
/* hugepages are never "special" */
VM_BUG_ON(pud_flags(pud) & _PAGE_SPECIAL);
_
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] x86/mm/gup: Simplify get_user_pages() PTE bit handling |
| Message-ID | <r3Dn4-2Bf-17@gated-at.bofh.it> |
| In reply to | #1333118 |
Commit-ID: 1874f6895c92d991ccf85edcc55a0d9dd552d71c
Gitweb: http://git.kernel.org/tip/1874f6895c92d991ccf85edcc55a0d9dd552d71c
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:02:18 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 18 Feb 2016 09:32:44 +0100
x86/mm/gup: Simplify get_user_pages() PTE bit handling
The current get_user_pages() code is a wee bit more complicated
than it needs to be for pte bit checking. Currently, it establishes
a mask of required pte _PAGE_* bits and ensures that the pte it
goes after has all those bits.
This consolidates the three identical copies of this code.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210218.3A2D4045@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/mm/gup.c | 38 ++++++++++++++++++++++----------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c
index ce5e454..2f0a329 100644
--- a/arch/x86/mm/gup.c
+++ b/arch/x86/mm/gup.c
@@ -75,6 +75,24 @@ static void undo_dev_pagemap(int *nr, int nr_start, struct page **pages)
}
/*
+ * 'pteval' can come from a pte, pmd or pud. We only check
+ * _PAGE_PRESENT, _PAGE_USER, and _PAGE_RW in here which are the
+ * same value on all 3 types.
+ */
+static inline int pte_allows_gup(unsigned long pteval, int write)
+{
+ unsigned long need_pte_bits = _PAGE_PRESENT|_PAGE_USER;
+
+ if (write)
+ need_pte_bits |= _PAGE_RW;
+
+ if ((pteval & need_pte_bits) != need_pte_bits)
+ return 0;
+
+ return 1;
+}
+
+/*
* The performance critical leaf functions are made noinline otherwise gcc
* inlines everything into a single function which results in too much
* register pressure.
@@ -83,14 +101,9 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
struct dev_pagemap *pgmap = NULL;
- unsigned long mask;
int nr_start = *nr;
pte_t *ptep;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
-
ptep = pte_offset_map(&pmd, addr);
do {
pte_t pte = gup_get_pte(ptep);
@@ -110,7 +123,8 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,
pte_unmap(ptep);
return 0;
}
- } else if ((pte_flags(pte) & (mask | _PAGE_SPECIAL)) != mask) {
+ } else if (!pte_allows_gup(pte_val(pte), write) ||
+ pte_special(pte)) {
pte_unmap(ptep);
return 0;
}
@@ -164,14 +178,10 @@ static int __gup_device_huge_pmd(pmd_t pmd, unsigned long addr,
static noinline int gup_huge_pmd(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pmd_flags(pmd) & mask) != mask)
+ if (!pte_allows_gup(pmd_val(pmd), write))
return 0;
VM_BUG_ON(!pfn_valid(pmd_pfn(pmd)));
@@ -231,14 +241,10 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,
static noinline int gup_huge_pud(pud_t pud, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pud_flags(pud) & mask) != mask)
+ if (!pte_allows_gup(pud_val(pud), write))
return 0;
/* hugepages are never "special" */
VM_BUG_ON(pud_flags(pud) & _PAGE_SPECIAL);
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 23/33] x86, pkeys: dump PKRU with other kernel registers |
| Message-ID | <r1t8w-3TO-55@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com> Protection Keys never affect kernel mappings. But, they can affect whether the kernel will fault when it touches a user mapping. The kernel doesn't touch user mappings without some careful choreography and these accesses don't generally result in oopses. But, if one does, we definitely want to have PKRU available so we can figure out if protection keys played a role. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/kernel/process_64.c | 2 ++ 1 file changed, 2 insertions(+) diff -puN arch/x86/kernel/process_64.c~pkeys-30-kernel-error-dumps arch/x86/kernel/process_64.c --- a/arch/x86/kernel/process_64.c~pkeys-30-kernel-error-dumps 2016-02-12 10:44:24.287614870 -0800 +++ b/arch/x86/kernel/process_64.c 2016-02-12 10:44:24.291615053 -0800 @@ -116,6 +116,8 @@ void __show_regs(struct pt_regs *regs, i printk(KERN_DEFAULT "DR0: %016lx DR1: %016lx DR2: %016lx\n", d0, d1, d2); printk(KERN_DEFAULT "DR3: %016lx DR6: %016lx DR7: %016lx\n", d3, d6, d7); + if (boot_cpu_has(X86_FEATURE_OSPKE)) + printk(KERN_DEFAULT "PKRU: %08x\n", read_pkru()); } void release_thread(struct task_struct *dead_task) _
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] x86/mm/pkeys: Dump PKRU with other kernel registers |
| Message-ID | <r3Dn4-2Bf-3@gated-at.bofh.it> |
| In reply to | #1333119 |
Commit-ID: c0b17b5bd4b7b98e7c6b67c9f69343b64711271b Gitweb: http://git.kernel.org/tip/c0b17b5bd4b7b98e7c6b67c9f69343b64711271b Author: Dave Hansen <dave.hansen@linux.intel.com> AuthorDate: Fri, 12 Feb 2016 13:02:25 -0800 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Thu, 18 Feb 2016 19:46:29 +0100 x86/mm/pkeys: Dump PKRU with other kernel registers Protection Keys never affect kernel mappings. But, they can affect whether the kernel will fault when it touches a user mapping. The kernel doesn't touch user mappings without some careful choreography and these accesses don't generally result in oopses. But, if one does, we definitely want to have PKRU available so we can figure out if protection keys played a role. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave@sr71.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/20160212210225.BF0D4482@viggo.jf.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/kernel/process_64.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index b9d99e0..776229e 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -116,6 +116,8 @@ void __show_regs(struct pt_regs *regs, int all) printk(KERN_DEFAULT "DR0: %016lx DR1: %016lx DR2: %016lx\n", d0, d1, d2); printk(KERN_DEFAULT "DR3: %016lx DR6: %016lx DR7: %016lx\n", d3, d6, d7); + if (boot_cpu_has(X86_FEATURE_OSPKE)) + printk(KERN_DEFAULT "PKRU: %08x\n", read_pkru()); } void release_thread(struct task_struct *dead_task)
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 30/33] x86, fpu: allow setting of XSAVE state |
| Message-ID | <r1t8w-3TO-53@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
We want to modify the Protection Key rights inside the kernel, so
we need to change PKRU's contents. But, if we do a plain
'wrpkru', when we return to userspace we might do an XRSTOR and
wipe out the kernel's 'wrpkru'. So, we need to go after PKRU in
the xsave buffer.
We do this by:
1. Ensuring that we have the XSAVE registers (fpregs) in the
kernel FPU buffer (fpstate)
2. Looking up the location of a given state in the buffer
3. Filling in the stat
4. Ensuring that the hardware knows that state is present there
(basically that the 'init optimization' is not in place).
5. Copying the newly-modified state back to the registers if
necessary.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/include/asm/fpu/internal.h | 2
b/arch/x86/kernel/fpu/core.c | 63 +++++++++++++++++++++
b/arch/x86/kernel/fpu/xstate.c | 98 +++++++++++++++++++++++++++++++++-
3 files changed, 161 insertions(+), 2 deletions(-)
diff -puN arch/x86/include/asm/fpu/internal.h~pkeys-76-xsave-set arch/x86/include/asm/fpu/internal.h
--- a/arch/x86/include/asm/fpu/internal.h~pkeys-76-xsave-set 2016-02-12 10:44:27.469760334 -0800
+++ b/arch/x86/include/asm/fpu/internal.h 2016-02-12 10:44:27.475760609 -0800
@@ -24,6 +24,8 @@
extern void fpu__activate_curr(struct fpu *fpu);
extern void fpu__activate_fpstate_read(struct fpu *fpu);
extern void fpu__activate_fpstate_write(struct fpu *fpu);
+extern void fpu__current_fpstate_write_begin(void);
+extern void fpu__current_fpstate_write_end(void);
extern void fpu__save(struct fpu *fpu);
extern void fpu__restore(struct fpu *fpu);
extern int fpu__restore_sig(void __user *buf, int ia32_frame);
diff -puN arch/x86/kernel/fpu/core.c~pkeys-76-xsave-set arch/x86/kernel/fpu/core.c
--- a/arch/x86/kernel/fpu/core.c~pkeys-76-xsave-set 2016-02-12 10:44:27.470760380 -0800
+++ b/arch/x86/kernel/fpu/core.c 2016-02-12 10:44:27.476760654 -0800
@@ -352,6 +352,69 @@ void fpu__activate_fpstate_write(struct
}
/*
+ * This function must be called before we write the current
+ * task's fpstate.
+ *
+ * This call gets the current FPU register state and moves
+ * it in to the 'fpstate'. Preemption is disabled so that
+ * no writes to the 'fpstate' can occur from context
+ * swiches.
+ *
+ * Must be followed by a fpu__current_fpstate_write_end().
+ */
+void fpu__current_fpstate_write_begin(void)
+{
+ struct fpu *fpu = ¤t->thread.fpu;
+
+ /*
+ * Ensure that the context-switching code does not write
+ * over the fpstate while we are doing our update.
+ */
+ preempt_disable();
+
+ /*
+ * Move the fpregs in to the fpu's 'fpstate'.
+ */
+ fpu__activate_fpstate_read(fpu);
+
+ /*
+ * The caller is about to write to 'fpu'. Ensure that no
+ * CPU thinks that its fpregs match the fpstate. This
+ * ensures we will not be lazy and skip a XRSTOR in the
+ * future.
+ */
+ fpu->last_cpu = -1;
+}
+
+/*
+ * This function must be paired with fpu__current_fpstate_write_begin()
+ *
+ * This will ensure that the modified fpstate gets placed back in
+ * the fpregs if necessary.
+ *
+ * Note: This function may be called whether or not an _actual_
+ * write to the fpstate occurred.
+ */
+void fpu__current_fpstate_write_end(void)
+{
+ struct fpu *fpu = ¤t->thread.fpu;
+
+ /*
+ * 'fpu' now has an updated copy of the state, but the
+ * registers may still be out of date. Update them with
+ * an XRSTOR if they are active.
+ */
+ if (fpregs_active())
+ copy_kernel_to_fpregs(&fpu->state);
+
+ /*
+ * Our update is done and the fpregs/fpstate are in sync
+ * if necessary. Context switches can happen again.
+ */
+ preempt_enable();
+}
+
+/*
* 'fpu__restore()' is called to copy FPU registers from
* the FPU fpstate to the live hw registers and to activate
* access to the hardware registers, so that FPU instructions
diff -puN arch/x86/kernel/fpu/xstate.c~pkeys-76-xsave-set arch/x86/kernel/fpu/xstate.c
--- a/arch/x86/kernel/fpu/xstate.c~pkeys-76-xsave-set 2016-02-12 10:44:27.472760471 -0800
+++ b/arch/x86/kernel/fpu/xstate.c 2016-02-12 10:44:27.476760654 -0800
@@ -679,6 +679,19 @@ void fpu__resume_cpu(void)
}
/*
+ * Given an xstate feature mask, calculate where in the xsave
+ * buffer the state is. Callers should ensure that the buffer
+ * is valid.
+ *
+ * Note: does not work for compacted buffers.
+ */
+void *__raw_xsave_addr(struct xregs_state *xsave, int xstate_feature_mask)
+{
+ int feature_nr = fls64(xstate_feature_mask) - 1;
+
+ return (void *)xsave + xstate_comp_offsets[feature_nr];
+}
+/*
* Given the xsave area and a state inside, this function returns the
* address of the state.
*
@@ -698,7 +711,6 @@ void fpu__resume_cpu(void)
*/
void *get_xsave_addr(struct xregs_state *xsave, int xstate_feature)
{
- int feature_nr = fls64(xstate_feature) - 1;
/*
* Do we even *have* xsave state?
*/
@@ -726,7 +738,7 @@ void *get_xsave_addr(struct xregs_state
if (!(xsave->header.xfeatures & xstate_feature))
return NULL;
- return (void *)xsave + xstate_comp_offsets[feature_nr];
+ return __raw_xsave_addr(xsave, xstate_feature);
}
EXPORT_SYMBOL_GPL(get_xsave_addr);
@@ -761,3 +773,85 @@ const void *get_xsave_field_ptr(int xsav
return get_xsave_addr(&fpu->state.xsave, xsave_state);
}
+
+
+/*
+ * Set xfeatures (aka XSTATE_BV) bit for a feature that we want
+ * to take out of its "init state". This will ensure that an
+ * XRSTOR actually restores the state.
+ */
+static void fpu__xfeature_set_non_init(struct xregs_state *xsave,
+ int xstate_feature_mask)
+{
+ xsave->header.xfeatures |= xstate_feature_mask;
+}
+
+/*
+ * This function is safe to call whether the FPU is in use or not.
+ *
+ * Note that this only works on the current task.
+ *
+ * Inputs:
+ * @xsave_state: state which is defined in xsave.h (e.g. XFEATURE_MASK_FP,
+ * XFEATURE_MASK_SSE, etc...)
+ * @xsave_state_ptr: a pointer to a copy of the state that you would
+ * like written in to the current task's FPU xsave state. This pointer
+ * must not be located in the current tasks's xsave area.
+ * Output:
+ * address of the state in the xsave area or NULL if the state
+ * is not present or is in its 'init state'.
+ */
+static void fpu__xfeature_set_state(int xstate_feature_mask,
+ void *xstate_feature_src, size_t len)
+{
+ struct xregs_state *xsave = ¤t->thread.fpu.state.xsave;
+ struct fpu *fpu = ¤t->thread.fpu;
+ void *dst;
+
+ if (!boot_cpu_has(X86_FEATURE_XSAVE)) {
+ WARN_ONCE(1, "%s() attempted with no xsave support", __func__);
+ return;
+ }
+
+ /*
+ * Tell the FPU code that we need the FPU state to be in
+ * 'fpu' (not in the registers), and that we need it to
+ * be stable while we write to it.
+ */
+ fpu__current_fpstate_write_begin();
+
+ /*
+ * This method *WILL* *NOT* work for compact-format
+ * buffers. If the 'xstate_feature_mask' is unset in
+ * xcomp_bv then we may need to move other feature state
+ * "up" in the buffer.
+ */
+ if (xsave->header.xcomp_bv & xstate_feature_mask) {
+ WARN_ON_ONCE(1);
+ goto out;
+ }
+
+ /* find the location in the xsave buffer of the desired state */
+ dst = __raw_xsave_addr(&fpu->state.xsave, xstate_feature_mask);
+
+ /*
+ * Make sure that the pointer being passed in did not
+ * come from the xsave buffer itself.
+ */
+ WARN_ONCE(xstate_feature_src == dst, "set from xsave buffer itself");
+
+ /* put the caller-provided data in the location */
+ memcpy(dst, xstate_feature_src, len);
+
+ /*
+ * Mark the xfeature so that the CPU knows there is state
+ * in the buffer now.
+ */
+ fpu__xfeature_set_non_init(xsave, xstate_feature_mask);
+out:
+ /*
+ * We are done writing to the 'fpu'. Reenable preeption
+ * and (possibly) move the fpstate back in to the fpregs.
+ */
+ fpu__current_fpstate_write_end();
+}
_
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] x86/fpu: Allow setting of XSAVE state |
| Message-ID | <r3Dn5-2Bf-25@gated-at.bofh.it> |
| In reply to | #1333120 |
Commit-ID: b8b9b6ba9dec3f155c7555cb208ba4078e97aedb
Gitweb: http://git.kernel.org/tip/b8b9b6ba9dec3f155c7555cb208ba4078e97aedb
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:02:35 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 18 Feb 2016 19:46:32 +0100
x86/fpu: Allow setting of XSAVE state
We want to modify the Protection Key rights inside the kernel, so
we need to change PKRU's contents. But, if we do a plain
'wrpkru', when we return to userspace we might do an XRSTOR and
wipe out the kernel's 'wrpkru'. So, we need to go after PKRU in
the xsave buffer.
We do this by:
1. Ensuring that we have the XSAVE registers (fpregs) in the
kernel FPU buffer (fpstate)
2. Looking up the location of a given state in the buffer
3. Filling in the stat
4. Ensuring that the hardware knows that state is present there
(basically that the 'init optimization' is not in place).
5. Copying the newly-modified state back to the registers if
necessary.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210235.5A3139BF@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/include/asm/fpu/internal.h | 2 +
arch/x86/kernel/fpu/core.c | 63 ++++++++++++++++++++++++
arch/x86/kernel/fpu/xstate.c | 98 ++++++++++++++++++++++++++++++++++++-
3 files changed, 161 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h
index a212434..31ac8e6 100644
--- a/arch/x86/include/asm/fpu/internal.h
+++ b/arch/x86/include/asm/fpu/internal.h
@@ -25,6 +25,8 @@
extern void fpu__activate_curr(struct fpu *fpu);
extern void fpu__activate_fpstate_read(struct fpu *fpu);
extern void fpu__activate_fpstate_write(struct fpu *fpu);
+extern void fpu__current_fpstate_write_begin(void);
+extern void fpu__current_fpstate_write_end(void);
extern void fpu__save(struct fpu *fpu);
extern void fpu__restore(struct fpu *fpu);
extern int fpu__restore_sig(void __user *buf, int ia32_frame);
diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c
index 299b58b..dea8e76 100644
--- a/arch/x86/kernel/fpu/core.c
+++ b/arch/x86/kernel/fpu/core.c
@@ -354,6 +354,69 @@ void fpu__activate_fpstate_write(struct fpu *fpu)
}
/*
+ * This function must be called before we write the current
+ * task's fpstate.
+ *
+ * This call gets the current FPU register state and moves
+ * it in to the 'fpstate'. Preemption is disabled so that
+ * no writes to the 'fpstate' can occur from context
+ * swiches.
+ *
+ * Must be followed by a fpu__current_fpstate_write_end().
+ */
+void fpu__current_fpstate_write_begin(void)
+{
+ struct fpu *fpu = ¤t->thread.fpu;
+
+ /*
+ * Ensure that the context-switching code does not write
+ * over the fpstate while we are doing our update.
+ */
+ preempt_disable();
+
+ /*
+ * Move the fpregs in to the fpu's 'fpstate'.
+ */
+ fpu__activate_fpstate_read(fpu);
+
+ /*
+ * The caller is about to write to 'fpu'. Ensure that no
+ * CPU thinks that its fpregs match the fpstate. This
+ * ensures we will not be lazy and skip a XRSTOR in the
+ * future.
+ */
+ fpu->last_cpu = -1;
+}
+
+/*
+ * This function must be paired with fpu__current_fpstate_write_begin()
+ *
+ * This will ensure that the modified fpstate gets placed back in
+ * the fpregs if necessary.
+ *
+ * Note: This function may be called whether or not an _actual_
+ * write to the fpstate occurred.
+ */
+void fpu__current_fpstate_write_end(void)
+{
+ struct fpu *fpu = ¤t->thread.fpu;
+
+ /*
+ * 'fpu' now has an updated copy of the state, but the
+ * registers may still be out of date. Update them with
+ * an XRSTOR if they are active.
+ */
+ if (fpregs_active())
+ copy_kernel_to_fpregs(&fpu->state);
+
+ /*
+ * Our update is done and the fpregs/fpstate are in sync
+ * if necessary. Context switches can happen again.
+ */
+ preempt_enable();
+}
+
+/*
* 'fpu__restore()' is called to copy FPU registers from
* the FPU fpstate to the live hw registers and to activate
* access to the hardware registers, so that FPU instructions
diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c
index a63ca80..30d144f 100644
--- a/arch/x86/kernel/fpu/xstate.c
+++ b/arch/x86/kernel/fpu/xstate.c
@@ -679,6 +679,19 @@ void fpu__resume_cpu(void)
}
/*
+ * Given an xstate feature mask, calculate where in the xsave
+ * buffer the state is. Callers should ensure that the buffer
+ * is valid.
+ *
+ * Note: does not work for compacted buffers.
+ */
+void *__raw_xsave_addr(struct xregs_state *xsave, int xstate_feature_mask)
+{
+ int feature_nr = fls64(xstate_feature_mask) - 1;
+
+ return (void *)xsave + xstate_comp_offsets[feature_nr];
+}
+/*
* Given the xsave area and a state inside, this function returns the
* address of the state.
*
@@ -698,7 +711,6 @@ void fpu__resume_cpu(void)
*/
void *get_xsave_addr(struct xregs_state *xsave, int xstate_feature)
{
- int feature_nr = fls64(xstate_feature) - 1;
/*
* Do we even *have* xsave state?
*/
@@ -726,7 +738,7 @@ void *get_xsave_addr(struct xregs_state *xsave, int xstate_feature)
if (!(xsave->header.xfeatures & xstate_feature))
return NULL;
- return (void *)xsave + xstate_comp_offsets[feature_nr];
+ return __raw_xsave_addr(xsave, xstate_feature);
}
EXPORT_SYMBOL_GPL(get_xsave_addr);
@@ -761,3 +773,85 @@ const void *get_xsave_field_ptr(int xsave_state)
return get_xsave_addr(&fpu->state.xsave, xsave_state);
}
+
+
+/*
+ * Set xfeatures (aka XSTATE_BV) bit for a feature that we want
+ * to take out of its "init state". This will ensure that an
+ * XRSTOR actually restores the state.
+ */
+static void fpu__xfeature_set_non_init(struct xregs_state *xsave,
+ int xstate_feature_mask)
+{
+ xsave->header.xfeatures |= xstate_feature_mask;
+}
+
+/*
+ * This function is safe to call whether the FPU is in use or not.
+ *
+ * Note that this only works on the current task.
+ *
+ * Inputs:
+ * @xsave_state: state which is defined in xsave.h (e.g. XFEATURE_MASK_FP,
+ * XFEATURE_MASK_SSE, etc...)
+ * @xsave_state_ptr: a pointer to a copy of the state that you would
+ * like written in to the current task's FPU xsave state. This pointer
+ * must not be located in the current tasks's xsave area.
+ * Output:
+ * address of the state in the xsave area or NULL if the state
+ * is not present or is in its 'init state'.
+ */
+static void fpu__xfeature_set_state(int xstate_feature_mask,
+ void *xstate_feature_src, size_t len)
+{
+ struct xregs_state *xsave = ¤t->thread.fpu.state.xsave;
+ struct fpu *fpu = ¤t->thread.fpu;
+ void *dst;
+
+ if (!boot_cpu_has(X86_FEATURE_XSAVE)) {
+ WARN_ONCE(1, "%s() attempted with no xsave support", __func__);
+ return;
+ }
+
+ /*
+ * Tell the FPU code that we need the FPU state to be in
+ * 'fpu' (not in the registers), and that we need it to
+ * be stable while we write to it.
+ */
+ fpu__current_fpstate_write_begin();
+
+ /*
+ * This method *WILL* *NOT* work for compact-format
+ * buffers. If the 'xstate_feature_mask' is unset in
+ * xcomp_bv then we may need to move other feature state
+ * "up" in the buffer.
+ */
+ if (xsave->header.xcomp_bv & xstate_feature_mask) {
+ WARN_ON_ONCE(1);
+ goto out;
+ }
+
+ /* find the location in the xsave buffer of the desired state */
+ dst = __raw_xsave_addr(&fpu->state.xsave, xstate_feature_mask);
+
+ /*
+ * Make sure that the pointer being passed in did not
+ * come from the xsave buffer itself.
+ */
+ WARN_ONCE(xstate_feature_src == dst, "set from xsave buffer itself");
+
+ /* put the caller-provided data in the location */
+ memcpy(dst, xstate_feature_src, len);
+
+ /*
+ * Mark the xfeature so that the CPU knows there is state
+ * in the buffer now.
+ */
+ fpu__xfeature_set_non_init(xsave, xstate_feature_mask);
+out:
+ /*
+ * We are done writing to the 'fpu'. Reenable preeption
+ * and (possibly) move the fpstate back in to the fpregs.
+ */
+ fpu__current_fpstate_write_end();
+}
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 03/33] mm, gup: switch callers of get_user_pages() to not pass tsk/mm |
| Message-ID | <r1t8w-3TO-57@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
We will soon modify the vanilla get_user_pages() so it can no
longer be used on mm/tasks other than 'current/current->mm',
which is by far the most common way it is called. For now,
we allow the old-style calls, but warn when they are used.
(implemented in previous patch)
This patch switches all callers of:
get_user_pages()
get_user_pages_unlocked()
get_user_pages_locked()
to stop passing tsk/mm so they will no longer see the warnings.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: jack@suse.cz
---
b/arch/cris/arch-v32/drivers/cryptocop.c | 8 ++------
b/arch/ia64/kernel/err_inject.c | 3 +--
b/arch/mips/mm/gup.c | 3 +--
b/arch/s390/mm/gup.c | 4 +---
b/arch/sh/mm/gup.c | 2 +-
b/arch/sparc/mm/gup.c | 2 +-
b/arch/x86/mm/gup.c | 2 +-
b/arch/x86/mm/mpx.c | 4 ++--
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +--
b/drivers/gpu/drm/radeon/radeon_ttm.c | 3 +--
b/drivers/gpu/drm/via/via_dmablit.c | 3 +--
b/drivers/infiniband/core/umem.c | 2 +-
b/drivers/infiniband/hw/mthca/mthca_memfree.c | 3 +--
b/drivers/infiniband/hw/qib/qib_user_pages.c | 3 +--
b/drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +-
b/drivers/media/pci/ivtv/ivtv-udma.c | 4 ++--
b/drivers/media/pci/ivtv/ivtv-yuv.c | 10 ++++------
b/drivers/media/v4l2-core/videobuf-dma-sg.c | 3 +--
b/drivers/misc/mic/scif/scif_rma.c | 2 --
b/drivers/misc/sgi-gru/grufault.c | 3 +--
b/drivers/scsi/st.c | 2 --
b/drivers/video/fbdev/pvr2fb.c | 4 ++--
b/drivers/virt/fsl_hypervisor.c | 5 ++---
b/mm/frame_vector.c | 2 +-
b/mm/gup.c | 6 ++++--
b/mm/ksm.c | 2 +-
b/mm/mempolicy.c | 6 +++---
b/net/ceph/pagevec.c | 2 +-
b/virt/kvm/kvm_main.c | 10 +++++-----
29 files changed, 44 insertions(+), 64 deletions(-)
diff -puN arch/cris/arch-v32/drivers/cryptocop.c~get_current_user_pages arch/cris/arch-v32/drivers/cryptocop.c
--- a/arch/cris/arch-v32/drivers/cryptocop.c~get_current_user_pages 2016-02-12 10:44:14.365161289 -0800
+++ b/arch/cris/arch-v32/drivers/cryptocop.c 2016-02-12 10:44:14.415163575 -0800
@@ -2719,9 +2719,7 @@ static int cryptocop_ioctl_process(struc
/* Acquire the mm page semaphore. */
down_read(¤t->mm->mmap_sem);
- err = get_user_pages(current,
- current->mm,
- (unsigned long int)(oper.indata + prev_ix),
+ err = get_user_pages((unsigned long int)(oper.indata + prev_ix),
noinpages,
0, /* read access only for in data */
0, /* no force */
@@ -2736,9 +2734,7 @@ static int cryptocop_ioctl_process(struc
}
noinpages = err;
if (oper.do_cipher){
- err = get_user_pages(current,
- current->mm,
- (unsigned long int)oper.cipher_outdata,
+ err = get_user_pages((unsigned long int)oper.cipher_outdata,
nooutpages,
1, /* write access for out data */
0, /* no force */
diff -puN arch/ia64/kernel/err_inject.c~get_current_user_pages arch/ia64/kernel/err_inject.c
--- a/arch/ia64/kernel/err_inject.c~get_current_user_pages 2016-02-12 10:44:14.367161380 -0800
+++ b/arch/ia64/kernel/err_inject.c 2016-02-12 10:44:14.416163620 -0800
@@ -142,8 +142,7 @@ store_virtual_to_phys(struct device *dev
u64 virt_addr=simple_strtoull(buf, NULL, 16);
int ret;
- ret = get_user_pages(current, current->mm, virt_addr,
- 1, VM_READ, 0, NULL, NULL);
+ ret = get_user_pages(virt_addr, 1, VM_READ, 0, NULL, NULL);
if (ret<=0) {
#ifdef ERR_INJ_DEBUG
printk("Virtual address %lx is not existing.\n",virt_addr);
diff -puN arch/mips/mm/gup.c~get_current_user_pages arch/mips/mm/gup.c
--- a/arch/mips/mm/gup.c~get_current_user_pages 2016-02-12 10:44:14.368161426 -0800
+++ b/arch/mips/mm/gup.c 2016-02-12 10:44:14.416163620 -0800
@@ -286,8 +286,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- (end - start) >> PAGE_SHIFT,
+ ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT,
write, 0, pages);
/* Have to be a bit careful with return values */
diff -puN arch/s390/mm/gup.c~get_current_user_pages arch/s390/mm/gup.c
--- a/arch/s390/mm/gup.c~get_current_user_pages 2016-02-12 10:44:14.370161517 -0800
+++ b/arch/s390/mm/gup.c 2016-02-12 10:44:14.416163620 -0800
@@ -210,7 +210,6 @@ int __get_user_pages_fast(unsigned long
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
- struct mm_struct *mm = current->mm;
int nr, ret;
might_sleep();
@@ -222,8 +221,7 @@ int get_user_pages_fast(unsigned long st
/* Try to get the remaining pages with get_user_pages */
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- nr_pages - nr, write, 0, pages);
+ ret = get_user_pages_unlocked(start, nr_pages - nr, write, 0, pages);
/* Have to be a bit careful with return values */
if (nr > 0)
ret = (ret < 0) ? nr : ret + nr;
diff -puN arch/sh/mm/gup.c~get_current_user_pages arch/sh/mm/gup.c
--- a/arch/sh/mm/gup.c~get_current_user_pages 2016-02-12 10:44:14.371161563 -0800
+++ b/arch/sh/mm/gup.c 2016-02-12 10:44:14.417163666 -0800
@@ -257,7 +257,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT, write, 0, pages);
/* Have to be a bit careful with return values */
diff -puN arch/sparc/mm/gup.c~get_current_user_pages arch/sparc/mm/gup.c
--- a/arch/sparc/mm/gup.c~get_current_user_pages 2016-02-12 10:44:14.373161655 -0800
+++ b/arch/sparc/mm/gup.c 2016-02-12 10:44:14.417163666 -0800
@@ -237,7 +237,7 @@ slow:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT, write, 0, pages);
/* Have to be a bit careful with return values */
diff -puN arch/x86/mm/gup.c~get_current_user_pages arch/x86/mm/gup.c
--- a/arch/x86/mm/gup.c~get_current_user_pages 2016-02-12 10:44:14.375161746 -0800
+++ b/arch/x86/mm/gup.c 2016-02-12 10:44:14.417163666 -0800
@@ -422,7 +422,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT,
write, 0, pages);
diff -puN arch/x86/mm/mpx.c~get_current_user_pages arch/x86/mm/mpx.c
--- a/arch/x86/mm/mpx.c~get_current_user_pages 2016-02-12 10:44:14.376161792 -0800
+++ b/arch/x86/mm/mpx.c 2016-02-12 10:44:14.418163712 -0800
@@ -546,8 +546,8 @@ static int mpx_resolve_fault(long __user
int nr_pages = 1;
int force = 0;
- gup_ret = get_user_pages(current, current->mm, (unsigned long)addr,
- nr_pages, write, force, NULL, NULL);
+ gup_ret = get_user_pages((unsigned long)addr, nr_pages, write,
+ force, NULL, NULL);
/*
* get_user_pages() returns number of pages gotten.
* 0 means we failed to fault in and get anything,
diff -puN drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c~get_current_user_pages drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c~get_current_user_pages 2016-02-12 10:44:14.378161883 -0800
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 2016-02-12 10:44:14.418163712 -0800
@@ -518,8 +518,7 @@ static int amdgpu_ttm_tt_pin_userptr(str
uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE;
struct page **pages = ttm->pages + pinned;
- r = get_user_pages(current, current->mm, userptr, num_pages,
- write, 0, pages, NULL);
+ r = get_user_pages(userptr, num_pages, write, 0, pages, NULL);
if (r < 0)
goto release_pages;
diff -puN drivers/gpu/drm/radeon/radeon_ttm.c~get_current_user_pages drivers/gpu/drm/radeon/radeon_ttm.c
--- a/drivers/gpu/drm/radeon/radeon_ttm.c~get_current_user_pages 2016-02-12 10:44:14.380161975 -0800
+++ b/drivers/gpu/drm/radeon/radeon_ttm.c 2016-02-12 10:44:14.419163758 -0800
@@ -554,8 +554,7 @@ static int radeon_ttm_tt_pin_userptr(str
uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE;
struct page **pages = ttm->pages + pinned;
- r = get_user_pages(current, current->mm, userptr, num_pages,
- write, 0, pages, NULL);
+ r = get_user_pages(userptr, num_pages, write, 0, pages, NULL);
if (r < 0)
goto release_pages;
diff -puN drivers/gpu/drm/via/via_dmablit.c~get_current_user_pages drivers/gpu/drm/via/via_dmablit.c
--- a/drivers/gpu/drm/via/via_dmablit.c~get_current_user_pages 2016-02-12 10:44:14.381162020 -0800
+++ b/drivers/gpu/drm/via/via_dmablit.c 2016-02-12 10:44:14.419163758 -0800
@@ -239,8 +239,7 @@ via_lock_all_dma_pages(drm_via_sg_info_t
if (NULL == vsg->pages)
return -ENOMEM;
down_read(¤t->mm->mmap_sem);
- ret = get_user_pages(current, current->mm,
- (unsigned long)xfer->mem_addr,
+ ret = get_user_pages((unsigned long)xfer->mem_addr,
vsg->num_pages,
(vsg->direction == DMA_FROM_DEVICE),
0, vsg->pages, NULL);
diff -puN drivers/infiniband/core/umem.c~get_current_user_pages drivers/infiniband/core/umem.c
--- a/drivers/infiniband/core/umem.c~get_current_user_pages 2016-02-12 10:44:14.383162112 -0800
+++ b/drivers/infiniband/core/umem.c 2016-02-12 10:44:14.420163803 -0800
@@ -188,7 +188,7 @@ struct ib_umem *ib_umem_get(struct ib_uc
sg_list_start = umem->sg_head.sgl;
while (npages) {
- ret = get_user_pages(current, current->mm, cur_base,
+ ret = get_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof (struct page *)),
1, !umem->writable, page_list, vma_list);
diff -puN drivers/infiniband/hw/mthca/mthca_memfree.c~get_current_user_pages drivers/infiniband/hw/mthca/mthca_memfree.c
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c~get_current_user_pages 2016-02-12 10:44:14.385162203 -0800
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c 2016-02-12 10:44:14.420163803 -0800
@@ -472,8 +472,7 @@ int mthca_map_user_db(struct mthca_dev *
goto out;
}
- ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- pages, NULL);
+ ret = get_user_pages(uaddr & PAGE_MASK, 1, 1, 0, pages, NULL);
if (ret < 0)
goto out;
diff -puN drivers/infiniband/hw/qib/qib_user_pages.c~get_current_user_pages drivers/infiniband/hw/qib/qib_user_pages.c
--- a/drivers/infiniband/hw/qib/qib_user_pages.c~get_current_user_pages 2016-02-12 10:44:14.386162249 -0800
+++ b/drivers/infiniband/hw/qib/qib_user_pages.c 2016-02-12 10:44:14.420163803 -0800
@@ -66,8 +66,7 @@ static int __qib_get_user_pages(unsigned
}
for (got = 0; got < num_pages; got += ret) {
- ret = get_user_pages(current, current->mm,
- start_page + got * PAGE_SIZE,
+ ret = get_user_pages(start_page + got * PAGE_SIZE,
num_pages - got, 1, 1,
p + got, NULL);
if (ret < 0)
diff -puN drivers/infiniband/hw/usnic/usnic_uiom.c~get_current_user_pages drivers/infiniband/hw/usnic/usnic_uiom.c
--- a/drivers/infiniband/hw/usnic/usnic_uiom.c~get_current_user_pages 2016-02-12 10:44:14.388162340 -0800
+++ b/drivers/infiniband/hw/usnic/usnic_uiom.c 2016-02-12 10:44:14.421163849 -0800
@@ -144,7 +144,7 @@ static int usnic_uiom_get_pages(unsigned
ret = 0;
while (npages) {
- ret = get_user_pages(current, current->mm, cur_base,
+ ret = get_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof(struct page *)),
1, !writable, page_list, NULL);
diff -puN drivers/media/pci/ivtv/ivtv-udma.c~get_current_user_pages drivers/media/pci/ivtv/ivtv-udma.c
--- a/drivers/media/pci/ivtv/ivtv-udma.c~get_current_user_pages 2016-02-12 10:44:14.389162386 -0800
+++ b/drivers/media/pci/ivtv/ivtv-udma.c 2016-02-12 10:44:14.421163849 -0800
@@ -124,8 +124,8 @@ int ivtv_udma_setup(struct ivtv *itv, un
}
/* Get user pages for DMA Xfer */
- err = get_user_pages_unlocked(current, current->mm,
- user_dma.uaddr, user_dma.page_count, 0, 1, dma->map);
+ err = get_user_pages_unlocked(user_dma.uaddr, user_dma.page_count, 0,
+ 1, dma->map);
if (user_dma.page_count != err) {
IVTV_DEBUG_WARN("failed to map user pages, returned %d instead of %d\n",
diff -puN drivers/media/pci/ivtv/ivtv-yuv.c~get_current_user_pages drivers/media/pci/ivtv/ivtv-yuv.c
--- a/drivers/media/pci/ivtv/ivtv-yuv.c~get_current_user_pages 2016-02-12 10:44:14.391162477 -0800
+++ b/drivers/media/pci/ivtv/ivtv-yuv.c 2016-02-12 10:44:14.422163895 -0800
@@ -75,14 +75,12 @@ static int ivtv_yuv_prep_user_dma(struct
ivtv_udma_get_page_info (&uv_dma, (unsigned long)args->uv_source, 360 * uv_decode_height);
/* Get user pages for DMA Xfer */
- y_pages = get_user_pages_unlocked(current, current->mm,
- y_dma.uaddr, y_dma.page_count, 0, 1,
- &dma->map[0]);
+ y_pages = get_user_pages_unlocked(y_dma.uaddr,
+ y_dma.page_count, 0, 1, &dma->map[0]);
uv_pages = 0; /* silence gcc. value is set and consumed only if: */
if (y_pages == y_dma.page_count) {
- uv_pages = get_user_pages_unlocked(current, current->mm,
- uv_dma.uaddr, uv_dma.page_count, 0, 1,
- &dma->map[y_pages]);
+ uv_pages = get_user_pages_unlocked(uv_dma.uaddr,
+ uv_dma.page_count, 0, 1, &dma->map[y_pages]);
}
if (y_pages != y_dma.page_count || uv_pages != uv_dma.page_count) {
diff -puN drivers/media/v4l2-core/videobuf-dma-sg.c~get_current_user_pages drivers/media/v4l2-core/videobuf-dma-sg.c
--- a/drivers/media/v4l2-core/videobuf-dma-sg.c~get_current_user_pages 2016-02-12 10:44:14.393162569 -0800
+++ b/drivers/media/v4l2-core/videobuf-dma-sg.c 2016-02-12 10:44:14.422163895 -0800
@@ -181,8 +181,7 @@ static int videobuf_dma_init_user_locked
dprintk(1, "init user [0x%lx+0x%lx => %d pages]\n",
data, size, dma->nr_pages);
- err = get_user_pages(current, current->mm,
- data & PAGE_MASK, dma->nr_pages,
+ err = get_user_pages(data & PAGE_MASK, dma->nr_pages,
rw == READ, 1, /* force */
dma->pages, NULL);
diff -puN drivers/misc/mic/scif/scif_rma.c~get_current_user_pages drivers/misc/mic/scif/scif_rma.c
--- a/drivers/misc/mic/scif/scif_rma.c~get_current_user_pages 2016-02-12 10:44:14.395162660 -0800
+++ b/drivers/misc/mic/scif/scif_rma.c 2016-02-12 10:44:14.423163940 -0800
@@ -1394,8 +1394,6 @@ retry:
}
pinned_pages->nr_pages = get_user_pages(
- current,
- mm,
(u64)addr,
nr_pages,
!!(prot & SCIF_PROT_WRITE),
diff -puN drivers/misc/sgi-gru/grufault.c~get_current_user_pages drivers/misc/sgi-gru/grufault.c
--- a/drivers/misc/sgi-gru/grufault.c~get_current_user_pages 2016-02-12 10:44:14.396162706 -0800
+++ b/drivers/misc/sgi-gru/grufault.c 2016-02-12 10:44:14.423163940 -0800
@@ -198,8 +198,7 @@ static int non_atomic_pte_lookup(struct
#else
*pageshift = PAGE_SHIFT;
#endif
- if (get_user_pages
- (current, current->mm, vaddr, 1, write, 0, &page, NULL) <= 0)
+ if (get_user_pages(vaddr, 1, write, 0, &page, NULL) <= 0)
return -EFAULT;
*paddr = page_to_phys(page);
put_page(page);
diff -puN drivers/scsi/st.c~get_current_user_pages drivers/scsi/st.c
--- a/drivers/scsi/st.c~get_current_user_pages 2016-02-12 10:44:14.398162797 -0800
+++ b/drivers/scsi/st.c 2016-02-12 10:44:14.424163986 -0800
@@ -4817,8 +4817,6 @@ static int sgl_map_user_pages(struct st_
/* Try to fault in all of the necessary pages */
/* rw==READ means read from drive, write into memory area */
res = get_user_pages_unlocked(
- current,
- current->mm,
uaddr,
nr_pages,
rw == READ,
diff -puN drivers/video/fbdev/pvr2fb.c~get_current_user_pages drivers/video/fbdev/pvr2fb.c
--- a/drivers/video/fbdev/pvr2fb.c~get_current_user_pages 2016-02-12 10:44:14.400162889 -0800
+++ b/drivers/video/fbdev/pvr2fb.c 2016-02-12 10:44:14.425164032 -0800
@@ -686,8 +686,8 @@ static ssize_t pvr2fb_write(struct fb_in
if (!pages)
return -ENOMEM;
- ret = get_user_pages_unlocked(current, current->mm, (unsigned long)buf,
- nr_pages, WRITE, 0, pages);
+ ret = get_user_pages_unlocked((unsigned long)buf, nr_pages, WRITE,
+ 0, pages);
if (ret < nr_pages) {
nr_pages = ret;
diff -puN drivers/virt/fsl_hypervisor.c~get_current_user_pages drivers/virt/fsl_hypervisor.c
--- a/drivers/virt/fsl_hypervisor.c~get_current_user_pages 2016-02-12 10:44:14.401162935 -0800
+++ b/drivers/virt/fsl_hypervisor.c 2016-02-12 10:44:14.425164032 -0800
@@ -244,9 +244,8 @@ static long ioctl_memcpy(struct fsl_hv_i
/* Get the physical addresses of the source buffer */
down_read(¤t->mm->mmap_sem);
- num_pinned = get_user_pages(current, current->mm,
- param.local_vaddr - lb_offset, num_pages,
- (param.source == -1) ? READ : WRITE,
+ num_pinned = get_user_pages(param.local_vaddr - lb_offset,
+ num_pages, (param.source == -1) ? READ : WRITE,
0, pages, NULL);
up_read(¤t->mm->mmap_sem);
diff -puN mm/frame_vector.c~get_current_user_pages mm/frame_vector.c
--- a/mm/frame_vector.c~get_current_user_pages 2016-02-12 10:44:14.403163026 -0800
+++ b/mm/frame_vector.c 2016-02-12 10:44:14.426164078 -0800
@@ -58,7 +58,7 @@ int get_vaddr_frames(unsigned long start
if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) {
vec->got_ref = true;
vec->is_pfns = false;
- ret = get_user_pages_locked(current, mm, start, nr_frames,
+ ret = get_user_pages_locked(start, nr_frames,
write, force, (struct page **)(vec->ptrs), &locked);
goto out;
}
diff -puN mm/gup.c~get_current_user_pages mm/gup.c
--- a/mm/gup.c~get_current_user_pages 2016-02-12 10:44:14.405163117 -0800
+++ b/mm/gup.c 2016-02-12 10:44:14.426164078 -0800
@@ -936,8 +936,10 @@ long get_user_pages_remote(struct task_s
EXPORT_SYMBOL(get_user_pages_remote);
/*
- * This is the same as get_user_pages_remote() for the time
- * being.
+ * This is the same as get_user_pages_remote(), just with a
+ * less-flexible calling convention where we assume that the task
+ * and mm being operated on are the current task's. We also
+ * obviously don't pass FOLL_REMOTE in here.
*/
long get_user_pages6(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
diff -puN mm/ksm.c~get_current_user_pages mm/ksm.c
--- a/mm/ksm.c~get_current_user_pages 2016-02-12 10:44:14.406163163 -0800
+++ b/mm/ksm.c 2016-02-12 10:44:14.427164123 -0800
@@ -352,7 +352,7 @@ static inline bool ksm_test_exit(struct
/*
* We use break_ksm to break COW on a ksm page: it's a stripped down
*
- * if (get_user_pages(current, mm, addr, 1, 1, 1, &page, NULL) == 1)
+ * if (get_user_pages(addr, 1, 1, 1, &page, NULL) == 1)
* put_page(page);
*
* but taking great care only to touch a ksm page, in a VM_MERGEABLE vma,
diff -puN mm/mempolicy.c~get_current_user_pages mm/mempolicy.c
--- a/mm/mempolicy.c~get_current_user_pages 2016-02-12 10:44:14.408163255 -0800
+++ b/mm/mempolicy.c 2016-02-12 10:44:14.428164169 -0800
@@ -844,12 +844,12 @@ static void get_policy_nodemask(struct m
}
}
-static int lookup_node(struct mm_struct *mm, unsigned long addr)
+static int lookup_node(unsigned long addr)
{
struct page *p;
int err;
- err = get_user_pages(current, mm, addr & PAGE_MASK, 1, 0, 0, &p, NULL);
+ err = get_user_pages(addr & PAGE_MASK, 1, 0, 0, &p, NULL);
if (err >= 0) {
err = page_to_nid(p);
put_page(p);
@@ -904,7 +904,7 @@ static long do_get_mempolicy(int *policy
if (flags & MPOL_F_NODE) {
if (flags & MPOL_F_ADDR) {
- err = lookup_node(mm, addr);
+ err = lookup_node(addr);
if (err < 0)
goto out;
*policy = err;
diff -puN net/ceph/pagevec.c~get_current_user_pages net/ceph/pagevec.c
--- a/net/ceph/pagevec.c~get_current_user_pages 2016-02-12 10:44:14.410163346 -0800
+++ b/net/ceph/pagevec.c 2016-02-12 10:44:14.428164169 -0800
@@ -24,7 +24,7 @@ struct page **ceph_get_direct_page_vecto
return ERR_PTR(-ENOMEM);
while (got < num_pages) {
- rc = get_user_pages_unlocked(current, current->mm,
+ rc = get_user_pages_unlocked(
(unsigned long)data + ((unsigned long)got * PAGE_SIZE),
num_pages - got, write_page, 0, pages + got);
if (rc < 0)
diff -puN virt/kvm/kvm_main.c~get_current_user_pages virt/kvm/kvm_main.c
--- a/virt/kvm/kvm_main.c~get_current_user_pages 2016-02-12 10:44:14.411163392 -0800
+++ b/virt/kvm/kvm_main.c 2016-02-12 10:44:14.429164215 -0800
@@ -1264,15 +1264,16 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(s
return gfn_to_hva_memslot_prot(slot, gfn, writable);
}
-static int get_user_page_nowait(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, int write, struct page **page)
+static int get_user_page_nowait(unsigned long start, int write,
+ struct page **page)
{
int flags = FOLL_TOUCH | FOLL_NOWAIT | FOLL_HWPOISON | FOLL_GET;
if (write)
flags |= FOLL_WRITE;
- return __get_user_pages(tsk, mm, start, 1, flags, page, NULL, NULL);
+ return __get_user_pages(current, current->mm, start, 1, flags, page,
+ NULL, NULL);
}
static inline int check_user_page_hwpoison(unsigned long addr)
@@ -1334,8 +1335,7 @@ static int hva_to_pfn_slow(unsigned long
if (async) {
down_read(¤t->mm->mmap_sem);
- npages = get_user_page_nowait(current, current->mm,
- addr, write_fault, page);
+ npages = get_user_page_nowait(addr, write_fault, page);
up_read(¤t->mm->mmap_sem);
} else
npages = __get_user_pages_unlocked(current, current->mm, addr, 1,
_
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:20 +0100 |
| Subject | [tip:mm/pkeys] mm/gup: Switch all callers of get_user_pages() to not pass tsk/mm |
| Message-ID | <r3Ddn-2wj-9@gated-at.bofh.it> |
| In reply to | #1333121 |
Commit-ID: d4edcf0d56958db0aca0196314ca38a5e730ea92
Gitweb: http://git.kernel.org/tip/d4edcf0d56958db0aca0196314ca38a5e730ea92
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:01:56 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 16 Feb 2016 10:11:12 +0100
mm/gup: Switch all callers of get_user_pages() to not pass tsk/mm
We will soon modify the vanilla get_user_pages() so it can no
longer be used on mm/tasks other than 'current/current->mm',
which is by far the most common way it is called. For now,
we allow the old-style calls, but warn when they are used.
(implemented in previous patch)
This patch switches all callers of:
get_user_pages()
get_user_pages_unlocked()
get_user_pages_locked()
to stop passing tsk/mm so they will no longer see the warnings.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: jack@suse.cz
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210156.113E9407@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/cris/arch-v32/drivers/cryptocop.c | 8 ++------
arch/ia64/kernel/err_inject.c | 3 +--
arch/mips/mm/gup.c | 3 +--
arch/s390/mm/gup.c | 4 +---
arch/sh/mm/gup.c | 2 +-
arch/sparc/mm/gup.c | 2 +-
arch/x86/mm/gup.c | 2 +-
arch/x86/mm/mpx.c | 4 ++--
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +--
drivers/gpu/drm/radeon/radeon_ttm.c | 3 +--
drivers/gpu/drm/via/via_dmablit.c | 3 +--
drivers/infiniband/core/umem.c | 2 +-
drivers/infiniband/hw/mthca/mthca_memfree.c | 3 +--
drivers/infiniband/hw/qib/qib_user_pages.c | 3 +--
drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +-
drivers/media/pci/ivtv/ivtv-udma.c | 4 ++--
drivers/media/pci/ivtv/ivtv-yuv.c | 10 ++++------
drivers/media/v4l2-core/videobuf-dma-sg.c | 3 +--
drivers/misc/mic/scif/scif_rma.c | 2 --
drivers/misc/sgi-gru/grufault.c | 3 +--
drivers/scsi/st.c | 2 --
drivers/video/fbdev/pvr2fb.c | 4 ++--
drivers/virt/fsl_hypervisor.c | 5 ++---
mm/frame_vector.c | 2 +-
mm/gup.c | 6 ++++--
mm/ksm.c | 2 +-
mm/mempolicy.c | 6 +++---
net/ceph/pagevec.c | 2 +-
virt/kvm/kvm_main.c | 10 +++++-----
29 files changed, 44 insertions(+), 64 deletions(-)
diff --git a/arch/cris/arch-v32/drivers/cryptocop.c b/arch/cris/arch-v32/drivers/cryptocop.c
index 877da19..617645d 100644
--- a/arch/cris/arch-v32/drivers/cryptocop.c
+++ b/arch/cris/arch-v32/drivers/cryptocop.c
@@ -2719,9 +2719,7 @@ static int cryptocop_ioctl_process(struct inode *inode, struct file *filp, unsig
/* Acquire the mm page semaphore. */
down_read(¤t->mm->mmap_sem);
- err = get_user_pages(current,
- current->mm,
- (unsigned long int)(oper.indata + prev_ix),
+ err = get_user_pages((unsigned long int)(oper.indata + prev_ix),
noinpages,
0, /* read access only for in data */
0, /* no force */
@@ -2736,9 +2734,7 @@ static int cryptocop_ioctl_process(struct inode *inode, struct file *filp, unsig
}
noinpages = err;
if (oper.do_cipher){
- err = get_user_pages(current,
- current->mm,
- (unsigned long int)oper.cipher_outdata,
+ err = get_user_pages((unsigned long int)oper.cipher_outdata,
nooutpages,
1, /* write access for out data */
0, /* no force */
diff --git a/arch/ia64/kernel/err_inject.c b/arch/ia64/kernel/err_inject.c
index 0c161ed..09f8457 100644
--- a/arch/ia64/kernel/err_inject.c
+++ b/arch/ia64/kernel/err_inject.c
@@ -142,8 +142,7 @@ store_virtual_to_phys(struct device *dev, struct device_attribute *attr,
u64 virt_addr=simple_strtoull(buf, NULL, 16);
int ret;
- ret = get_user_pages(current, current->mm, virt_addr,
- 1, VM_READ, 0, NULL, NULL);
+ ret = get_user_pages(virt_addr, 1, VM_READ, 0, NULL, NULL);
if (ret<=0) {
#ifdef ERR_INJ_DEBUG
printk("Virtual address %lx is not existing.\n",virt_addr);
diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c
index 1afd87c..982e83f 100644
--- a/arch/mips/mm/gup.c
+++ b/arch/mips/mm/gup.c
@@ -286,8 +286,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- (end - start) >> PAGE_SHIFT,
+ ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT,
write, 0, pages);
/* Have to be a bit careful with return values */
diff --git a/arch/s390/mm/gup.c b/arch/s390/mm/gup.c
index 13dab0c..49a1c84 100644
--- a/arch/s390/mm/gup.c
+++ b/arch/s390/mm/gup.c
@@ -210,7 +210,6 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
- struct mm_struct *mm = current->mm;
int nr, ret;
might_sleep();
@@ -222,8 +221,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,
/* Try to get the remaining pages with get_user_pages */
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- nr_pages - nr, write, 0, pages);
+ ret = get_user_pages_unlocked(start, nr_pages - nr, write, 0, pages);
/* Have to be a bit careful with return values */
if (nr > 0)
ret = (ret < 0) ? nr : ret + nr;
diff --git a/arch/sh/mm/gup.c b/arch/sh/mm/gup.c
index e7af6a6..40fa6c8 100644
--- a/arch/sh/mm/gup.c
+++ b/arch/sh/mm/gup.c
@@ -257,7 +257,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT, write, 0, pages);
/* Have to be a bit careful with return values */
diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c
index eb3d8e8..4e06750 100644
--- a/arch/sparc/mm/gup.c
+++ b/arch/sparc/mm/gup.c
@@ -237,7 +237,7 @@ slow:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT, write, 0, pages);
/* Have to be a bit careful with return values */
diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c
index 6d5eb59..ce5e454 100644
--- a/arch/x86/mm/gup.c
+++ b/arch/x86/mm/gup.c
@@ -422,7 +422,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT,
write, 0, pages);
diff --git a/arch/x86/mm/mpx.c b/arch/x86/mm/mpx.c
index b2fd67d..84fa4a4 100644
--- a/arch/x86/mm/mpx.c
+++ b/arch/x86/mm/mpx.c
@@ -546,8 +546,8 @@ static int mpx_resolve_fault(long __user *addr, int write)
int nr_pages = 1;
int force = 0;
- gup_ret = get_user_pages(current, current->mm, (unsigned long)addr,
- nr_pages, write, force, NULL, NULL);
+ gup_ret = get_user_pages((unsigned long)addr, nr_pages, write,
+ force, NULL, NULL);
/*
* get_user_pages() returns number of pages gotten.
* 0 means we failed to fault in and get anything,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 6442a06..5fedfb6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -518,8 +518,7 @@ static int amdgpu_ttm_tt_pin_userptr(struct ttm_tt *ttm)
uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE;
struct page **pages = ttm->pages + pinned;
- r = get_user_pages(current, current->mm, userptr, num_pages,
- write, 0, pages, NULL);
+ r = get_user_pages(userptr, num_pages, write, 0, pages, NULL);
if (r < 0)
goto release_pages;
diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c
index e343074..927a9f2 100644
--- a/drivers/gpu/drm/radeon/radeon_ttm.c
+++ b/drivers/gpu/drm/radeon/radeon_ttm.c
@@ -554,8 +554,7 @@ static int radeon_ttm_tt_pin_userptr(struct ttm_tt *ttm)
uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE;
struct page **pages = ttm->pages + pinned;
- r = get_user_pages(current, current->mm, userptr, num_pages,
- write, 0, pages, NULL);
+ r = get_user_pages(userptr, num_pages, write, 0, pages, NULL);
if (r < 0)
goto release_pages;
diff --git a/drivers/gpu/drm/via/via_dmablit.c b/drivers/gpu/drm/via/via_dmablit.c
index d0cbd5e..e797dfc 100644
--- a/drivers/gpu/drm/via/via_dmablit.c
+++ b/drivers/gpu/drm/via/via_dmablit.c
@@ -239,8 +239,7 @@ via_lock_all_dma_pages(drm_via_sg_info_t *vsg, drm_via_dmablit_t *xfer)
if (NULL == vsg->pages)
return -ENOMEM;
down_read(¤t->mm->mmap_sem);
- ret = get_user_pages(current, current->mm,
- (unsigned long)xfer->mem_addr,
+ ret = get_user_pages((unsigned long)xfer->mem_addr,
vsg->num_pages,
(vsg->direction == DMA_FROM_DEVICE),
0, vsg->pages, NULL);
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 38acb3c..fe4d2e1 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -188,7 +188,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
sg_list_start = umem->sg_head.sgl;
while (npages) {
- ret = get_user_pages(current, current->mm, cur_base,
+ ret = get_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof (struct page *)),
1, !umem->writable, page_list, vma_list);
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index 7d2e42d..6c00d04 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -472,8 +472,7 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
goto out;
}
- ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- pages, NULL);
+ ret = get_user_pages(uaddr & PAGE_MASK, 1, 1, 0, pages, NULL);
if (ret < 0)
goto out;
diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c
index 74f90b2..2d2b94f 100644
--- a/drivers/infiniband/hw/qib/qib_user_pages.c
+++ b/drivers/infiniband/hw/qib/qib_user_pages.c
@@ -66,8 +66,7 @@ static int __qib_get_user_pages(unsigned long start_page, size_t num_pages,
}
for (got = 0; got < num_pages; got += ret) {
- ret = get_user_pages(current, current->mm,
- start_page + got * PAGE_SIZE,
+ ret = get_user_pages(start_page + got * PAGE_SIZE,
num_pages - got, 1, 1,
p + got, NULL);
if (ret < 0)
diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c b/drivers/infiniband/hw/usnic/usnic_uiom.c
index 645a5f6..7209fbc 100644
--- a/drivers/infiniband/hw/usnic/usnic_uiom.c
+++ b/drivers/infiniband/hw/usnic/usnic_uiom.c
@@ -144,7 +144,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable,
ret = 0;
while (npages) {
- ret = get_user_pages(current, current->mm, cur_base,
+ ret = get_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof(struct page *)),
1, !writable, page_list, NULL);
diff --git a/drivers/media/pci/ivtv/ivtv-udma.c b/drivers/media/pci/ivtv/ivtv-udma.c
index 24152ac..4769469 100644
--- a/drivers/media/pci/ivtv/ivtv-udma.c
+++ b/drivers/media/pci/ivtv/ivtv-udma.c
@@ -124,8 +124,8 @@ int ivtv_udma_setup(struct ivtv *itv, unsigned long ivtv_dest_addr,
}
/* Get user pages for DMA Xfer */
- err = get_user_pages_unlocked(current, current->mm,
- user_dma.uaddr, user_dma.page_count, 0, 1, dma->map);
+ err = get_user_pages_unlocked(user_dma.uaddr, user_dma.page_count, 0,
+ 1, dma->map);
if (user_dma.page_count != err) {
IVTV_DEBUG_WARN("failed to map user pages, returned %d instead of %d\n",
diff --git a/drivers/media/pci/ivtv/ivtv-yuv.c b/drivers/media/pci/ivtv/ivtv-yuv.c
index 2b8e7b2..b094054 100644
--- a/drivers/media/pci/ivtv/ivtv-yuv.c
+++ b/drivers/media/pci/ivtv/ivtv-yuv.c
@@ -75,14 +75,12 @@ static int ivtv_yuv_prep_user_dma(struct ivtv *itv, struct ivtv_user_dma *dma,
ivtv_udma_get_page_info (&uv_dma, (unsigned long)args->uv_source, 360 * uv_decode_height);
/* Get user pages for DMA Xfer */
- y_pages = get_user_pages_unlocked(current, current->mm,
- y_dma.uaddr, y_dma.page_count, 0, 1,
- &dma->map[0]);
+ y_pages = get_user_pages_unlocked(y_dma.uaddr,
+ y_dma.page_count, 0, 1, &dma->map[0]);
uv_pages = 0; /* silence gcc. value is set and consumed only if: */
if (y_pages == y_dma.page_count) {
- uv_pages = get_user_pages_unlocked(current, current->mm,
- uv_dma.uaddr, uv_dma.page_count, 0, 1,
- &dma->map[y_pages]);
+ uv_pages = get_user_pages_unlocked(uv_dma.uaddr,
+ uv_dma.page_count, 0, 1, &dma->map[y_pages]);
}
if (y_pages != y_dma.page_count || uv_pages != uv_dma.page_count) {
diff --git a/drivers/media/v4l2-core/videobuf-dma-sg.c b/drivers/media/v4l2-core/videobuf-dma-sg.c
index f669ced..df4c052c 100644
--- a/drivers/media/v4l2-core/videobuf-dma-sg.c
+++ b/drivers/media/v4l2-core/videobuf-dma-sg.c
@@ -181,8 +181,7 @@ static int videobuf_dma_init_user_locked(struct videobuf_dmabuf *dma,
dprintk(1, "init user [0x%lx+0x%lx => %d pages]\n",
data, size, dma->nr_pages);
- err = get_user_pages(current, current->mm,
- data & PAGE_MASK, dma->nr_pages,
+ err = get_user_pages(data & PAGE_MASK, dma->nr_pages,
rw == READ, 1, /* force */
dma->pages, NULL);
diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c
index 8310b4d..0fa0d242 100644
--- a/drivers/misc/mic/scif/scif_rma.c
+++ b/drivers/misc/mic/scif/scif_rma.c
@@ -1394,8 +1394,6 @@ retry:
}
pinned_pages->nr_pages = get_user_pages(
- current,
- mm,
(u64)addr,
nr_pages,
!!(prot & SCIF_PROT_WRITE),
diff --git a/drivers/misc/sgi-gru/grufault.c b/drivers/misc/sgi-gru/grufault.c
index f74fc0c..a2d97b9 100644
--- a/drivers/misc/sgi-gru/grufault.c
+++ b/drivers/misc/sgi-gru/grufault.c
@@ -198,8 +198,7 @@ static int non_atomic_pte_lookup(struct vm_area_struct *vma,
#else
*pageshift = PAGE_SHIFT;
#endif
- if (get_user_pages
- (current, current->mm, vaddr, 1, write, 0, &page, NULL) <= 0)
+ if (get_user_pages(vaddr, 1, write, 0, &page, NULL) <= 0)
return -EFAULT;
*paddr = page_to_phys(page);
put_page(page);
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 2e52295..664852a 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -4817,8 +4817,6 @@ static int sgl_map_user_pages(struct st_buffer *STbp,
/* Try to fault in all of the necessary pages */
/* rw==READ means read from drive, write into memory area */
res = get_user_pages_unlocked(
- current,
- current->mm,
uaddr,
nr_pages,
rw == READ,
diff --git a/drivers/video/fbdev/pvr2fb.c b/drivers/video/fbdev/pvr2fb.c
index 0e24eb9..71a923e 100644
--- a/drivers/video/fbdev/pvr2fb.c
+++ b/drivers/video/fbdev/pvr2fb.c
@@ -686,8 +686,8 @@ static ssize_t pvr2fb_write(struct fb_info *info, const char *buf,
if (!pages)
return -ENOMEM;
- ret = get_user_pages_unlocked(current, current->mm, (unsigned long)buf,
- nr_pages, WRITE, 0, pages);
+ ret = get_user_pages_unlocked((unsigned long)buf, nr_pages, WRITE,
+ 0, pages);
if (ret < nr_pages) {
nr_pages = ret;
diff --git a/drivers/virt/fsl_hypervisor.c b/drivers/virt/fsl_hypervisor.c
index 32c8fc5..60bdad3 100644
--- a/drivers/virt/fsl_hypervisor.c
+++ b/drivers/virt/fsl_hypervisor.c
@@ -244,9 +244,8 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p)
/* Get the physical addresses of the source buffer */
down_read(¤t->mm->mmap_sem);
- num_pinned = get_user_pages(current, current->mm,
- param.local_vaddr - lb_offset, num_pages,
- (param.source == -1) ? READ : WRITE,
+ num_pinned = get_user_pages(param.local_vaddr - lb_offset,
+ num_pages, (param.source == -1) ? READ : WRITE,
0, pages, NULL);
up_read(¤t->mm->mmap_sem);
diff --git a/mm/frame_vector.c b/mm/frame_vector.c
index 7cf2b71..381bb07 100644
--- a/mm/frame_vector.c
+++ b/mm/frame_vector.c
@@ -58,7 +58,7 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) {
vec->got_ref = true;
vec->is_pfns = false;
- ret = get_user_pages_locked(current, mm, start, nr_frames,
+ ret = get_user_pages_locked(start, nr_frames,
write, force, (struct page **)(vec->ptrs), &locked);
goto out;
}
diff --git a/mm/gup.c b/mm/gup.c
index 8a035e0..de24ef4 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -936,8 +936,10 @@ long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
EXPORT_SYMBOL(get_user_pages_remote);
/*
- * This is the same as get_user_pages_remote() for the time
- * being.
+ * This is the same as get_user_pages_remote(), just with a
+ * less-flexible calling convention where we assume that the task
+ * and mm being operated on are the current task's. We also
+ * obviously don't pass FOLL_REMOTE in here.
*/
long get_user_pages6(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
diff --git a/mm/ksm.c b/mm/ksm.c
index ca6d2a0..c2013f6 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -352,7 +352,7 @@ static inline bool ksm_test_exit(struct mm_struct *mm)
/*
* We use break_ksm to break COW on a ksm page: it's a stripped down
*
- * if (get_user_pages(current, mm, addr, 1, 1, 1, &page, NULL) == 1)
+ * if (get_user_pages(addr, 1, 1, 1, &page, NULL) == 1)
* put_page(page);
*
* but taking great care only to touch a ksm page, in a VM_MERGEABLE vma,
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 4c4187c..dd0ce7f 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -844,12 +844,12 @@ static void get_policy_nodemask(struct mempolicy *p, nodemask_t *nodes)
}
}
-static int lookup_node(struct mm_struct *mm, unsigned long addr)
+static int lookup_node(unsigned long addr)
{
struct page *p;
int err;
- err = get_user_pages(current, mm, addr & PAGE_MASK, 1, 0, 0, &p, NULL);
+ err = get_user_pages(addr & PAGE_MASK, 1, 0, 0, &p, NULL);
if (err >= 0) {
err = page_to_nid(p);
put_page(p);
@@ -904,7 +904,7 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask,
if (flags & MPOL_F_NODE) {
if (flags & MPOL_F_ADDR) {
- err = lookup_node(mm, addr);
+ err = lookup_node(addr);
if (err < 0)
goto out;
*policy = err;
diff --git a/net/ceph/pagevec.c b/net/ceph/pagevec.c
index d4f5f22..10297f7 100644
--- a/net/ceph/pagevec.c
+++ b/net/ceph/pagevec.c
@@ -24,7 +24,7 @@ struct page **ceph_get_direct_page_vector(const void __user *data,
return ERR_PTR(-ENOMEM);
while (got < num_pages) {
- rc = get_user_pages_unlocked(current, current->mm,
+ rc = get_user_pages_unlocked(
(unsigned long)data + ((unsigned long)got * PAGE_SIZE),
num_pages - got, write_page, 0, pages + got);
if (rc < 0)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index a11cfd2..0253ad9 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1264,15 +1264,16 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w
return gfn_to_hva_memslot_prot(slot, gfn, writable);
}
-static int get_user_page_nowait(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, int write, struct page **page)
+static int get_user_page_nowait(unsigned long start, int write,
+ struct page **page)
{
int flags = FOLL_TOUCH | FOLL_NOWAIT | FOLL_HWPOISON | FOLL_GET;
if (write)
flags |= FOLL_WRITE;
- return __get_user_pages(tsk, mm, start, 1, flags, page, NULL, NULL);
+ return __get_user_pages(current, current->mm, start, 1, flags, page,
+ NULL, NULL);
}
static inline int check_user_page_hwpoison(unsigned long addr)
@@ -1334,8 +1335,7 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault,
if (async) {
down_read(¤t->mm->mmap_sem);
- npages = get_user_page_nowait(current, current->mm,
- addr, write_fault, page);
+ npages = get_user_page_nowait(addr, write_fault, page);
up_read(¤t->mm->mmap_sem);
} else
npages = __get_user_pages_unlocked(current, current->mm, addr, 1,
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 19/33] x86, pkeys: check VMAs and PTEs for protection keys |
| Message-ID | <r1t8w-3TO-61@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
Today, for normal faults and page table walks, we check the VMA
and/or PTE to ensure that it is compatible with the action. For
instance, if we get a write fault on a non-writeable VMA, we
SIGSEGV.
We try to do the same thing for protection keys. Basically, we
try to make sure that if a user does this:
mprotect(ptr, size, PROT_NONE);
*ptr = foo;
they see the same effects with protection keys when they do this:
mprotect(ptr, size, PROT_READ|PROT_WRITE);
set_pkey(ptr, size, 4);
wrpkru(0xffffff3f); // access disable pkey 4
*ptr = foo;
The state to do that checking is in the VMA, but we also
sometimes have to do it on the page tables only, like when doing
a get_user_pages_fast() where we have no VMA.
We add two functions and expose them to generic code:
arch_pte_access_permitted(pte_flags, write)
arch_vma_access_permitted(vma, write)
These are, of course, backed up in x86 arch code with checks
against the PTE or VMA's protection key.
But, there are also cases where we do not want to respect
protection keys. When we ptrace(), for instance, we do not want
to apply the tracer's PKRU permissions to the PTEs from the
process being traced.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/powerpc/include/asm/mmu_context.h | 11 ++++++
b/arch/s390/include/asm/mmu_context.h | 11 ++++++
b/arch/unicore32/include/asm/mmu_context.h | 11 ++++++
b/arch/x86/include/asm/mmu_context.h | 49 +++++++++++++++++++++++++++++
b/arch/x86/include/asm/pgtable.h | 29 +++++++++++++++++
b/arch/x86/mm/fault.c | 21 +++++++++++-
b/arch/x86/mm/gup.c | 5 ++
b/include/asm-generic/mm_hooks.h | 11 ++++++
b/mm/gup.c | 18 ++++++++--
b/mm/memory.c | 4 ++
10 files changed, 166 insertions(+), 4 deletions(-)
diff -puN arch/powerpc/include/asm/mmu_context.h~pkeys-13-pte-fault arch/powerpc/include/asm/mmu_context.h
--- a/arch/powerpc/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-02-12 10:44:21.985509635 -0800
+++ b/arch/powerpc/include/asm/mmu_context.h 2016-02-12 10:44:22.002510412 -0800
@@ -148,5 +148,16 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* __KERNEL__ */
#endif /* __ASM_POWERPC_MMU_CONTEXT_H */
diff -puN arch/s390/include/asm/mmu_context.h~pkeys-13-pte-fault arch/s390/include/asm/mmu_context.h
--- a/arch/s390/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-02-12 10:44:21.986509681 -0800
+++ b/arch/s390/include/asm/mmu_context.h 2016-02-12 10:44:22.003510458 -0800
@@ -130,4 +130,15 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* __S390_MMU_CONTEXT_H */
diff -puN arch/unicore32/include/asm/mmu_context.h~pkeys-13-pte-fault arch/unicore32/include/asm/mmu_context.h
--- a/arch/unicore32/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-02-12 10:44:21.988509772 -0800
+++ b/arch/unicore32/include/asm/mmu_context.h 2016-02-12 10:44:22.003510458 -0800
@@ -97,4 +97,15 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif
diff -puN arch/x86/include/asm/mmu_context.h~pkeys-13-pte-fault arch/x86/include/asm/mmu_context.h
--- a/arch/x86/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-02-12 10:44:21.990509864 -0800
+++ b/arch/x86/include/asm/mmu_context.h 2016-02-12 10:44:22.003510458 -0800
@@ -286,4 +286,53 @@ static inline int vma_pkey(struct vm_are
return pkey;
}
+static inline bool __pkru_allows_pkey(u16 pkey, bool write)
+{
+ u32 pkru = read_pkru();
+
+ if (!__pkru_allows_read(pkru, pkey))
+ return false;
+ if (write && !__pkru_allows_write(pkru, pkey))
+ return false;
+
+ return true;
+}
+
+/*
+ * We only want to enforce protection keys on the current process
+ * because we effectively have no access to PKRU for other
+ * processes or any way to tell *which * PKRU in a threaded
+ * process we could use.
+ *
+ * So do not enforce things if the VMA is not from the current
+ * mm, or if we are in a kernel thread.
+ */
+static inline bool vma_is_foreign(struct vm_area_struct *vma)
+{
+ if (!current->mm)
+ return true;
+ /*
+ * Should PKRU be enforced on the access to this VMA? If
+ * the VMA is from another process, then PKRU has no
+ * relevance and should not be enforced.
+ */
+ if (current->mm != vma->vm_mm)
+ return true;
+
+ return false;
+}
+
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* allow access if the VMA is not one from this process */
+ if (vma_is_foreign(vma))
+ return true;
+ return __pkru_allows_pkey(vma_pkey(vma), write);
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ return __pkru_allows_pkey(pte_flags_pkey(pte_flags(pte)), write);
+}
+
#endif /* _ASM_X86_MMU_CONTEXT_H */
diff -puN arch/x86/include/asm/pgtable.h~pkeys-13-pte-fault arch/x86/include/asm/pgtable.h
--- a/arch/x86/include/asm/pgtable.h~pkeys-13-pte-fault 2016-02-12 10:44:21.991509909 -0800
+++ b/arch/x86/include/asm/pgtable.h 2016-02-12 10:44:22.004510503 -0800
@@ -919,6 +919,35 @@ static inline pte_t pte_swp_clear_soft_d
}
#endif
+#define PKRU_AD_BIT 0x1
+#define PKRU_WD_BIT 0x2
+
+static inline bool __pkru_allows_read(u32 pkru, u16 pkey)
+{
+ int pkru_pkey_bits = pkey * 2;
+ return !(pkru & (PKRU_AD_BIT << pkru_pkey_bits));
+}
+
+static inline bool __pkru_allows_write(u32 pkru, u16 pkey)
+{
+ int pkru_pkey_bits = pkey * 2;
+ /*
+ * Access-disable disables writes too so we need to check
+ * both bits here.
+ */
+ return !(pkru & ((PKRU_AD_BIT|PKRU_WD_BIT) << pkru_pkey_bits));
+}
+
+static inline u16 pte_flags_pkey(unsigned long pte_flags)
+{
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+ /* ifdef to avoid doing 59-bit shift on 32-bit values */
+ return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
+#else
+ return 0;
+#endif
+}
+
#include <asm-generic/pgtable.h>
#endif /* __ASSEMBLY__ */
diff -puN arch/x86/mm/fault.c~pkeys-13-pte-fault arch/x86/mm/fault.c
--- a/arch/x86/mm/fault.c~pkeys-13-pte-fault 2016-02-12 10:44:21.993510001 -0800
+++ b/arch/x86/mm/fault.c 2016-02-12 10:44:22.004510503 -0800
@@ -897,6 +897,16 @@ bad_area(struct pt_regs *regs, unsigned
__bad_area(regs, error_code, address, NULL, SEGV_MAPERR);
}
+static inline bool bad_area_access_from_pkeys(unsigned long error_code,
+ struct vm_area_struct *vma)
+{
+ if (!boot_cpu_has(X86_FEATURE_OSPKE))
+ return false;
+ if (error_code & PF_PK)
+ return true;
+ return false;
+}
+
static noinline void
bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
unsigned long address, struct vm_area_struct *vma)
@@ -906,7 +916,7 @@ bad_area_access_error(struct pt_regs *re
* But, doing it this way allows compiler optimizations
* if pkeys are compiled out.
*/
- if (boot_cpu_has(X86_FEATURE_OSPKE) && (error_code & PF_PK))
+ if (bad_area_access_from_pkeys(error_code, vma))
__bad_area(regs, error_code, address, vma, SEGV_PKUERR);
else
__bad_area(regs, error_code, address, vma, SEGV_ACCERR);
@@ -1081,6 +1091,15 @@ int show_unhandled_signals = 1;
static inline int
access_error(unsigned long error_code, struct vm_area_struct *vma)
{
+ /*
+ * Access or read was blocked by protection keys. We do
+ * this check before any others because we do not want
+ * to, for instance, confuse a protection-key-denied
+ * write with one for which we should do a COW.
+ */
+ if (error_code & PF_PK)
+ return 1;
+
if (error_code & PF_WRITE) {
/* write, present and write, not present: */
if (unlikely(!(vma->vm_flags & VM_WRITE)))
diff -puN arch/x86/mm/gup.c~pkeys-13-pte-fault arch/x86/mm/gup.c
--- a/arch/x86/mm/gup.c~pkeys-13-pte-fault 2016-02-12 10:44:21.994510046 -0800
+++ b/arch/x86/mm/gup.c 2016-02-12 10:44:22.005510549 -0800
@@ -11,6 +11,7 @@
#include <linux/swap.h>
#include <linux/memremap.h>
+#include <asm/mmu_context.h>
#include <asm/pgtable.h>
static inline pte_t gup_get_pte(pte_t *ptep)
@@ -89,6 +90,10 @@ static inline int pte_allows_gup(unsigne
if ((pteval & need_pte_bits) != need_pte_bits)
return 0;
+ /* Check memory protection keys permissions. */
+ if (!__pkru_allows_pkey(pte_flags_pkey(pteval), write))
+ return 0;
+
return 1;
}
diff -puN include/asm-generic/mm_hooks.h~pkeys-13-pte-fault include/asm-generic/mm_hooks.h
--- a/include/asm-generic/mm_hooks.h~pkeys-13-pte-fault 2016-02-12 10:44:21.996510138 -0800
+++ b/include/asm-generic/mm_hooks.h 2016-02-12 10:44:22.005510549 -0800
@@ -26,4 +26,15 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* _ASM_GENERIC_MM_HOOKS_H */
diff -puN mm/gup.c~pkeys-13-pte-fault mm/gup.c
--- a/mm/gup.c~pkeys-13-pte-fault 2016-02-12 10:44:21.997510184 -0800
+++ b/mm/gup.c 2016-02-12 10:44:22.006510595 -0800
@@ -15,6 +15,7 @@
#include <linux/rwsem.h>
#include <linux/hugetlb.h>
+#include <asm/mmu_context.h>
#include <asm/pgtable.h>
#include <asm/tlbflush.h>
@@ -444,6 +445,8 @@ static int check_vma_flags(struct vm_are
if (!(vm_flags & VM_MAYREAD))
return -EFAULT;
}
+ if (!arch_vma_access_permitted(vma, (gup_flags & FOLL_WRITE)))
+ return -EFAULT;
return 0;
}
@@ -612,13 +615,19 @@ EXPORT_SYMBOL(__get_user_pages);
bool vma_permits_fault(struct vm_area_struct *vma, unsigned int fault_flags)
{
- vm_flags_t vm_flags;
-
- vm_flags = (fault_flags & FAULT_FLAG_WRITE) ? VM_WRITE : VM_READ;
+ bool write = !!(fault_flags & FAULT_FLAG_WRITE);
+ vm_flags_t vm_flags = write ? VM_WRITE : VM_READ;
if (!(vm_flags & vma->vm_flags))
return false;
+ /*
+ * The architecture might have a hardware protection
+ * mechanism other than read/write that can deny access
+ */
+ if (!arch_vma_access_permitted(vma, write))
+ return false;
+
return true;
}
@@ -1172,6 +1181,9 @@ static int gup_pte_range(pmd_t pmd, unsi
pte_protnone(pte) || (write && !pte_write(pte)))
goto pte_unmap;
+ if (!arch_pte_access_permitted(pte, write))
+ goto pte_unmap;
+
VM_BUG_ON(!pfn_valid(pte_pfn(pte)));
page = pte_page(pte);
head = compound_head(page);
diff -puN mm/memory.c~pkeys-13-pte-fault mm/memory.c
--- a/mm/memory.c~pkeys-13-pte-fault 2016-02-12 10:44:21.999510275 -0800
+++ b/mm/memory.c 2016-02-12 10:44:22.007510641 -0800
@@ -65,6 +65,7 @@
#include <linux/userfaultfd_k.h>
#include <asm/io.h>
+#include <asm/mmu_context.h>
#include <asm/pgalloc.h>
#include <asm/uaccess.h>
#include <asm/tlb.h>
@@ -3357,6 +3358,9 @@ static int __handle_mm_fault(struct mm_s
pmd_t *pmd;
pte_t *pte;
+ if (!arch_vma_access_permitted(vma, flags & FAULT_FLAG_WRITE))
+ return VM_FAULT_SIGSEGV;
+
if (unlikely(is_vm_hugetlb_page(vma)))
return hugetlb_fault(mm, vma, address, flags);
_
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] mm/gup, x86/mm/pkeys: Check VMAs and PTEs for protection keys |
| Message-ID | <r3Dn4-2Bf-15@gated-at.bofh.it> |
| In reply to | #1333122 |
Commit-ID: 33a709b25a760b91184bb335cf7d7c32b8123013
Gitweb: http://git.kernel.org/tip/33a709b25a760b91184bb335cf7d7c32b8123013
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:02:19 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 18 Feb 2016 09:32:44 +0100
mm/gup, x86/mm/pkeys: Check VMAs and PTEs for protection keys
Today, for normal faults and page table walks, we check the VMA
and/or PTE to ensure that it is compatible with the action. For
instance, if we get a write fault on a non-writeable VMA, we
SIGSEGV.
We try to do the same thing for protection keys. Basically, we
try to make sure that if a user does this:
mprotect(ptr, size, PROT_NONE);
*ptr = foo;
they see the same effects with protection keys when they do this:
mprotect(ptr, size, PROT_READ|PROT_WRITE);
set_pkey(ptr, size, 4);
wrpkru(0xffffff3f); // access disable pkey 4
*ptr = foo;
The state to do that checking is in the VMA, but we also
sometimes have to do it on the page tables only, like when doing
a get_user_pages_fast() where we have no VMA.
We add two functions and expose them to generic code:
arch_pte_access_permitted(pte_flags, write)
arch_vma_access_permitted(vma, write)
These are, of course, backed up in x86 arch code with checks
against the PTE or VMA's protection key.
But, there are also cases where we do not want to respect
protection keys. When we ptrace(), for instance, we do not want
to apply the tracer's PKRU permissions to the PTEs from the
process being traced.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Boaz Harrosh <boaz@plexistor.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: David Hildenbrand <dahi@linux.vnet.ibm.com>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Dominik Dingel <dingel@linux.vnet.ibm.com>
Cc: Dominik Vogt <vogt@linux.vnet.ibm.com>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jason Low <jason.low2@hp.com>
Cc: Jerome Marchand <jmarchan@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Matthew Wilcox <willy@linux.intel.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mikulas Patocka <mpatocka@redhat.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Shachar Raindel <raindel@mellanox.com>
Cc: Stephen Smalley <sds@tycho.nsa.gov>
Cc: Toshi Kani <toshi.kani@hpe.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: linux-s390@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/20160212210219.14D5D715@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/powerpc/include/asm/mmu_context.h | 11 +++++++
arch/s390/include/asm/mmu_context.h | 11 +++++++
arch/unicore32/include/asm/mmu_context.h | 11 +++++++
arch/x86/include/asm/mmu_context.h | 49 ++++++++++++++++++++++++++++++++
arch/x86/include/asm/pgtable.h | 29 +++++++++++++++++++
arch/x86/mm/fault.c | 21 +++++++++++++-
arch/x86/mm/gup.c | 5 ++++
include/asm-generic/mm_hooks.h | 11 +++++++
mm/gup.c | 18 ++++++++++--
mm/memory.c | 4 +++
10 files changed, 166 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h
index 878c277..a0f1838 100644
--- a/arch/powerpc/include/asm/mmu_context.h
+++ b/arch/powerpc/include/asm/mmu_context.h
@@ -148,5 +148,16 @@ static inline void arch_bprm_mm_init(struct mm_struct *mm,
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* __KERNEL__ */
#endif /* __ASM_POWERPC_MMU_CONTEXT_H */
diff --git a/arch/s390/include/asm/mmu_context.h b/arch/s390/include/asm/mmu_context.h
index fb1b93e..2627b33 100644
--- a/arch/s390/include/asm/mmu_context.h
+++ b/arch/s390/include/asm/mmu_context.h
@@ -130,4 +130,15 @@ static inline void arch_bprm_mm_init(struct mm_struct *mm,
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* __S390_MMU_CONTEXT_H */
diff --git a/arch/unicore32/include/asm/mmu_context.h b/arch/unicore32/include/asm/mmu_context.h
index 1cb5220..3133f94 100644
--- a/arch/unicore32/include/asm/mmu_context.h
+++ b/arch/unicore32/include/asm/mmu_context.h
@@ -97,4 +97,15 @@ static inline void arch_bprm_mm_init(struct mm_struct *mm,
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif
diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index 94c4c8b..19036cd 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -286,4 +286,53 @@ static inline int vma_pkey(struct vm_area_struct *vma)
return pkey;
}
+static inline bool __pkru_allows_pkey(u16 pkey, bool write)
+{
+ u32 pkru = read_pkru();
+
+ if (!__pkru_allows_read(pkru, pkey))
+ return false;
+ if (write && !__pkru_allows_write(pkru, pkey))
+ return false;
+
+ return true;
+}
+
+/*
+ * We only want to enforce protection keys on the current process
+ * because we effectively have no access to PKRU for other
+ * processes or any way to tell *which * PKRU in a threaded
+ * process we could use.
+ *
+ * So do not enforce things if the VMA is not from the current
+ * mm, or if we are in a kernel thread.
+ */
+static inline bool vma_is_foreign(struct vm_area_struct *vma)
+{
+ if (!current->mm)
+ return true;
+ /*
+ * Should PKRU be enforced on the access to this VMA? If
+ * the VMA is from another process, then PKRU has no
+ * relevance and should not be enforced.
+ */
+ if (current->mm != vma->vm_mm)
+ return true;
+
+ return false;
+}
+
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* allow access if the VMA is not one from this process */
+ if (vma_is_foreign(vma))
+ return true;
+ return __pkru_allows_pkey(vma_pkey(vma), write);
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ return __pkru_allows_pkey(pte_flags_pkey(pte_flags(pte)), write);
+}
+
#endif /* _ASM_X86_MMU_CONTEXT_H */
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index e997dcc..3cbfae8 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -919,6 +919,35 @@ static inline pte_t pte_swp_clear_soft_dirty(pte_t pte)
}
#endif
+#define PKRU_AD_BIT 0x1
+#define PKRU_WD_BIT 0x2
+
+static inline bool __pkru_allows_read(u32 pkru, u16 pkey)
+{
+ int pkru_pkey_bits = pkey * 2;
+ return !(pkru & (PKRU_AD_BIT << pkru_pkey_bits));
+}
+
+static inline bool __pkru_allows_write(u32 pkru, u16 pkey)
+{
+ int pkru_pkey_bits = pkey * 2;
+ /*
+ * Access-disable disables writes too so we need to check
+ * both bits here.
+ */
+ return !(pkru & ((PKRU_AD_BIT|PKRU_WD_BIT) << pkru_pkey_bits));
+}
+
+static inline u16 pte_flags_pkey(unsigned long pte_flags)
+{
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+ /* ifdef to avoid doing 59-bit shift on 32-bit values */
+ return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
+#else
+ return 0;
+#endif
+}
+
#include <asm-generic/pgtable.h>
#endif /* __ASSEMBLY__ */
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 6e71dcf..319331a 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -897,6 +897,16 @@ bad_area(struct pt_regs *regs, unsigned long error_code, unsigned long address)
__bad_area(regs, error_code, address, NULL, SEGV_MAPERR);
}
+static inline bool bad_area_access_from_pkeys(unsigned long error_code,
+ struct vm_area_struct *vma)
+{
+ if (!boot_cpu_has(X86_FEATURE_OSPKE))
+ return false;
+ if (error_code & PF_PK)
+ return true;
+ return false;
+}
+
static noinline void
bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
unsigned long address, struct vm_area_struct *vma)
@@ -906,7 +916,7 @@ bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
* But, doing it this way allows compiler optimizations
* if pkeys are compiled out.
*/
- if (boot_cpu_has(X86_FEATURE_OSPKE) && (error_code & PF_PK))
+ if (bad_area_access_from_pkeys(error_code, vma))
__bad_area(regs, error_code, address, vma, SEGV_PKUERR);
else
__bad_area(regs, error_code, address, vma, SEGV_ACCERR);
@@ -1081,6 +1091,15 @@ int show_unhandled_signals = 1;
static inline int
access_error(unsigned long error_code, struct vm_area_struct *vma)
{
+ /*
+ * Access or read was blocked by protection keys. We do
+ * this check before any others because we do not want
+ * to, for instance, confuse a protection-key-denied
+ * write with one for which we should do a COW.
+ */
+ if (error_code & PF_PK)
+ return 1;
+
if (error_code & PF_WRITE) {
/* write, present and write, not present: */
if (unlikely(!(vma->vm_flags & VM_WRITE)))
diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c
index 2f0a329..bab259e 100644
--- a/arch/x86/mm/gup.c
+++ b/arch/x86/mm/gup.c
@@ -11,6 +11,7 @@
#include <linux/swap.h>
#include <linux/memremap.h>
+#include <asm/mmu_context.h>
#include <asm/pgtable.h>
static inline pte_t gup_get_pte(pte_t *ptep)
@@ -89,6 +90,10 @@ static inline int pte_allows_gup(unsigned long pteval, int write)
if ((pteval & need_pte_bits) != need_pte_bits)
return 0;
+ /* Check memory protection keys permissions. */
+ if (!__pkru_allows_pkey(pte_flags_pkey(pteval), write))
+ return 0;
+
return 1;
}
diff --git a/include/asm-generic/mm_hooks.h b/include/asm-generic/mm_hooks.h
index 866aa46..c1fc5af 100644
--- a/include/asm-generic/mm_hooks.h
+++ b/include/asm-generic/mm_hooks.h
@@ -26,4 +26,15 @@ static inline void arch_bprm_mm_init(struct mm_struct *mm,
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* _ASM_GENERIC_MM_HOOKS_H */
diff --git a/mm/gup.c b/mm/gup.c
index b935c2c..e0f5f357 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -15,6 +15,7 @@
#include <linux/rwsem.h>
#include <linux/hugetlb.h>
+#include <asm/mmu_context.h>
#include <asm/pgtable.h>
#include <asm/tlbflush.h>
@@ -444,6 +445,8 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)
if (!(vm_flags & VM_MAYREAD))
return -EFAULT;
}
+ if (!arch_vma_access_permitted(vma, (gup_flags & FOLL_WRITE)))
+ return -EFAULT;
return 0;
}
@@ -612,13 +615,19 @@ EXPORT_SYMBOL(__get_user_pages);
bool vma_permits_fault(struct vm_area_struct *vma, unsigned int fault_flags)
{
- vm_flags_t vm_flags;
-
- vm_flags = (fault_flags & FAULT_FLAG_WRITE) ? VM_WRITE : VM_READ;
+ bool write = !!(fault_flags & FAULT_FLAG_WRITE);
+ vm_flags_t vm_flags = write ? VM_WRITE : VM_READ;
if (!(vm_flags & vma->vm_flags))
return false;
+ /*
+ * The architecture might have a hardware protection
+ * mechanism other than read/write that can deny access
+ */
+ if (!arch_vma_access_permitted(vma, write))
+ return false;
+
return true;
}
@@ -1172,6 +1181,9 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
pte_protnone(pte) || (write && !pte_write(pte)))
goto pte_unmap;
+ if (!arch_pte_access_permitted(pte, write))
+ goto pte_unmap;
+
VM_BUG_ON(!pfn_valid(pte_pfn(pte)));
page = pte_page(pte);
head = compound_head(page);
diff --git a/mm/memory.c b/mm/memory.c
index 8bfbad0..d7e84fe 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -65,6 +65,7 @@
#include <linux/userfaultfd_k.h>
#include <asm/io.h>
+#include <asm/mmu_context.h>
#include <asm/pgalloc.h>
#include <asm/uaccess.h>
#include <asm/tlb.h>
@@ -3378,6 +3379,9 @@ static int __handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
pmd_t *pmd;
pte_t *pte;
+ if (!arch_vma_access_permitted(vma, flags & FAULT_FLAG_WRITE))
+ return VM_FAULT_SIGSEGV;
+
if (unlikely(is_vm_hugetlb_page(vma)))
return hugetlb_fault(mm, vma, address, flags);
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 26/33] x86, pkeys: actually enable Memory Protection Keys in CPU |
| Message-ID | <r1t8w-3TO-59@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
This sets the bit in 'cr4' to actually enable the protection
keys feature. We also include a boot-time disable for the
feature "nopku".
Seting X86_CR4_PKE will cause the X86_FEATURE_OSPKE cpuid
bit to appear set. At this point in boot, identify_cpu()
has already run the actual CPUID instructions and populated
the "cpu features" structures. We need to go back and
re-run identify_cpu() to make sure it gets updated values.
We *could* simply re-populate the 11th word of the cpuid
data, but this is probably quick enough.
Also note that with the cpu_has() check and X86_FEATURE_PKU
present in disabled-features.h, we do not need an #ifdef
for setup_pku().
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/Documentation/kernel-parameters.txt | 3 ++
b/arch/x86/kernel/cpu/common.c | 41 ++++++++++++++++++++++++++++++++++
2 files changed, 44 insertions(+)
diff -puN arch/x86/kernel/cpu/common.c~pkeys-50-should-be-last-patch arch/x86/kernel/cpu/common.c
--- a/arch/x86/kernel/cpu/common.c~pkeys-50-should-be-last-patch 2016-02-12 10:44:25.545672379 -0800
+++ b/arch/x86/kernel/cpu/common.c 2016-02-12 10:44:25.551672653 -0800
@@ -288,6 +288,46 @@ static __always_inline void setup_smap(s
}
/*
+ * Protection Keys are not available in 32-bit mode.
+ */
+static bool pku_disabled;
+static __always_inline void setup_pku(struct cpuinfo_x86 *c)
+{
+ if (!cpu_has(c, X86_FEATURE_PKU))
+ return;
+ if (pku_disabled)
+ return;
+
+ cr4_set_bits(X86_CR4_PKE);
+ /*
+ * Seting X86_CR4_PKE will cause the X86_FEATURE_OSPKE
+ * cpuid bit to be set. We need to ensure that we
+ * update that bit in this CPU's "cpu_info".
+ */
+ get_cpu_cap(c);
+}
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+static __init int setup_disable_pku(char *arg)
+{
+ /*
+ * Do not clear the X86_FEATURE_PKU bit. All of the
+ * runtime checks are against OSPKE so clearing the
+ * bit does nothing.
+ *
+ * This way, we will see "pku" in cpuinfo, but not
+ * "ospke", which is exactly what we want. It shows
+ * that the CPU has PKU, but the OS has not enabled it.
+ * This happens to be exactly how a system would look
+ * if we disabled the config option.
+ */
+ pr_info("x86: 'nopku' specified, disabling Memory Protection Keys\n");
+ pku_disabled = true;
+ return 1;
+}
+__setup("nopku", setup_disable_pku);
+#endif /* CONFIG_X86_64 */
+
+/*
* Some CPU features depend on higher CPUID levels, which may not always
* be available due to CPUID level capping or broken virtualization
* software. Add those features to this table to auto-disable them.
@@ -944,6 +984,7 @@ static void identify_cpu(struct cpuinfo_
init_hypervisor(c);
x86_init_rdrand(c);
x86_init_cache_qos(c);
+ setup_pku(c);
/*
* Clear/Set all flags overriden by options, need do it
diff -puN Documentation/kernel-parameters.txt~pkeys-50-should-be-last-patch Documentation/kernel-parameters.txt
--- a/Documentation/kernel-parameters.txt~pkeys-50-should-be-last-patch 2016-02-12 10:44:25.547672471 -0800
+++ b/Documentation/kernel-parameters.txt 2016-02-12 10:44:25.552672699 -0800
@@ -976,6 +976,9 @@ bytes respectively. Such letter suffixes
See Documentation/x86/intel_mpx.txt for more
information about the feature.
+ nopku [X86] Disable Memory Protection Keys CPU feature found
+ in some Intel CPUs.
+
eagerfpu= [X86]
on enable eager fpu restore
off disable eager fpu restore
_
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] x86/mm/pkeys: Actually enable Memory Protection Keys in the CPU |
| Message-ID | <r3Dn6-2Bf-49@gated-at.bofh.it> |
| In reply to | #1333124 |
Commit-ID: 0697694564c84f4c9320e5d103d0191297a20023
Gitweb: http://git.kernel.org/tip/0697694564c84f4c9320e5d103d0191297a20023
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:02:29 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 18 Feb 2016 19:46:30 +0100
x86/mm/pkeys: Actually enable Memory Protection Keys in the CPU
This sets the bit in 'cr4' to actually enable the protection
keys feature. We also include a boot-time disable for the
feature "nopku".
Seting X86_CR4_PKE will cause the X86_FEATURE_OSPKE cpuid
bit to appear set. At this point in boot, identify_cpu()
has already run the actual CPUID instructions and populated
the "cpu features" structures. We need to go back and
re-run identify_cpu() to make sure it gets updated values.
We *could* simply re-populate the 11th word of the cpuid
data, but this is probably quick enough.
Also note that with the cpu_has() check and X86_FEATURE_PKU
present in disabled-features.h, we do not need an #ifdef
for setup_pku().
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210229.6708027C@viggo.jf.intel.com
[ Small readability edits. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
Documentation/kernel-parameters.txt | 3 +++
arch/x86/kernel/cpu/common.c | 43 +++++++++++++++++++++++++++++++++++++
2 files changed, 46 insertions(+)
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index a37b5bb..acf467d 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -976,6 +976,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
See Documentation/x86/intel_mpx.txt for more
information about the feature.
+ nopku [X86] Disable Memory Protection Keys CPU feature found
+ in some Intel CPUs.
+
eagerfpu= [X86]
on enable eager fpu restore
off disable eager fpu restore
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index a719ad7..4fac263 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -304,6 +304,48 @@ static __always_inline void setup_smap(struct cpuinfo_x86 *c)
}
/*
+ * Protection Keys are not available in 32-bit mode.
+ */
+static bool pku_disabled;
+
+static __always_inline void setup_pku(struct cpuinfo_x86 *c)
+{
+ if (!cpu_has(c, X86_FEATURE_PKU))
+ return;
+ if (pku_disabled)
+ return;
+
+ cr4_set_bits(X86_CR4_PKE);
+ /*
+ * Seting X86_CR4_PKE will cause the X86_FEATURE_OSPKE
+ * cpuid bit to be set. We need to ensure that we
+ * update that bit in this CPU's "cpu_info".
+ */
+ get_cpu_cap(c);
+}
+
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+static __init int setup_disable_pku(char *arg)
+{
+ /*
+ * Do not clear the X86_FEATURE_PKU bit. All of the
+ * runtime checks are against OSPKE so clearing the
+ * bit does nothing.
+ *
+ * This way, we will see "pku" in cpuinfo, but not
+ * "ospke", which is exactly what we want. It shows
+ * that the CPU has PKU, but the OS has not enabled it.
+ * This happens to be exactly how a system would look
+ * if we disabled the config option.
+ */
+ pr_info("x86: 'nopku' specified, disabling Memory Protection Keys\n");
+ pku_disabled = true;
+ return 1;
+}
+__setup("nopku", setup_disable_pku);
+#endif /* CONFIG_X86_64 */
+
+/*
* Some CPU features depend on higher CPUID levels, which may not always
* be available due to CPUID level capping or broken virtualization
* software. Add those features to this table to auto-disable them.
@@ -960,6 +1002,7 @@ static void identify_cpu(struct cpuinfo_x86 *c)
init_hypervisor(c);
x86_init_rdrand(c);
x86_init_cache_qos(c);
+ setup_pku(c);
/*
* Clear/Set all flags overriden by options, need do it
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 07/33] x86, pkeys: define new CR4 bit |
| Message-ID | <r1t8w-3TO-65@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com> There is a new bit in CR4 for enabling protection keys. We will actually enable it later in the series. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/include/uapi/asm/processor-flags.h | 2 ++ 1 file changed, 2 insertions(+) diff -puN arch/x86/include/uapi/asm/processor-flags.h~pkeys-02-cr4 arch/x86/include/uapi/asm/processor-flags.h --- a/arch/x86/include/uapi/asm/processor-flags.h~pkeys-02-cr4 2016-02-12 10:44:16.803272741 -0800 +++ b/arch/x86/include/uapi/asm/processor-flags.h 2016-02-12 10:44:16.807272924 -0800 @@ -118,6 +118,8 @@ #define X86_CR4_SMEP _BITUL(X86_CR4_SMEP_BIT) #define X86_CR4_SMAP_BIT 21 /* enable SMAP support */ #define X86_CR4_SMAP _BITUL(X86_CR4_SMAP_BIT) +#define X86_CR4_PKE_BIT 22 /* enable Protection Keys support */ +#define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT) /* * x86-64 Task Priority Register, CR8 _
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:20 +0100 |
| Subject | [tip:mm/pkeys] x86/cpu, x86/mm/pkeys: Define new CR4 bit |
| Message-ID | <r3Ddp-2wj-59@gated-at.bofh.it> |
| In reply to | #1333125 |
Commit-ID: f28b49d2bcdb9ef9e771b3d6750f40be9d453316 Gitweb: http://git.kernel.org/tip/f28b49d2bcdb9ef9e771b3d6750f40be9d453316 Author: Dave Hansen <dave.hansen@linux.intel.com> AuthorDate: Fri, 12 Feb 2016 13:02:02 -0800 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Tue, 16 Feb 2016 10:11:14 +0100 x86/cpu, x86/mm/pkeys: Define new CR4 bit There is a new bit in CR4 for enabling protection keys. We will actually enable it later in the series. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave@sr71.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/20160212210202.3CFC3DB2@viggo.jf.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/include/uapi/asm/processor-flags.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/include/uapi/asm/processor-flags.h b/arch/x86/include/uapi/asm/processor-flags.h index 79887ab..567de50 100644 --- a/arch/x86/include/uapi/asm/processor-flags.h +++ b/arch/x86/include/uapi/asm/processor-flags.h @@ -118,6 +118,8 @@ #define X86_CR4_SMEP _BITUL(X86_CR4_SMEP_BIT) #define X86_CR4_SMAP_BIT 21 /* enable SMAP support */ #define X86_CR4_SMAP _BITUL(X86_CR4_SMAP_BIT) +#define X86_CR4_PKE_BIT 22 /* enable Protection Keys support */ +#define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT) /* * x86-64 Task Priority Register, CR8
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:10 +0100 |
| Subject | [PATCH 13/33] x86, pkeys: pass VMA down in to fault signal generation code |
| Message-ID | <r1t8x-3TO-67@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
During a page fault, we look up the VMA to ensure that the fault
is in a region with a valid mapping. But, in the top-level page
fault code we don't need the VMA for much else. Once we have
decided that an access is bad, we are going to send a signal no
matter what and do not need the VMA any more. So we do not pass
it down in to the signal generation code.
But, for protection keys, we need the VMA. It tells us *which*
protection key we violated if we get a PF_PK. So, we need to
pass the VMA down and fill in siginfo->si_pkey.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/mm/fault.c | 50 ++++++++++++++++++++++++++++----------------------
1 file changed, 28 insertions(+), 22 deletions(-)
diff -puN arch/x86/mm/fault.c~pkeys-08-pass-down-vma arch/x86/mm/fault.c
--- a/arch/x86/mm/fault.c~pkeys-08-pass-down-vma 2016-02-12 10:44:19.441393337 -0800
+++ b/arch/x86/mm/fault.c 2016-02-12 10:44:19.445393520 -0800
@@ -171,7 +171,8 @@ is_prefetch(struct pt_regs *regs, unsign
static void
force_sig_info_fault(int si_signo, int si_code, unsigned long address,
- struct task_struct *tsk, int fault)
+ struct task_struct *tsk, struct vm_area_struct *vma,
+ int fault)
{
unsigned lsb = 0;
siginfo_t info;
@@ -656,6 +657,8 @@ no_context(struct pt_regs *regs, unsigne
struct task_struct *tsk = current;
unsigned long flags;
int sig;
+ /* No context means no VMA to pass down */
+ struct vm_area_struct *vma = NULL;
/* Are we prepared to handle this kernel fault? */
if (fixup_exception(regs)) {
@@ -679,7 +682,8 @@ no_context(struct pt_regs *regs, unsigne
tsk->thread.cr2 = address;
/* XXX: hwpoison faults will set the wrong code. */
- force_sig_info_fault(signal, si_code, address, tsk, 0);
+ force_sig_info_fault(signal, si_code, address,
+ tsk, vma, 0);
}
/*
@@ -756,7 +760,8 @@ show_signal_msg(struct pt_regs *regs, un
static void
__bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, int si_code)
+ unsigned long address, struct vm_area_struct *vma,
+ int si_code)
{
struct task_struct *tsk = current;
@@ -799,7 +804,7 @@ __bad_area_nosemaphore(struct pt_regs *r
tsk->thread.error_code = error_code;
tsk->thread.trap_nr = X86_TRAP_PF;
- force_sig_info_fault(SIGSEGV, si_code, address, tsk, 0);
+ force_sig_info_fault(SIGSEGV, si_code, address, tsk, vma, 0);
return;
}
@@ -812,14 +817,14 @@ __bad_area_nosemaphore(struct pt_regs *r
static noinline void
bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
- unsigned long address)
+ unsigned long address, struct vm_area_struct *vma)
{
- __bad_area_nosemaphore(regs, error_code, address, SEGV_MAPERR);
+ __bad_area_nosemaphore(regs, error_code, address, vma, SEGV_MAPERR);
}
static void
__bad_area(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, int si_code)
+ unsigned long address, struct vm_area_struct *vma, int si_code)
{
struct mm_struct *mm = current->mm;
@@ -829,25 +834,25 @@ __bad_area(struct pt_regs *regs, unsigne
*/
up_read(&mm->mmap_sem);
- __bad_area_nosemaphore(regs, error_code, address, si_code);
+ __bad_area_nosemaphore(regs, error_code, address, vma, si_code);
}
static noinline void
bad_area(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
- __bad_area(regs, error_code, address, SEGV_MAPERR);
+ __bad_area(regs, error_code, address, NULL, SEGV_MAPERR);
}
static noinline void
bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
- unsigned long address)
+ unsigned long address, struct vm_area_struct *vma)
{
- __bad_area(regs, error_code, address, SEGV_ACCERR);
+ __bad_area(regs, error_code, address, vma, SEGV_ACCERR);
}
static void
do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address,
- unsigned int fault)
+ struct vm_area_struct *vma, unsigned int fault)
{
struct task_struct *tsk = current;
int code = BUS_ADRERR;
@@ -874,12 +879,13 @@ do_sigbus(struct pt_regs *regs, unsigned
code = BUS_MCEERR_AR;
}
#endif
- force_sig_info_fault(SIGBUS, code, address, tsk, fault);
+ force_sig_info_fault(SIGBUS, code, address, tsk, vma, fault);
}
static noinline void
mm_fault_error(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, unsigned int fault)
+ unsigned long address, struct vm_area_struct *vma,
+ unsigned int fault)
{
if (fatal_signal_pending(current) && !(error_code & PF_USER)) {
no_context(regs, error_code, address, 0, 0);
@@ -903,9 +909,9 @@ mm_fault_error(struct pt_regs *regs, uns
} else {
if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|
VM_FAULT_HWPOISON_LARGE))
- do_sigbus(regs, error_code, address, fault);
+ do_sigbus(regs, error_code, address, vma, fault);
else if (fault & VM_FAULT_SIGSEGV)
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, vma);
else
BUG();
}
@@ -1119,7 +1125,7 @@ __do_page_fault(struct pt_regs *regs, un
* Don't take the mm semaphore here. If we fixup a prefetch
* fault we could otherwise deadlock:
*/
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1132,7 +1138,7 @@ __do_page_fault(struct pt_regs *regs, un
pgtable_bad(regs, error_code, address);
if (unlikely(smap_violation(error_code, regs))) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1141,7 +1147,7 @@ __do_page_fault(struct pt_regs *regs, un
* in a region with pagefaults disabled then we must not take the fault
*/
if (unlikely(faulthandler_disabled() || !mm)) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1185,7 +1191,7 @@ __do_page_fault(struct pt_regs *regs, un
if (unlikely(!down_read_trylock(&mm->mmap_sem))) {
if ((error_code & PF_USER) == 0 &&
!search_exception_tables(regs->ip)) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
retry:
@@ -1233,7 +1239,7 @@ retry:
*/
good_area:
if (unlikely(access_error(error_code, vma))) {
- bad_area_access_error(regs, error_code, address);
+ bad_area_access_error(regs, error_code, address, vma);
return;
}
@@ -1271,7 +1277,7 @@ good_area:
up_read(&mm->mmap_sem);
if (unlikely(fault & VM_FAULT_ERROR)) {
- mm_fault_error(regs, error_code, address, fault);
+ mm_fault_error(regs, error_code, address, vma, fault);
return;
}
_
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] x86/mm/pkeys: Pass VMA down in to fault signal generation code |
| Message-ID | <r3Dn4-2Bf-9@gated-at.bofh.it> |
| In reply to | #1333126 |
Commit-ID: 7b2d0dbac4890c8ca4a8acc57709639fc8b158e9
Gitweb: http://git.kernel.org/tip/7b2d0dbac4890c8ca4a8acc57709639fc8b158e9
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:02:11 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 18 Feb 2016 09:31:51 +0100
x86/mm/pkeys: Pass VMA down in to fault signal generation code
During a page fault, we look up the VMA to ensure that the fault
is in a region with a valid mapping. But, in the top-level page
fault code we don't need the VMA for much else. Once we have
decided that an access is bad, we are going to send a signal no
matter what and do not need the VMA any more. So we do not pass
it down in to the signal generation code.
But, for protection keys, we need the VMA. It tells us *which*
protection key we violated if we get a PF_PK. So, we need to
pass the VMA down and fill in siginfo->si_pkey.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210211.AD3B36A3@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/mm/fault.c | 50 ++++++++++++++++++++++++++++----------------------
1 file changed, 28 insertions(+), 22 deletions(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 9f72f9c..3c51c66 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -171,7 +171,8 @@ is_prefetch(struct pt_regs *regs, unsigned long error_code, unsigned long addr)
static void
force_sig_info_fault(int si_signo, int si_code, unsigned long address,
- struct task_struct *tsk, int fault)
+ struct task_struct *tsk, struct vm_area_struct *vma,
+ int fault)
{
unsigned lsb = 0;
siginfo_t info;
@@ -656,6 +657,8 @@ no_context(struct pt_regs *regs, unsigned long error_code,
struct task_struct *tsk = current;
unsigned long flags;
int sig;
+ /* No context means no VMA to pass down */
+ struct vm_area_struct *vma = NULL;
/* Are we prepared to handle this kernel fault? */
if (fixup_exception(regs)) {
@@ -679,7 +682,8 @@ no_context(struct pt_regs *regs, unsigned long error_code,
tsk->thread.cr2 = address;
/* XXX: hwpoison faults will set the wrong code. */
- force_sig_info_fault(signal, si_code, address, tsk, 0);
+ force_sig_info_fault(signal, si_code, address,
+ tsk, vma, 0);
}
/*
@@ -756,7 +760,8 @@ show_signal_msg(struct pt_regs *regs, unsigned long error_code,
static void
__bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, int si_code)
+ unsigned long address, struct vm_area_struct *vma,
+ int si_code)
{
struct task_struct *tsk = current;
@@ -799,7 +804,7 @@ __bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
tsk->thread.error_code = error_code;
tsk->thread.trap_nr = X86_TRAP_PF;
- force_sig_info_fault(SIGSEGV, si_code, address, tsk, 0);
+ force_sig_info_fault(SIGSEGV, si_code, address, tsk, vma, 0);
return;
}
@@ -812,14 +817,14 @@ __bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
static noinline void
bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
- unsigned long address)
+ unsigned long address, struct vm_area_struct *vma)
{
- __bad_area_nosemaphore(regs, error_code, address, SEGV_MAPERR);
+ __bad_area_nosemaphore(regs, error_code, address, vma, SEGV_MAPERR);
}
static void
__bad_area(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, int si_code)
+ unsigned long address, struct vm_area_struct *vma, int si_code)
{
struct mm_struct *mm = current->mm;
@@ -829,25 +834,25 @@ __bad_area(struct pt_regs *regs, unsigned long error_code,
*/
up_read(&mm->mmap_sem);
- __bad_area_nosemaphore(regs, error_code, address, si_code);
+ __bad_area_nosemaphore(regs, error_code, address, vma, si_code);
}
static noinline void
bad_area(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
- __bad_area(regs, error_code, address, SEGV_MAPERR);
+ __bad_area(regs, error_code, address, NULL, SEGV_MAPERR);
}
static noinline void
bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
- unsigned long address)
+ unsigned long address, struct vm_area_struct *vma)
{
- __bad_area(regs, error_code, address, SEGV_ACCERR);
+ __bad_area(regs, error_code, address, vma, SEGV_ACCERR);
}
static void
do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address,
- unsigned int fault)
+ struct vm_area_struct *vma, unsigned int fault)
{
struct task_struct *tsk = current;
int code = BUS_ADRERR;
@@ -874,12 +879,13 @@ do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address,
code = BUS_MCEERR_AR;
}
#endif
- force_sig_info_fault(SIGBUS, code, address, tsk, fault);
+ force_sig_info_fault(SIGBUS, code, address, tsk, vma, fault);
}
static noinline void
mm_fault_error(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, unsigned int fault)
+ unsigned long address, struct vm_area_struct *vma,
+ unsigned int fault)
{
if (fatal_signal_pending(current) && !(error_code & PF_USER)) {
no_context(regs, error_code, address, 0, 0);
@@ -903,9 +909,9 @@ mm_fault_error(struct pt_regs *regs, unsigned long error_code,
} else {
if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|
VM_FAULT_HWPOISON_LARGE))
- do_sigbus(regs, error_code, address, fault);
+ do_sigbus(regs, error_code, address, vma, fault);
else if (fault & VM_FAULT_SIGSEGV)
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, vma);
else
BUG();
}
@@ -1119,7 +1125,7 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code,
* Don't take the mm semaphore here. If we fixup a prefetch
* fault we could otherwise deadlock:
*/
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1132,7 +1138,7 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code,
pgtable_bad(regs, error_code, address);
if (unlikely(smap_violation(error_code, regs))) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1141,7 +1147,7 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code,
* in a region with pagefaults disabled then we must not take the fault
*/
if (unlikely(faulthandler_disabled() || !mm)) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1185,7 +1191,7 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code,
if (unlikely(!down_read_trylock(&mm->mmap_sem))) {
if ((error_code & PF_USER) == 0 &&
!search_exception_tables(regs->ip)) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
retry:
@@ -1233,7 +1239,7 @@ retry:
*/
good_area:
if (unlikely(access_error(error_code, vma))) {
- bad_area_access_error(regs, error_code, address);
+ bad_area_access_error(regs, error_code, address, vma);
return;
}
@@ -1271,7 +1277,7 @@ good_area:
up_read(&mm->mmap_sem);
if (unlikely(fault & VM_FAULT_ERROR)) {
- mm_fault_error(regs, error_code, address, fault);
+ mm_fault_error(regs, error_code, address, vma, fault);
return;
}
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:20 +0100 |
| Subject | [PATCH 11/33] x86, pkeys: store protection in high VMA flags |
| Message-ID | <r1tia-3WT-1@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com> vma->vm_flags is an 'unsigned long', so has space for 32 flags on 32-bit architectures. The high 32 bits are unused on 64-bit platforms. We've steered away from using the unused high VMA bits for things because we would have difficulty supporting it on 32-bit. Protection Keys are not available in 32-bit mode, so there is no concern about supporting this feature in 32-bit mode or on 32-bit CPUs. This patch carves out 4 bits from the high half of vma->vm_flags and allows architectures to set config option to make them available. Sparse complains about these constants unless we explicitly call them "UL". Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/Kconfig | 1 + b/include/linux/mm.h | 11 +++++++++++ b/mm/Kconfig | 3 +++ 3 files changed, 15 insertions(+) diff -puN arch/x86/Kconfig~pkeys-06-eat-high-vma-flags arch/x86/Kconfig --- a/arch/x86/Kconfig~pkeys-06-eat-high-vma-flags 2016-02-12 10:44:18.494350045 -0800 +++ b/arch/x86/Kconfig 2016-02-12 10:44:18.502350411 -0800 @@ -155,6 +155,7 @@ config X86 select VIRT_TO_BUS select X86_DEV_DMA_OPS if X86_64 select X86_FEATURE_NAMES if PROC_FS + select ARCH_USES_HIGH_VMA_FLAGS if X86_INTEL_MEMORY_PROTECTION_KEYS config INSTRUCTION_DECODER def_bool y diff -puN include/linux/mm.h~pkeys-06-eat-high-vma-flags include/linux/mm.h --- a/include/linux/mm.h~pkeys-06-eat-high-vma-flags 2016-02-12 10:44:18.496350136 -0800 +++ b/include/linux/mm.h 2016-02-12 10:44:18.503350456 -0800 @@ -170,6 +170,17 @@ extern unsigned int kobjsize(const void #define VM_NOHUGEPAGE 0x40000000 /* MADV_NOHUGEPAGE marked this vma */ #define VM_MERGEABLE 0x80000000 /* KSM may merge identical pages */ +#ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS +#define VM_HIGH_ARCH_BIT_0 32 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_1 33 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_2 34 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_3 35 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_0 BIT(VM_HIGH_ARCH_BIT_0) +#define VM_HIGH_ARCH_1 BIT(VM_HIGH_ARCH_BIT_1) +#define VM_HIGH_ARCH_2 BIT(VM_HIGH_ARCH_BIT_2) +#define VM_HIGH_ARCH_3 BIT(VM_HIGH_ARCH_BIT_3) +#endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */ + #if defined(CONFIG_X86) # define VM_PAT VM_ARCH_1 /* PAT reserves whole VMA at once (x86) */ #elif defined(CONFIG_PPC) diff -puN mm/Kconfig~pkeys-06-eat-high-vma-flags mm/Kconfig --- a/mm/Kconfig~pkeys-06-eat-high-vma-flags 2016-02-12 10:44:18.498350228 -0800 +++ b/mm/Kconfig 2016-02-12 10:44:18.503350456 -0800 @@ -669,3 +669,6 @@ config ZONE_DEVICE config FRAME_VECTOR bool + +config ARCH_USES_HIGH_VMA_FLAGS + bool _
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Date | 2016-02-18 21:30 +0100 |
| Subject | [tip:mm/pkeys] mm/core, x86/mm/pkeys: Store protection bits in high VMA flags |
| Message-ID | <r3Dn7-2Bf-65@gated-at.bofh.it> |
| In reply to | #1333127 |
Commit-ID: 63c17fb8e5a46a16e10e82005748837fd11a2024 Gitweb: http://git.kernel.org/tip/63c17fb8e5a46a16e10e82005748837fd11a2024 Author: Dave Hansen <dave.hansen@linux.intel.com> AuthorDate: Fri, 12 Feb 2016 13:02:08 -0800 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Thu, 18 Feb 2016 09:31:50 +0100 mm/core, x86/mm/pkeys: Store protection bits in high VMA flags vma->vm_flags is an 'unsigned long', so has space for 32 flags on 32-bit architectures. The high 32 bits are unused on 64-bit platforms. We've steered away from using the unused high VMA bits for things because we would have difficulty supporting it on 32-bit. Protection Keys are not available in 32-bit mode, so there is no concern about supporting this feature in 32-bit mode or on 32-bit CPUs. This patch carves out 4 bits from the high half of vma->vm_flags and allows architectures to set config option to make them available. Sparse complains about these constants unless we explicitly call them "UL". Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Dave Hansen <dave@sr71.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Jan Kara <jack@suse.cz> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Valentin Rothberg <valentinrothberg@gmail.com> Cc: Vladimir Davydov <vdavydov@parallels.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Xie XiuQi <xiexiuqi@huawei.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/20160212210208.81AF00D5@viggo.jf.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/Kconfig | 1 + include/linux/mm.h | 11 +++++++++++ mm/Kconfig | 3 +++ 3 files changed, 15 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 3632cdd..fb2ebeb 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -155,6 +155,7 @@ config X86 select VIRT_TO_BUS select X86_DEV_DMA_OPS if X86_64 select X86_FEATURE_NAMES if PROC_FS + select ARCH_USES_HIGH_VMA_FLAGS if X86_INTEL_MEMORY_PROTECTION_KEYS config INSTRUCTION_DECODER def_bool y diff --git a/include/linux/mm.h b/include/linux/mm.h index 4c73178..54d173b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -170,6 +170,17 @@ extern unsigned int kobjsize(const void *objp); #define VM_NOHUGEPAGE 0x40000000 /* MADV_NOHUGEPAGE marked this vma */ #define VM_MERGEABLE 0x80000000 /* KSM may merge identical pages */ +#ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS +#define VM_HIGH_ARCH_BIT_0 32 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_1 33 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_2 34 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_3 35 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_0 BIT(VM_HIGH_ARCH_BIT_0) +#define VM_HIGH_ARCH_1 BIT(VM_HIGH_ARCH_BIT_1) +#define VM_HIGH_ARCH_2 BIT(VM_HIGH_ARCH_BIT_2) +#define VM_HIGH_ARCH_3 BIT(VM_HIGH_ARCH_BIT_3) +#endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */ + #if defined(CONFIG_X86) # define VM_PAT VM_ARCH_1 /* PAT reserves whole VMA at once (x86) */ #elif defined(CONFIG_PPC) diff --git a/mm/Kconfig b/mm/Kconfig index 03cbfa0..6cf4399 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -669,3 +669,6 @@ config ZONE_DEVICE config FRAME_VECTOR bool + +config ARCH_USES_HIGH_VMA_FLAGS + bool
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-02-12 22:20 +0100 |
| Subject | [PATCH 10/33] x86, pkeys: new page fault error code bit: PF_PK |
| Message-ID | <r1tia-3WT-9@gated-at.bofh.it> |
| In reply to | #1333117 |
From: Dave Hansen <dave.hansen@linux.intel.com>
Note: "PK" is how the Intel SDM refers to this bit, so we also
use that nomenclature.
This only defines the bit, it does not plumb it anywhere to be
handled.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/mm/fault.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff -puN arch/x86/mm/fault.c~pkeys-05-pfec arch/x86/mm/fault.c
--- a/arch/x86/mm/fault.c~pkeys-05-pfec 2016-02-12 10:44:18.080331119 -0800
+++ b/arch/x86/mm/fault.c 2016-02-12 10:44:18.084331302 -0800
@@ -33,6 +33,7 @@
* bit 2 == 0: kernel-mode access 1: user-mode access
* bit 3 == 1: use of reserved bit detected
* bit 4 == 1: fault was an instruction fetch
+ * bit 5 == 1: protection keys block access
*/
enum x86_pf_error_code {
@@ -41,6 +42,7 @@ enum x86_pf_error_code {
PF_USER = 1 << 2,
PF_RSVD = 1 << 3,
PF_INSTR = 1 << 4,
+ PF_PK = 1 << 5,
};
/*
@@ -916,6 +918,12 @@ static int spurious_fault_check(unsigned
if ((error_code & PF_INSTR) && !pte_exec(*pte))
return 0;
+ /*
+ * Note: We do not do lazy flushing on protection key
+ * changes, so no spurious fault will ever set PF_PK.
+ */
+ if ((error_code & PF_PK))
+ return 1;
return 1;
}
_
[toc] | [prev] | [next] | [standalone]
Page 1 of 3 [1] 2 3 Next page →
Back to top | Article view | linux.kernel
csiph-web