Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1303164 > unrolled thread
| Started by | Dave Hansen <dave@sr71.net> |
|---|---|
| First post | 2016-01-07 01:10 +0100 |
| Last post | 2016-01-07 01:20 +0100 |
| Articles | 20 on this page of 42 — 5 participants |
Back to article view | Back to linux.kernel
[PATCH 00/31] x86: Memory Protection Keys (v8) Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 22/31] x86, pkeys: dump pkey from VMA in /proc/pid/smaps Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 28/31] x86, fpu: allow setting of XSAVE state Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 06/31] x86, pkeys: add PKRU xsave fields and data structure(s) Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 07/31] x86, pkeys: PTE bits for storing protection key Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 01/31] mm, gup: introduce concept of "foreign" get_user_pages() Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
Re: [PATCH 01/31] mm, gup: introduce concept of "foreign" get_user_pages() Vlastimil Babka <vbabka@suse.cz> - 2016-01-13 20:10 +0100
Re: [PATCH 01/31] mm, gup: introduce concept of "foreign" get_user_pages() Dave Hansen <dave@sr71.net> - 2016-01-13 20:20 +0100
[PATCH 17/31] x86, pkeys: check VMAs and PTEs for protection keys Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 03/31] x86, pkeys: Add Kconfig option Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 29/31] x86, pkeys: allow kernel to modify user pkey rights register Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 21/31] x86, pkeys: dump PKRU with other kernel registers Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 11/31] x86, pkeys: pass VMA down in to fault signal generation code Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 10/31] x86, pkeys: arch-specific protection bits Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 19/31] x86, pkeys: optimize fault handling in access_error() Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 05/31] x86, pkeys: define new CR4 bit Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 09/31] x86, pkeys: store protection in high VMA flags Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 16/31] x86, mm: simplify get_user_pages() PTE bit handling Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 27/31] x86: separate out LDT init from context init Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 14/31] x86, pkeys: add functions to fetch PKRU Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
Re: [PATCH 14/31] x86, pkeys: add functions to fetch PKRU Thomas Gleixner <tglx@linutronix.de> - 2016-01-08 20:40 +0100
[PATCH 31/31] x86, pkeys: execute-only support Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
Re: [PATCH 31/31] x86, pkeys: execute-only support Kees Cook <keescook@google.com> - 2016-01-07 22:10 +0100
Re: [PATCH 31/31] x86, pkeys: execute-only support Dave Hansen <dave@sr71.net> - 2016-01-07 23:30 +0100
Re: [PATCH 31/31] x86, pkeys: execute-only support Andy Lutomirski <luto@amacapital.net> - 2016-01-07 22:20 +0100
Re: [PATCH 31/31] x86, pkeys: execute-only support Dave Hansen <dave@sr71.net> - 2016-01-07 23:20 +0100
Re: [PATCH 31/31] x86, pkeys: execute-only support Andy Lutomirski <luto@amacapital.net> - 2016-01-07 23:50 +0100
Re: [PATCH 31/31] x86, pkeys: execute-only support Thomas Gleixner <tglx@linutronix.de> - 2016-01-08 21:00 +0100
[PATCH 24/31] x86, pkeys: actually enable Memory Protection Keys in CPU Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 30/31] x86, pkeys: create an x86 arch_calc_vm_prot_bits() for VMA flags Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 18/31] mm: add gup flag to indicate "foreign" mm access Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 26/31] x86, pkeys: add arch_validate_pkey() Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
Re: [PATCH 26/31] x86, pkeys: add arch_validate_pkey() Thomas Gleixner <tglx@linutronix.de> - 2016-01-08 20:40 +0100
[PATCH 08/31] x86, pkeys: new page fault error code bit: PF_PK Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 13/31] x86, pkeys: fill in pkey field in siginfo Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 12/31] signals, pkeys: notify userspace about protection key faults Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 23/31] x86, pkeys: add Kconfig prompt to existing config option Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 15/31] mm: factor out VMA fault permission checking Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 04/31] x86, pkeys: cpuid bit definition Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 20/31] x86, pkeys: differentiate instruction fetches Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 25/31] mm, multi-arch: pass a protection key in to calc_vm_flag_bits() Dave Hansen <dave@sr71.net> - 2016-01-07 01:10 +0100
[PATCH 02/31] x86, fpu: add placeholder for Processor Trace XSAVE state Dave Hansen <dave@sr71.net> - 2016-01-07 01:20 +0100
Page 1 of 3 [1] 2 3 Next page →
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 00/31] x86: Memory Protection Keys (v8) |
| Message-ID | <qO6jn-8sH-3@gated-at.bofh.it> |
Memory Protection Keys for User pages is a CPU feature which will
first appear on Skylake Servers, but will also be supported on
future non-server parts (there is also a QEMU implementation). It
provides a mechanism for enforcing page-based protections, but
without requiring modification of the page tables when an
application changes protection domains.
This set introduces supported limited to:
1. Allows "execute-only" memory
2. Enables KVM to run Protection-Key-enabled guests
My preference would be to merge this part by itself (presumably
for 4.6, *not* 4.5). This set contains the vast majority of
of the code, with the small but tricky explicit user interface
parts left off. We can have a more focused review on those at
a later time in a (much smaller) follow-on series.
Changes from v7:
* Fixed merge issue with cpu feature bitmap definitions
* Fixed up some comments in get_user_pages() and smaps patches
(thanks Vlastimil!)
Changes from v6:
* fix up ??'s showing up in in smaps' VmFlags field
* added execute-only support
* removed all the new syscalls from this set. We can discuss
them in detail after this is merged.
Changes from v5:
* make types in read_pkru() u32's, not ints
* rework VM_* bits to avoid using __ffsl() and clean up
vma_pkey()
* rework pte_allows_gup() to use p??_val() instead of passing
around p{te,md,ud}_t types.
* Fix up some inconsistent bool vs. int usage
* corrected name of ARCH_VM_PKEY_FLAGS in patch description
* remove NR_PKEYS... config option. Just define it directly
Changes from v4:
* Made "allow setting of XSAVE state" safe if we got preempted
between when we saved our FPU state and when we restore it.
(I would appreciate a look from Ingo on this patch).
* Fixed up a few things from Thomas's latest comments: splt up
siginfo in to x86 and generic, removed extra 'eax' variable
in rdpkru function, reworked vm_flags assignment, reworded
a comment in pte_allows_gup()
* Add missing DISABLED/REQUIRED_MASK14 in cpufeature.h
* Added comment about compile optimization in fault path
* Left get_user_pages_locked() alone. Andrea thinks we need it.
Changes from RFCv3:
* Added 'current' and 'foreign' variants of get_user_pages() to
help indicate whether protection keys should be enforced.
Thanks to Jerome Glisse for pointing out this issue.
* Added "allocation" and set/get system calls so that we can do
management of proection keys in the kernel. This opens the
door to use of specific protection keys for kernel use in the
future, such as for execute-only memory.
* Removed the kselftest code for the moment. It will be
submitted separately.
Thanks Ingo and Thomas for most of these):
Changes from RFCv2 (Thanks Ingo and Thomas for most of these):
* few minor compile warnings
* changed 'nopku' interaction with cpuid bits. Now, we do not
clear the PKU cpuid bit, we just skip enabling it.
* changed __pkru_allows_write() to also check access disable bit
* removed the unused write_pkru()
* made si_pkey a u64 and added some patch description details.
Also made it share space in siginfo with MPX and clarified
comments.
* give some real text for the Processor Trace xsave state
* made vma_pkey() less ugly (and much more optimized actually)
* added SEGV_PKUERR to copy_siginfo_to_user()
* remove page table walk when filling in si_pkey, added some
big fat comments about it being inherently racy.
* added self test code
This code is not runnable to anyone outside of Intel unless they
have some special hardware or a fancy simulator. There is a qemu
model to emulate the feature, but it is not currently implemented
fully enough to be usable. If you are interested in running this
for real, please get in touch with me. Hardware is available to a
very small but nonzero number of people.
This set is also available here:
git://git.kernel.org/pub/scm/linux/kernel/git/daveh/x86-pkeys.git pkeys-v019
=== diffstat ===
Dave Hansen (31):
mm, gup: introduce concept of "foreign" get_user_pages()
x86, fpu: add placeholder for Processor Trace XSAVE state
x86, pkeys: Add Kconfig option
x86, pkeys: cpuid bit definition
x86, pkeys: define new CR4 bit
x86, pkeys: add PKRU xsave fields and data structure(s)
x86, pkeys: PTE bits for storing protection key
x86, pkeys: new page fault error code bit: PF_PK
x86, pkeys: store protection in high VMA flags
x86, pkeys: arch-specific protection bits
x86, pkeys: pass VMA down in to fault signal generation code
signals, pkeys: notify userspace about protection key faults
x86, pkeys: fill in pkey field in siginfo
x86, pkeys: add functions to fetch PKRU
mm: factor out VMA fault permission checking
x86, mm: simplify get_user_pages() PTE bit handling
x86, pkeys: check VMAs and PTEs for protection keys
mm: add gup flag to indicate "foreign" mm access
x86, pkeys: optimize fault handling in access_error()
x86, pkeys: differentiate instruction fetches
x86, pkeys: dump PKRU with other kernel registers
x86, pkeys: dump pkey from VMA in /proc/pid/smaps
x86, pkeys: add Kconfig prompt to existing config option
x86, pkeys: actually enable Memory Protection Keys in CPU
mm, multi-arch: pass a protection key in to calc_vm_flag_bits()
x86, pkeys: add arch_validate_pkey()
x86: separate out LDT init from context init
x86, fpu: allow setting of XSAVE state
x86, pkeys: allow kernel to modify user pkey rights register
x86, pkeys: create an x86 arch_calc_vm_prot_bits() for VMA flags
x86, pkeys: execute-only support
Documentation/kernel-parameters.txt | 3 +
arch/mips/mm/gup.c | 3 +-
arch/powerpc/include/asm/mman.h | 5 +-
arch/powerpc/include/asm/mmu_context.h | 12 ++
arch/s390/include/asm/mmu_context.h | 12 ++
arch/s390/mm/gup.c | 4 +-
arch/sh/mm/gup.c | 2 +-
arch/sparc/mm/gup.c | 2 +-
arch/unicore32/include/asm/mmu_context.h | 12 ++
arch/x86/Kconfig | 16 ++
arch/x86/include/asm/cpufeature.h | 56 +++---
arch/x86/include/asm/disabled-features.h | 13 ++
arch/x86/include/asm/fpu/internal.h | 2 +
arch/x86/include/asm/fpu/types.h | 12 ++
arch/x86/include/asm/fpu/xstate.h | 4 +-
arch/x86/include/asm/mmu_context.h | 85 ++++++++-
arch/x86/include/asm/pgtable.h | 38 ++++
arch/x86/include/asm/pgtable_types.h | 34 +++-
arch/x86/include/asm/pkeys.h | 34 ++++
arch/x86/include/asm/required-features.h | 5 +
arch/x86/include/asm/special_insns.h | 22 +++
arch/x86/include/uapi/asm/mman.h | 22 +++
arch/x86/include/uapi/asm/processor-flags.h | 2 +
arch/x86/kernel/cpu/common.c | 42 +++++
arch/x86/kernel/fpu/core.c | 63 +++++++
arch/x86/kernel/fpu/init.c | 3 +
arch/x86/kernel/fpu/xstate.c | 185 +++++++++++++++++++-
arch/x86/kernel/ldt.c | 4 +-
arch/x86/kernel/process_64.c | 2 +
arch/x86/kernel/setup.c | 9 +
arch/x86/mm/Makefile | 2 +
arch/x86/mm/fault.c | 171 +++++++++++++++---
arch/x86/mm/gup.c | 46 +++--
arch/x86/mm/mpx.c | 4 +-
arch/x86/mm/pkeys.c | 101 +++++++++++
drivers/char/agp/frontend.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 4 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 2 +-
drivers/gpu/drm/radeon/radeon_ttm.c | 4 +-
drivers/gpu/drm/via/via_dmablit.c | 3 +-
drivers/infiniband/core/umem.c | 2 +-
drivers/infiniband/core/umem_odp.c | 8 +-
drivers/infiniband/hw/mthca/mthca_memfree.c | 3 +-
drivers/infiniband/hw/qib/qib_user_pages.c | 3 +-
drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +-
drivers/iommu/amd_iommu_v2.c | 8 +-
drivers/media/pci/ivtv/ivtv-udma.c | 4 +-
drivers/media/pci/ivtv/ivtv-yuv.c | 10 +-
drivers/media/v4l2-core/videobuf-dma-sg.c | 3 +-
drivers/misc/sgi-gru/grufault.c | 3 +-
drivers/scsi/st.c | 2 -
drivers/staging/android/ashmem.c | 4 +-
drivers/video/fbdev/pvr2fb.c | 4 +-
drivers/virt/fsl_hypervisor.c | 5 +-
fs/exec.c | 8 +-
fs/proc/task_mmu.c | 14 ++
include/asm-generic/mm_hooks.h | 12 ++
include/linux/mm.h | 60 +++++--
include/linux/mman.h | 6 +-
include/linux/pkeys.h | 33 ++++
include/uapi/asm-generic/siginfo.h | 17 +-
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: aarcange@redhat.com
Cc: akpm@linux-foundation.org
Cc: jack@suse.cz
Cc: kirill.shutemov@linux.intel.com
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: n-horiguchi@ah.jp.nec.com
Cc: x86@kernel.org
Cc: torvalds@linux-foundation.org
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 22/31] x86, pkeys: dump pkey from VMA in /proc/pid/smaps |
| Message-ID | <qO6jo-8sH-5@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
The protection key can now be just as important as read/write
permissions on a VMA. We need some debug mechanism to help
figure out if it is in play. smaps seems like a logical
place to expose it.
arch/x86/kernel/setup.c is a bit of a weirdo place to put
this code, but it already had seq_file.h and there was not
a much better existing place to put it.
We also use no #ifdef. If protection keys is .config'd out we
will effectively get the same function as if we used the weak
generic function.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: vbabka@suse.cz
---
b/arch/x86/kernel/setup.c | 9 +++++++++
b/fs/proc/task_mmu.c | 14 ++++++++++++++
2 files changed, 23 insertions(+)
diff -puN arch/x86/kernel/setup.c~pkeys-40-smaps arch/x86/kernel/setup.c
--- a/arch/x86/kernel/setup.c~pkeys-40-smaps 2016-01-06 15:50:12.674474474 -0800
+++ b/arch/x86/kernel/setup.c 2016-01-06 15:50:12.679474700 -0800
@@ -112,6 +112,7 @@
#include <asm/alternative.h>
#include <asm/prom.h>
#include <asm/microcode.h>
+#include <asm/mmu_context.h>
/*
* max_low_pfn_mapped: highest direct mapped pfn under 4GB
@@ -1282,3 +1283,11 @@ static int __init register_kernel_offset
return 0;
}
__initcall(register_kernel_offset_dumper);
+
+void arch_show_smap(struct seq_file *m, struct vm_area_struct *vma)
+{
+ if (!boot_cpu_has(X86_FEATURE_OSPKE))
+ return;
+
+ seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma));
+}
diff -puN fs/proc/task_mmu.c~pkeys-40-smaps fs/proc/task_mmu.c
--- a/fs/proc/task_mmu.c~pkeys-40-smaps 2016-01-06 15:50:12.675474519 -0800
+++ b/fs/proc/task_mmu.c 2016-01-06 15:50:12.679474700 -0800
@@ -615,11 +615,20 @@ static void show_smap_vma_flags(struct s
[ilog2(VM_MERGEABLE)] = "mg",
[ilog2(VM_UFFD_MISSING)]= "um",
[ilog2(VM_UFFD_WP)] = "uw",
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+ /* These come out via ProtectionKey: */
+ [ilog2(VM_PKEY_BIT0)] = "",
+ [ilog2(VM_PKEY_BIT1)] = "",
+ [ilog2(VM_PKEY_BIT2)] = "",
+ [ilog2(VM_PKEY_BIT3)] = "",
+#endif
};
size_t i;
seq_puts(m, "VmFlags: ");
for (i = 0; i < BITS_PER_LONG; i++) {
+ if (!mnemonics[i][0])
+ continue;
if (vma->vm_flags & (1UL << i)) {
seq_printf(m, "%c%c ",
mnemonics[i][0], mnemonics[i][1]);
@@ -657,6 +666,10 @@ static int smaps_hugetlb_range(pte_t *pt
}
#endif /* HUGETLB_PAGE */
+void __weak arch_show_smap(struct seq_file *m, struct vm_area_struct *vma)
+{
+}
+
static int show_smap(struct seq_file *m, void *v, int is_pid)
{
struct vm_area_struct *vma = v;
@@ -713,6 +726,7 @@ static int show_smap(struct seq_file *m,
(vma->vm_flags & VM_LOCKED) ?
(unsigned long)(mss.pss >> (10 + PSS_SHIFT)) : 0);
+ arch_show_smap(m, vma);
show_smap_vma_flags(m, vma);
m_cache_vma(m, vma);
return 0;
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 28/31] x86, fpu: allow setting of XSAVE state |
| Message-ID | <qO6jo-8sH-13@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
We want to modify the Protection Key rights inside the kernel, so
we need to change PKRU's contents. But, if we do a plain
'wrpkru', when we return to userspace we might do an XRSTOR and
wipe out the kernel's 'wrpkru'. So, we need to go after PKRU in
the xsave buffer.
We do this by:
1. Ensuring that we have the XSAVE registers (fpregs) in the
kernel FPU buffer (fpstate)
2. Looking up the location of a given state in the buffer
3. Filling in the stat
4. Ensuring that the hardware knows that state is present there
(basically that the 'init optimization' is not in place).
5. Copying the newly-modified state back to the registers if
necessary.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/include/asm/fpu/internal.h | 2
b/arch/x86/kernel/fpu/core.c | 63 +++++++++++++++++++++
b/arch/x86/kernel/fpu/xstate.c | 98 +++++++++++++++++++++++++++++++++-
3 files changed, 161 insertions(+), 2 deletions(-)
diff -puN arch/x86/include/asm/fpu/internal.h~pkeys-76-xsave-set arch/x86/include/asm/fpu/internal.h
--- a/arch/x86/include/asm/fpu/internal.h~pkeys-76-xsave-set 2016-01-06 15:50:15.440599181 -0800
+++ b/arch/x86/include/asm/fpu/internal.h 2016-01-06 15:50:15.446599452 -0800
@@ -24,6 +24,8 @@
extern void fpu__activate_curr(struct fpu *fpu);
extern void fpu__activate_fpstate_read(struct fpu *fpu);
extern void fpu__activate_fpstate_write(struct fpu *fpu);
+extern void fpu__current_fpstate_write_begin(void);
+extern void fpu__current_fpstate_write_end(void);
extern void fpu__save(struct fpu *fpu);
extern void fpu__restore(struct fpu *fpu);
extern int fpu__restore_sig(void __user *buf, int ia32_frame);
diff -puN arch/x86/kernel/fpu/core.c~pkeys-76-xsave-set arch/x86/kernel/fpu/core.c
--- a/arch/x86/kernel/fpu/core.c~pkeys-76-xsave-set 2016-01-06 15:50:15.441599227 -0800
+++ b/arch/x86/kernel/fpu/core.c 2016-01-06 15:50:15.447599497 -0800
@@ -352,6 +352,69 @@ void fpu__activate_fpstate_write(struct
}
/*
+ * This function must be called before we write the current
+ * task's fpstate.
+ *
+ * This call gets the current FPU register state and moves
+ * it in to the 'fpstate'. Preemption is disabled so that
+ * no writes to the 'fpstate' can occur from context
+ * swiches.
+ *
+ * Must be followed by a fpu__current_fpstate_write_end().
+ */
+void fpu__current_fpstate_write_begin(void)
+{
+ struct fpu *fpu = ¤t->thread.fpu;
+
+ /*
+ * Ensure that the context-switching code does not write
+ * over the fpstate while we are doing our update.
+ */
+ preempt_disable();
+
+ /*
+ * Move the fpregs in to the fpu's 'fpstate'.
+ */
+ fpu__activate_fpstate_read(fpu);
+
+ /*
+ * The caller is about to write to 'fpu'. Ensure that no
+ * CPU thinks that its fpregs match the fpstate. This
+ * ensures we will not be lazy and skip a XRSTOR in the
+ * future.
+ */
+ fpu->last_cpu = -1;
+}
+
+/*
+ * This function must be paired with fpu__current_fpstate_write_begin()
+ *
+ * This will ensure that the modified fpstate gets placed back in
+ * the fpregs if necessary.
+ *
+ * Note: This function may be called whether or not an _actual_
+ * write to the fpstate occurred.
+ */
+void fpu__current_fpstate_write_end(void)
+{
+ struct fpu *fpu = ¤t->thread.fpu;
+
+ /*
+ * 'fpu' now has an updated copy of the state, but the
+ * registers may still be out of date. Update them with
+ * an XRSTOR if they are active.
+ */
+ if (fpregs_active())
+ copy_kernel_to_fpregs(&fpu->state);
+
+ /*
+ * Our update is done and the fpregs/fpstate are in sync
+ * if necessary. Context switches can happen again.
+ */
+ preempt_enable();
+}
+
+/*
* 'fpu__restore()' is called to copy FPU registers from
* the FPU fpstate to the live hw registers and to activate
* access to the hardware registers, so that FPU instructions
diff -puN arch/x86/kernel/fpu/xstate.c~pkeys-76-xsave-set arch/x86/kernel/fpu/xstate.c
--- a/arch/x86/kernel/fpu/xstate.c~pkeys-76-xsave-set 2016-01-06 15:50:15.443599317 -0800
+++ b/arch/x86/kernel/fpu/xstate.c 2016-01-06 15:50:15.447599497 -0800
@@ -679,6 +679,19 @@ void fpu__resume_cpu(void)
}
/*
+ * Given an xstate feature mask, calculate where in the xsave
+ * buffer the state is. Callers should ensure that the buffer
+ * is valid.
+ *
+ * Note: does not work for compacted buffers.
+ */
+void *__raw_xsave_addr(struct xregs_state *xsave, int xstate_feature_mask)
+{
+ int feature_nr = fls64(xstate_feature_mask) - 1;
+
+ return (void *)xsave + xstate_comp_offsets[feature_nr];
+}
+/*
* Given the xsave area and a state inside, this function returns the
* address of the state.
*
@@ -698,7 +711,6 @@ void fpu__resume_cpu(void)
*/
void *get_xsave_addr(struct xregs_state *xsave, int xstate_feature)
{
- int feature_nr = fls64(xstate_feature) - 1;
/*
* Do we even *have* xsave state?
*/
@@ -727,7 +739,7 @@ void *get_xsave_addr(struct xregs_state
if (!(xsave->header.xfeatures & xstate_feature))
return NULL;
- return (void *)xsave + xstate_comp_offsets[feature_nr];
+ return __raw_xsave_addr(xsave, xstate_feature);
}
EXPORT_SYMBOL_GPL(get_xsave_addr);
@@ -762,3 +774,85 @@ const void *get_xsave_field_ptr(int xsav
return get_xsave_addr(&fpu->state.xsave, xsave_state);
}
+
+
+/*
+ * Set xfeatures (aka XSTATE_BV) bit for a feature that we want
+ * to take out of its "init state". This will ensure that an
+ * XRSTOR actually restores the state.
+ */
+static void fpu__xfeature_set_non_init(struct xregs_state *xsave,
+ int xstate_feature_mask)
+{
+ xsave->header.xfeatures |= xstate_feature_mask;
+}
+
+/*
+ * This function is safe to call whether the FPU is in use or not.
+ *
+ * Note that this only works on the current task.
+ *
+ * Inputs:
+ * @xsave_state: state which is defined in xsave.h (e.g. XFEATURE_MASK_FP,
+ * XFEATURE_MASK_SSE, etc...)
+ * @xsave_state_ptr: a pointer to a copy of the state that you would
+ * like written in to the current task's FPU xsave state. This pointer
+ * must not be located in the current tasks's xsave area.
+ * Output:
+ * address of the state in the xsave area or NULL if the state
+ * is not present or is in its 'init state'.
+ */
+static void fpu__xfeature_set_state(int xstate_feature_mask,
+ void *xstate_feature_src, size_t len)
+{
+ struct xregs_state *xsave = ¤t->thread.fpu.state.xsave;
+ struct fpu *fpu = ¤t->thread.fpu;
+ void *dst;
+
+ if (!boot_cpu_has(X86_FEATURE_XSAVE)) {
+ WARN_ONCE(1, "%s() attempted with no xsave support", __func__);
+ return;
+ }
+
+ /*
+ * Tell the FPU code that we need the FPU state to be in
+ * 'fpu' (not in the registers), and that we need it to
+ * be stable while we write to it.
+ */
+ fpu__current_fpstate_write_begin();
+
+ /*
+ * This method *WILL* *NOT* work for compact-format
+ * buffers. If the 'xstate_feature_mask' is unset in
+ * xcomp_bv then we may need to move other feature state
+ * "up" in the buffer.
+ */
+ if (xsave->header.xcomp_bv & xstate_feature_mask) {
+ WARN_ON_ONCE(1);
+ goto out;
+ }
+
+ /* find the location in the xsave buffer of the desired state */
+ dst = __raw_xsave_addr(&fpu->state.xsave, xstate_feature_mask);
+
+ /*
+ * Make sure that the pointer being passed in did not
+ * come from the xsave buffer itself.
+ */
+ WARN_ONCE(xstate_feature_src == dst, "set from xsave buffer itself");
+
+ /* put the caller-provided data in the location */
+ memcpy(dst, xstate_feature_src, len);
+
+ /*
+ * Mark the xfeature so that the CPU knows there is state
+ * in the buffer now.
+ */
+ fpu__xfeature_set_non_init(xsave, xstate_feature_mask);
+out:
+ /*
+ * We are done writing to the 'fpu'. Reenable preeption
+ * and (possibly) move the fpstate back in to the fpregs.
+ */
+ fpu__current_fpstate_write_end();
+}
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 06/31] x86, pkeys: add PKRU xsave fields and data structure(s) |
| Message-ID | <qO6jo-8sH-15@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
The protection keys register (PKRU) is saved and restored using
xsave. Define the data structure that we will use to access it
inside the xsave buffer.
Note that we also have to widen the printk of the xsave feature
masks since this is feature 0x200 and we only did two characters
before.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/include/asm/fpu/types.h | 11 +++++++++++
b/arch/x86/include/asm/fpu/xstate.h | 4 +++-
b/arch/x86/kernel/fpu/xstate.c | 7 ++++++-
3 files changed, 20 insertions(+), 2 deletions(-)
diff -puN arch/x86/include/asm/fpu/types.h~pkeys-03-xsave arch/x86/include/asm/fpu/types.h
--- a/arch/x86/include/asm/fpu/types.h~pkeys-03-xsave 2016-01-06 15:50:05.206137774 -0800
+++ b/arch/x86/include/asm/fpu/types.h 2016-01-06 15:50:05.212138044 -0800
@@ -109,6 +109,7 @@ enum xfeature {
XFEATURE_ZMM_Hi256,
XFEATURE_Hi16_ZMM,
XFEATURE_PT_UNIMPLEMENTED_SO_FAR,
+ XFEATURE_PKRU,
XFEATURE_MAX,
};
@@ -121,6 +122,7 @@ enum xfeature {
#define XFEATURE_MASK_OPMASK (1 << XFEATURE_OPMASK)
#define XFEATURE_MASK_ZMM_Hi256 (1 << XFEATURE_ZMM_Hi256)
#define XFEATURE_MASK_Hi16_ZMM (1 << XFEATURE_Hi16_ZMM)
+#define XFEATURE_MASK_PKRU (1 << XFEATURE_PKRU)
#define XFEATURE_MASK_FPSSE (XFEATURE_MASK_FP | XFEATURE_MASK_SSE)
#define XFEATURE_MASK_AVX512 (XFEATURE_MASK_OPMASK \
@@ -213,6 +215,15 @@ struct avx_512_hi16_state {
struct reg_512_bit hi16_zmm[16];
} __packed;
+/*
+ * State component 9: 32-bit PKRU register. The state is
+ * 8 bytes long but only 4 bytes is used currently.
+ */
+struct pkru_state {
+ u32 pkru;
+ u32 pad;
+} __packed;
+
struct xstate_header {
u64 xfeatures;
u64 xcomp_bv;
diff -puN arch/x86/include/asm/fpu/xstate.h~pkeys-03-xsave arch/x86/include/asm/fpu/xstate.h
--- a/arch/x86/include/asm/fpu/xstate.h~pkeys-03-xsave 2016-01-06 15:50:05.208137864 -0800
+++ b/arch/x86/include/asm/fpu/xstate.h 2016-01-06 15:50:05.213138090 -0800
@@ -27,7 +27,9 @@
XFEATURE_MASK_Hi16_ZMM)
/* Supported features which require eager state saving */
-#define XFEATURE_MASK_EAGER (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_BNDCSR)
+#define XFEATURE_MASK_EAGER (XFEATURE_MASK_BNDREGS | \
+ XFEATURE_MASK_BNDCSR | \
+ XFEATURE_MASK_PKRU)
/* All currently supported features */
#define XCNTXT_MASK (XFEATURE_MASK_LAZY | XFEATURE_MASK_EAGER)
diff -puN arch/x86/kernel/fpu/xstate.c~pkeys-03-xsave arch/x86/kernel/fpu/xstate.c
--- a/arch/x86/kernel/fpu/xstate.c~pkeys-03-xsave 2016-01-06 15:50:05.209137909 -0800
+++ b/arch/x86/kernel/fpu/xstate.c 2016-01-06 15:50:05.213138090 -0800
@@ -29,6 +29,8 @@ static const char *xfeature_names[] =
"AVX-512 Hi256" ,
"AVX-512 ZMM_Hi256" ,
"Processor Trace (unused)" ,
+ "Protection Keys User registers",
+ "unknown xstate feature" ,
};
/*
@@ -57,6 +59,7 @@ void fpu__xstate_clear_all_cpu_caps(void
setup_clear_cpu_cap(X86_FEATURE_AVX512ER);
setup_clear_cpu_cap(X86_FEATURE_AVX512CD);
setup_clear_cpu_cap(X86_FEATURE_MPX);
+ setup_clear_cpu_cap(X86_FEATURE_PKU);
}
/*
@@ -235,7 +238,7 @@ static void __init print_xstate_feature(
const char *feature_name;
if (cpu_has_xfeatures(xstate_mask, &feature_name))
- pr_info("x86/fpu: Supporting XSAVE feature 0x%02Lx: '%s'\n", xstate_mask, feature_name);
+ pr_info("x86/fpu: Supporting XSAVE feature 0x%03Lx: '%s'\n", xstate_mask, feature_name);
}
/*
@@ -251,6 +254,7 @@ static void __init print_xstate_features
print_xstate_feature(XFEATURE_MASK_OPMASK);
print_xstate_feature(XFEATURE_MASK_ZMM_Hi256);
print_xstate_feature(XFEATURE_MASK_Hi16_ZMM);
+ print_xstate_feature(XFEATURE_MASK_PKRU);
}
/*
@@ -467,6 +471,7 @@ static void check_xstate_against_struct(
XCHECK_SZ(sz, nr, XFEATURE_OPMASK, struct avx_512_opmask_state);
XCHECK_SZ(sz, nr, XFEATURE_ZMM_Hi256, struct avx_512_zmm_uppers_state);
XCHECK_SZ(sz, nr, XFEATURE_Hi16_ZMM, struct avx_512_hi16_state);
+ XCHECK_SZ(sz, nr, XFEATURE_PKRU, struct pkru_state);
/*
* Make *SURE* to add any feature numbers in below if
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 07/31] x86, pkeys: PTE bits for storing protection key |
| Message-ID | <qO6jo-8sH-19@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com> Previous documentation has referred to these 4 bits as "ignored". That means that software could have made use of them. But, as far as I know, the kernel never used them. They are still ignored when protection keys is not enabled, so they could theoretically still get used for software purposes. We also implement "empty" versions so that code that references to them can be optimized away by the compiler when the config option is not enabled. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/include/asm/pgtable_types.h | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff -puN arch/x86/include/asm/pgtable_types.h~pkeys-04-ptebits arch/x86/include/asm/pgtable_types.h --- a/arch/x86/include/asm/pgtable_types.h~pkeys-04-ptebits 2016-01-06 15:50:05.662158333 -0800 +++ b/arch/x86/include/asm/pgtable_types.h 2016-01-06 15:50:05.665158468 -0800 @@ -25,7 +25,11 @@ #define _PAGE_BIT_SPLITTING _PAGE_BIT_SOFTW2 /* only valid on a PSE pmd */ #define _PAGE_BIT_HIDDEN _PAGE_BIT_SOFTW3 /* hidden by kmemcheck */ #define _PAGE_BIT_SOFT_DIRTY _PAGE_BIT_SOFTW3 /* software dirty tracking */ -#define _PAGE_BIT_NX 63 /* No execute: only valid after cpuid check */ +#define _PAGE_BIT_PKEY_BIT0 59 /* Protection Keys, bit 1/4 */ +#define _PAGE_BIT_PKEY_BIT1 60 /* Protection Keys, bit 2/4 */ +#define _PAGE_BIT_PKEY_BIT2 61 /* Protection Keys, bit 3/4 */ +#define _PAGE_BIT_PKEY_BIT3 62 /* Protection Keys, bit 4/4 */ +#define _PAGE_BIT_NX 63 /* No execute: only valid after cpuid check */ /* If _PAGE_BIT_PRESENT is clear, we use these: */ /* - if the user mapped it with PROT_NONE; pte_present gives true */ @@ -47,6 +51,17 @@ #define _PAGE_SPECIAL (_AT(pteval_t, 1) << _PAGE_BIT_SPECIAL) #define _PAGE_CPA_TEST (_AT(pteval_t, 1) << _PAGE_BIT_CPA_TEST) #define _PAGE_SPLITTING (_AT(pteval_t, 1) << _PAGE_BIT_SPLITTING) +#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS +#define _PAGE_PKEY_BIT0 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0) +#define _PAGE_PKEY_BIT1 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1) +#define _PAGE_PKEY_BIT2 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT2) +#define _PAGE_PKEY_BIT3 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT3) +#else +#define _PAGE_PKEY_BIT0 (_AT(pteval_t, 0)) +#define _PAGE_PKEY_BIT1 (_AT(pteval_t, 0)) +#define _PAGE_PKEY_BIT2 (_AT(pteval_t, 0)) +#define _PAGE_PKEY_BIT3 (_AT(pteval_t, 0)) +#endif #define __HAVE_ARCH_PTE_SPECIAL #ifdef CONFIG_KMEMCHECK _ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 01/31] mm, gup: introduce concept of "foreign" get_user_pages() |
| Message-ID | <qO6jo-8sH-7@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
For protection keys, we need to understand whether protections
should be enforced in software or not. In general, we enforce
protections when working on our own task, but not when on others.
We call these "current" and "foreign" operations.
This introduces two new get_user_pages() variants:
get_current_user_pages()
get_foreign_user_pages()
get_current_user_pages() is a drop-in replacement for when
get_user_pages() was called with (current, current->mm, ...) as
arguments. Using it makes a few of the call sites look a bit
nicer.
get_foreign_user_pages() is a replacement for when
get_user_pages() is called on non-current tsk/mm.
We leave a stub get_user_pages() around with a __deprecated
warning.
This also effectively turns get_user_pages_unlocked() in to
get_user_pages_unlocked_current() since it no longer gets a
tsk/mm passed in. I thought that would be too long of a name if
we added "_current" on there. BTW, if someone wants the
get_user_pages_unlocked() behavior with a non-current tsk/mm,
they just have to use __get_user_pages_unlocked() directly.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: vbabka@suse.cz
---
b/arch/mips/mm/gup.c | 3 -
b/arch/s390/mm/gup.c | 4 --
b/arch/sh/mm/gup.c | 2 -
b/arch/sparc/mm/gup.c | 2 -
b/arch/x86/mm/gup.c | 2 -
b/arch/x86/mm/mpx.c | 4 +-
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 4 +-
b/drivers/gpu/drm/i915/i915_gem_userptr.c | 2 -
b/drivers/gpu/drm/radeon/radeon_ttm.c | 4 +-
b/drivers/gpu/drm/via/via_dmablit.c | 3 -
b/drivers/infiniband/core/umem.c | 2 -
b/drivers/infiniband/core/umem_odp.c | 8 ++--
b/drivers/infiniband/hw/mthca/mthca_memfree.c | 3 -
b/drivers/infiniband/hw/qib/qib_user_pages.c | 3 -
b/drivers/infiniband/hw/usnic/usnic_uiom.c | 2 -
b/drivers/media/pci/ivtv/ivtv-udma.c | 4 +-
b/drivers/media/pci/ivtv/ivtv-yuv.c | 10 ++---
b/drivers/media/v4l2-core/videobuf-dma-sg.c | 3 -
b/drivers/misc/sgi-gru/grufault.c | 3 -
b/drivers/scsi/st.c | 2 -
b/drivers/video/fbdev/pvr2fb.c | 4 +-
b/drivers/virt/fsl_hypervisor.c | 5 +-
b/fs/exec.c | 8 +++-
b/include/linux/mm.h | 39 +++++++++++++------
b/kernel/events/uprobes.c | 4 +-
b/mm/frame_vector.c | 2 -
b/mm/gup.c | 51 ++++++++++++++++----------
b/mm/memory.c | 2 -
b/mm/mempolicy.c | 6 +--
b/mm/nommu.c | 34 ++++++++++-------
b/mm/process_vm_access.c | 6 ++-
b/mm/util.c | 4 --
b/net/ceph/pagevec.c | 2 -
b/security/tomoyo/domain.c | 9 ++++
b/virt/kvm/async_pf.c | 2 -
b/virt/kvm/kvm_main.c | 13 +++---
36 files changed, 147 insertions(+), 114 deletions(-)
diff -puN arch/mips/mm/gup.c~get_current_user_pages arch/mips/mm/gup.c
--- a/arch/mips/mm/gup.c~get_current_user_pages 2016-01-06 15:50:02.181001390 -0800
+++ b/arch/mips/mm/gup.c 2016-01-06 15:50:02.243004185 -0800
@@ -301,8 +301,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- (end - start) >> PAGE_SHIFT,
+ ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT,
write, 0, pages);
/* Have to be a bit careful with return values */
diff -puN arch/s390/mm/gup.c~get_current_user_pages arch/s390/mm/gup.c
--- a/arch/s390/mm/gup.c~get_current_user_pages 2016-01-06 15:50:02.183001480 -0800
+++ b/arch/s390/mm/gup.c 2016-01-06 15:50:02.243004185 -0800
@@ -230,7 +230,6 @@ int __get_user_pages_fast(unsigned long
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
- struct mm_struct *mm = current->mm;
int nr, ret;
start &= PAGE_MASK;
@@ -241,8 +240,7 @@ int get_user_pages_fast(unsigned long st
/* Try to get the remaining pages with get_user_pages */
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- nr_pages - nr, write, 0, pages);
+ ret = get_user_pages_unlocked(start, nr_pages - nr, write, 0, pages);
/* Have to be a bit careful with return values */
if (nr > 0)
ret = (ret < 0) ? nr : ret + nr;
diff -puN arch/sh/mm/gup.c~get_current_user_pages arch/sh/mm/gup.c
--- a/arch/sh/mm/gup.c~get_current_user_pages 2016-01-06 15:50:02.185001570 -0800
+++ b/arch/sh/mm/gup.c 2016-01-06 15:50:02.243004185 -0800
@@ -257,7 +257,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT, write, 0, pages);
/* Have to be a bit careful with return values */
diff -puN arch/sparc/mm/gup.c~get_current_user_pages arch/sparc/mm/gup.c
--- a/arch/sparc/mm/gup.c~get_current_user_pages 2016-01-06 15:50:02.186001615 -0800
+++ b/arch/sparc/mm/gup.c 2016-01-06 15:50:02.244004230 -0800
@@ -249,7 +249,7 @@ slow:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT, write, 0, pages);
/* Have to be a bit careful with return values */
diff -puN arch/x86/mm/gup.c~get_current_user_pages arch/x86/mm/gup.c
--- a/arch/x86/mm/gup.c~get_current_user_pages 2016-01-06 15:50:02.188001705 -0800
+++ b/arch/x86/mm/gup.c 2016-01-06 15:50:02.244004230 -0800
@@ -386,7 +386,7 @@ slow_irqon:
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
+ ret = get_user_pages_unlocked(start,
(end - start) >> PAGE_SHIFT,
write, 0, pages);
diff -puN arch/x86/mm/mpx.c~get_current_user_pages arch/x86/mm/mpx.c
--- a/arch/x86/mm/mpx.c~get_current_user_pages 2016-01-06 15:50:02.189001750 -0800
+++ b/arch/x86/mm/mpx.c 2016-01-06 15:50:02.245004275 -0800
@@ -546,8 +546,8 @@ static int mpx_resolve_fault(long __user
int nr_pages = 1;
int force = 0;
- gup_ret = get_user_pages(current, current->mm, (unsigned long)addr,
- nr_pages, write, force, NULL, NULL);
+ gup_ret = get_current_user_pages((unsigned long)addr, nr_pages, write,
+ force, NULL, NULL);
/*
* get_user_pages() returns number of pages gotten.
* 0 means we failed to fault in and get anything,
diff -puN drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c~get_current_user_pages drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c~get_current_user_pages 2016-01-06 15:50:02.191001840 -0800
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 2016-01-06 15:50:02.245004275 -0800
@@ -518,8 +518,8 @@ static int amdgpu_ttm_tt_pin_userptr(str
uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE;
struct page **pages = ttm->pages + pinned;
- r = get_user_pages(current, current->mm, userptr, num_pages,
- write, 0, pages, NULL);
+ r = get_current_user_pages(userptr, num_pages, write, 0, pages,
+ NULL);
if (r < 0)
goto release_pages;
diff -puN drivers/gpu/drm/i915/i915_gem_userptr.c~get_current_user_pages drivers/gpu/drm/i915/i915_gem_userptr.c
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c~get_current_user_pages 2016-01-06 15:50:02.193001931 -0800
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c 2016-01-06 15:50:02.246004320 -0800
@@ -587,7 +587,7 @@ __i915_gem_userptr_get_pages_worker(stru
down_read(&mm->mmap_sem);
while (pinned < num_pages) {
- ret = get_user_pages(work->task, mm,
+ ret = get_foreign_user_pages(work->task, mm,
obj->userptr.ptr + pinned * PAGE_SIZE,
num_pages - pinned,
!obj->userptr.read_only, 0,
diff -puN drivers/gpu/drm/radeon/radeon_ttm.c~get_current_user_pages drivers/gpu/drm/radeon/radeon_ttm.c
--- a/drivers/gpu/drm/radeon/radeon_ttm.c~get_current_user_pages 2016-01-06 15:50:02.194001976 -0800
+++ b/drivers/gpu/drm/radeon/radeon_ttm.c 2016-01-06 15:50:02.246004320 -0800
@@ -554,8 +554,8 @@ static int radeon_ttm_tt_pin_userptr(str
uint64_t userptr = gtt->userptr + pinned * PAGE_SIZE;
struct page **pages = ttm->pages + pinned;
- r = get_user_pages(current, current->mm, userptr, num_pages,
- write, 0, pages, NULL);
+ r = get_current_user_pages(userptr, num_pages, write, 0, pages,
+ NULL);
if (r < 0)
goto release_pages;
diff -puN drivers/gpu/drm/via/via_dmablit.c~get_current_user_pages drivers/gpu/drm/via/via_dmablit.c
--- a/drivers/gpu/drm/via/via_dmablit.c~get_current_user_pages 2016-01-06 15:50:02.196002066 -0800
+++ b/drivers/gpu/drm/via/via_dmablit.c 2016-01-06 15:50:02.247004365 -0800
@@ -239,8 +239,7 @@ via_lock_all_dma_pages(drm_via_sg_info_t
if (NULL == vsg->pages)
return -ENOMEM;
down_read(¤t->mm->mmap_sem);
- ret = get_user_pages(current, current->mm,
- (unsigned long)xfer->mem_addr,
+ ret = get_current_user_pages((unsigned long)xfer->mem_addr,
vsg->num_pages,
(vsg->direction == DMA_FROM_DEVICE),
0, vsg->pages, NULL);
diff -puN drivers/infiniband/core/umem.c~get_current_user_pages drivers/infiniband/core/umem.c
--- a/drivers/infiniband/core/umem.c~get_current_user_pages 2016-01-06 15:50:02.198002156 -0800
+++ b/drivers/infiniband/core/umem.c 2016-01-06 15:50:02.247004365 -0800
@@ -188,7 +188,7 @@ struct ib_umem *ib_umem_get(struct ib_uc
sg_list_start = umem->sg_head.sgl;
while (npages) {
- ret = get_user_pages(current, current->mm, cur_base,
+ ret = get_current_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof (struct page *)),
1, !umem->writable, page_list, vma_list);
diff -puN drivers/infiniband/core/umem_odp.c~get_current_user_pages drivers/infiniband/core/umem_odp.c
--- a/drivers/infiniband/core/umem_odp.c~get_current_user_pages 2016-01-06 15:50:02.199002201 -0800
+++ b/drivers/infiniband/core/umem_odp.c 2016-01-06 15:50:02.247004365 -0800
@@ -572,10 +572,10 @@ int ib_umem_odp_map_dma_pages(struct ib_
* complex (and doesn't gain us much performance in most use
* cases).
*/
- npages = get_user_pages(owning_process, owning_mm, user_virt,
- gup_num_pages,
- access_mask & ODP_WRITE_ALLOWED_BIT, 0,
- local_page_list, NULL);
+ npages = get_foreign_user_pages(owning_process, owning_mm,
+ user_virt, gup_num_pages,
+ access_mask & ODP_WRITE_ALLOWED_BIT,
+ 0, local_page_list, NULL);
up_read(&owning_mm->mmap_sem);
if (npages < 0)
diff -puN drivers/infiniband/hw/mthca/mthca_memfree.c~get_current_user_pages drivers/infiniband/hw/mthca/mthca_memfree.c
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c~get_current_user_pages 2016-01-06 15:50:02.201002291 -0800
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c 2016-01-06 15:50:02.248004410 -0800
@@ -472,8 +472,7 @@ int mthca_map_user_db(struct mthca_dev *
goto out;
}
- ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- pages, NULL);
+ ret = get_current_user_pages(uaddr & PAGE_MASK, 1, 1, 0, pages, NULL);
if (ret < 0)
goto out;
diff -puN drivers/infiniband/hw/qib/qib_user_pages.c~get_current_user_pages drivers/infiniband/hw/qib/qib_user_pages.c
--- a/drivers/infiniband/hw/qib/qib_user_pages.c~get_current_user_pages 2016-01-06 15:50:02.203002381 -0800
+++ b/drivers/infiniband/hw/qib/qib_user_pages.c 2016-01-06 15:50:02.248004410 -0800
@@ -66,8 +66,7 @@ static int __qib_get_user_pages(unsigned
}
for (got = 0; got < num_pages; got += ret) {
- ret = get_user_pages(current, current->mm,
- start_page + got * PAGE_SIZE,
+ ret = get_current_user_pages(start_page + got * PAGE_SIZE,
num_pages - got, 1, 1,
p + got, NULL);
if (ret < 0)
diff -puN drivers/infiniband/hw/usnic/usnic_uiom.c~get_current_user_pages drivers/infiniband/hw/usnic/usnic_uiom.c
--- a/drivers/infiniband/hw/usnic/usnic_uiom.c~get_current_user_pages 2016-01-06 15:50:02.204002427 -0800
+++ b/drivers/infiniband/hw/usnic/usnic_uiom.c 2016-01-06 15:50:02.248004410 -0800
@@ -144,7 +144,7 @@ static int usnic_uiom_get_pages(unsigned
ret = 0;
while (npages) {
- ret = get_user_pages(current, current->mm, cur_base,
+ ret = get_current_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof(struct page *)),
1, !writable, page_list, NULL);
diff -puN drivers/media/pci/ivtv/ivtv-udma.c~get_current_user_pages drivers/media/pci/ivtv/ivtv-udma.c
--- a/drivers/media/pci/ivtv/ivtv-udma.c~get_current_user_pages 2016-01-06 15:50:02.206002517 -0800
+++ b/drivers/media/pci/ivtv/ivtv-udma.c 2016-01-06 15:50:02.249004455 -0800
@@ -124,8 +124,8 @@ int ivtv_udma_setup(struct ivtv *itv, un
}
/* Get user pages for DMA Xfer */
- err = get_user_pages_unlocked(current, current->mm,
- user_dma.uaddr, user_dma.page_count, 0, 1, dma->map);
+ err = get_user_pages_unlocked(user_dma.uaddr, user_dma.page_count, 0,
+ 1, dma->map);
if (user_dma.page_count != err) {
IVTV_DEBUG_WARN("failed to map user pages, returned %d instead of %d\n",
diff -puN drivers/media/pci/ivtv/ivtv-yuv.c~get_current_user_pages drivers/media/pci/ivtv/ivtv-yuv.c
--- a/drivers/media/pci/ivtv/ivtv-yuv.c~get_current_user_pages 2016-01-06 15:50:02.208002607 -0800
+++ b/drivers/media/pci/ivtv/ivtv-yuv.c 2016-01-06 15:50:02.249004455 -0800
@@ -75,14 +75,12 @@ static int ivtv_yuv_prep_user_dma(struct
ivtv_udma_get_page_info (&uv_dma, (unsigned long)args->uv_source, 360 * uv_decode_height);
/* Get user pages for DMA Xfer */
- y_pages = get_user_pages_unlocked(current, current->mm,
- y_dma.uaddr, y_dma.page_count, 0, 1,
- &dma->map[0]);
+ y_pages = get_user_pages_unlocked(y_dma.uaddr,
+ y_dma.page_count, 0, 1, &dma->map[0]);
uv_pages = 0; /* silence gcc. value is set and consumed only if: */
if (y_pages == y_dma.page_count) {
- uv_pages = get_user_pages_unlocked(current, current->mm,
- uv_dma.uaddr, uv_dma.page_count, 0, 1,
- &dma->map[y_pages]);
+ uv_pages = get_user_pages_unlocked(uv_dma.uaddr,
+ uv_dma.page_count, 0, 1, &dma->map[y_pages]);
}
if (y_pages != y_dma.page_count || uv_pages != uv_dma.page_count) {
diff -puN drivers/media/v4l2-core/videobuf-dma-sg.c~get_current_user_pages drivers/media/v4l2-core/videobuf-dma-sg.c
--- a/drivers/media/v4l2-core/videobuf-dma-sg.c~get_current_user_pages 2016-01-06 15:50:02.209002652 -0800
+++ b/drivers/media/v4l2-core/videobuf-dma-sg.c 2016-01-06 15:50:02.250004500 -0800
@@ -181,8 +181,7 @@ static int videobuf_dma_init_user_locked
dprintk(1, "init user [0x%lx+0x%lx => %d pages]\n",
data, size, dma->nr_pages);
- err = get_user_pages(current, current->mm,
- data & PAGE_MASK, dma->nr_pages,
+ err = get_current_user_pages(data & PAGE_MASK, dma->nr_pages,
rw == READ, 1, /* force */
dma->pages, NULL);
diff -puN drivers/misc/sgi-gru/grufault.c~get_current_user_pages drivers/misc/sgi-gru/grufault.c
--- a/drivers/misc/sgi-gru/grufault.c~get_current_user_pages 2016-01-06 15:50:02.211002742 -0800
+++ b/drivers/misc/sgi-gru/grufault.c 2016-01-06 15:50:02.250004500 -0800
@@ -198,8 +198,7 @@ static int non_atomic_pte_lookup(struct
#else
*pageshift = PAGE_SHIFT;
#endif
- if (get_user_pages
- (current, current->mm, vaddr, 1, write, 0, &page, NULL) <= 0)
+ if (get_current_user_pages(vaddr, 1, write, 0, &page, NULL) <= 0)
return -EFAULT;
*paddr = page_to_phys(page);
put_page(page);
diff -puN drivers/scsi/st.c~get_current_user_pages drivers/scsi/st.c
--- a/drivers/scsi/st.c~get_current_user_pages 2016-01-06 15:50:02.213002832 -0800
+++ b/drivers/scsi/st.c 2016-01-06 15:50:02.252004591 -0800
@@ -4786,8 +4786,6 @@ static int sgl_map_user_pages(struct st_
/* Try to fault in all of the necessary pages */
/* rw==READ means read from drive, write into memory area */
res = get_user_pages_unlocked(
- current,
- current->mm,
uaddr,
nr_pages,
rw == READ,
diff -puN drivers/video/fbdev/pvr2fb.c~get_current_user_pages drivers/video/fbdev/pvr2fb.c
--- a/drivers/video/fbdev/pvr2fb.c~get_current_user_pages 2016-01-06 15:50:02.214002877 -0800
+++ b/drivers/video/fbdev/pvr2fb.c 2016-01-06 15:50:02.252004591 -0800
@@ -686,8 +686,8 @@ static ssize_t pvr2fb_write(struct fb_in
if (!pages)
return -ENOMEM;
- ret = get_user_pages_unlocked(current, current->mm, (unsigned long)buf,
- nr_pages, WRITE, 0, pages);
+ ret = get_user_pages_unlocked((unsigned long)buf, nr_pages, WRITE,
+ 0, pages);
if (ret < nr_pages) {
nr_pages = ret;
diff -puN drivers/virt/fsl_hypervisor.c~get_current_user_pages drivers/virt/fsl_hypervisor.c
--- a/drivers/virt/fsl_hypervisor.c~get_current_user_pages 2016-01-06 15:50:02.216002967 -0800
+++ b/drivers/virt/fsl_hypervisor.c 2016-01-06 15:50:02.253004636 -0800
@@ -244,9 +244,8 @@ static long ioctl_memcpy(struct fsl_hv_i
/* Get the physical addresses of the source buffer */
down_read(¤t->mm->mmap_sem);
- num_pinned = get_user_pages(current, current->mm,
- param.local_vaddr - lb_offset, num_pages,
- (param.source == -1) ? READ : WRITE,
+ num_pinned = get_current_user_pages(param.local_vaddr - lb_offset,
+ num_pages, (param.source == -1) ? READ : WRITE,
0, pages, NULL);
up_read(¤t->mm->mmap_sem);
diff -puN fs/exec.c~get_current_user_pages fs/exec.c
--- a/fs/exec.c~get_current_user_pages 2016-01-06 15:50:02.218003058 -0800
+++ b/fs/exec.c 2016-01-06 15:50:02.253004636 -0800
@@ -198,8 +198,12 @@ static struct page *get_arg_page(struct
return NULL;
}
#endif
- ret = get_user_pages(current, bprm->mm, pos,
- 1, write, 1, &page, NULL);
+ /*
+ * We are doing an exec(). 'current' is the process
+ * doing the exec and bprm->mm is the new process's mm.
+ */
+ ret = get_foreign_user_pages(current, bprm->mm, pos, 1, write,
+ 1, &page, NULL);
if (ret <= 0)
return NULL;
diff -puN include/linux/mm.h~get_current_user_pages include/linux/mm.h
--- a/include/linux/mm.h~get_current_user_pages 2016-01-06 15:50:02.219003103 -0800
+++ b/include/linux/mm.h 2016-01-06 15:50:02.254004681 -0800
@@ -1191,24 +1191,39 @@ long __get_user_pages(struct task_struct
unsigned long start, unsigned long nr_pages,
unsigned int foll_flags, struct page **pages,
struct vm_area_struct **vmas, int *nonblocking);
-long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
- int write, int force, struct page **pages,
- struct vm_area_struct **vmas);
-long get_user_pages_locked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
- int write, int force, struct page **pages,
- int *locked);
-long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_foreign_user_pages(struct task_struct *tsk, struct mm_struct *mm,
+ unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages,
+ struct vm_area_struct **vmas);
+long get_current_user_pages(unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages,
+ struct vm_area_struct **vmas);
+long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages, int *locked);
+long __get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
unsigned int gup_flags);
-long get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages);
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages);
+/*
+ * All callers should use get_foreign_user_pages() or
+ * get_current_user_pages(). The foreign variant is the most
+ * permissive and is the least likely to break something in
+ * a negative way.
+ */
+static inline __deprecated
+long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
+ unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages,
+ struct vm_area_struct **vmas)
+{
+ return get_foreign_user_pages(tsk, mm, start, nr_pages, write, force,
+ pages, vmas);
+}
+
/* Container for pinned pfns / pages */
struct frame_vector {
unsigned int nr_allocated; /* Number of frames we have space for */
diff -puN kernel/events/uprobes.c~get_current_user_pages kernel/events/uprobes.c
--- a/kernel/events/uprobes.c~get_current_user_pages 2016-01-06 15:50:02.221003193 -0800
+++ b/kernel/events/uprobes.c 2016-01-06 15:50:02.255004726 -0800
@@ -298,7 +298,7 @@ int uprobe_write_opcode(struct mm_struct
retry:
/* Read the page with vaddr into memory */
- ret = get_user_pages(NULL, mm, vaddr, 1, 0, 1, &old_page, &vma);
+ ret = get_foreign_user_pages(NULL, mm, vaddr, 1, 0, 1, &old_page, &vma);
if (ret <= 0)
return ret;
@@ -1699,7 +1699,7 @@ static int is_trap_at_addr(struct mm_str
if (likely(result == 0))
goto out;
- result = get_user_pages(NULL, mm, vaddr, 1, 0, 1, &page, NULL);
+ result = get_current_user_pages(vaddr, 1, 0, 1, &page, NULL);
if (result < 0)
return result;
diff -puN mm/frame_vector.c~get_current_user_pages mm/frame_vector.c
--- a/mm/frame_vector.c~get_current_user_pages 2016-01-06 15:50:02.223003283 -0800
+++ b/mm/frame_vector.c 2016-01-06 15:50:02.255004726 -0800
@@ -58,7 +58,7 @@ int get_vaddr_frames(unsigned long start
if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) {
vec->got_ref = true;
vec->is_pfns = false;
- ret = get_user_pages_locked(current, mm, start, nr_frames,
+ ret = get_user_pages_locked(start, nr_frames,
write, force, (struct page **)(vec->ptrs), &locked);
goto out;
}
diff -puN mm/gup.c~get_current_user_pages mm/gup.c
--- a/mm/gup.c~get_current_user_pages 2016-01-06 15:50:02.224003328 -0800
+++ b/mm/gup.c 2016-01-06 15:50:02.256004771 -0800
@@ -735,13 +735,13 @@ static __always_inline long __get_user_p
* if (locked)
* up_read(&mm->mmap_sem);
*/
-long get_user_pages_locked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
int *locked)
{
- return __get_user_pages_locked(tsk, mm, start, nr_pages, write, force,
- pages, NULL, locked, true, FOLL_TOUCH);
+ return __get_user_pages_locked(current, current->mm, start, nr_pages,
+ write, force, pages, NULL, locked, true,
+ FOLL_TOUCH);
}
EXPORT_SYMBOL(get_user_pages_locked);
@@ -755,11 +755,12 @@ EXPORT_SYMBOL(get_user_pages_locked);
* according to the parameters "pages", "write", "force"
* respectively.
*/
-__always_inline long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+__always_inline long __get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
unsigned int gup_flags)
{
+ struct task_struct *tsk = current;
+ struct mm_struct *mm = tsk->mm;
long ret;
int locked = 1;
down_read(&mm->mmap_sem);
@@ -788,17 +789,16 @@ EXPORT_SYMBOL(__get_user_pages_unlocked)
* or if "force" shall be set to 1 (get_user_pages_fast misses the
* "force" parameter).
*/
-long get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages)
{
- return __get_user_pages_unlocked(tsk, mm, start, nr_pages, write,
+ return __get_user_pages_unlocked(start, nr_pages, write,
force, pages, FOLL_TOUCH);
}
EXPORT_SYMBOL(get_user_pages_unlocked);
/*
- * get_user_pages() - pin user pages in memory
+ * get_foreign_user_pages() - pin user pages in memory
* @tsk: the task_struct to use for page fault accounting, or
* NULL if faults are not to be recorded.
* @mm: mm_struct of target mm
@@ -852,14 +852,30 @@ EXPORT_SYMBOL(get_user_pages_unlocked);
* should use get_user_pages because it cannot pass
* FAULT_FLAG_ALLOW_RETRY to handle_mm_fault.
*/
-long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages, int write,
- int force, struct page **pages, struct vm_area_struct **vmas)
+long get_foreign_user_pages(struct task_struct *tsk, struct mm_struct *mm,
+ unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages,
+ struct vm_area_struct **vmas)
{
return __get_user_pages_locked(tsk, mm, start, nr_pages, write, force,
- pages, vmas, NULL, false, FOLL_TOUCH);
+ pages, vmas, NULL, false, FOLL_TOUCH);
}
-EXPORT_SYMBOL(get_user_pages);
+EXPORT_SYMBOL(get_foreign_user_pages);
+
+/*
+ * This is exactly the same as get_foreign_user_pages(), just
+ * with a less-flexible calling convention where we assume that
+ * the task and mm being operated on are the current task's.
+ */
+long get_current_user_pages(unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages,
+ struct vm_area_struct **vmas)
+{
+ return get_foreign_user_pages(current, current->mm,
+ start, nr_pages, write, force,
+ pages, vmas);
+}
+EXPORT_SYMBOL(get_current_user_pages);
/**
* populate_vma_page_range() - populate a range of pages in the vma.
@@ -1395,7 +1411,6 @@ int __get_user_pages_fast(unsigned long
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
- struct mm_struct *mm = current->mm;
int nr, ret;
start &= PAGE_MASK;
@@ -1407,8 +1422,8 @@ int get_user_pages_fast(unsigned long st
start += nr << PAGE_SHIFT;
pages += nr;
- ret = get_user_pages_unlocked(current, mm, start,
- nr_pages - nr, write, 0, pages);
+ ret = get_user_pages_unlocked(start, nr_pages - nr, write, 0,
+ pages);
/* Have to be a bit careful with return values */
if (nr > 0) {
diff -puN mm/memory.c~get_current_user_pages mm/memory.c
--- a/mm/memory.c~get_current_user_pages 2016-01-06 15:50:02.226003418 -0800
+++ b/mm/memory.c 2016-01-06 15:50:02.257004816 -0800
@@ -3659,7 +3659,7 @@ static int __access_remote_vm(struct tas
void *maddr;
struct page *page = NULL;
- ret = get_user_pages(tsk, mm, addr, 1,
+ ret = get_foreign_user_pages(tsk, mm, addr, 1,
write, 1, &page, &vma);
if (ret <= 0) {
#ifndef CONFIG_HAVE_IOREMAP_PROT
diff -puN mm/mempolicy.c~get_current_user_pages mm/mempolicy.c
--- a/mm/mempolicy.c~get_current_user_pages 2016-01-06 15:50:02.228003508 -0800
+++ b/mm/mempolicy.c 2016-01-06 15:50:02.258004861 -0800
@@ -813,12 +813,12 @@ static void get_policy_nodemask(struct m
}
}
-static int lookup_node(struct mm_struct *mm, unsigned long addr)
+static int lookup_node(unsigned long addr)
{
struct page *p;
int err;
- err = get_user_pages(current, mm, addr & PAGE_MASK, 1, 0, 0, &p, NULL);
+ err = get_current_user_pages(addr & PAGE_MASK, 1, 0, 0, &p, NULL);
if (err >= 0) {
err = page_to_nid(p);
put_page(p);
@@ -873,7 +873,7 @@ static long do_get_mempolicy(int *policy
if (flags & MPOL_F_NODE) {
if (flags & MPOL_F_ADDR) {
- err = lookup_node(mm, addr);
+ err = lookup_node(addr);
if (err < 0)
goto out;
*policy = err;
diff -puN mm/nommu.c~get_current_user_pages mm/nommu.c
--- a/mm/nommu.c~get_current_user_pages 2016-01-06 15:50:02.230003599 -0800
+++ b/mm/nommu.c 2016-01-06 15:50:02.259004906 -0800
@@ -182,7 +182,7 @@ finish_or_fault:
* slab page or a secondary page from a compound page
* - don't permit access to VMAs that don't support it, such as I/O mappings
*/
-long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
+long get_foreign_user_pages(struct task_struct *tsk, struct mm_struct *mm,
unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
struct vm_area_struct **vmas)
@@ -199,35 +199,41 @@ long get_user_pages(struct task_struct *
}
EXPORT_SYMBOL(get_user_pages);
-long get_user_pages_locked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
int *locked)
{
- return get_user_pages(tsk, mm, start, nr_pages, write, force,
- pages, NULL);
+ return get_user_pages(current, current->mm, start, nr_pages, write,
+ force, pages, NULL);
}
EXPORT_SYMBOL(get_user_pages_locked);
-long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_current_user_pages(unsigned long start, unsigned long nr_pages,
+ int write, int force, struct page **pages,
+ struct vm_area_struct **vmas)
+{
+ return get_foreign_user_pages(current, current->mm, start, nr_pages,
+ write, force, pages, vmas);
+}
+EXPORT_SYMBOL(get_current_user_pages);
+
+long __get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages,
unsigned int gup_flags)
{
long ret;
- down_read(&mm->mmap_sem);
- ret = get_user_pages(tsk, mm, start, nr_pages, write, force,
- pages, NULL);
- up_read(&mm->mmap_sem);
+ down_read(¤t->mm->mmap_sem);
+ ret = get_current_user_pages(start, nr_pages, write, force,
+ pages, NULL);
+ up_read(¤t->mm->mmap_sem);
return ret;
}
EXPORT_SYMBOL(__get_user_pages_unlocked);
-long get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, unsigned long nr_pages,
+long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
int write, int force, struct page **pages)
{
- return __get_user_pages_unlocked(tsk, mm, start, nr_pages, write,
+ return __get_user_pages_unlocked(start, nr_pages, write,
force, pages, 0);
}
EXPORT_SYMBOL(get_user_pages_unlocked);
diff -puN mm/process_vm_access.c~get_current_user_pages mm/process_vm_access.c
--- a/mm/process_vm_access.c~get_current_user_pages 2016-01-06 15:50:02.231003644 -0800
+++ b/mm/process_vm_access.c 2016-01-06 15:50:02.259004906 -0800
@@ -99,8 +99,10 @@ static int process_vm_rw_single_vec(unsi
size_t bytes;
/* Get the pages we're interested in */
- pages = get_user_pages_unlocked(task, mm, pa, pages,
- vm_write, 0, process_pages);
+ down_read(&mm->mmap_sem);
+ pages = get_foreign_user_pages(task, mm, pa, pages, vm_write,
+ 0, process_pages, NULL);
+ up_read(&mm->mmap_sem);
if (pages <= 0)
return -EFAULT;
diff -puN mm/util.c~get_current_user_pages mm/util.c
--- a/mm/util.c~get_current_user_pages 2016-01-06 15:50:02.233003734 -0800
+++ b/mm/util.c 2016-01-06 15:50:02.259004906 -0800
@@ -277,9 +277,7 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast)
int __weak get_user_pages_fast(unsigned long start,
int nr_pages, int write, struct page **pages)
{
- struct mm_struct *mm = current->mm;
- return get_user_pages_unlocked(current, mm, start, nr_pages,
- write, 0, pages);
+ return get_user_pages_unlocked(start, nr_pages, write, 0, pages);
}
EXPORT_SYMBOL_GPL(get_user_pages_fast);
diff -puN net/ceph/pagevec.c~get_current_user_pages net/ceph/pagevec.c
--- a/net/ceph/pagevec.c~get_current_user_pages 2016-01-06 15:50:02.234003779 -0800
+++ b/net/ceph/pagevec.c 2016-01-06 15:50:02.260004951 -0800
@@ -24,7 +24,7 @@ struct page **ceph_get_direct_page_vecto
return ERR_PTR(-ENOMEM);
while (got < num_pages) {
- rc = get_user_pages_unlocked(current, current->mm,
+ rc = get_user_pages_unlocked(
(unsigned long)data + ((unsigned long)got * PAGE_SIZE),
num_pages - got, write_page, 0, pages + got);
if (rc < 0)
diff -puN security/tomoyo/domain.c~get_current_user_pages security/tomoyo/domain.c
--- a/security/tomoyo/domain.c~get_current_user_pages 2016-01-06 15:50:02.236003869 -0800
+++ b/security/tomoyo/domain.c 2016-01-06 15:50:02.260004951 -0800
@@ -874,7 +874,14 @@ bool tomoyo_dump_page(struct linux_binpr
}
/* Same with get_arg_page(bprm, pos, 0) in fs/exec.c */
#ifdef CONFIG_MMU
- if (get_user_pages(current, bprm->mm, pos, 1, 0, 1, &page, NULL) <= 0)
+ /*
+ * This is called at execve() time in order to dig around
+ * in the argv/environment of the new proceess
+ * (represented by bprm). 'current' is the process doing
+ * the execve().
+ */
+ if (get_foreign_user_pages(current, bprm->mm, pos, 1,
+ 0, 1, &page, NULL) <= 0)
return false;
#else
page = bprm->page[pos / PAGE_SIZE];
diff -puN virt/kvm/async_pf.c~get_current_user_pages virt/kvm/async_pf.c
--- a/virt/kvm/async_pf.c~get_current_user_pages 2016-01-06 15:50:02.238003960 -0800
+++ b/virt/kvm/async_pf.c 2016-01-06 15:50:02.260004951 -0800
@@ -80,7 +80,7 @@ static void async_pf_execute(struct work
might_sleep();
- get_user_pages_unlocked(NULL, mm, addr, 1, 1, 0, NULL);
+ get_user_pages_unlocked(addr, 1, 1, 0, NULL);
kvm_async_page_present_sync(vcpu, apf);
spin_lock(&vcpu->async_pf.lock);
diff -puN virt/kvm/kvm_main.c~get_current_user_pages virt/kvm/kvm_main.c
--- a/virt/kvm/kvm_main.c~get_current_user_pages 2016-01-06 15:50:02.239004004 -0800
+++ b/virt/kvm/kvm_main.c 2016-01-06 15:50:02.262005041 -0800
@@ -1274,15 +1274,16 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(s
return gfn_to_hva_memslot_prot(slot, gfn, writable);
}
-static int get_user_page_nowait(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long start, int write, struct page **page)
+static int get_user_page_nowait(unsigned long start, int write,
+ struct page **page)
{
int flags = FOLL_TOUCH | FOLL_NOWAIT | FOLL_HWPOISON | FOLL_GET;
if (write)
flags |= FOLL_WRITE;
- return __get_user_pages(tsk, mm, start, 1, flags, page, NULL, NULL);
+ return __get_user_pages(current, current->mm, start, 1, flags, page,
+ NULL, NULL);
}
static inline int check_user_page_hwpoison(unsigned long addr)
@@ -1344,12 +1345,10 @@ static int hva_to_pfn_slow(unsigned long
if (async) {
down_read(¤t->mm->mmap_sem);
- npages = get_user_page_nowait(current, current->mm,
- addr, write_fault, page);
+ npages = get_user_page_nowait(addr, write_fault, page);
up_read(¤t->mm->mmap_sem);
} else
- npages = __get_user_pages_unlocked(current, current->mm, addr, 1,
- write_fault, 0, page,
+ npages = __get_user_pages_unlocked(addr, 1, write_fault, 0, page,
FOLL_TOUCH|FOLL_HWPOISON);
if (npages != 1)
return npages;
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Vlastimil Babka <vbabka@suse.cz> |
|---|---|
| Date | 2016-01-13 20:10 +0100 |
| Subject | Re: [PATCH 01/31] mm, gup: introduce concept of "foreign" get_user_pages() |
| Message-ID | <qQyXU-41l-13@gated-at.bofh.it> |
| In reply to | #1303170 |
On 01/07/2016 01:01 AM, Dave Hansen wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> For protection keys, we need to understand whether protections
> should be enforced in software or not. In general, we enforce
> protections when working on our own task, but not when on others.
> We call these "current" and "foreign" operations.
>
> This introduces two new get_user_pages() variants:
>
> get_current_user_pages()
> get_foreign_user_pages()
>
> get_current_user_pages() is a drop-in replacement for when
> get_user_pages() was called with (current, current->mm, ...) as
> arguments. Using it makes a few of the call sites look a bit
> nicer.
>
> get_foreign_user_pages() is a replacement for when
> get_user_pages() is called on non-current tsk/mm.
>
> We leave a stub get_user_pages() around with a __deprecated
> warning.
Hm when replying to previous version I assumed this is because there are many
get_user_pages() callers remaining. But now I see there are just 3 drivers not
converted by this patch? In that case I would favor to convert get_user_pages()
to become what is now get_current_user_pages(). This would be much more
consistent IMHO. We don't need to cater to out-of-tree modules?
Sorry, I should have looked thoroughly on the previous reply, not just assume.
> This also effectively turns get_user_pages_unlocked() in to
> get_user_pages_unlocked_current() since it no longer gets a
> tsk/mm passed in. I thought that would be too long of a name if
> we added "_current" on there. BTW, if someone wants the
> get_user_pages_unlocked() behavior with a non-current tsk/mm,
> they just have to use __get_user_pages_unlocked() directly.
>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Andrea Arcangeli <aarcange@redhat.com>
> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Cc: vbabka@suse.cz
> ---
Also (but moot if you accept my suggestion):
> diff -puN mm/nommu.c~get_current_user_pages mm/nommu.c
> --- a/mm/nommu.c~get_current_user_pages 2016-01-06 15:50:02.230003599 -0800
> +++ b/mm/nommu.c 2016-01-06 15:50:02.259004906 -0800
> @@ -182,7 +182,7 @@ finish_or_fault:
> * slab page or a secondary page from a compound page
> * - don't permit access to VMAs that don't support it, such as I/O mappings
> */
> -long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
> +long get_foreign_user_pages(struct task_struct *tsk, struct mm_struct *mm,
> unsigned long start, unsigned long nr_pages,
> int write, int force, struct page **pages,
> struct vm_area_struct **vmas)
> @@ -199,35 +199,41 @@ long get_user_pages(struct task_struct *
> }
> EXPORT_SYMBOL(get_user_pages);
I think you need to change the export here as you did in gup.c
>
> -long get_user_pages_locked(struct task_struct *tsk, struct mm_struct *mm,
> - unsigned long start, unsigned long nr_pages,
> +long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
> int write, int force, struct page **pages,
> int *locked)
> {
> - return get_user_pages(tsk, mm, start, nr_pages, write, force,
> - pages, NULL);
> + return get_user_pages(current, current->mm, start, nr_pages, write,
> + force, pages, NULL);
Why not use the _current variant here?
> }
> EXPORT_SYMBOL(get_user_pages_locked);
>
> -long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
> - unsigned long start, unsigned long nr_pages,
> +long get_current_user_pages(unsigned long start, unsigned long nr_pages,
> + int write, int force, struct page **pages,
> + struct vm_area_struct **vmas)
> +{
> + return get_foreign_user_pages(current, current->mm, start, nr_pages,
> + write, force, pages, vmas);
> +}
> +EXPORT_SYMBOL(get_current_user_pages);
> +
> +long __get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
> int write, int force, struct page **pages,
> unsigned int gup_flags)
> {
> long ret;
> - down_read(&mm->mmap_sem);
> - ret = get_user_pages(tsk, mm, start, nr_pages, write, force,
> - pages, NULL);
> - up_read(&mm->mmap_sem);
> + down_read(¤t->mm->mmap_sem);
> + ret = get_current_user_pages(start, nr_pages, write, force,
> + pages, NULL);
> + up_read(¤t->mm->mmap_sem);
> return ret;
> }
> EXPORT_SYMBOL(__get_user_pages_unlocked);
>
> -long get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
> - unsigned long start, unsigned long nr_pages,
> +long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
> int write, int force, struct page **pages)
> {
> - return __get_user_pages_unlocked(tsk, mm, start, nr_pages, write,
> + return __get_user_pages_unlocked(start, nr_pages, write,
> force, pages, 0);
> }
> EXPORT_SYMBOL(get_user_pages_unlocked);
> diff -puN mm/process_vm_access.c~get_current_user_pages mm/process_vm_access.c
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-13 20:20 +0100 |
| Subject | Re: [PATCH 01/31] mm, gup: introduce concept of "foreign" get_user_pages() |
| Message-ID | <qQz7A-44Y-15@gated-at.bofh.it> |
| In reply to | #1308738 |
On 01/13/2016 11:00 AM, Vlastimil Babka wrote: >> > We leave a stub get_user_pages() around with a __deprecated >> > warning. > Hm when replying to previous version I assumed this is because there are many > get_user_pages() callers remaining. But now I see there are just 3 drivers not > converted by this patch? In that case I would favor to convert get_user_pages() > to become what is now get_current_user_pages(). This would be much more > consistent IMHO. We don't need to cater to out-of-tree modules? > > Sorry, I should have looked thoroughly on the previous reply, not just assume. It's really hard to submit a set of patches that remove a well-known API. New (in-tree) callers are always popping up, and you can see that a few have popped up since I updated this the last time. Without leaving the old stub around, it virtually guarantees that this patch will cause breakage in -next for a release or two. I'll fix up the other bits you commented on, btw!
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 17/31] x86, pkeys: check VMAs and PTEs for protection keys |
| Message-ID | <qO6jo-8sH-11@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
Today, for normal faults and page table walks, we check the VMA
and/or PTE to ensure that it is compatible with the action. For
instance, if we get a write fault on a non-writeable VMA, we
SIGSEGV.
We try to do the same thing for protection keys. Basically, we
try to make sure that if a user does this:
mprotect(ptr, size, PROT_NONE);
*ptr = foo;
they see the same effects with protection keys when they do this:
mprotect(ptr, size, PROT_READ|PROT_WRITE);
set_pkey(ptr, size, 4);
wrpkru(0xffffff3f); // access disable pkey 4
*ptr = foo;
The state to do that checking is in the VMA, but we also
sometimes have to do it on the page tables only, like when doing
a get_user_pages_fast() where we have no VMA.
We add two functions and expose them to generic code:
arch_pte_access_permitted(pte_flags, write)
arch_vma_access_permitted(vma, write)
These are, of course, backed up in x86 arch code with checks
against the PTE or VMA's protection key.
But, there are also cases where we do not want to respect
protection keys. When we ptrace(), for instance, we do not want
to apply the tracer's PKRU permissions to the PTEs from the
process being traced.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/powerpc/include/asm/mmu_context.h | 11 ++++++
b/arch/s390/include/asm/mmu_context.h | 11 ++++++
b/arch/unicore32/include/asm/mmu_context.h | 11 ++++++
b/arch/x86/include/asm/mmu_context.h | 49 +++++++++++++++++++++++++++++
b/arch/x86/include/asm/pgtable.h | 29 +++++++++++++++++
b/arch/x86/mm/fault.c | 21 +++++++++++-
b/arch/x86/mm/gup.c | 5 ++
b/include/asm-generic/mm_hooks.h | 11 ++++++
b/mm/gup.c | 18 ++++++++--
b/mm/memory.c | 4 ++
10 files changed, 166 insertions(+), 4 deletions(-)
diff -puN arch/powerpc/include/asm/mmu_context.h~pkeys-13-pte-fault arch/powerpc/include/asm/mmu_context.h
--- a/arch/powerpc/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-01-06 15:50:09.964352292 -0800
+++ b/arch/powerpc/include/asm/mmu_context.h 2016-01-06 15:50:09.981353058 -0800
@@ -148,5 +148,16 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* __KERNEL__ */
#endif /* __ASM_POWERPC_MMU_CONTEXT_H */
diff -puN arch/s390/include/asm/mmu_context.h~pkeys-13-pte-fault arch/s390/include/asm/mmu_context.h
--- a/arch/s390/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-01-06 15:50:09.965352337 -0800
+++ b/arch/s390/include/asm/mmu_context.h 2016-01-06 15:50:09.981353058 -0800
@@ -130,4 +130,15 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* __S390_MMU_CONTEXT_H */
diff -puN arch/unicore32/include/asm/mmu_context.h~pkeys-13-pte-fault arch/unicore32/include/asm/mmu_context.h
--- a/arch/unicore32/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-01-06 15:50:09.967352427 -0800
+++ b/arch/unicore32/include/asm/mmu_context.h 2016-01-06 15:50:09.981353058 -0800
@@ -97,4 +97,15 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif
diff -puN arch/x86/include/asm/mmu_context.h~pkeys-13-pte-fault arch/x86/include/asm/mmu_context.h
--- a/arch/x86/include/asm/mmu_context.h~pkeys-13-pte-fault 2016-01-06 15:50:09.968352472 -0800
+++ b/arch/x86/include/asm/mmu_context.h 2016-01-06 15:50:09.982353103 -0800
@@ -254,4 +254,53 @@ static inline int vma_pkey(struct vm_are
return pkey;
}
+static inline bool __pkru_allows_pkey(u16 pkey, bool write)
+{
+ u32 pkru = read_pkru();
+
+ if (!__pkru_allows_read(pkru, pkey))
+ return false;
+ if (write && !__pkru_allows_write(pkru, pkey))
+ return false;
+
+ return true;
+}
+
+/*
+ * We only want to enforce protection keys on the current process
+ * because we effectively have no access to PKRU for other
+ * processes or any way to tell *which * PKRU in a threaded
+ * process we could use.
+ *
+ * So do not enforce things if the VMA is not from the current
+ * mm, or if we are in a kernel thread.
+ */
+static inline bool vma_is_foreign(struct vm_area_struct *vma)
+{
+ if (!current->mm)
+ return true;
+ /*
+ * Should PKRU be enforced on the access to this VMA? If
+ * the VMA is from another process, then PKRU has no
+ * relevance and should not be enforced.
+ */
+ if (current->mm != vma->vm_mm)
+ return true;
+
+ return false;
+}
+
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* allow access if the VMA is not one from this process */
+ if (vma_is_foreign(vma))
+ return true;
+ return __pkru_allows_pkey(vma_pkey(vma), write);
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ return __pkru_allows_pkey(pte_flags_pkey(pte_flags(pte)), write);
+}
+
#endif /* _ASM_X86_MMU_CONTEXT_H */
diff -puN arch/x86/include/asm/pgtable.h~pkeys-13-pte-fault arch/x86/include/asm/pgtable.h
--- a/arch/x86/include/asm/pgtable.h~pkeys-13-pte-fault 2016-01-06 15:50:09.970352562 -0800
+++ b/arch/x86/include/asm/pgtable.h 2016-01-06 15:50:09.982353103 -0800
@@ -910,6 +910,35 @@ static inline pte_t pte_swp_clear_soft_d
}
#endif
+#define PKRU_AD_BIT 0x1
+#define PKRU_WD_BIT 0x2
+
+static inline bool __pkru_allows_read(u32 pkru, u16 pkey)
+{
+ int pkru_pkey_bits = pkey * 2;
+ return !(pkru & (PKRU_AD_BIT << pkru_pkey_bits));
+}
+
+static inline bool __pkru_allows_write(u32 pkru, u16 pkey)
+{
+ int pkru_pkey_bits = pkey * 2;
+ /*
+ * Access-disable disables writes too so we need to check
+ * both bits here.
+ */
+ return !(pkru & ((PKRU_AD_BIT|PKRU_WD_BIT) << pkru_pkey_bits));
+}
+
+static inline u16 pte_flags_pkey(unsigned long pte_flags)
+{
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+ /* ifdef to avoid doing 59-bit shift on 32-bit values */
+ return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
+#else
+ return 0;
+#endif
+}
+
#include <asm-generic/pgtable.h>
#endif /* __ASSEMBLY__ */
diff -puN arch/x86/mm/fault.c~pkeys-13-pte-fault arch/x86/mm/fault.c
--- a/arch/x86/mm/fault.c~pkeys-13-pte-fault 2016-01-06 15:50:09.971352607 -0800
+++ b/arch/x86/mm/fault.c 2016-01-06 15:50:09.983353149 -0800
@@ -897,6 +897,16 @@ bad_area(struct pt_regs *regs, unsigned
__bad_area(regs, error_code, address, NULL, SEGV_MAPERR);
}
+static inline bool bad_area_access_from_pkeys(unsigned long error_code,
+ struct vm_area_struct *vma)
+{
+ if (!boot_cpu_has(X86_FEATURE_OSPKE))
+ return false;
+ if (error_code & PF_PK)
+ return true;
+ return false;
+}
+
static noinline void
bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
unsigned long address, struct vm_area_struct *vma)
@@ -906,7 +916,7 @@ bad_area_access_error(struct pt_regs *re
* But, doing it this way allows compiler optimizations
* if pkeys are compiled out.
*/
- if (boot_cpu_has(X86_FEATURE_OSPKE) && (error_code & PF_PK))
+ if (bad_area_access_from_pkeys(error_code, vma))
__bad_area(regs, error_code, address, vma, SEGV_PKUERR);
else
__bad_area(regs, error_code, address, vma, SEGV_ACCERR);
@@ -1081,6 +1091,15 @@ int show_unhandled_signals = 1;
static inline int
access_error(unsigned long error_code, struct vm_area_struct *vma)
{
+ /*
+ * Access or read was blocked by protection keys. We do
+ * this check before any others because we do not want
+ * to, for instance, confuse a protection-key-denied
+ * write with one for which we should do a COW.
+ */
+ if (error_code & PF_PK)
+ return 1;
+
if (error_code & PF_WRITE) {
/* write, present and write, not present: */
if (unlikely(!(vma->vm_flags & VM_WRITE)))
diff -puN arch/x86/mm/gup.c~pkeys-13-pte-fault arch/x86/mm/gup.c
--- a/arch/x86/mm/gup.c~pkeys-13-pte-fault 2016-01-06 15:50:09.973352698 -0800
+++ b/arch/x86/mm/gup.c 2016-01-06 15:50:09.983353149 -0800
@@ -10,6 +10,7 @@
#include <linux/highmem.h>
#include <linux/swap.h>
+#include <asm/mmu_context.h>
#include <asm/pgtable.h>
static inline pte_t gup_get_pte(pte_t *ptep)
@@ -78,6 +79,10 @@ static inline int pte_allows_gup(unsigne
if ((pteval & need_pte_bits) != need_pte_bits)
return 0;
+ /* Check memory protection keys permissions. */
+ if (!__pkru_allows_pkey(pte_flags_pkey(pteval), write))
+ return 0;
+
return 1;
}
diff -puN include/asm-generic/mm_hooks.h~pkeys-13-pte-fault include/asm-generic/mm_hooks.h
--- a/include/asm-generic/mm_hooks.h~pkeys-13-pte-fault 2016-01-06 15:50:09.974352743 -0800
+++ b/include/asm-generic/mm_hooks.h 2016-01-06 15:50:09.983353149 -0800
@@ -26,4 +26,15 @@ static inline void arch_bprm_mm_init(str
{
}
+static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
+
+static inline bool arch_pte_access_permitted(pte_t pte, bool write)
+{
+ /* by default, allow everything */
+ return true;
+}
#endif /* _ASM_GENERIC_MM_HOOKS_H */
diff -puN mm/gup.c~pkeys-13-pte-fault mm/gup.c
--- a/mm/gup.c~pkeys-13-pte-fault 2016-01-06 15:50:09.976352833 -0800
+++ b/mm/gup.c 2016-01-06 15:50:09.984353194 -0800
@@ -13,6 +13,7 @@
#include <linux/rwsem.h>
#include <linux/hugetlb.h>
+#include <asm/mmu_context.h>
#include <asm/pgtable.h>
#include <asm/tlbflush.h>
@@ -391,6 +392,8 @@ static int check_vma_flags(struct vm_are
if (!(vm_flags & VM_MAYREAD))
return -EFAULT;
}
+ if (!arch_vma_access_permitted(vma, (gup_flags & FOLL_WRITE)))
+ return -EFAULT;
return 0;
}
@@ -559,13 +562,19 @@ EXPORT_SYMBOL(__get_user_pages);
bool vma_permits_fault(struct vm_area_struct *vma, unsigned int fault_flags)
{
- vm_flags_t vm_flags;
-
- vm_flags = (fault_flags & FAULT_FLAG_WRITE) ? VM_WRITE : VM_READ;
+ bool write = !!(fault_flags & FAULT_FLAG_WRITE);
+ vm_flags_t vm_flags = write ? VM_WRITE : VM_READ;
if (!(vm_flags & vma->vm_flags))
return false;
+ /*
+ * The architecture might have a hardware protection
+ * mechanism other than read/write that can deny access
+ */
+ if (!arch_vma_access_permitted(vma, write))
+ return false;
+
return true;
}
@@ -1102,6 +1111,9 @@ static int gup_pte_range(pmd_t pmd, unsi
pte_protnone(pte) || (write && !pte_write(pte)))
goto pte_unmap;
+ if (!arch_pte_access_permitted(pte, write))
+ goto pte_unmap;
+
VM_BUG_ON(!pfn_valid(pte_pfn(pte)));
page = pte_page(pte);
diff -puN mm/memory.c~pkeys-13-pte-fault mm/memory.c
--- a/mm/memory.c~pkeys-13-pte-fault 2016-01-06 15:50:09.977352878 -0800
+++ b/mm/memory.c 2016-01-06 15:50:09.985353239 -0800
@@ -64,6 +64,7 @@
#include <linux/userfaultfd_k.h>
#include <asm/io.h>
+#include <asm/mmu_context.h>
#include <asm/pgalloc.h>
#include <asm/uaccess.h>
#include <asm/tlb.h>
@@ -3344,6 +3345,9 @@ static int __handle_mm_fault(struct mm_s
pmd_t *pmd;
pte_t *pte;
+ if (!arch_vma_access_permitted(vma, flags & FAULT_FLAG_WRITE))
+ return VM_FAULT_SIGSEGV;
+
if (unlikely(is_vm_hugetlb_page(vma)))
return hugetlb_fault(mm, vma, address, flags);
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 03/31] x86, pkeys: Add Kconfig option |
| Message-ID | <qO6jp-8sH-21@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com> I don't have a strong opinion on whether we need a Kconfig prompt or not. Protection Keys has relatively little code associated with it, and it is not a heavyweight feature to keep enabled. However, I can imagine that folks would still appreciate being able to disable it. Note that, with disabled-features.h, the checks in the code for protection keys are always the same: cpu_has(c, X86_FEATURE_PKU) With the config option disabled, this essentially turns into an #ifdef. We will hide the prompt for now. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/Kconfig | 4 ++++ 1 file changed, 4 insertions(+) diff -puN arch/x86/Kconfig~pkeys-01-kconfig arch/x86/Kconfig --- a/arch/x86/Kconfig~pkeys-01-kconfig 2016-01-06 15:50:03.902078982 -0800 +++ b/arch/x86/Kconfig 2016-01-06 15:50:03.905079117 -0800 @@ -1680,6 +1680,10 @@ config X86_INTEL_MPX If unsure, say N. +config X86_INTEL_MEMORY_PROTECTION_KEYS + def_bool y + depends on CPU_SUP_INTEL && X86_64 + config EFI bool "EFI runtime service support" depends on ACPI _ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 29/31] x86, pkeys: allow kernel to modify user pkey rights register |
| Message-ID | <qO6jp-8sH-23@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
The Protection Key Rights for User memory (PKRU) is a 32-bit
user-accessible register. It contains two bits for each
protection key: one to write-disable (WD) access to memory
covered by the key and another to access-disable (AD).
Userspace can read/write the register with the RDPKRU and WRPKRU
instructions. But, the register is saved and restored with the
XSAVE family of instructions, which means we have to treat it
like a floating point register.
The kernel needs to write to the register if it wants to
implement execute-only memory or if it implements a system call
to change PKRU.
To do this, we need to create a 'pkru_state' buffer, read the old
contents in to it, modify it, and then tell the FPU code that
there is modified data in there so it can (possibly) move the
buffer back in to the registers.
This uses the fpu__xfeature_set_state() function that we defined
in the previous patch.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
---
b/arch/x86/include/asm/pgtable.h | 5 +-
b/arch/x86/include/asm/pkeys.h | 3 +
b/arch/x86/kernel/fpu/xstate.c | 74 +++++++++++++++++++++++++++++++++++++++
b/include/linux/pkeys.h | 5 ++
4 files changed, 85 insertions(+), 2 deletions(-)
diff -puN arch/x86/include/asm/pgtable.h~pkeys-77-arch_set_user_pkey_access arch/x86/include/asm/pgtable.h
--- a/arch/x86/include/asm/pgtable.h~pkeys-77-arch_set_user_pkey_access 2016-01-06 15:50:15.900619921 -0800
+++ b/arch/x86/include/asm/pgtable.h 2016-01-06 15:50:15.909620327 -0800
@@ -912,16 +912,17 @@ static inline pte_t pte_swp_clear_soft_d
#define PKRU_AD_BIT 0x1
#define PKRU_WD_BIT 0x2
+#define PKRU_BITS_PER_PKEY 2
static inline bool __pkru_allows_read(u32 pkru, u16 pkey)
{
- int pkru_pkey_bits = pkey * 2;
+ int pkru_pkey_bits = pkey * PKRU_BITS_PER_PKEY;
return !(pkru & (PKRU_AD_BIT << pkru_pkey_bits));
}
static inline bool __pkru_allows_write(u32 pkru, u16 pkey)
{
- int pkru_pkey_bits = pkey * 2;
+ int pkru_pkey_bits = pkey * PKRU_BITS_PER_PKEY;
/*
* Access-disable disables writes too so we need to check
* both bits here.
diff -puN arch/x86/include/asm/pkeys.h~pkeys-77-arch_set_user_pkey_access arch/x86/include/asm/pkeys.h
--- a/arch/x86/include/asm/pkeys.h~pkeys-77-arch_set_user_pkey_access 2016-01-06 15:50:15.902620011 -0800
+++ b/arch/x86/include/asm/pkeys.h 2016-01-06 15:50:15.909620327 -0800
@@ -3,4 +3,7 @@
#define arch_max_pkey() (boot_cpu_has(X86_FEATURE_OSPKE) ? 16 : 1)
+extern int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,
+ unsigned long init_val);
+
#endif /*_ASM_X86_PKEYS_H */
diff -puN arch/x86/kernel/fpu/xstate.c~pkeys-77-arch_set_user_pkey_access arch/x86/kernel/fpu/xstate.c
--- a/arch/x86/kernel/fpu/xstate.c~pkeys-77-arch_set_user_pkey_access 2016-01-06 15:50:15.904620101 -0800
+++ b/arch/x86/kernel/fpu/xstate.c 2016-01-06 15:50:15.909620327 -0800
@@ -5,6 +5,7 @@
*/
#include <linux/compat.h>
#include <linux/cpu.h>
+#include <linux/pkeys.h>
#include <asm/fpu/api.h>
#include <asm/fpu/internal.h>
@@ -856,3 +857,76 @@ out:
*/
fpu__current_fpstate_write_end();
}
+
+#define NR_VALID_PKRU_BITS (CONFIG_NR_PROTECTION_KEYS * 2)
+#define PKRU_VALID_MASK (NR_VALID_PKRU_BITS - 1)
+
+/*
+ * This will go out and modify the XSAVE buffer so that PKRU is
+ * set to a particular state for access to 'pkey'.
+ *
+ * PKRU state does affect kernel access to user memory. We do
+ * not modfiy PKRU *itself* here, only the XSAVE state that will
+ * be restored in to PKRU when we return back to userspace.
+ */
+int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,
+ unsigned long init_val)
+{
+ struct xregs_state *xsave = &tsk->thread.fpu.state.xsave;
+ struct pkru_state *old_pkru_state;
+ struct pkru_state new_pkru_state;
+ int pkey_shift = (pkey * PKRU_BITS_PER_PKEY);
+ u32 new_pkru_bits = 0;
+
+ if (!validate_pkey(pkey))
+ return -EINVAL;
+ /*
+ * This check implies XSAVE support. OSPKE only gets
+ * set if we enable XSAVE and we enable PKU in XCR0.
+ */
+ if (!boot_cpu_has(X86_FEATURE_OSPKE))
+ return -EINVAL;
+
+ /* Set the bits we need in PKRU */
+ if (init_val & PKEY_DISABLE_ACCESS)
+ new_pkru_bits |= PKRU_AD_BIT;
+ if (init_val & PKEY_DISABLE_WRITE)
+ new_pkru_bits |= PKRU_WD_BIT;
+
+ /* Shift the bits in to the correct place in PKRU for pkey. */
+ new_pkru_bits <<= pkey_shift;
+
+ /* Locate old copy of the state in the xsave buffer */
+ old_pkru_state = get_xsave_addr(xsave, XFEATURE_MASK_PKRU);
+
+ /*
+ * When state is not in the buffer, it is in the init
+ * state, set it manually. Otherwise, copy out the old
+ * state.
+ */
+ if (!old_pkru_state)
+ new_pkru_state.pkru = 0;
+ else
+ new_pkru_state.pkru = old_pkru_state->pkru;
+
+ /* mask off any old bits in place */
+ new_pkru_state.pkru &= ~((PKRU_AD_BIT|PKRU_WD_BIT) << pkey_shift);
+ /* Set the newly-requested bits */
+ new_pkru_state.pkru |= new_pkru_bits;
+
+ /*
+ * We could theoretically live without zeroing pkru.pad.
+ * The current XSAVE feature state definition says that
+ * only bytes 0->3 are used. But we do not want to
+ * chance leaking kernel stack out to userspace in case a
+ * memcpy() of the whole xsave buffer was done.
+ *
+ * They're in the same cacheline anyway.
+ */
+ new_pkru_state.pad = 0;
+
+ fpu__xfeature_set_state(XFEATURE_MASK_PKRU, &new_pkru_state,
+ sizeof(new_pkru_state));
+
+ return 0;
+}
diff -puN include/linux/pkeys.h~pkeys-77-arch_set_user_pkey_access include/linux/pkeys.h
--- a/include/linux/pkeys.h~pkeys-77-arch_set_user_pkey_access 2016-01-06 15:50:15.905620146 -0800
+++ b/include/linux/pkeys.h 2016-01-06 15:50:15.910620372 -0800
@@ -4,6 +4,11 @@
#include <linux/mm_types.h>
#include <asm/mmu_context.h>
+#define PKEY_DISABLE_ACCESS 0x1
+#define PKEY_DISABLE_WRITE 0x2
+#define PKEY_ACCESS_MASK (PKEY_DISABLE_ACCESS |\
+ PKEY_DISABLE_WRITE)
+
#ifdef CONFIG_ARCH_HAS_PKEYS
#include <asm/pkeys.h>
#else /* ! CONFIG_ARCH_HAS_PKEYS */
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 21/31] x86, pkeys: dump PKRU with other kernel registers |
| Message-ID | <qO6jp-8sH-25@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com> Protection Keys never affect kernel mappings. But, they can affect whether the kernel will fault when it touches a user mapping. The kernel doesn't touch user mappings without some careful choreography and these accesses don't generally result in oopses. But, if one does, we definitely want to have PKRU available so we can figure out if protection keys played a role. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/kernel/process_64.c | 2 ++ 1 file changed, 2 insertions(+) diff -puN arch/x86/kernel/process_64.c~pkeys-30-kernel-error-dumps arch/x86/kernel/process_64.c --- a/arch/x86/kernel/process_64.c~pkeys-30-kernel-error-dumps 2016-01-06 15:50:12.265456034 -0800 +++ b/arch/x86/kernel/process_64.c 2016-01-06 15:50:12.268456170 -0800 @@ -116,6 +116,8 @@ void __show_regs(struct pt_regs *regs, i printk(KERN_DEFAULT "DR0: %016lx DR1: %016lx DR2: %016lx\n", d0, d1, d2); printk(KERN_DEFAULT "DR3: %016lx DR6: %016lx DR7: %016lx\n", d3, d6, d7); + if (boot_cpu_has(X86_FEATURE_OSPKE)) + printk(KERN_DEFAULT "PKRU: %08x\n", read_pkru()); } void release_thread(struct task_struct *dead_task) _ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 11/31] x86, pkeys: pass VMA down in to fault signal generation code |
| Message-ID | <qO6jp-8sH-29@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
During a page fault, we look up the VMA to ensure that the fault
is in a region with a valid mapping. But, in the top-level page
fault code we don't need the VMA for much else. Once we have
decided that an access is bad, we are going to send a signal no
matter what and do not need the VMA any more. So we do not pass
it down in to the signal generation code.
But, for protection keys, we need the VMA. It tells us *which*
protection key we violated if we get a PF_PK. So, we need to
pass the VMA down and fill in siginfo->si_pkey.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/mm/fault.c | 50 ++++++++++++++++++++++++++++----------------------
1 file changed, 28 insertions(+), 22 deletions(-)
diff -puN arch/x86/mm/fault.c~pkeys-08-pass-down-vma arch/x86/mm/fault.c
--- a/arch/x86/mm/fault.c~pkeys-08-pass-down-vma 2016-01-06 15:50:07.422237684 -0800
+++ b/arch/x86/mm/fault.c 2016-01-06 15:50:07.425237819 -0800
@@ -171,7 +171,8 @@ is_prefetch(struct pt_regs *regs, unsign
static void
force_sig_info_fault(int si_signo, int si_code, unsigned long address,
- struct task_struct *tsk, int fault)
+ struct task_struct *tsk, struct vm_area_struct *vma,
+ int fault)
{
unsigned lsb = 0;
siginfo_t info;
@@ -656,6 +657,8 @@ no_context(struct pt_regs *regs, unsigne
struct task_struct *tsk = current;
unsigned long flags;
int sig;
+ /* No context means no VMA to pass down */
+ struct vm_area_struct *vma = NULL;
/* Are we prepared to handle this kernel fault? */
if (fixup_exception(regs)) {
@@ -679,7 +682,8 @@ no_context(struct pt_regs *regs, unsigne
tsk->thread.cr2 = address;
/* XXX: hwpoison faults will set the wrong code. */
- force_sig_info_fault(signal, si_code, address, tsk, 0);
+ force_sig_info_fault(signal, si_code, address,
+ tsk, vma, 0);
}
/*
@@ -756,7 +760,8 @@ show_signal_msg(struct pt_regs *regs, un
static void
__bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, int si_code)
+ unsigned long address, struct vm_area_struct *vma,
+ int si_code)
{
struct task_struct *tsk = current;
@@ -799,7 +804,7 @@ __bad_area_nosemaphore(struct pt_regs *r
tsk->thread.error_code = error_code;
tsk->thread.trap_nr = X86_TRAP_PF;
- force_sig_info_fault(SIGSEGV, si_code, address, tsk, 0);
+ force_sig_info_fault(SIGSEGV, si_code, address, tsk, vma, 0);
return;
}
@@ -812,14 +817,14 @@ __bad_area_nosemaphore(struct pt_regs *r
static noinline void
bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
- unsigned long address)
+ unsigned long address, struct vm_area_struct *vma)
{
- __bad_area_nosemaphore(regs, error_code, address, SEGV_MAPERR);
+ __bad_area_nosemaphore(regs, error_code, address, vma, SEGV_MAPERR);
}
static void
__bad_area(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, int si_code)
+ unsigned long address, struct vm_area_struct *vma, int si_code)
{
struct mm_struct *mm = current->mm;
@@ -829,25 +834,25 @@ __bad_area(struct pt_regs *regs, unsigne
*/
up_read(&mm->mmap_sem);
- __bad_area_nosemaphore(regs, error_code, address, si_code);
+ __bad_area_nosemaphore(regs, error_code, address, vma, si_code);
}
static noinline void
bad_area(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
- __bad_area(regs, error_code, address, SEGV_MAPERR);
+ __bad_area(regs, error_code, address, NULL, SEGV_MAPERR);
}
static noinline void
bad_area_access_error(struct pt_regs *regs, unsigned long error_code,
- unsigned long address)
+ unsigned long address, struct vm_area_struct *vma)
{
- __bad_area(regs, error_code, address, SEGV_ACCERR);
+ __bad_area(regs, error_code, address, vma, SEGV_ACCERR);
}
static void
do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address,
- unsigned int fault)
+ struct vm_area_struct *vma, unsigned int fault)
{
struct task_struct *tsk = current;
int code = BUS_ADRERR;
@@ -874,12 +879,13 @@ do_sigbus(struct pt_regs *regs, unsigned
code = BUS_MCEERR_AR;
}
#endif
- force_sig_info_fault(SIGBUS, code, address, tsk, fault);
+ force_sig_info_fault(SIGBUS, code, address, tsk, vma, fault);
}
static noinline void
mm_fault_error(struct pt_regs *regs, unsigned long error_code,
- unsigned long address, unsigned int fault)
+ unsigned long address, struct vm_area_struct *vma,
+ unsigned int fault)
{
if (fatal_signal_pending(current) && !(error_code & PF_USER)) {
no_context(regs, error_code, address, 0, 0);
@@ -903,9 +909,9 @@ mm_fault_error(struct pt_regs *regs, uns
} else {
if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|
VM_FAULT_HWPOISON_LARGE))
- do_sigbus(regs, error_code, address, fault);
+ do_sigbus(regs, error_code, address, vma, fault);
else if (fault & VM_FAULT_SIGSEGV)
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, vma);
else
BUG();
}
@@ -1119,7 +1125,7 @@ __do_page_fault(struct pt_regs *regs, un
* Don't take the mm semaphore here. If we fixup a prefetch
* fault we could otherwise deadlock:
*/
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1132,7 +1138,7 @@ __do_page_fault(struct pt_regs *regs, un
pgtable_bad(regs, error_code, address);
if (unlikely(smap_violation(error_code, regs))) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1141,7 +1147,7 @@ __do_page_fault(struct pt_regs *regs, un
* in a region with pagefaults disabled then we must not take the fault
*/
if (unlikely(faulthandler_disabled() || !mm)) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
@@ -1185,7 +1191,7 @@ __do_page_fault(struct pt_regs *regs, un
if (unlikely(!down_read_trylock(&mm->mmap_sem))) {
if ((error_code & PF_USER) == 0 &&
!search_exception_tables(regs->ip)) {
- bad_area_nosemaphore(regs, error_code, address);
+ bad_area_nosemaphore(regs, error_code, address, NULL);
return;
}
retry:
@@ -1233,7 +1239,7 @@ retry:
*/
good_area:
if (unlikely(access_error(error_code, vma))) {
- bad_area_access_error(regs, error_code, address);
+ bad_area_access_error(regs, error_code, address, vma);
return;
}
@@ -1271,7 +1277,7 @@ good_area:
up_read(&mm->mmap_sem);
if (unlikely(fault & VM_FAULT_ERROR)) {
- mm_fault_error(regs, error_code, address, fault);
+ mm_fault_error(regs, error_code, address, vma, fault);
return;
}
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 10/31] x86, pkeys: arch-specific protection bits |
| Message-ID | <qO6jp-8sH-31@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
Lots of things seem to do:
vma->vm_page_prot = vm_get_page_prot(flags);
and the ptes get created right from things we pull out
of ->vm_page_prot. So it is very convenient if we can
store the protection key in flags and vm_page_prot, just
like the existing permission bits (_PAGE_RW/PRESENT). It
greatly reduces the amount of plumbing and arch-specific
hacking we have to do in generic code.
This also takes the new PROT_PKEY{0,1,2,3} flags and
turns *those* in to VM_ flags for vma->vm_flags.
The protection key values are stored in 4 places:
1. "prot" argument to system calls
2. vma->vm_flags, filled from the mmap "prot"
3. vma->vm_page prot, filled from vma->vm_flags
4. the PTE itself.
The pseudocode for these for steps are as follows:
mmap(PROT_PKEY*)
vma->vm_flags = ... | arch_calc_vm_prot_bits(mmap_prot);
vma->vm_page_prot = ... | arch_vm_get_page_prot(vma->vm_flags);
pte = pfn | vma->vm_page_prot
Note that this provides a new definitions for x86:
arch_vm_get_page_prot()
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
---
b/arch/x86/include/asm/mmu_context.h | 11 +++++++++++
b/arch/x86/include/asm/pgtable_types.h | 12 ++++++++++--
b/arch/x86/include/uapi/asm/mman.h | 16 ++++++++++++++++
b/include/linux/mm.h | 7 +++++++
4 files changed, 44 insertions(+), 2 deletions(-)
diff -puN arch/x86/include/asm/mmu_context.h~pkeys-07-store-pkey-in-vma arch/x86/include/asm/mmu_context.h
--- a/arch/x86/include/asm/mmu_context.h~pkeys-07-store-pkey-in-vma 2016-01-06 15:50:06.939215908 -0800
+++ b/arch/x86/include/asm/mmu_context.h 2016-01-06 15:50:06.947216268 -0800
@@ -243,4 +243,15 @@ static inline void arch_unmap(struct mm_
mpx_notify_unmap(mm, vma, start, end);
}
+static inline int vma_pkey(struct vm_area_struct *vma)
+{
+ u16 pkey = 0;
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+ unsigned long vma_pkey_mask = VM_PKEY_BIT0 | VM_PKEY_BIT1 |
+ VM_PKEY_BIT2 | VM_PKEY_BIT3;
+ pkey = (vma->vm_flags & vma_pkey_mask) >> VM_PKEY_SHIFT;
+#endif
+ return pkey;
+}
+
#endif /* _ASM_X86_MMU_CONTEXT_H */
diff -puN arch/x86/include/asm/pgtable_types.h~pkeys-07-store-pkey-in-vma arch/x86/include/asm/pgtable_types.h
--- a/arch/x86/include/asm/pgtable_types.h~pkeys-07-store-pkey-in-vma 2016-01-06 15:50:06.941215998 -0800
+++ b/arch/x86/include/asm/pgtable_types.h 2016-01-06 15:50:06.948216313 -0800
@@ -111,7 +111,12 @@
#define _KERNPG_TABLE (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED | \
_PAGE_DIRTY)
-/* Set of bits not changed in pte_modify */
+/*
+ * Set of bits not changed in pte_modify. The pte's
+ * protection key is treated like _PAGE_RW, for
+ * instance, and is *not* included in this mask since
+ * pte_modify() does modify it.
+ */
#define _PAGE_CHG_MASK (PTE_PFN_MASK | _PAGE_PCD | _PAGE_PWT | \
_PAGE_SPECIAL | _PAGE_ACCESSED | _PAGE_DIRTY | \
_PAGE_SOFT_DIRTY)
@@ -227,7 +232,10 @@ enum page_cache_mode {
/* Extracts the PFN from a (pte|pmd|pud|pgd)val_t of a 4KB page */
#define PTE_PFN_MASK ((pteval_t)PHYSICAL_PAGE_MASK)
-/* Extracts the flags from a (pte|pmd|pud|pgd)val_t of a 4KB page */
+/*
+ * Extracts the flags from a (pte|pmd|pud|pgd)val_t
+ * This includes the protection key value.
+ */
#define PTE_FLAGS_MASK (~PTE_PFN_MASK)
typedef struct pgprot { pgprotval_t pgprot; } pgprot_t;
diff -puN arch/x86/include/uapi/asm/mman.h~pkeys-07-store-pkey-in-vma arch/x86/include/uapi/asm/mman.h
--- a/arch/x86/include/uapi/asm/mman.h~pkeys-07-store-pkey-in-vma 2016-01-06 15:50:06.942216043 -0800
+++ b/arch/x86/include/uapi/asm/mman.h 2016-01-06 15:50:06.948216313 -0800
@@ -6,6 +6,22 @@
#define MAP_HUGE_2MB (21 << MAP_HUGE_SHIFT)
#define MAP_HUGE_1GB (30 << MAP_HUGE_SHIFT)
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+/*
+ * Take the 4 protection key bits out of the vma->vm_flags
+ * value and turn them in to the bits that we can put in
+ * to a pte.
+ *
+ * Only override these if Protection Keys are available
+ * (which is only on 64-bit).
+ */
+#define arch_vm_get_page_prot(vm_flags) __pgprot( \
+ ((vm_flags) & VM_PKEY_BIT0 ? _PAGE_PKEY_BIT0 : 0) | \
+ ((vm_flags) & VM_PKEY_BIT1 ? _PAGE_PKEY_BIT1 : 0) | \
+ ((vm_flags) & VM_PKEY_BIT2 ? _PAGE_PKEY_BIT2 : 0) | \
+ ((vm_flags) & VM_PKEY_BIT3 ? _PAGE_PKEY_BIT3 : 0))
+#endif
+
#include <asm-generic/mman.h>
#endif /* _ASM_X86_MMAN_H */
diff -puN include/linux/mm.h~pkeys-07-store-pkey-in-vma include/linux/mm.h
--- a/include/linux/mm.h~pkeys-07-store-pkey-in-vma 2016-01-06 15:50:06.944216133 -0800
+++ b/include/linux/mm.h 2016-01-06 15:50:06.949216358 -0800
@@ -171,6 +171,13 @@ extern unsigned int kobjsize(const void
#if defined(CONFIG_X86)
# define VM_PAT VM_ARCH_1 /* PAT reserves whole VMA at once (x86) */
+#if defined (CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS)
+# define VM_PKEY_SHIFT VM_HIGH_ARCH_BIT_0
+# define VM_PKEY_BIT0 VM_HIGH_ARCH_0 /* A protection key is a 4-bit value */
+# define VM_PKEY_BIT1 VM_HIGH_ARCH_1
+# define VM_PKEY_BIT2 VM_HIGH_ARCH_2
+# define VM_PKEY_BIT3 VM_HIGH_ARCH_3
+#endif
#elif defined(CONFIG_PPC)
# define VM_SAO VM_ARCH_1 /* Strong Access Ordering (powerpc) */
#elif defined(CONFIG_PARISC)
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 19/31] x86, pkeys: optimize fault handling in access_error() |
| Message-ID | <qO6jp-8sH-33@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
We might not strictly have to make modifictions to
access_error() to check the VMA here.
If we do not, we will do this:
1. app sets VMA pkey to K
2. app touches a !present page
3. do_page_fault(), allocates and maps page, sets pte.pkey=K
4. return to userspace
5. touch instruction reexecutes, but triggers PF_PK
6. do PKEY signal
What happens with this patch applied:
1. app sets VMA pkey to K
2. app touches a !present page
3. do_page_fault() notices that K is inaccessible
4. do PKEY signal
We basically skip the fault that does an allocation.
So what this lets us do is protect areas from even being
*populated* unless it is accessible according to protection
keys. That seems handy to me and makes protection keys work
more like an mprotect()'d mapping.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/mm/fault.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff -puN arch/x86/mm/fault.c~pkeys-15-access_error arch/x86/mm/fault.c
--- a/arch/x86/mm/fault.c~pkeys-15-access_error 2016-01-06 15:50:11.270411174 -0800
+++ b/arch/x86/mm/fault.c 2016-01-06 15:50:11.274411354 -0800
@@ -900,10 +900,16 @@ bad_area(struct pt_regs *regs, unsigned
static inline bool bad_area_access_from_pkeys(unsigned long error_code,
struct vm_area_struct *vma)
{
+ /* This code is always called on the current mm */
+ bool foreign = false;
+
if (!boot_cpu_has(X86_FEATURE_OSPKE))
return false;
if (error_code & PF_PK)
return true;
+ /* this checks permission keys on the VMA: */
+ if (!arch_vma_access_permitted(vma, (error_code & PF_WRITE), foreign))
+ return true;
return false;
}
@@ -1091,6 +1097,8 @@ int show_unhandled_signals = 1;
static inline int
access_error(unsigned long error_code, struct vm_area_struct *vma)
{
+ /* This is only called for the current mm, so: */
+ bool foreign = false;
/*
* Access or read was blocked by protection keys. We do
* this check before any others because we do not want
@@ -1099,6 +1107,13 @@ access_error(unsigned long error_code, s
*/
if (error_code & PF_PK)
return 1;
+ /*
+ * Make sure to check the VMA so that we do not perform
+ * faults just to hit a PF_PK as soon as we fill in a
+ * page.
+ */
+ if (!arch_vma_access_permitted(vma, (error_code & PF_WRITE), foreign))
+ return 1;
if (error_code & PF_WRITE) {
/* write, present and write, not present: */
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 05/31] x86, pkeys: define new CR4 bit |
| Message-ID | <qO6jq-8sH-45@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com> There is a new bit in CR4 for enabling protection keys. We will actually enable it later in the series. Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/include/uapi/asm/processor-flags.h | 2 ++ 1 file changed, 2 insertions(+) diff -puN arch/x86/include/uapi/asm/processor-flags.h~pkeys-02-cr4 arch/x86/include/uapi/asm/processor-flags.h --- a/arch/x86/include/uapi/asm/processor-flags.h~pkeys-02-cr4 2016-01-06 15:50:04.798119379 -0800 +++ b/arch/x86/include/uapi/asm/processor-flags.h 2016-01-06 15:50:04.801119514 -0800 @@ -118,6 +118,8 @@ #define X86_CR4_SMEP _BITUL(X86_CR4_SMEP_BIT) #define X86_CR4_SMAP_BIT 21 /* enable SMAP support */ #define X86_CR4_SMAP _BITUL(X86_CR4_SMAP_BIT) +#define X86_CR4_PKE_BIT 22 /* enable Protection Keys support */ +#define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT) /* * x86-64 Task Priority Register, CR8 _ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 09/31] x86, pkeys: store protection in high VMA flags |
| Message-ID | <qO6jq-8sH-37@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com> vma->vm_flags is an 'unsigned long', so has space for 32 flags on 32-bit architectures. The high 32 bits are unused on 64-bit platforms. We've steered away from using the unused high VMA bits for things because we would have difficulty supporting it on 32-bit. Protection Keys are not available in 32-bit mode, so there is no concern about supporting this feature in 32-bit mode or on 32-bit CPUs. This patch carves out 4 bits from the high half of vma->vm_flags and allows architectures to set config option to make them available. Sparse complains about these constants unless we explicitly call them "UL". Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> --- b/arch/x86/Kconfig | 1 + b/include/linux/mm.h | 11 +++++++++++ b/mm/Kconfig | 3 +++ 3 files changed, 15 insertions(+) diff -puN arch/x86/Kconfig~pkeys-06-eat-high-vma-flags arch/x86/Kconfig --- a/arch/x86/Kconfig~pkeys-06-eat-high-vma-flags 2016-01-06 15:50:06.481195258 -0800 +++ b/arch/x86/Kconfig 2016-01-06 15:50:06.488195574 -0800 @@ -152,6 +152,7 @@ config X86 select VIRT_TO_BUS select X86_DEV_DMA_OPS if X86_64 select X86_FEATURE_NAMES if PROC_FS + select ARCH_USES_HIGH_VMA_FLAGS if X86_INTEL_MEMORY_PROTECTION_KEYS config INSTRUCTION_DECODER def_bool y diff -puN include/linux/mm.h~pkeys-06-eat-high-vma-flags include/linux/mm.h --- a/include/linux/mm.h~pkeys-06-eat-high-vma-flags 2016-01-06 15:50:06.482195303 -0800 +++ b/include/linux/mm.h 2016-01-06 15:50:06.489195619 -0800 @@ -158,6 +158,17 @@ extern unsigned int kobjsize(const void #define VM_NOHUGEPAGE 0x40000000 /* MADV_NOHUGEPAGE marked this vma */ #define VM_MERGEABLE 0x80000000 /* KSM may merge identical pages */ +#ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS +#define VM_HIGH_ARCH_BIT_0 32 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_1 33 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_2 34 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_3 35 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_0 BIT(VM_HIGH_ARCH_BIT_0) +#define VM_HIGH_ARCH_1 BIT(VM_HIGH_ARCH_BIT_1) +#define VM_HIGH_ARCH_2 BIT(VM_HIGH_ARCH_BIT_2) +#define VM_HIGH_ARCH_3 BIT(VM_HIGH_ARCH_BIT_3) +#endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */ + #if defined(CONFIG_X86) # define VM_PAT VM_ARCH_1 /* PAT reserves whole VMA at once (x86) */ #elif defined(CONFIG_PPC) diff -puN mm/Kconfig~pkeys-06-eat-high-vma-flags mm/Kconfig --- a/mm/Kconfig~pkeys-06-eat-high-vma-flags 2016-01-06 15:50:06.484195393 -0800 +++ b/mm/Kconfig 2016-01-06 15:50:06.489195619 -0800 @@ -668,3 +668,6 @@ config ZONE_DEVICE config FRAME_VECTOR bool + +config ARCH_USES_HIGH_VMA_FLAGS + bool _ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 16/31] x86, mm: simplify get_user_pages() PTE bit handling |
| Message-ID | <qO6jq-8sH-39@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
The current get_user_pages() code is a wee bit more complicated
than it needs to be for pte bit checking. Currently, it establishes
a mask of required pte _PAGE_* bits and ensures that the pte it
goes after has all those bits.
This consolidates the three identical copies of this code.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/mm/gup.c | 39 ++++++++++++++++++++++-----------------
1 file changed, 22 insertions(+), 17 deletions(-)
diff -puN arch/x86/mm/gup.c~pkeys-12-gup-swizzle arch/x86/mm/gup.c
--- a/arch/x86/mm/gup.c~pkeys-12-gup-swizzle 2016-01-06 15:50:09.552333717 -0800
+++ b/arch/x86/mm/gup.c 2016-01-06 15:50:09.555333852 -0800
@@ -64,6 +64,24 @@ retry:
}
/*
+ * 'pteval' can come from a pte, pmd or pud. We only check
+ * _PAGE_PRESENT, _PAGE_USER, and _PAGE_RW in here which are the
+ * same value on all 3 types.
+ */
+static inline int pte_allows_gup(unsigned long pteval, int write)
+{
+ unsigned long need_pte_bits = _PAGE_PRESENT|_PAGE_USER;
+
+ if (write)
+ need_pte_bits |= _PAGE_RW;
+
+ if ((pteval & need_pte_bits) != need_pte_bits)
+ return 0;
+
+ return 1;
+}
+
+/*
* The performance critical leaf functions are made noinline otherwise gcc
* inlines everything into a single function which results in too much
* register pressure.
@@ -71,13 +89,8 @@ retry:
static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
pte_t *ptep;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
-
ptep = pte_offset_map(&pmd, addr);
do {
pte_t pte = gup_get_pte(ptep);
@@ -88,8 +101,8 @@ static noinline int gup_pte_range(pmd_t
pte_unmap(ptep);
return 0;
}
-
- if ((pte_flags(pte) & (mask | _PAGE_SPECIAL)) != mask) {
+ if (!pte_allows_gup(pte_val(pte), write) ||
+ pte_special(pte)) {
pte_unmap(ptep);
return 0;
}
@@ -117,14 +130,10 @@ static inline void get_head_page_multipl
static noinline int gup_huge_pmd(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pmd_flags(pmd) & mask) != mask)
+ if (!pte_allows_gup(pmd_val(pmd), write))
return 0;
/* hugepages are never "special" */
VM_BUG_ON(pmd_flags(pmd) & _PAGE_SPECIAL);
@@ -193,14 +202,10 @@ static int gup_pmd_range(pud_t pud, unsi
static noinline int gup_huge_pud(pud_t pud, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pud_flags(pud) & mask) != mask)
+ if (!pte_allows_gup(pud_val(pud), write))
return 0;
/* hugepages are never "special" */
VM_BUG_ON(pud_flags(pud) & _PAGE_SPECIAL);
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 27/31] x86: separate out LDT init from context init |
| Message-ID | <qO6jq-8sH-41@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
The arch-specific mm_context_t is a great place to put
protection-key allocation state.
But, we need to initialize the allocation state because pkey 0 is
always "allocated". All of the runtime initialization of
mm_context_t is done in *_ldt() manipulation functions. This
renames the existing LDT functions like this:
init_new_context() -> init_new_context_ldt()
destroy_context() -> destroy_context_ldt()
and makes init_new_context() and destroy_context() available for
generic use.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
b/arch/x86/include/asm/mmu_context.h | 21 ++++++++++++++++-----
b/arch/x86/kernel/ldt.c | 4 ++--
2 files changed, 18 insertions(+), 7 deletions(-)
diff -puN arch/x86/include/asm/mmu_context.h~pkeys-72-init-ldt-extricate arch/x86/include/asm/mmu_context.h
--- a/arch/x86/include/asm/mmu_context.h~pkeys-72-init-ldt-extricate 2016-01-06 15:50:15.004579524 -0800
+++ b/arch/x86/include/asm/mmu_context.h 2016-01-06 15:50:15.008579705 -0800
@@ -52,15 +52,15 @@ struct ldt_struct {
/*
* Used for LDT copy/destruction.
*/
-int init_new_context(struct task_struct *tsk, struct mm_struct *mm);
-void destroy_context(struct mm_struct *mm);
+int init_new_context_ldt(struct task_struct *tsk, struct mm_struct *mm);
+void destroy_context_ldt(struct mm_struct *mm);
#else /* CONFIG_MODIFY_LDT_SYSCALL */
-static inline int init_new_context(struct task_struct *tsk,
- struct mm_struct *mm)
+static inline int init_new_context_ldt(struct task_struct *tsk,
+ struct mm_struct *mm)
{
return 0;
}
-static inline void destroy_context(struct mm_struct *mm) {}
+static inline void destroy_context_ldt(struct mm_struct *mm) {}
#endif
static inline void load_mm_ldt(struct mm_struct *mm)
@@ -104,6 +104,17 @@ static inline void enter_lazy_tlb(struct
#endif
}
+static inline int init_new_context(struct task_struct *tsk,
+ struct mm_struct *mm)
+{
+ init_new_context_ldt(tsk, mm);
+ return 0;
+}
+static inline void destroy_context(struct mm_struct *mm)
+{
+ destroy_context_ldt(mm);
+}
+
static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
struct task_struct *tsk)
{
diff -puN arch/x86/kernel/ldt.c~pkeys-72-init-ldt-extricate arch/x86/kernel/ldt.c
--- a/arch/x86/kernel/ldt.c~pkeys-72-init-ldt-extricate 2016-01-06 15:50:15.005579569 -0800
+++ b/arch/x86/kernel/ldt.c 2016-01-06 15:50:15.009579749 -0800
@@ -103,7 +103,7 @@ static void free_ldt_struct(struct ldt_s
* we do not have to muck with descriptors here, that is
* done in switch_mm() as needed.
*/
-int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
+int init_new_context_ldt(struct task_struct *tsk, struct mm_struct *mm)
{
struct ldt_struct *new_ldt;
struct mm_struct *old_mm;
@@ -144,7 +144,7 @@ out_unlock:
*
* 64bit: Don't touch the LDT register - we're already in the next thread.
*/
-void destroy_context(struct mm_struct *mm)
+void destroy_context_ldt(struct mm_struct *mm)
{
free_ldt_struct(mm->context.ldt);
mm->context.ldt = NULL;
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Dave Hansen <dave@sr71.net> |
|---|---|
| Date | 2016-01-07 01:10 +0100 |
| Subject | [PATCH 14/31] x86, pkeys: add functions to fetch PKRU |
| Message-ID | <qO6jq-8sH-43@gated-at.bofh.it> |
| In reply to | #1303164 |
From: Dave Hansen <dave.hansen@linux.intel.com>
This adds the raw instruction to access PKRU as well as some
accessor functions that correctly handle when the CPU does not
support the instruction. We don't use it here, but we will use
read_pkru() in the next patch.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
---
b/arch/x86/include/asm/pgtable.h | 8 ++++++++
b/arch/x86/include/asm/special_insns.h | 22 ++++++++++++++++++++++
2 files changed, 30 insertions(+)
diff -puN arch/x86/include/asm/pgtable.h~pkeys-10-kernel-pkru-instructions arch/x86/include/asm/pgtable.h
--- a/arch/x86/include/asm/pgtable.h~pkeys-10-kernel-pkru-instructions 2016-01-06 15:50:08.711295799 -0800
+++ b/arch/x86/include/asm/pgtable.h 2016-01-06 15:50:08.716296025 -0800
@@ -102,6 +102,14 @@ static inline int pte_dirty(pte_t pte)
return pte_flags(pte) & _PAGE_DIRTY;
}
+
+static inline u32 read_pkru(void)
+{
+ if (boot_cpu_has(X86_FEATURE_OSPKE))
+ return __read_pkru();
+ return 0;
+}
+
static inline int pte_young(pte_t pte)
{
return pte_flags(pte) & _PAGE_ACCESSED;
diff -puN arch/x86/include/asm/special_insns.h~pkeys-10-kernel-pkru-instructions arch/x86/include/asm/special_insns.h
--- a/arch/x86/include/asm/special_insns.h~pkeys-10-kernel-pkru-instructions 2016-01-06 15:50:08.713295890 -0800
+++ b/arch/x86/include/asm/special_insns.h 2016-01-06 15:50:08.717296070 -0800
@@ -98,6 +98,28 @@ static inline void native_write_cr8(unsi
}
#endif
+#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
+static inline u32 __read_pkru(void)
+{
+ u32 ecx = 0;
+ u32 edx, pkru;
+
+ /*
+ * "rdpkru" instruction. Places PKRU contents in to EAX,
+ * clears EDX and requires that ecx=0.
+ */
+ asm volatile(".byte 0x0f,0x01,0xee\n\t"
+ : "=a" (pkru), "=d" (edx)
+ : "c" (ecx));
+ return pkru;
+}
+#else
+static inline u32 __read_pkru(void)
+{
+ return 0;
+}
+#endif
+
static inline void native_wbinvd(void)
{
asm volatile("wbinvd": : :"memory");
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
Page 1 of 3 [1] 2 3 Next page →
Back to top | Article view | linux.kernel
csiph-web