Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1337700
| From | tip-bot for Dave Hansen <tipbot@zytor.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [tip:mm/pkeys] x86/mm/gup: Simplify get_user_pages() PTE bit handling |
| Date | 2016-02-18 21:30 +0100 |
| Message-ID | <r3Dn4-2Bf-17@gated-at.bofh.it> (permalink) |
| References | <r1t8w-3TO-51@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
Commit-ID: 1874f6895c92d991ccf85edcc55a0d9dd552d71c
Gitweb: http://git.kernel.org/tip/1874f6895c92d991ccf85edcc55a0d9dd552d71c
Author: Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate: Fri, 12 Feb 2016 13:02:18 -0800
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 18 Feb 2016 09:32:44 +0100
x86/mm/gup: Simplify get_user_pages() PTE bit handling
The current get_user_pages() code is a wee bit more complicated
than it needs to be for pte bit checking. Currently, it establishes
a mask of required pte _PAGE_* bits and ensures that the pte it
goes after has all those bits.
This consolidates the three identical copies of this code.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210218.3A2D4045@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/mm/gup.c | 38 ++++++++++++++++++++++----------------
1 file changed, 22 insertions(+), 16 deletions(-)
diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c
index ce5e454..2f0a329 100644
--- a/arch/x86/mm/gup.c
+++ b/arch/x86/mm/gup.c
@@ -75,6 +75,24 @@ static void undo_dev_pagemap(int *nr, int nr_start, struct page **pages)
}
/*
+ * 'pteval' can come from a pte, pmd or pud. We only check
+ * _PAGE_PRESENT, _PAGE_USER, and _PAGE_RW in here which are the
+ * same value on all 3 types.
+ */
+static inline int pte_allows_gup(unsigned long pteval, int write)
+{
+ unsigned long need_pte_bits = _PAGE_PRESENT|_PAGE_USER;
+
+ if (write)
+ need_pte_bits |= _PAGE_RW;
+
+ if ((pteval & need_pte_bits) != need_pte_bits)
+ return 0;
+
+ return 1;
+}
+
+/*
* The performance critical leaf functions are made noinline otherwise gcc
* inlines everything into a single function which results in too much
* register pressure.
@@ -83,14 +101,9 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
struct dev_pagemap *pgmap = NULL;
- unsigned long mask;
int nr_start = *nr;
pte_t *ptep;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
-
ptep = pte_offset_map(&pmd, addr);
do {
pte_t pte = gup_get_pte(ptep);
@@ -110,7 +123,8 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,
pte_unmap(ptep);
return 0;
}
- } else if ((pte_flags(pte) & (mask | _PAGE_SPECIAL)) != mask) {
+ } else if (!pte_allows_gup(pte_val(pte), write) ||
+ pte_special(pte)) {
pte_unmap(ptep);
return 0;
}
@@ -164,14 +178,10 @@ static int __gup_device_huge_pmd(pmd_t pmd, unsigned long addr,
static noinline int gup_huge_pmd(pmd_t pmd, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pmd_flags(pmd) & mask) != mask)
+ if (!pte_allows_gup(pmd_val(pmd), write))
return 0;
VM_BUG_ON(!pfn_valid(pmd_pfn(pmd)));
@@ -231,14 +241,10 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,
static noinline int gup_huge_pud(pud_t pud, unsigned long addr,
unsigned long end, int write, struct page **pages, int *nr)
{
- unsigned long mask;
struct page *head, *page;
int refs;
- mask = _PAGE_PRESENT|_PAGE_USER;
- if (write)
- mask |= _PAGE_RW;
- if ((pud_flags(pud) & mask) != mask)
+ if (!pte_allows_gup(pud_val(pud), write))
return 0;
/* hugepages are never "special" */
VM_BUG_ON(pud_flags(pud) & _PAGE_SPECIAL);
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH 00/33] x86: Memory Protection Keys (v10) Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[PATCH 18/33] x86, mm: simplify get_user_pages() PTE bit handling Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/gup: Simplify get_user_pages() PTE bit handling tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 23/33] x86, pkeys: dump PKRU with other kernel registers Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/pkeys: Dump PKRU with other kernel registers tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 30/33] x86, fpu: allow setting of XSAVE state Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/fpu: Allow setting of XSAVE state tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 03/33] mm, gup: switch callers of get_user_pages() to not pass tsk/mm Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] mm/gup: Switch all callers of get_user_pages() to not pass tsk/mm tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 19/33] x86, pkeys: check VMAs and PTEs for protection keys Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] mm/gup, x86/mm/pkeys: Check VMAs and PTEs for protection keys tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 26/33] x86, pkeys: actually enable Memory Protection Keys in CPU Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/pkeys: Actually enable Memory Protection Keys in the CPU tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 07/33] x86, pkeys: define new CR4 bit Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/cpu, x86/mm/pkeys: Define new CR4 bit tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 13/33] x86, pkeys: pass VMA down in to fault signal generation code Dave Hansen <dave@sr71.net> - 2016-02-12 22:10 +0100
[tip:mm/pkeys] x86/mm/pkeys: Pass VMA down in to fault signal generation code tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 11/33] x86, pkeys: store protection in high VMA flags Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] mm/core, x86/mm/pkeys: Store protection bits in high VMA flags tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:30 +0100
[PATCH 10/33] x86, pkeys: new page fault error code bit: PF_PK Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/mm/pkeys: Add new 'PF_PK' page fault error code bit tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 09/33] x86, pkeys: PTE bits for storing protection key Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/mm/pkeys: Add PTE bits for storing protection key tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 06/33] x86, pkeys: cpuid bit definition Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/cpufeature, x86/mm/pkeys: Add protection keys related CPUID definitions tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 02/33] mm: overload get_user_pages() functions Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
Re: [PATCH 02/33] mm: overload get_user_pages() functions Ingo Molnar <mingo@kernel.org> - 2016-02-16 09:40 +0100
Re: [PATCH 02/33] mm: overload get_user_pages() functions Dave Hansen <dave@sr71.net> - 2016-02-17 19:20 +0100
[tip:mm/pkeys] mm/gup: Overload get_user_pages() functions tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH 01/33] mm: introduce get_user_pages_remote() Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
Re: [PATCH 01/33] mm: introduce get_user_pages_remote() Balbir Singh <bsingharora@gmail.com> - 2016-02-15 07:10 +0100
Re: [PATCH 01/33] mm: introduce get_user_pages_remote() Dave Hansen <dave@sr71.net> - 2016-02-15 17:30 +0100
Re: [PATCH 01/33] mm: introduce get_user_pages_remote() Srikar Dronamraju <srikar@linux.vnet.ibm.com> - 2016-02-15 07:20 +0100
[tip:x86/pkeys] mm/gup: Introduce get_user_pages_remote() tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-16 13:20 +0100
Re: [tip:x86/pkeys] mm/gup: Introduce get_user_pages_remote() Konstantin Khlebnikov <koct9i@gmail.com> - 2016-02-20 07:30 +0100
[PATCH 05/33] x86, pkeys: Add Kconfig option Dave Hansen <dave@sr71.net> - 2016-02-12 22:20 +0100
[tip:mm/pkeys] x86/mm/pkeys: Add Kconfig option tip-bot for Dave Hansen <tipbot@zytor.com> - 2016-02-18 21:20 +0100
[PATCH] x86/mm/pkeys: Do not enable them by default Borislav Petkov <bp@alien8.de> - 2016-02-19 12:30 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Dave Hansen <dave@sr71.net> - 2016-02-19 18:20 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Borislav Petkov <bp@alien8.de> - 2016-02-19 18:30 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Dave Hansen <dave@sr71.net> - 2016-02-19 18:50 +0100
Re: [PATCH] x86/mm/pkeys: Do not enable them by default Borislav Petkov <bp@alien8.de> - 2016-02-19 19:40 +0100
Re: [PATCH 00/33] x86: Memory Protection Keys (v10) Ingo Molnar <mingo@kernel.org> - 2016-02-16 10:30 +0100
csiph-web