Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1253181

[PATCH v11 09/15] HMM: add mm page table iterator helpers.

Show all headers | View raw

Because inside the mmu_notifier callback we do not have access to the
vma nor do we know which lock we are holding (the mmap semaphore or
the i_mmap_lock) we can not rely on the regular page table walk (nor
do we want as we have to be carefull to not split huge page).

So this patch introduce an helper to iterate of the cpu page table
content in an efficient way for the situation we are in. Which is we
know that none of the page table entry might vanish from below us
and thus it is safe to walk the page table.

The only added value of the iterator is that it keeps the page table
entry level map accross call which fit well with the HMM mirror page
table update code.

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
---
 mm/hmm.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 95 insertions(+)

diff --git a/mm/hmm.c b/mm/hmm.c
index cad25d9..a3815bd 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -403,6 +403,101 @@ static struct mmu_notifier_ops hmm_notifier_ops = {
 };
 
 
+struct mm_pt_iter {
+	struct mm_struct	*mm;
+	pte_t			*ptep;
+	unsigned long		addr;
+};
+
+static void mm_pt_iter_init(struct mm_pt_iter *pt_iter, struct mm_struct *mm)
+{
+	pt_iter->mm = mm;
+	pt_iter->ptep = NULL;
+	pt_iter->addr = -1UL;
+}
+
+static void mm_pt_iter_fini(struct mm_pt_iter *pt_iter)
+{
+	pte_unmap(pt_iter->ptep);
+	pt_iter->ptep = NULL;
+	pt_iter->addr = -1UL;
+	pt_iter->mm = NULL;
+}
+
+static inline bool mm_pt_iter_in_range(struct mm_pt_iter *pt_iter,
+				       unsigned long addr)
+{
+	return (addr >= pt_iter->addr && addr < (pt_iter->addr + PMD_SIZE));
+}
+
+static struct page *mm_pt_iter_page(struct mm_pt_iter *pt_iter,
+				    unsigned long addr)
+{
+	pgd_t *pgdp;
+	pud_t *pudp;
+	pmd_t *pmdp;
+
+again:
+	/*
+	 * What we are doing here is only valid if we old either the mmap
+	 * semaphore or the i_mmap_lock of vma->address_space the address
+	 * belongs to. Sadly because we can not easily get the vma struct
+	 * we can not sanity test that either of those lock is taken.
+	 *
+	 * We have to rely on people using this code knowing what they do.
+	 */
+	if (mm_pt_iter_in_range(pt_iter, addr) && likely(pt_iter->ptep)) {
+		pte_t pte = *(pt_iter->ptep + pte_index(addr));
+		unsigned long pfn;
+
+		if (pte_none(pte) || !pte_present(pte))
+			return NULL;
+		if (unlikely(pte_special(pte)))
+			return NULL;
+
+		pfn = pte_pfn(pte);
+		if (is_zero_pfn(pfn))
+			return NULL;
+		return pfn_to_page(pfn);
+	}
+
+	if (pt_iter->ptep) {
+		pte_unmap(pt_iter->ptep);
+		pt_iter->ptep = NULL;
+		pt_iter->addr = -1UL;
+	}
+
+	pgdp = pgd_offset(pt_iter->mm, addr);
+	if (pgd_none_or_clear_bad(pgdp))
+		return NULL;
+	pudp = pud_offset(pgdp, addr);
+	if (pud_none_or_clear_bad(pudp))
+		return NULL;
+	pmdp = pmd_offset(pudp, addr);
+	/*
+	 * Because we either have the mmap semaphore or the i_mmap_lock we know
+	 * that pmd can not vanish from under us, thus if pmd exist then it is
+	 * either a huge page or a valid pmd. It might also be in the splitting
+	 * transitory state.
+	 */
+	if (pmd_none(*pmdp) || unlikely(pmd_bad(*pmdp)))
+		return NULL;
+	if (pmd_trans_splitting(*pmdp))
+		/*
+		 * FIXME idealy we would wait but we have no easy mean to get a
+		 * hold of the vma. So for now busy loop until the splitting is
+		 * done.
+		 */
+		goto again;
+	if (pmd_huge(*pmdp))
+		return pmd_page(*pmdp) + pte_index(addr);
+	/* Regular pmd and it can not morph. */
+	pt_iter->ptep = pte_offset_map(pmdp, addr & PMD_MASK);
+	pt_iter->addr = addr & PMD_MASK;
+	goto again;
+}
+
+
 /* hmm_mirror - per device mirroring functions.
  *
  * Each device that mirror a process has a uniq hmm_mirror struct. A process
-- 
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH v11 00/15] HMM (Heterogeneous Memory Management) Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 03/15] mmu_notifier: pass page pointer to mmu_notifier_invalidate_page() v2 Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 06/15] HMM: add HMM page table v4. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 07/15] HMM: add per mirror page table v4. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 15/15] HMM: add documentation explaining HMM internals and how to use it. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
    Re: [PATCH v11 15/15] HMM: add documentation explaining HMM internals  and how to use it. Randy Dunlap <rdunlap@infradead.org> - 2015-10-22 05:30 +0200
      Re: [PATCH v11 15/15] HMM: add documentation explaining HMM  internals and how to use it. Jerome Glisse <jglisse@redhat.com> - 2015-10-22 16:20 +0200
      Re: [PATCH v11 15/15] HMM: add documentation explaining HMM  internals and how to use it. David Woodhouse <dwmw2@infradead.org> - 2015-10-28 02:20 +0100
        Re: [PATCH v11 15/15] HMM: add documentation explaining HMM internals  and how to use it. Randy Dunlap <rdunlap@infradead.org> - 2015-10-28 18:20 +0100
  [PATCH v11 04/15] mmu_notifier: allow range invalidation to exclude a specific mmu_notifier Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 05/15] HMM: introduce heterogeneous memory management v5. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
    Re: [PATCH v11 05/15] HMM: introduce heterogeneous memory management  v5. Randy Dunlap <rdunlap@infradead.org> - 2015-10-21 22:20 +0200
  [PATCH v11 11/15] HMM: add discard range helper (to clear and free resources for a range). Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 14/15] HMM: Add support for hugetlb. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 13/15] HMM: DMA map memory on behalf of device driver v2. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 09/15] HMM: add mm page table iterator helpers. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200
  [PATCH v11 08/15] HMM: add device page fault support v6. Jérôme Glisse <jglisse@redhat.com> - 2015-10-21 22:10 +0200

csiph-web