Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1218368

[PATCHv10 05/36] mm: adjust FOLL_SPLIT for new refcounting

Show all headers | View raw

We need to prepare kernel to allow transhuge pages to be mapped with
ptes too. We need to handle FOLL_SPLIT in follow_page_pte().

Also we use split_huge_page() directly instead of split_huge_page_pmd().
split_huge_page_pmd() will gone.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Sasha Levin <sasha.levin@oracle.com>
Tested-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Jerome Marchand <jmarchan@redhat.com>
---
 mm/gup.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 49 insertions(+), 18 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index deafa2c91b36..745a50f2d57d 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -116,6 +116,19 @@ retry:
 		}
 	}
 
+	if (flags & FOLL_SPLIT && PageTransCompound(page)) {
+		int ret;
+		get_page(page);
+		pte_unmap_unlock(ptep, ptl);
+		lock_page(page);
+		ret = split_huge_page(page);
+		unlock_page(page);
+		put_page(page);
+		if (ret)
+			return ERR_PTR(ret);
+		goto retry;
+	}
+
 	if (flags & FOLL_GET)
 		get_page_foll(page);
 	if (flags & FOLL_TOUCH) {
@@ -220,27 +233,45 @@ struct page *follow_page_mask(struct vm_area_struct *vma,
 	}
 	if ((flags & FOLL_NUMA) && pmd_protnone(*pmd))
 		return no_page_table(vma, flags);
-	if (pmd_trans_huge(*pmd)) {
-		if (flags & FOLL_SPLIT) {
+	if (likely(!pmd_trans_huge(*pmd)))
+		return follow_page_pte(vma, address, pmd, flags);
+
+	ptl = pmd_lock(mm, pmd);
+	if (unlikely(!pmd_trans_huge(*pmd))) {
+		spin_unlock(ptl);
+		return follow_page_pte(vma, address, pmd, flags);
+	}
+
+	if (unlikely(pmd_trans_splitting(*pmd))) {
+		spin_unlock(ptl);
+		wait_split_huge_page(vma->anon_vma, pmd);
+		return follow_page_pte(vma, address, pmd, flags);
+	}
+
+	if (flags & FOLL_SPLIT) {
+		int ret;
+		page = pmd_page(*pmd);
+		if (is_huge_zero_page(page)) {
+			spin_unlock(ptl);
+			ret = 0;
 			split_huge_page_pmd(vma, address, pmd);
-			return follow_page_pte(vma, address, pmd, flags);
-		}
-		ptl = pmd_lock(mm, pmd);
-		if (likely(pmd_trans_huge(*pmd))) {
-			if (unlikely(pmd_trans_splitting(*pmd))) {
-				spin_unlock(ptl);
-				wait_split_huge_page(vma->anon_vma, pmd);
-			} else {
-				page = follow_trans_huge_pmd(vma, address,
-							     pmd, flags);
-				spin_unlock(ptl);
-				*page_mask = HPAGE_PMD_NR - 1;
-				return page;
-			}
-		} else
+		} else {
+			get_page(page);
 			spin_unlock(ptl);
+			lock_page(page);
+			ret = split_huge_page(page);
+			unlock_page(page);
+			put_page(page);
+		}
+
+		return ret ? ERR_PTR(ret) :
+			follow_page_pte(vma, address, pmd, flags);
 	}
-	return follow_page_pte(vma, address, pmd, flags);
+
+	page = follow_trans_huge_pmd(vma, address, pmd, flags);
+	spin_unlock(ptl);
+	*page_mask = HPAGE_PMD_NR - 1;
+	return page;
 }
 
 static int get_gate_page(struct mm_struct *mm, unsigned long address,
-- 
2.5.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCHv10 00/36] THP refcounting redesign "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 23/36] tile, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 11/36] mm: temporally mark THP broken "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 12/36] thp: drop all split_huge_page()-related code "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 05/36] mm: adjust FOLL_SPLIT for new refcounting "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 20/36] powerpc, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 31/36] thp, mm: split_huge_page(): caller need to lock page "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 18/36] arm, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 04/36] mm, thp: adjust conditions when we can reuse the page on WP fault "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 16/36] mm, thp: remove compound_lock "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 24/36] x86, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200
  [PATCHv10 22/36] sparc, thp: remove infrastructure for handling splitting PMDs "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2015-09-03 17:30 +0200

csiph-web