Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1567382
| From | "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCHv6 17/37] fs: make block_read_full_page() be able to read huge page |
| Date | 2017-01-26 13:10 +0100 |
| Message-ID | <t3R2j-79n-35@gated-at.bofh.it> (permalink) |
| References | <t3QSB-6QS-5@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
The approach is straight-forward: for compound pages we read out whole
huge page.
For huge page we cannot have array of buffer head pointers on stack --
it's 4096 pointers on x86-64 -- 'arr' is allocated with kmalloc() for
huge pages.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
fs/buffer.c | 22 +++++++++++++++++-----
include/linux/buffer_head.h | 9 +++++----
include/linux/page-flags.h | 2 +-
3 files changed, 23 insertions(+), 10 deletions(-)
diff --git a/fs/buffer.c b/fs/buffer.c
index 0e87401cf335..72462beca909 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -871,7 +871,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
try_again:
head = NULL;
- offset = PAGE_SIZE;
+ offset = hpage_size(page);
while ((offset -= size) >= 0) {
bh = alloc_buffer_head(GFP_NOFS);
if (!bh)
@@ -1466,7 +1466,7 @@ void set_bh_page(struct buffer_head *bh,
struct page *page, unsigned long offset)
{
bh->b_page = page;
- BUG_ON(offset >= PAGE_SIZE);
+ BUG_ON(offset >= hpage_size(page));
if (PageHighMem(page))
/*
* This catches illegal uses and preserves the offset:
@@ -2280,11 +2280,13 @@ int block_read_full_page(struct page *page, get_block_t *get_block)
{
struct inode *inode = page->mapping->host;
sector_t iblock, lblock;
- struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE];
+ struct buffer_head *arr_on_stack[MAX_BUF_PER_PAGE];
+ struct buffer_head *bh, *head, **arr = arr_on_stack;
unsigned int blocksize, bbits;
int nr, i;
int fully_mapped = 1;
+ VM_BUG_ON_PAGE(PageTail(page), page);
head = create_page_buffers(page, inode, 0);
blocksize = head->b_size;
bbits = block_size_bits(blocksize);
@@ -2295,6 +2297,11 @@ int block_read_full_page(struct page *page, get_block_t *get_block)
nr = 0;
i = 0;
+ if (PageTransHuge(page)) {
+ arr = kmalloc(sizeof(struct buffer_head *) * HPAGE_PMD_NR *
+ MAX_BUF_PER_PAGE, GFP_NOFS);
+ }
+
do {
if (buffer_uptodate(bh))
continue;
@@ -2310,7 +2317,9 @@ int block_read_full_page(struct page *page, get_block_t *get_block)
SetPageError(page);
}
if (!buffer_mapped(bh)) {
- zero_user(page, i * blocksize, blocksize);
+ zero_user(page + (i * blocksize / PAGE_SIZE),
+ i * blocksize % PAGE_SIZE,
+ blocksize);
if (!err)
set_buffer_uptodate(bh);
continue;
@@ -2336,7 +2345,7 @@ int block_read_full_page(struct page *page, get_block_t *get_block)
if (!PageError(page))
SetPageUptodate(page);
unlock_page(page);
- return 0;
+ goto out;
}
/* Stage two: lock the buffers */
@@ -2358,6 +2367,9 @@ int block_read_full_page(struct page *page, get_block_t *get_block)
else
submit_bh(REQ_OP_READ, 0, bh);
}
+out:
+ if (arr != arr_on_stack)
+ kfree(arr);
return 0;
}
EXPORT_SYMBOL(block_read_full_page);
diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h
index fd4134ce9c54..f12f6293ed44 100644
--- a/include/linux/buffer_head.h
+++ b/include/linux/buffer_head.h
@@ -131,13 +131,14 @@ BUFFER_FNS(Meta, meta)
BUFFER_FNS(Prio, prio)
BUFFER_FNS(Defer_Completion, defer_completion)
-#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~PAGE_MASK)
+#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~hpage_mask(bh->b_page))
/* If we *know* page->private refers to buffer_heads */
-#define page_buffers(page) \
+#define page_buffers(__page) \
({ \
- BUG_ON(!PagePrivate(page)); \
- ((struct buffer_head *)page_private(page)); \
+ struct page *p = compound_head(__page); \
+ BUG_ON(!PagePrivate(p)); \
+ ((struct buffer_head *)page_private(p)); \
})
#define page_has_buffers(page) PagePrivate(page)
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index b85b73cfb1b3..23534bd47c08 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -746,7 +746,7 @@ static inline void ClearPageSlabPfmemalloc(struct page *page)
*/
static inline int page_has_private(struct page *page)
{
- return !!(page->flags & PAGE_FLAGS_PRIVATE);
+ return !!(compound_head(page)->flags & PAGE_FLAGS_PRIVATE);
}
#undef PF_ANY
--
2.11.0
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCHv6 00/37] ext4: support of huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
[PATCHv6 03/37] page-flags: relax page flag policy for few flags "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
[PATCHv6 36/37] mm, fs, ext4: expand use of page_mapping() and page_to_pgoff() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
[PATCHv6 06/37] thp: handle write-protection faults for file THP "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
Re: [PATCHv6 06/37] thp: handle write-protection faults for file THP Matthew Wilcox <willy@infradead.org> - 2017-01-26 16:50 +0100
Re: [PATCHv6 06/37] thp: handle write-protection faults for file THP "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-01-26 17:00 +0100
[PATCHv6 09/37] filemap: allocate huge page in pagecache_get_page(), if allowed "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
[PATCHv6 34/37] ext4: make fallocate() operations work with huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
[PATCHv6 31/37] ext4: handle writeback with huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:00 +0100
[PATCHv6 10/37] filemap: handle huge pages in filemap_fdatawait_range() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 15/37] thp: do not threat slab pages as huge in hpage_{nr_pages,size,mask} "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 20/37] truncate: make truncate_inode_pages_range() aware about huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 13/37] mm: make write_cache_pages() work on huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 18/37] fs: make block_write_{begin,end}() be able to handle huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 32/37] ext4: make EXT4_IOC_MOVE_EXT work with huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 37/37] ext4, vfs: add huge= mount option "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 19/37] fs: make block_page_mkwrite() aware about huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 14/37] thp: introduce hpage_size() and hpage_mask() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 02/37] Revert "radix-tree: implement radix_tree_maybe_preload_order()" "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
Re: [PATCHv6 02/37] Revert "radix-tree: implement radix_tree_maybe_preload_order()" Matthew Wilcox <willy@infradead.org> - 2017-01-26 16:40 +0100
[PATCHv6 26/37] ext4: handle huge pages in ext4_page_mkwrite() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 12/37] brd: make it handle huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 27/37] ext4: handle huge pages in __ext4_block_zero_page_range() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 29/37] ext4: handle huge pages in ext4_da_write_end() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 11/37] HACK: readahead: alloc huge pages, if allowed "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 33/37] ext4: fix SEEK_DATA/SEEK_HOLE for huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 24/37] ext4: make ext4_mpage_readpages() hugepage-aware "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 17/37] fs: make block_read_full_page() be able to read huge page "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 35/37] ext4: reserve larger jounral transaction for huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 07/37] filemap: allocate huge page in page_cache_read(), if allowed "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 23/37] mm: account huge pages to dirty, writaback, reclaimable, etc. "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 01/37] mm, shmem: swich huge tmpfs to multi-order radix-tree entries "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 04/37] mm, rmap: account file thp pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 22/37] mm, hugetlb: switch hugetlbfs to multi-order radix-tree entries "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 05/37] thp: try to free page's buffers before attempt split "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 28/37] ext4: make ext4_block_write_begin() aware about huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 08/37] filemap: handle huge pages in do_generic_file_read() "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 16/37] thp: make thp_get_unmapped_area() respect S_HUGE_MODE "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 21/37] truncate: make invalidate_inode_pages2_range() aware about huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
[PATCHv6 25/37] ext4: make ext4_writepage() work on huge pages "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> - 2017-01-26 13:10 +0100
csiph-web