Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1375571 > unrolled thread
| Started by | Mel Gorman <mgorman@techsingularity.net> |
|---|---|
| First post | 2016-04-11 10:20 +0200 |
| Last post | 2016-04-11 10:40 +0200 |
| Articles | 3 on this page of 23 — 1 participant |
Back to article view | Back to linux.kernel
[PATCH 00/21] Optimise page alloc/free fast paths Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 08/22] mm, page_alloc: Convert alloc_flags to unsigned Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 04/22] mm, page_alloc: Inline zone_statistics Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 21/22] cpuset: use static key better and convert to new API Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 11/22] mm, page_alloc: Remove unnecessary initialisation in get_page_from_freelist Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 14/22] mm, page_alloc: Simplify last cpupid reset Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 13/22] mm, page_alloc: Remove redundant check for empty zonelist Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 09/22] mm, page_alloc: Convert nr_fair_skipped to bool Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 15/22] mm, page_alloc: Move might_sleep_if check to the allocator slowpath Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 20/22] mm, page_alloc: Check multiple page fields with a single branch Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 12/22] mm, page_alloc: Remove unnecessary initialisation from __alloc_pages_nodemask() Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 19/22] mm, page_alloc: Avoid looking up the first zone in a zonelist twice Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 05/22] mm, page_alloc: Inline the fast path of the zonelist iterator Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 02/22] mm, page_alloc: Use new PageAnonHead helper in the free page fast path Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 16/22] mm, page_alloc: Move __GFP_HARDWALL modifications out of the fastpath Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 17/22] mm, page_alloc: Reduce cost of fair zone allocation policy retry Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:20 +0200
[PATCH 18/22] mm, page_alloc: Shortcut watermark checks for order-0 pages Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:30 +0200
Re: [PATCH 00/21] Optimise page alloc/free fast paths Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:30 +0200
[PATCH 06/22] mm, page_alloc: Use __dec_zone_state for order-0 page allocation Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:30 +0200
[PATCH 10/22] mm, page_alloc: Remove unnecessary local variable in get_page_from_freelist Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:30 +0200
[PATCH 07/22] mm, page_alloc: Avoid unnecessary zone lookups during pageblock operations Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:30 +0200
[PATCH 01/22] mm, page_alloc: Only check PageCompound for high-order pages Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:40 +0200
[PATCH 03/22] mm, page_alloc: Reduce branches in zone_statistics Mel Gorman <mgorman@techsingularity.net> - 2016-04-11 10:40 +0200
Page 2 of 2 — ← Prev page 1 [2]
| From | Mel Gorman <mgorman@techsingularity.net> |
|---|---|
| Date | 2016-04-11 10:30 +0200 |
| Subject | [PATCH 07/22] mm, page_alloc: Avoid unnecessary zone lookups during pageblock operations |
| Message-ID | <rmFon-2Ng-35@gated-at.bofh.it> |
| In reply to | #1375571 |
Pageblocks have an associated bitmap to store migrate types and whether
the pageblock should be skipped during compaction. The bitmap may be
associated with a memory section or a zone but the zone is looked up
unconditionally. The compiler should optimise this away automatically so
this is a cosmetic patch only in many cases.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
mm/page_alloc.c | 22 +++++++++-------------
1 file changed, 9 insertions(+), 13 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index ab16560b76e6..d00847bb1612 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -6759,23 +6759,23 @@ void *__init alloc_large_system_hash(const char *tablename,
}
/* Return a pointer to the bitmap storing bits affecting a block of pages */
-static inline unsigned long *get_pageblock_bitmap(struct zone *zone,
+static inline unsigned long *get_pageblock_bitmap(struct page *page,
unsigned long pfn)
{
#ifdef CONFIG_SPARSEMEM
return __pfn_to_section(pfn)->pageblock_flags;
#else
- return zone->pageblock_flags;
+ return page_zone(page)->pageblock_flags;
#endif /* CONFIG_SPARSEMEM */
}
-static inline int pfn_to_bitidx(struct zone *zone, unsigned long pfn)
+static inline int pfn_to_bitidx(struct page *page, unsigned long pfn)
{
#ifdef CONFIG_SPARSEMEM
pfn &= (PAGES_PER_SECTION-1);
return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS;
#else
- pfn = pfn - round_down(zone->zone_start_pfn, pageblock_nr_pages);
+ pfn = pfn - round_down(page_zone(page)->zone_start_pfn, pageblock_nr_pages);
return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS;
#endif /* CONFIG_SPARSEMEM */
}
@@ -6793,14 +6793,12 @@ unsigned long get_pfnblock_flags_mask(struct page *page, unsigned long pfn,
unsigned long end_bitidx,
unsigned long mask)
{
- struct zone *zone;
unsigned long *bitmap;
unsigned long bitidx, word_bitidx;
unsigned long word;
- zone = page_zone(page);
- bitmap = get_pageblock_bitmap(zone, pfn);
- bitidx = pfn_to_bitidx(zone, pfn);
+ bitmap = get_pageblock_bitmap(page, pfn);
+ bitidx = pfn_to_bitidx(page, pfn);
word_bitidx = bitidx / BITS_PER_LONG;
bitidx &= (BITS_PER_LONG-1);
@@ -6822,20 +6820,18 @@ void set_pfnblock_flags_mask(struct page *page, unsigned long flags,
unsigned long end_bitidx,
unsigned long mask)
{
- struct zone *zone;
unsigned long *bitmap;
unsigned long bitidx, word_bitidx;
unsigned long old_word, word;
BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 4);
- zone = page_zone(page);
- bitmap = get_pageblock_bitmap(zone, pfn);
- bitidx = pfn_to_bitidx(zone, pfn);
+ bitmap = get_pageblock_bitmap(page, pfn);
+ bitidx = pfn_to_bitidx(page, pfn);
word_bitidx = bitidx / BITS_PER_LONG;
bitidx &= (BITS_PER_LONG-1);
- VM_BUG_ON_PAGE(!zone_spans_pfn(zone, pfn), page);
+ VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);
bitidx += end_bitidx;
mask <<= (BITS_PER_LONG - bitidx - 1);
--
2.6.4
[toc] | [prev] | [next] | [standalone]
| From | Mel Gorman <mgorman@techsingularity.net> |
|---|---|
| Date | 2016-04-11 10:40 +0200 |
| Subject | [PATCH 01/22] mm, page_alloc: Only check PageCompound for high-order pages |
| Message-ID | <rmFy2-2Qu-5@gated-at.bofh.it> |
| In reply to | #1375571 |
order-0 pages by definition cannot be compound so avoid the check in the
fast path for those pages.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
mm/page_alloc.c | 25 +++++++++++++++++--------
1 file changed, 17 insertions(+), 8 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 59de90d5d3a3..5d205bcfe10d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1024,24 +1024,33 @@ void __meminit reserve_bootmem_region(unsigned long start, unsigned long end)
static bool free_pages_prepare(struct page *page, unsigned int order)
{
- bool compound = PageCompound(page);
- int i, bad = 0;
+ int bad = 0;
VM_BUG_ON_PAGE(PageTail(page), page);
- VM_BUG_ON_PAGE(compound && compound_order(page) != order, page);
trace_mm_page_free(page, order);
kmemcheck_free_shadow(page, order);
kasan_free_pages(page, order);
+ /*
+ * Check tail pages before head page information is cleared to
+ * avoid checking PageCompound for order-0 pages.
+ */
+ if (order) {
+ bool compound = PageCompound(page);
+ int i;
+
+ VM_BUG_ON_PAGE(compound && compound_order(page) != order, page);
+
+ for (i = 1; i < (1 << order); i++) {
+ if (compound)
+ bad += free_tail_pages_check(page, page + i);
+ bad += free_pages_check(page + i);
+ }
+ }
if (PageAnon(page))
page->mapping = NULL;
bad += free_pages_check(page);
- for (i = 1; i < (1 << order); i++) {
- if (compound)
- bad += free_tail_pages_check(page, page + i);
- bad += free_pages_check(page + i);
- }
if (bad)
return false;
--
2.6.4
[toc] | [prev] | [next] | [standalone]
| From | Mel Gorman <mgorman@techsingularity.net> |
|---|---|
| Date | 2016-04-11 10:40 +0200 |
| Subject | [PATCH 03/22] mm, page_alloc: Reduce branches in zone_statistics |
| Message-ID | <rmFy2-2Qu-15@gated-at.bofh.it> |
| In reply to | #1375571 |
zone_statistics has more branches than it really needs to take an
unlikely GFP flag into account. Reduce the number and annotate
the unlikely flag.
The performance difference on a page allocator microbenchmark is;
4.6.0-rc2 4.6.0-rc2
nocompound-v1r10 statbranch-v1r10
Min alloc-odr0-1 417.00 ( 0.00%) 419.00 ( -0.48%)
Min alloc-odr0-2 308.00 ( 0.00%) 305.00 ( 0.97%)
Min alloc-odr0-4 253.00 ( 0.00%) 250.00 ( 1.19%)
Min alloc-odr0-8 221.00 ( 0.00%) 219.00 ( 0.90%)
Min alloc-odr0-16 205.00 ( 0.00%) 203.00 ( 0.98%)
Min alloc-odr0-32 199.00 ( 0.00%) 195.00 ( 2.01%)
Min alloc-odr0-64 193.00 ( 0.00%) 191.00 ( 1.04%)
Min alloc-odr0-128 191.00 ( 0.00%) 189.00 ( 1.05%)
Min alloc-odr0-256 200.00 ( 0.00%) 198.00 ( 1.00%)
Min alloc-odr0-512 212.00 ( 0.00%) 210.00 ( 0.94%)
Min alloc-odr0-1024 219.00 ( 0.00%) 216.00 ( 1.37%)
Min alloc-odr0-2048 225.00 ( 0.00%) 221.00 ( 1.78%)
Min alloc-odr0-4096 231.00 ( 0.00%) 227.00 ( 1.73%)
Min alloc-odr0-8192 234.00 ( 0.00%) 232.00 ( 0.85%)
Min alloc-odr0-16384 234.00 ( 0.00%) 232.00 ( 0.85%)
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
mm/vmstat.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 5e4300482897..2e58ead9bcf5 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -581,17 +581,21 @@ void drain_zonestat(struct zone *zone, struct per_cpu_pageset *pset)
*/
void zone_statistics(struct zone *preferred_zone, struct zone *z, gfp_t flags)
{
- if (z->zone_pgdat == preferred_zone->zone_pgdat) {
+ int local_nid = numa_node_id();
+ enum zone_stat_item local_stat = NUMA_LOCAL;
+
+ if (unlikely(flags & __GFP_OTHER_NODE)) {
+ local_stat = NUMA_OTHER;
+ local_nid = preferred_zone->node;
+ }
+
+ if (z->node == local_nid) {
__inc_zone_state(z, NUMA_HIT);
+ __inc_zone_state(z, local_stat);
} else {
__inc_zone_state(z, NUMA_MISS);
__inc_zone_state(preferred_zone, NUMA_FOREIGN);
}
- if (z->node == ((flags & __GFP_OTHER_NODE) ?
- preferred_zone->node : numa_node_id()))
- __inc_zone_state(z, NUMA_LOCAL);
- else
- __inc_zone_state(z, NUMA_OTHER);
}
/*
--
2.6.4
[toc] | [prev] | [standalone]
Page 2 of 2 — ← Prev page 1 [2]
Back to top | Article view | linux.kernel
csiph-web