Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1500681 > unrolled thread
| Started by | js1304@gmail.com |
|---|---|
| First post | 2016-10-14 05:10 +0200 |
| Last post | 2016-10-14 05:10 +0200 |
| Articles | 4 — 1 participant |
Back to article view | Back to linux.kernel
[PATCH v6 0/6] Introduce ZONE_CMA js1304@gmail.com - 2016-10-14 05:10 +0200
[PATCH v6 1/6] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request js1304@gmail.com - 2016-10-14 05:10 +0200
[PATCH v6 5/6] mm/cma: remove MIGRATE_CMA js1304@gmail.com - 2016-10-14 05:10 +0200
[PATCH v6 6/6] mm/cma: remove per zone CMA stat js1304@gmail.com - 2016-10-14 05:10 +0200
| From | js1304@gmail.com |
|---|---|
| Date | 2016-10-14 05:10 +0200 |
| Subject | [PATCH v6 0/6] Introduce ZONE_CMA |
| Message-ID | <ss12F-45Z-3@gated-at.bofh.it> |
From: Joonsoo Kim <iamjoonsoo.kim@lge.com> Hello, Changes from v5 o Add acked/reviewed-by tag from Vlastimil and Aneesh o Rebase on next-20161013 o Cosmetic change on patch 1 o Optimize span of ZONE_CMA on multiple node system Changes from v4 o Rebase on next-20160825 o Add general fix patch for lowmem reserve o Fix lowmem reserve ratio o Fix zone span optimizaion per Vlastimil o Fix pageset initialization o Change invocation timing on cma_init_reserved_areas() Changes from v3 o Rebase on next-20160805 o Split first patch per Vlastimil o Remove useless function parameter per Vlastimil o Add code comment per Vlastimil o Add following description on cover-letter This is the 4th version of ZONE_CMA patchset. Most of changes are due to rebase and there is no major change. CMA has many problems and I mentioned them on the bottom of the cover letter. These problems comes from limitation of CMA memory that should be always migratable for device usage. I think that introducing a new zone is the best approach to solve them. Here are the reasons. Zone is introduced to solve some issues due to H/W addressing limitation. MM subsystem is implemented to work efficiently with these zones. Allocation/reclaim logic in MM consider this limitation very much. What I did in this patchset is introducing a new zone and extending zone's concept slightly. New concept is that zone can have not only H/W addressing limitation but also S/W limitation to guarantee page migration. This concept is originated from ZONE_MOVABLE and it works well for a long time. So, ZONE_CMA should not be special at this moment. There is a major concern from Mel that ZONE_MOVABLE which has S/W limitation causes highmem/lowmem problem. Highmem/lowmem problem is that some of memory cannot be usable for kernel memory due to limitation of the zone. It causes to break LRU ordering and makes hard to find kernel usable memory when memory pressure. However, important point is that this problem doesn't come from implementation detail (ZONE_MOVABLE/MIGRATETYPE). Even if we implement it by MIGRATETYPE instead of by ZONE_MOVABLE, we cannot use that type of memory for kernel allocation because it isn't migratable. So, it will cause to break LRU ordering, too. We cannot avoid the problem in any case. Therefore, we should focus on which solution is better for maintainance and not intrusive for MM subsystem. In this viewpoint, I think that zone approach is better. As mentioned earlier, MM subsystem already have many infrastructures to deal with zone's H/W addressing limitation. Adding S/W limitation on zone concept and adding a new zone doesn't change anything. It will work by itself. My patchset can remove many hooks related to CMA area management in MM while solving the problems. More hooks are required to solve the problems if we choose MIGRATETYPE approach. Although Mel withdrew the review, Vlastimil expressed an agreement on this new zone approach [6]. "I realize I differ here from much more experienced mm guys, and will probably deservingly regret it later on, but I think that the ZONE_CMA approach could work indeed better than current MIGRATE_CMA pageblocks." If anyone has a different opinion, please let me know. Thanks. Changes from v2 o Rebase on next-20160525 o No other changes except following description There was a discussion with Mel [5] after LSF/MM 2016. I could summarise it to help merge decision but it's better to read by yourself since if I summarise it, it would be biased for me. But, if anyone hope the summary, I will do it. :) Anyway, Mel's position on this patchset seems to be neutral. He saids: "I'm not going to outright NAK your series but I won't ACK it either" We can fix the problems with any approach but I hope to go a new zone approach because it is less error-prone. It reduces some corner case handling for now and remove need for potential corner case handling to fix problems. Note that our company is already using ZONE_CMA and there is no problem. If anyone has a different opinion, please let me know and let's discuss together. Andrew, if there is something to do for merge, please let me know. Changes from v1 o Separate some patches which deserve to submit independently o Modify description to reflect current kernel state (e.g. high-order watermark problem disappeared by Mel's work) o Don't increase SECTION_SIZE_BITS to make a room in page flags (detailed reason is on the patch that adds ZONE_CMA) o Adjust ZONE_CMA population code This series try to solve problems of current CMA implementation. CMA is introduced to provide physically contiguous pages at runtime without exclusive reserved memory area. But, current implementation works like as previous reserved memory approach, because freepages on CMA region are used only if there is no movable freepage. In other words, freepages on CMA region are only used as fallback. In that situation where freepages on CMA region are used as fallback, kswapd would be woken up easily since there is no unmovable and reclaimable freepage, too. If kswapd starts to reclaim memory, fallback allocation to MIGRATE_CMA doesn't occur any more since movable freepages are already refilled by kswapd and then most of freepage on CMA are left to be in free. This situation looks like exclusive reserved memory case. In my experiment, I found that if system memory has 1024 MB memory and 512 MB is reserved for CMA, kswapd is mostly woken up when roughly 512 MB free memory is left. Detailed reason is that for keeping enough free memory for unmovable and reclaimable allocation, kswapd uses below equation when calculating free memory and it easily go under the watermark. Free memory for unmovable and reclaimable = Free total - Free CMA pages This is derivated from the property of CMA freepage that CMA freepage can't be used for unmovable and reclaimable allocation. Anyway, in this case, kswapd are woken up when (FreeTotal - FreeCMA) is lower than low watermark and tries to make free memory until (FreeTotal - FreeCMA) is higher than high watermark. That results in that FreeTotal is moving around 512MB boundary consistently. It then means that we can't utilize full memory capacity. To fix this problem, I submitted some patches [1] about 10 months ago, but, found some more problems to be fixed before solving this problem. It requires many hooks in allocator hotpath so some developers doesn't like it. Instead, some of them suggest different approach [2] to fix all the problems related to CMA, that is, introducing a new zone to deal with free CMA pages. I agree that it is the best way to go so implement here. Although properties of ZONE_MOVABLE and ZONE_CMA is similar, I decide to add a new zone rather than piggyback on ZONE_MOVABLE since they have some differences. First, reserved CMA pages should not be offlined. If freepage for CMA is managed by ZONE_MOVABLE, we need to keep MIGRATE_CMA migratetype and insert many hooks on memory hotplug code to distiguish hotpluggable memory and reserved memory for CMA in the same zone. It would make memory hotplug code which is already complicated more complicated. Second, cma_alloc() can be called more frequently than memory hotplug operation and possibly we need to control allocation rate of ZONE_CMA to optimize latency in the future. In this case, separate zone approach is easy to modify. Third, I'd like to see statistics for CMA, separately. Sometimes, we need to debug why cma_alloc() is failed and separate statistics would be more helpful in this situtaion. Anyway, this patchset solves four problems related to CMA implementation. 1) Utilization problem As mentioned above, we can't utilize full memory capacity due to the limitation of CMA freepage and fallback policy. This patchset implements a new zone for CMA and uses it for GFP_HIGHUSER_MOVABLE request. This typed allocation is used for page cache and anonymous pages which occupies most of memory usage in normal case so we can utilize full memory capacity. Below is the experiment result about this problem. 8 CPUs, 1024 MB, VIRTUAL MACHINE make -j16 <Before this series> CMA reserve: 0 MB 512 MB Elapsed-time: 92.4 186.5 pswpin: 82 18647 pswpout: 160 69839 <After this series> CMA reserve: 0 MB 512 MB Elapsed-time: 93.1 93.4 pswpin: 84 46 pswpout: 183 92 FYI, there is another attempt [3] trying to solve this problem in lkml. And, as far as I know, Qualcomm also has out-of-tree solution for this problem. 2) Reclaim problem Currently, there is no logic to distinguish CMA pages in reclaim path. If reclaim is initiated for unmovable and reclaimable allocation, reclaiming CMA pages doesn't help to satisfy the request and reclaiming CMA page is just waste. By managing CMA pages in the new zone, we can skip to reclaim ZONE_CMA completely if it is unnecessary. 3) Atomic allocation failure problem Kswapd isn't started to reclaim pages when allocation request is movable type and there is enough free page in the CMA region. After bunch of consecutive movable allocation requests, free pages in ordinary region (not CMA region) would be exhausted without waking up kswapd. At that time, if atomic unmovable allocation comes, it can't be successful since there is not enough page in ordinary region. This problem is reported by Aneesh [4] and can be solved by this patchset. 4) Inefficiently work of compaction Usual high-order allocation request is unmovable type and it cannot be serviced from CMA area. In compaction, migration scanner doesn't distinguish migratable pages on the CMA area and do migration. In this case, even if we make high-order page on that region, it cannot be used due to type mismatch. This patch will solve this problem by separating CMA pages from ordinary zones. I passed boot test on x86_64, x86_32, arm and arm64. I did some stress tests on x86_64 and x86_32 and there is no problem. Feel free to enjoy and please give me a feedback. :) This patchset is based on linux-next-20160330. Thanks. [1] https://lkml.org/lkml/2014/5/28/64 [2] https://lkml.org/lkml/2014/11/4/55 [3] https://lkml.org/lkml/2014/10/15/623 [4] http://www.spinics.net/lists/linux-mm/msg100562.html [5] https://lkml.kernel.org/r/20160425053653.GA25662@js1304-P5Q-DELUXE [6] https://lkml.kernel.org/r/1919a85d-6e1e-374f-b8c3-1236c36b0393@suse.cz Joonsoo Kim (6): mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request mm/cma: introduce new zone, ZONE_CMA mm/cma: populate ZONE_CMA mm/cma: remove ALLOC_CMA mm/cma: remove MIGRATE_CMA mm/cma: remove per zone CMA stat arch/powerpc/mm/mmu_context_iommu.c | 2 +- arch/x86/mm/highmem_32.c | 8 ++ fs/proc/meminfo.c | 2 +- include/linux/cma.h | 6 + include/linux/gfp.h | 32 +++--- include/linux/memory_hotplug.h | 3 - include/linux/mempolicy.h | 2 +- include/linux/mm.h | 1 + include/linux/mmzone.h | 58 +++++----- include/linux/page-isolation.h | 5 +- include/linux/vm_event_item.h | 10 +- include/linux/vmstat.h | 8 -- include/trace/events/compaction.h | 10 +- kernel/power/snapshot.c | 8 ++ mm/cma.c | 80 ++++++++++++- mm/compaction.c | 14 +-- mm/hugetlb.c | 2 +- mm/internal.h | 4 +- mm/memory_hotplug.c | 10 +- mm/page_alloc.c | 224 ++++++++++++++++++------------------ mm/page_isolation.c | 15 +-- mm/page_owner.c | 6 +- mm/usercopy.c | 4 +- mm/vmstat.c | 10 +- 24 files changed, 313 insertions(+), 211 deletions(-) -- 1.9.1
[toc] | [next] | [standalone]
| From | js1304@gmail.com |
|---|---|
| Date | 2016-10-14 05:10 +0200 |
| Subject | [PATCH v6 1/6] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <ss12F-45Z-29@gated-at.bofh.it> |
| In reply to | #1500681 |
From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that
important to reserve. When ZONE_MOVABLE is used, this problem would
theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE
allocation request which is mainly used for page cache and anon page
allocation. So, fix it.
And, defining sysctl_lowmem_reserve_ratio array by MAX_NR_ZONES - 1 size
makes code complex. For example, if there is highmem system, following
reserve ratio is activated for *NORMAL ZONE* which would be easyily
misleading people.
#ifdef CONFIG_HIGHMEM
32
#endif
This patch also fix this situation by defining sysctl_lowmem_reserve_ratio
array by MAX_NR_ZONES and place "#ifdef" to right place.
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
---
include/linux/mmzone.h | 2 +-
mm/page_alloc.c | 11 ++++++-----
2 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 7f2ae99..bd30fc1 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -889,7 +889,7 @@ int min_free_kbytes_sysctl_handler(struct ctl_table *, int,
void __user *, size_t *, loff_t *);
int watermark_scale_factor_sysctl_handler(struct ctl_table *, int,
void __user *, size_t *, loff_t *);
-extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1];
+extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES];
int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *, int,
void __user *, size_t *, loff_t *);
int percpu_pagelist_fraction_sysctl_handler(struct ctl_table *, int,
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1790391..92b68cc 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -198,17 +198,18 @@ bool pm_suspended_storage(void)
* TBD: should special case ZONE_DMA32 machines here - in those we normally
* don't need any ZONE_NORMAL reservation
*/
-int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1] = {
+int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES] = {
#ifdef CONFIG_ZONE_DMA
- 256,
+ [ZONE_DMA] = 256,
#endif
#ifdef CONFIG_ZONE_DMA32
- 256,
+ [ZONE_DMA32] = 256,
#endif
+ [ZONE_NORMAL] = 32,
#ifdef CONFIG_HIGHMEM
- 32,
+ [ZONE_HIGHMEM] = INT_MAX,
#endif
- 32,
+ [ZONE_MOVABLE] = INT_MAX,
};
EXPORT_SYMBOL(totalram_pages);
--
1.9.1
[toc] | [prev] | [next] | [standalone]
| From | js1304@gmail.com |
|---|---|
| Date | 2016-10-14 05:10 +0200 |
| Subject | [PATCH v6 5/6] mm/cma: remove MIGRATE_CMA |
| Message-ID | <ss12F-45Z-13@gated-at.bofh.it> |
| In reply to | #1500681 |
From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Now, all reserved pages for CMA region are belong to the ZONE_CMA
and there is no other type of pages. Therefore, we don't need to
use MIGRATE_CMA to distinguish and handle differently for CMA pages
and ordinary pages. Remove MIGRATE_CMA.
Unfortunately, this patch make free CMA counter incorrect because
we count it when pages are on the MIGRATE_CMA. It will be fixed
by next patch. I can squash next patch here but it makes changes
complicated and hard to review so I separate that.
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
---
arch/powerpc/mm/mmu_context_iommu.c | 2 +-
include/linux/gfp.h | 3 +-
include/linux/mmzone.h | 24 ----------
include/linux/page-isolation.h | 5 +--
include/linux/vmstat.h | 8 ----
mm/cma.c | 2 +-
mm/compaction.c | 10 +----
mm/hugetlb.c | 2 +-
mm/memory_hotplug.c | 7 ++-
mm/page_alloc.c | 89 ++++++++++---------------------------
mm/page_isolation.c | 15 +++----
mm/page_owner.c | 6 +--
mm/usercopy.c | 4 +-
13 files changed, 44 insertions(+), 133 deletions(-)
diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c
index e0f1c33..06498b2 100644
--- a/arch/powerpc/mm/mmu_context_iommu.c
+++ b/arch/powerpc/mm/mmu_context_iommu.c
@@ -190,7 +190,7 @@ long mm_iommu_get(unsigned long ua, unsigned long entries,
* of the CMA zone if possible. NOTE: faulting in + migration
* can be expensive. Batching can be considered later
*/
- if (get_pageblock_migratetype(page) == MIGRATE_CMA) {
+ if (is_zone_cma(page_zone(page))) {
if (mm_iommu_move_page_from_cma(page))
goto populate;
if (1 != get_user_pages_fast(ua + (i << PAGE_SHIFT),
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index b86e0c2..815d756 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -553,8 +553,7 @@ static inline bool pm_suspended_storage(void)
#if (defined(CONFIG_MEMORY_ISOLATION) && defined(CONFIG_COMPACTION)) || defined(CONFIG_CMA)
/* The below functions must be run on a range from a single zone. */
-extern int alloc_contig_range(unsigned long start, unsigned long end,
- unsigned migratetype);
+extern int alloc_contig_range(unsigned long start, unsigned long end);
extern void free_contig_range(unsigned long pfn, unsigned nr_pages);
#endif
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 41faf59..dd37da9 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -41,22 +41,6 @@ enum {
MIGRATE_RECLAIMABLE,
MIGRATE_PCPTYPES, /* the number of types on the pcp lists */
MIGRATE_HIGHATOMIC = MIGRATE_PCPTYPES,
-#ifdef CONFIG_CMA
- /*
- * MIGRATE_CMA migration type is designed to mimic the way
- * ZONE_MOVABLE works. Only movable pages can be allocated
- * from MIGRATE_CMA pageblocks and page allocator never
- * implicitly change migration type of MIGRATE_CMA pageblock.
- *
- * The way to use it is to change migratetype of a range of
- * pageblocks to MIGRATE_CMA which can be done by
- * __free_pageblock_cma() function. What is important though
- * is that a range of pageblocks must be aligned to
- * MAX_ORDER_NR_PAGES should biggest page be bigger then
- * a single pageblock.
- */
- MIGRATE_CMA,
-#endif
#ifdef CONFIG_MEMORY_ISOLATION
MIGRATE_ISOLATE, /* can't allocate from here */
#endif
@@ -66,14 +50,6 @@ enum {
/* In mm/page_alloc.c; keep in sync also with show_migration_types() there */
extern char * const migratetype_names[MIGRATE_TYPES];
-#ifdef CONFIG_CMA
-# define is_migrate_cma(migratetype) unlikely((migratetype) == MIGRATE_CMA)
-# define is_migrate_cma_page(_page) (get_pageblock_migratetype(_page) == MIGRATE_CMA)
-#else
-# define is_migrate_cma(migratetype) false
-# define is_migrate_cma_page(_page) false
-#endif
-
#define for_each_migratetype_order(order, type) \
for (order = 0; order < MAX_ORDER; order++) \
for (type = 0; type < MIGRATE_TYPES; type++)
diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h
index 047d647..1db9759 100644
--- a/include/linux/page-isolation.h
+++ b/include/linux/page-isolation.h
@@ -49,15 +49,14 @@ int move_freepages(struct zone *zone,
*/
int
start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
- unsigned migratetype, bool skip_hwpoisoned_pages);
+ bool skip_hwpoisoned_pages);
/*
* Changes MIGRATE_ISOLATE to MIGRATE_MOVABLE.
* target range is [start_pfn, end_pfn)
*/
int
-undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
- unsigned migratetype);
+undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn);
/*
* Test all pages in [start_pfn, end_pfn) are isolated or not.
diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h
index 6137719..ac6db88 100644
--- a/include/linux/vmstat.h
+++ b/include/linux/vmstat.h
@@ -341,14 +341,6 @@ static inline void drain_zonestat(struct zone *zone,
struct per_cpu_pageset *pset) { }
#endif /* CONFIG_SMP */
-static inline void __mod_zone_freepage_state(struct zone *zone, int nr_pages,
- int migratetype)
-{
- __mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
- if (is_migrate_cma(migratetype))
- __mod_zone_page_state(zone, NR_FREE_CMA_PAGES, nr_pages);
-}
-
extern const char * const vmstat_text[];
#endif /* _LINUX_VMSTAT_H */
diff --git a/mm/cma.c b/mm/cma.c
index ba7c340..adc2785 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -457,7 +457,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align)
pfn = cma->base_pfn + (bitmap_no << cma->order_per_bit);
mutex_lock(&cma_mutex);
- ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
+ ret = alloc_contig_range(pfn, pfn + count);
mutex_unlock(&cma_mutex);
if (ret == 0) {
page = pfn_to_page(pfn);
diff --git a/mm/compaction.c b/mm/compaction.c
index 2b31088..6dfd971 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -90,7 +90,7 @@ static void map_pages(struct list_head *list)
static inline bool migrate_async_suitable(int migratetype)
{
- return is_migrate_cma(migratetype) || migratetype == MIGRATE_MOVABLE;
+ return migratetype == MIGRATE_MOVABLE;
}
#ifdef CONFIG_COMPACTION
@@ -1014,7 +1014,7 @@ static bool suitable_migration_target(struct compact_control *cc,
return false;
}
- /* If the block is MIGRATE_MOVABLE or MIGRATE_CMA, allow migration */
+ /* If the block is MIGRATE_MOVABLE, allow migration */
if (migrate_async_suitable(get_pageblock_migratetype(page)))
return true;
@@ -1335,12 +1335,6 @@ static enum compact_result __compact_finished(struct zone *zone, struct compact_
if (!list_empty(&area->free_list[migratetype]))
return COMPACT_SUCCESS;
-#ifdef CONFIG_CMA
- /* MIGRATE_MOVABLE can fallback on MIGRATE_CMA */
- if (migratetype == MIGRATE_MOVABLE &&
- !list_empty(&area->free_list[MIGRATE_CMA]))
- return COMPACT_SUCCESS;
-#endif
/*
* Job done if allocation would steal freepages from
* other migratetype buddy lists.
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index ec49d9e..87b8232 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1051,7 +1051,7 @@ static int __alloc_gigantic_page(unsigned long start_pfn,
unsigned long nr_pages)
{
unsigned long end_pfn = start_pfn + nr_pages;
- return alloc_contig_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
+ return alloc_contig_range(start_pfn, end_pfn);
}
static bool pfn_range_valid_gigantic(struct zone *z,
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index d941b6e..868bbf0 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1891,8 +1891,7 @@ static int __ref __offline_pages(unsigned long start_pfn,
return -EINVAL;
/* set above range as isolated */
- ret = start_isolate_page_range(start_pfn, end_pfn,
- MIGRATE_MOVABLE, true);
+ ret = start_isolate_page_range(start_pfn, end_pfn, true);
if (ret)
return ret;
@@ -1962,7 +1961,7 @@ static int __ref __offline_pages(unsigned long start_pfn,
We cannot do rollback at this point. */
offline_isolated_pages(start_pfn, end_pfn);
/* reset pagetype flags and makes migrate type to be MOVABLE */
- undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
+ undo_isolate_page_range(start_pfn, end_pfn);
/* removal success */
adjust_managed_page_count(pfn_to_page(start_pfn), -offlined_pages);
zone->present_pages -= offlined_pages;
@@ -1999,7 +1998,7 @@ static int __ref __offline_pages(unsigned long start_pfn,
((unsigned long long) end_pfn << PAGE_SHIFT) - 1);
memory_notify(MEM_CANCEL_OFFLINE, &arg);
/* pushback to free area */
- undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
+ undo_isolate_page_range(start_pfn, end_pfn);
return ret;
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a1b3264..6df8533 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -131,8 +131,8 @@
* put on a pcplist. Used to avoid the pageblock migratetype lookup when
* freeing from pcplists in most cases, at the cost of possibly becoming stale.
* Also the migratetype set in the page does not necessarily match the pcplist
- * index, e.g. page might have MIGRATE_CMA set but be on a pcplist with any
- * other index - this ensures that it will be put on the correct CMA freelist.
+ * index, e.g. page might have MIGRATE_MOVABLE set but be on a pcplist with any
+ * other index - this ensures that it will be put on the correct freelist.
*/
static inline int get_pcppage_migratetype(struct page *page)
{
@@ -242,9 +242,6 @@ bool pm_suspended_storage(void)
"Movable",
"Reclaimable",
"HighAtomic",
-#ifdef CONFIG_CMA
- "CMA",
-#endif
#ifdef CONFIG_MEMORY_ISOLATION
"Isolate",
#endif
@@ -676,7 +673,7 @@ static inline bool set_page_guard(struct zone *zone, struct page *page,
INIT_LIST_HEAD(&page->lru);
set_page_private(page, order);
/* Guard pages are not available for any usage */
- __mod_zone_freepage_state(zone, -(1 << order), migratetype);
+ __mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << order));
return true;
}
@@ -697,7 +694,7 @@ static inline void clear_page_guard(struct zone *zone, struct page *page,
set_page_private(page, 0);
if (!is_migrate_isolate(migratetype))
- __mod_zone_freepage_state(zone, (1 << order), migratetype);
+ __mod_zone_page_state(zone, NR_FREE_PAGES, (1 << order));
}
#else
struct page_ext_operations debug_guardpage_ops;
@@ -808,7 +805,7 @@ static inline void __free_one_page(struct page *page,
VM_BUG_ON(migratetype == -1);
if (likely(!is_migrate_isolate(migratetype)))
- __mod_zone_freepage_state(zone, 1 << order, migratetype);
+ __mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
page_idx = pfn & ((1 << MAX_ORDER) - 1);
@@ -1618,7 +1615,7 @@ static void __init adjust_present_page_count(struct page *page, long count)
zone->present_pages += count;
}
-/* Free whole pageblock and set its migration type to MIGRATE_CMA. */
+/* Free whole pageblock and set its migration type to MIGRATE_MOVABLE. */
void __init init_cma_reserved_pageblock(struct page *page)
{
unsigned i = pageblock_nr_pages;
@@ -1643,7 +1640,7 @@ void __init init_cma_reserved_pageblock(struct page *page)
adjust_present_page_count(page, pageblock_nr_pages);
- set_pageblock_migratetype(page, MIGRATE_CMA);
+ set_pageblock_migratetype(page, MIGRATE_MOVABLE);
if (pageblock_order >= MAX_ORDER) {
i = pageblock_nr_pages;
@@ -1870,25 +1867,11 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
[MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE, MIGRATE_TYPES },
[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE, MIGRATE_TYPES },
[MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE, MIGRATE_TYPES },
-#ifdef CONFIG_CMA
- [MIGRATE_CMA] = { MIGRATE_TYPES }, /* Never used */
-#endif
#ifdef CONFIG_MEMORY_ISOLATION
[MIGRATE_ISOLATE] = { MIGRATE_TYPES }, /* Never used */
#endif
};
-#ifdef CONFIG_CMA
-static struct page *__rmqueue_cma_fallback(struct zone *zone,
- unsigned int order)
-{
- return __rmqueue_smallest(zone, order, MIGRATE_CMA);
-}
-#else
-static inline struct page *__rmqueue_cma_fallback(struct zone *zone,
- unsigned int order) { return NULL; }
-#endif
-
/*
* Move the free pages in a range to the free lists of the requested type.
* Note that start_page and end_pages are not aligned on a pageblock
@@ -2093,7 +2076,7 @@ static void reserve_highatomic_pageblock(struct page *page, struct zone *zone,
/* Yoink! */
mt = get_pageblock_migratetype(page);
if (mt != MIGRATE_HIGHATOMIC &&
- !is_migrate_isolate(mt) && !is_migrate_cma(mt)) {
+ !is_migrate_isolate(mt)) {
zone->nr_reserved_highatomic += pageblock_nr_pages;
set_pageblock_migratetype(page, MIGRATE_HIGHATOMIC);
move_freepages_block(zone, page, MIGRATE_HIGHATOMIC);
@@ -2196,9 +2179,7 @@ static void unreserve_highatomic_pageblock(const struct alloc_context *ac)
/*
* The pcppage_migratetype may differ from pageblock's
* migratetype depending on the decisions in
- * find_suitable_fallback(). This is OK as long as it does not
- * differ for MIGRATE_CMA pageblocks. Those can be used as
- * fallback only via special __rmqueue_cma_fallback() function
+ * find_suitable_fallback(). This is OK.
*/
set_pcppage_migratetype(page, start_migratetype);
@@ -2221,13 +2202,8 @@ static struct page *__rmqueue(struct zone *zone, unsigned int order,
struct page *page;
page = __rmqueue_smallest(zone, order, migratetype);
- if (unlikely(!page)) {
- if (migratetype == MIGRATE_MOVABLE)
- page = __rmqueue_cma_fallback(zone, order);
-
- if (!page)
- page = __rmqueue_fallback(zone, order, migratetype);
- }
+ if (unlikely(!page))
+ page = __rmqueue_fallback(zone, order, migratetype);
trace_mm_page_alloc_zone_locked(page, order, migratetype);
return page;
@@ -2267,9 +2243,6 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
else
list_add_tail(&page->lru, list);
list = &page->lru;
- if (is_migrate_cma(get_pcppage_migratetype(page)))
- __mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
- -(1 << order));
}
__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
spin_unlock(&zone->lock);
@@ -2568,7 +2541,7 @@ int __isolate_free_page(struct page *page, unsigned int order)
if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
return 0;
- __mod_zone_freepage_state(zone, -(1UL << order), mt);
+ __mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
}
/* Remove page from free list */
@@ -2584,7 +2557,7 @@ int __isolate_free_page(struct page *page, unsigned int order)
struct page *endpage = page + (1 << order) - 1;
for (; page < endpage; page += pageblock_nr_pages) {
int mt = get_pageblock_migratetype(page);
- if (!is_migrate_isolate(mt) && !is_migrate_cma(mt))
+ if (!is_migrate_isolate(mt))
set_pageblock_migratetype(page,
MIGRATE_MOVABLE);
}
@@ -2684,8 +2657,8 @@ struct page *buffered_rmqueue(struct zone *preferred_zone,
spin_unlock(&zone->lock);
if (!page)
goto failed;
- __mod_zone_freepage_state(zone, -(1 << order),
- get_pcppage_migratetype(page));
+
+ __mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << order));
}
__count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order);
@@ -2835,11 +2808,6 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
if (!list_empty(&area->free_list[mt]))
return true;
}
-
-#ifdef CONFIG_CMA
- if (!list_empty(&area->free_list[MIGRATE_CMA]))
- return true;
-#endif
}
return false;
}
@@ -4235,9 +4203,6 @@ static void show_migration_types(unsigned char type)
[MIGRATE_MOVABLE] = 'M',
[MIGRATE_RECLAIMABLE] = 'E',
[MIGRATE_HIGHATOMIC] = 'H',
-#ifdef CONFIG_CMA
- [MIGRATE_CMA] = 'C',
-#endif
#ifdef CONFIG_MEMORY_ISOLATION
[MIGRATE_ISOLATE] = 'I',
#endif
@@ -7200,7 +7165,7 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
return false;
mt = get_pageblock_migratetype(page);
- if (mt == MIGRATE_MOVABLE || is_migrate_cma(mt))
+ if (mt == MIGRATE_MOVABLE)
return false;
pfn = page_to_pfn(page);
@@ -7348,15 +7313,11 @@ static int __alloc_contig_migrate_range(struct compact_control *cc,
* alloc_contig_range() -- tries to allocate given range of pages
* @start: start PFN to allocate
* @end: one-past-the-last PFN to allocate
- * @migratetype: migratetype of the underlaying pageblocks (either
- * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks
- * in range must have the same migratetype and it must
- * be either of the two.
*
* The PFN range does not have to be pageblock or MAX_ORDER_NR_PAGES
* aligned, however it's the caller's responsibility to guarantee that
* we are the only thread that changes migrate type of pageblocks the
- * pages fall in.
+ * pages fall in and it should be MIGRATE_MOVABLE.
*
* The PFN range must belong to a single zone.
*
@@ -7364,8 +7325,7 @@ static int __alloc_contig_migrate_range(struct compact_control *cc,
* pages which PFN is in [start, end) are allocated for the caller and
* need to be freed with free_contig_range().
*/
-int alloc_contig_range(unsigned long start, unsigned long end,
- unsigned migratetype)
+int alloc_contig_range(unsigned long start, unsigned long end)
{
unsigned long outer_start, outer_end;
unsigned int order;
@@ -7398,15 +7358,14 @@ int alloc_contig_range(unsigned long start, unsigned long end,
* allocator removing them from the buddy system. This way
* page allocator will never consider using them.
*
- * This lets us mark the pageblocks back as
- * MIGRATE_CMA/MIGRATE_MOVABLE so that free pages in the
- * aligned range but not in the unaligned, original range are
- * put back to page allocator so that buddy can use them.
+ * This lets us mark the pageblocks back as MIGRATE_MOVABLE
+ * so that free pages in the aligned range but not in the
+ * unaligned, original range are put back to page allocator
+ * so that buddy can use them.
*/
ret = start_isolate_page_range(pfn_max_align_down(start),
- pfn_max_align_up(end), migratetype,
- false);
+ pfn_max_align_up(end), false);
if (ret)
return ret;
@@ -7484,7 +7443,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
done:
undo_isolate_page_range(pfn_max_align_down(start),
- pfn_max_align_up(end), migratetype);
+ pfn_max_align_up(end));
return ret;
}
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index a5594bf..5882174 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -62,13 +62,12 @@ static int set_migratetype_isolate(struct page *page,
out:
if (!ret) {
unsigned long nr_pages;
- int migratetype = get_pageblock_migratetype(page);
set_pageblock_migratetype(page, MIGRATE_ISOLATE);
zone->nr_isolate_pageblock++;
nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
- __mod_zone_freepage_state(zone, -nr_pages, migratetype);
+ __mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
}
spin_unlock_irqrestore(&zone->lock, flags);
@@ -121,7 +120,7 @@ static void unset_migratetype_isolate(struct page *page, unsigned migratetype)
*/
if (!isolated_page) {
nr_pages = move_freepages_block(zone, page, migratetype);
- __mod_zone_freepage_state(zone, nr_pages, migratetype);
+ __mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
}
set_pageblock_migratetype(page, migratetype);
zone->nr_isolate_pageblock--;
@@ -150,7 +149,6 @@ static void unset_migratetype_isolate(struct page *page, unsigned migratetype)
* to be MIGRATE_ISOLATE.
* @start_pfn: The lower PFN of the range to be isolated.
* @end_pfn: The upper PFN of the range to be isolated.
- * @migratetype: migrate type to set in error recovery.
*
* Making page-allocation-type to be MIGRATE_ISOLATE means free pages in
* the range will never be allocated. Any free pages and pages freed in the
@@ -160,7 +158,7 @@ static void unset_migratetype_isolate(struct page *page, unsigned migratetype)
* Returns 0 on success and -EBUSY if any part of range cannot be isolated.
*/
int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
- unsigned migratetype, bool skip_hwpoisoned_pages)
+ bool skip_hwpoisoned_pages)
{
unsigned long pfn;
unsigned long undo_pfn;
@@ -184,7 +182,7 @@ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
for (pfn = start_pfn;
pfn < undo_pfn;
pfn += pageblock_nr_pages)
- unset_migratetype_isolate(pfn_to_page(pfn), migratetype);
+ unset_migratetype_isolate(pfn_to_page(pfn), MIGRATE_MOVABLE);
return -EBUSY;
}
@@ -192,8 +190,7 @@ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
/*
* Make isolated pages available again.
*/
-int undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
- unsigned migratetype)
+int undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn)
{
unsigned long pfn;
struct page *page;
@@ -207,7 +204,7 @@ int undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
page = __first_valid_page(pfn, pageblock_nr_pages);
if (!page || get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
continue;
- unset_migratetype_isolate(page, migratetype);
+ unset_migratetype_isolate(page, MIGRATE_MOVABLE);
}
return 0;
}
diff --git a/mm/page_owner.c b/mm/page_owner.c
index c3cee24..4016815 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -299,11 +299,7 @@ void pagetypeinfo_showmixedcount_print(struct seq_file *m,
page_mt = gfpflags_to_migratetype(
page_owner->gfp_mask);
if (pageblock_mt != page_mt) {
- if (is_migrate_cma(pageblock_mt))
- count[MIGRATE_MOVABLE]++;
- else
- count[pageblock_mt]++;
-
+ count[pageblock_mt]++;
pfn = block_end_pfn;
break;
}
diff --git a/mm/usercopy.c b/mm/usercopy.c
index 3c8da0a..a16691e 100644
--- a/mm/usercopy.c
+++ b/mm/usercopy.c
@@ -182,7 +182,7 @@ static inline const char *check_page_span(const void *ptr, unsigned long n,
* several independently allocated pages.
*/
is_reserved = PageReserved(page);
- is_cma = is_migrate_cma_page(page);
+ is_cma = is_zone_cma(page_zone(page));
if (!is_reserved && !is_cma)
return "<spans multiple pages>";
@@ -190,7 +190,7 @@ static inline const char *check_page_span(const void *ptr, unsigned long n,
page = virt_to_head_page(ptr);
if (is_reserved && !PageReserved(page))
return "<spans Reserved and non-Reserved pages>";
- if (is_cma && !is_migrate_cma_page(page))
+ if (is_cma && !is_zone_cma(page_zone(page)))
return "<spans CMA and non-CMA pages>";
}
#endif
--
1.9.1
[toc] | [prev] | [next] | [standalone]
| From | js1304@gmail.com |
|---|---|
| Date | 2016-10-14 05:10 +0200 |
| Subject | [PATCH v6 6/6] mm/cma: remove per zone CMA stat |
| Message-ID | <ss12F-45Z-23@gated-at.bofh.it> |
| In reply to | #1500681 |
From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Now, all reserved pages for CMA region are belong to the ZONE_CMA
so we don't need to maintain CMA stat in other zones. Remove it.
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
---
fs/proc/meminfo.c | 2 +-
include/linux/cma.h | 6 ++++++
include/linux/mmzone.h | 1 -
mm/cma.c | 15 +++++++++++++++
mm/page_alloc.c | 7 +++----
mm/vmstat.c | 1 -
6 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index 8a42849..0ca6f38 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -151,7 +151,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
#ifdef CONFIG_CMA
show_val_kb(m, "CmaTotal: ", totalcma_pages);
show_val_kb(m, "CmaFree: ",
- global_page_state(NR_FREE_CMA_PAGES));
+ cma_get_free());
#endif
hugetlb_report_meminfo(m);
diff --git a/include/linux/cma.h b/include/linux/cma.h
index 29f9e77..816290c 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -28,4 +28,10 @@ extern int cma_init_reserved_mem(phys_addr_t base, phys_addr_t size,
struct cma **res_cma);
extern struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align);
extern bool cma_release(struct cma *cma, const struct page *pages, unsigned int count);
+
+#ifdef CONFIG_CMA
+extern unsigned long cma_get_free(void);
+#else
+static inline unsigned long cma_get_free(void) { return 0; }
+#endif
#endif
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index dd37da9..f5cd27a4 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -113,7 +113,6 @@ enum zone_stat_item {
NUMA_LOCAL, /* allocation from local node */
NUMA_OTHER, /* allocation from other node */
#endif
- NR_FREE_CMA_PAGES,
NR_VM_ZONE_STAT_ITEMS };
enum node_stat_item {
diff --git a/mm/cma.c b/mm/cma.c
index adc2785..cd869d7 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -54,6 +54,21 @@ unsigned long cma_get_size(const struct cma *cma)
return cma->count << PAGE_SHIFT;
}
+unsigned long cma_get_free(void)
+{
+ struct zone *zone;
+ unsigned long freecma = 0;
+
+ for_each_populated_zone(zone) {
+ if (!is_zone_cma(zone))
+ continue;
+
+ freecma += zone_page_state(zone, NR_FREE_PAGES);
+ }
+
+ return freecma;
+}
+
static unsigned long cma_bitmap_aligned_mask(const struct cma *cma,
int align_order)
{
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 6df8533..45a35de 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -65,6 +65,7 @@
#include <linux/kthread.h>
#include <linux/memcontrol.h>
#include <linux/random.h>
+#include <linux/cma.h>
#include <asm/sections.h>
#include <asm/tlbflush.h>
@@ -4268,7 +4269,7 @@ void show_free_areas(unsigned int filter)
global_page_state(NR_BOUNCE),
global_page_state(NR_FREE_PAGES),
free_pcp,
- global_page_state(NR_FREE_CMA_PAGES));
+ cma_get_free());
for_each_online_pgdat(pgdat) {
printk("Node %d"
@@ -4349,7 +4350,6 @@ void show_free_areas(unsigned int filter)
" bounce:%lukB"
" free_pcp:%lukB"
" local_pcp:%ukB"
- " free_cma:%lukB"
"\n",
zone->name,
K(zone_page_state(zone, NR_FREE_PAGES)),
@@ -4371,8 +4371,7 @@ void show_free_areas(unsigned int filter)
K(zone_page_state(zone, NR_PAGETABLE)),
K(zone_page_state(zone, NR_BOUNCE)),
K(free_pcp),
- K(this_cpu_read(zone->pageset->pcp.count)),
- K(zone_page_state(zone, NR_FREE_CMA_PAGES)));
+ K(this_cpu_read(zone->pageset->pcp.count)));
printk("lowmem_reserve[]:");
for (i = 0; i < MAX_NR_ZONES; i++)
printk(" %ld", zone->lowmem_reserve[i]);
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 429742f..c1ead13 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -951,7 +951,6 @@ int fragmentation_index(struct zone *zone, unsigned int order)
"numa_local",
"numa_other",
#endif
- "nr_free_cma",
/* Node-based counters */
"nr_inactive_anon",
--
1.9.1
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web