Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1427795 > unrolled thread

[PATCH 00/27] Move LRU page reclaim from zones to nodes v7

Back to article view | Back to linux.kernel

Contents

  [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 07/27] mm, vmscan: Remove balance gap Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 09/27] mm, vmscan: By default have direct reclaim only shrink once per node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 06/27] mm, vmscan: Make kswapd reclaim in terms of nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 02/27] mm, vmscan: Move lru_lock to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 10/27] mm, vmscan: Remove duplicate logic clearing node congestion and dirty state Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 11/27] mm: vmscan: Do not reclaim from kswapd if there is any eligible zone Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
    [PATCH 14/27] mm, workingset: Make working set detection node-aware Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    [PATCH 17/27] mm: Rename NR_ANON_PAGES to NR_ANON_MAPPED Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
      Re: [PATCH 17/27] mm: Rename NR_ANON_PAGES to NR_ANON_MAPPED Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:30 +0200
    [PATCH 12/27] mm, vmscan: Make shrink_node decisions more node-centric Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
      Re: [PATCH 12/27] mm, vmscan: Make shrink_node decisions more  node-centric Michal Hocko <mhocko@kernel.org> - 2016-06-22 15:30 +0200
      Re: [PATCH 12/27] mm, vmscan: Make shrink_node decisions more  node-centric Vlastimil Babka <vbabka@suse.cz> - 2016-06-22 17:50 +0200
    [PATCH 16/27] mm: Move page mapped accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
      Re: [PATCH 16/27] mm: Move page mapped accounting to the node Andrew Morton <akpm@linux-foundation.org> - 2016-06-22 00:50 +0200
        Re: [PATCH 16/27] mm: Move page mapped accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 10:40 +0200
      Re: [PATCH 16/27] mm: Move page mapped accounting to the node Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:40 +0200
    [PATCH 13/27] mm, memcg: Move memcg limit enforcement from zones to nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
      Re: [PATCH 13/27] mm, memcg: Move memcg limit enforcement from zones  to nodes Michal Hocko <mhocko@kernel.org> - 2016-06-22 15:20 +0200
    [PATCH 22/27] mm: Convert zone_reclaim to node_reclaim Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
      Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting  to the node Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:50 +0200
        Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting  to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 16:00 +0200
          Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting to  the node Vlastimil Babka <vbabka@suse.cz> - 2016-06-23 16:10 +0200
            Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting  to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 18:10 +0200
    [PATCH 24/27] mm, page_alloc: Remove fair zone allocation policy Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    [PATCH 18/27] mm: Move most file-based accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
      Re: [PATCH 18/27] mm: Move most file-based accounting to the node Michal Hocko <mhocko@kernel.org> - 2016-06-22 17:30 +0200
    [PATCH 23/27] mm, vmscan: Add classzone information to tracepoints Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
    [PATCH 20/27] mm, vmscan: Update classzone_idx if buffer_heads_over_limit Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
      Re: [PATCH 20/27] mm, vmscan: Update classzone_idx if  buffer_heads_over_limit Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:50 +0200
    [PATCH 27/27] mm: vmstat: Account per-zone stalls and pages skipped during reclaim Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
    [PATCH 26/27] mm: vmstat: Replace __count_zone_vm_events with a zone id equivalent Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
    [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
      Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:30 +0200
        Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:30 +0200
          Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 15:00 +0200
            Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Michal Hocko <mhocko@kernel.org> - 2016-06-23 15:20 +0200
    [PATCH 25/27] mm: page_alloc: Cache the last node whose dirty limit is reached Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:50 +0200
    [PATCH 21/27] mm, vmscan: Only wakeup kswapd once per node for the requested classzone Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:50 +0200
      Re: [PATCH 21/27] mm, vmscan: Only wakeup kswapd once per node for  the requested classzone Vlastimil Babka <vbabka@suse.cz> - 2016-06-22 18:10 +0200
    Re: [PATCH 03/27] mm, vmscan: Move LRU lists to node Vlastimil Babka <vbabka@suse.cz> - 2016-06-22 15:00 +0200
    Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 12:30 +0200
      Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Michal Hocko <mhocko@kernel.org> - 2016-06-23 13:30 +0200
        Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 14:40 +0200
          Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Michal Hocko <mhocko@kernel.org> - 2016-06-23 14:50 +0200

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#1427817 — [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node

As reclaim is now node-based, it follows that page write activity
due to page reclaim should also be accounted for on the node. For
consistency, also account page writes and page dirtying on a per-node
basis.

After this patch, there are a few remaining zone counters that may
appear strange but are fine. NUMA stats are still per-zone as this is a
user-space interface that tools consume. NR_MLOCK, NR_SLAB_*, NR_PAGETABLE,
NR_KERNEL_STACK and NR_BOUNCE are all allocations that potentially pin
low memory and cannot trivially be reclaimed on demand. This information
is still useful for debugging a page allocation failure warning.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/linux/mmzone.h           | 8 ++++----
 include/trace/events/writeback.h | 4 ++--
 mm/page-writeback.c              | 6 +++---
 mm/vmscan.c                      | 4 ++--
 mm/vmstat.c                      | 8 ++++----
 5 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 9924b46e3a13..9c59b8540cb7 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -121,10 +121,6 @@ enum zone_stat_item {
 	NR_KERNEL_STACK,
 	/* Second 128 byte cacheline */
 	NR_BOUNCE,
-	NR_VMSCAN_WRITE,
-	NR_VMSCAN_IMMEDIATE,	/* Prioritise for reclaim when writeback ends */
-	NR_DIRTIED,		/* page dirtyings since bootup */
-	NR_WRITTEN,		/* page writings since bootup */
 #if IS_ENABLED(CONFIG_ZSMALLOC)
 	NR_ZSPAGES,		/* allocated in zsmalloc */
 #endif
@@ -162,6 +158,10 @@ enum node_stat_item {
 	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
 	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
 	NR_UNSTABLE_NFS,	/* NFS unstable pages */
+	NR_VMSCAN_WRITE,
+	NR_VMSCAN_IMMEDIATE,	/* Prioritise for reclaim when writeback ends */
+	NR_DIRTIED,		/* page dirtyings since bootup */
+	NR_WRITTEN,		/* page writings since bootup */
 	NR_VM_NODE_STAT_ITEMS
 };
 
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index c581d9c04ca5..a1002cfede1d 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -415,8 +415,8 @@ TRACE_EVENT(global_dirty_state,
 		__entry->nr_dirty	= global_node_page_state(NR_FILE_DIRTY);
 		__entry->nr_writeback	= global_node_page_state(NR_WRITEBACK);
 		__entry->nr_unstable	= global_node_page_state(NR_UNSTABLE_NFS);
-		__entry->nr_dirtied	= global_page_state(NR_DIRTIED);
-		__entry->nr_written	= global_page_state(NR_WRITTEN);
+		__entry->nr_dirtied	= global_node_page_state(NR_DIRTIED);
+		__entry->nr_written	= global_node_page_state(NR_WRITTEN);
 		__entry->background_thresh = background_thresh;
 		__entry->dirty_thresh	= dirty_thresh;
 		__entry->dirty_limit	= global_wb_domain.dirty_limit;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index aa9fa1eb8b80..95f38a59f9a0 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2433,7 +2433,7 @@ void account_page_dirtied(struct page *page, struct address_space *mapping)
 
 		mem_cgroup_inc_page_stat(page, MEM_CGROUP_STAT_DIRTY);
 		__inc_node_page_state(page, NR_FILE_DIRTY);
-		__inc_zone_page_state(page, NR_DIRTIED);
+		__inc_node_page_state(page, NR_DIRTIED);
 		__inc_wb_stat(wb, WB_RECLAIMABLE);
 		__inc_wb_stat(wb, WB_DIRTIED);
 		task_io_account_write(PAGE_SIZE);
@@ -2521,7 +2521,7 @@ void account_page_redirty(struct page *page)
 
 		wb = unlocked_inode_to_wb_begin(inode, &locked);
 		current->nr_dirtied--;
-		dec_zone_page_state(page, NR_DIRTIED);
+		dec_node_page_state(page, NR_DIRTIED);
 		dec_wb_stat(wb, WB_DIRTIED);
 		unlocked_inode_to_wb_end(inode, locked);
 	}
@@ -2750,7 +2750,7 @@ int test_clear_page_writeback(struct page *page)
 	if (ret) {
 		mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_WRITEBACK);
 		dec_node_page_state(page, NR_WRITEBACK);
-		inc_zone_page_state(page, NR_WRITTEN);
+		inc_node_page_state(page, NR_WRITTEN);
 	}
 	unlock_page_memcg(page);
 	return ret;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4501a9ab1d3f..1f7c1262c0a3 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -612,7 +612,7 @@ static pageout_t pageout(struct page *page, struct address_space *mapping,
 			ClearPageReclaim(page);
 		}
 		trace_mm_vmscan_writepage(page);
-		inc_zone_page_state(page, NR_VMSCAN_WRITE);
+		inc_node_page_state(page, NR_VMSCAN_WRITE);
 		return PAGE_SUCCESS;
 	}
 
@@ -1111,7 +1111,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 				 * except we already have the page isolated
 				 * and know it's dirty
 				 */
-				inc_zone_page_state(page, NR_VMSCAN_IMMEDIATE);
+				inc_node_page_state(page, NR_VMSCAN_IMMEDIATE);
 				SetPageReclaim(page);
 
 				goto keep_locked;
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 32c499251174..b58ceb8c872c 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -929,10 +929,6 @@ const char * const vmstat_text[] = {
 	"nr_page_table_pages",
 	"nr_kernel_stack",
 	"nr_bounce",
-	"nr_vmscan_write",
-	"nr_vmscan_immediate_reclaim",
-	"nr_dirtied",
-	"nr_written",
 #if IS_ENABLED(CONFIG_ZSMALLOC)
 	"nr_zspages",
 #endif
@@ -967,6 +963,10 @@ const char * const vmstat_text[] = {
 	"nr_writeback_temp",
 	"nr_shmem",
 	"nr_unstable",
+	"nr_vmscan_write",
+	"nr_vmscan_immediate_reclaim",
+	"nr_dirtied",
+	"nr_written",
 
 	/* enum writeback_stat_item counters */
 	"nr_dirty_threshold",
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1428868 — Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node

On Tue 21-06-16 15:15:58, Mel Gorman wrote:
> As reclaim is now node-based, it follows that page write activity
> due to page reclaim should also be accounted for on the node. For
> consistency, also account page writes and page dirtying on a per-node
> basis.
> 
> After this patch, there are a few remaining zone counters that may
> appear strange but are fine. NUMA stats are still per-zone as this is a
> user-space interface that tools consume. NR_MLOCK, NR_SLAB_*, NR_PAGETABLE,
> NR_KERNEL_STACK and NR_BOUNCE are all allocations that potentially pin
> low memory and cannot trivially be reclaimed on demand. This information
> is still useful for debugging a page allocation failure warning.

As I've said in other patch. I think we will need to provide
/proc/nodeinfo to fill the gap.

> Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  include/linux/mmzone.h           | 8 ++++----
>  include/trace/events/writeback.h | 4 ++--
>  mm/page-writeback.c              | 6 +++---
>  mm/vmscan.c                      | 4 ++--
>  mm/vmstat.c                      | 8 ++++----
>  5 files changed, 15 insertions(+), 15 deletions(-)
> 
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 9924b46e3a13..9c59b8540cb7 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -121,10 +121,6 @@ enum zone_stat_item {
>  	NR_KERNEL_STACK,
>  	/* Second 128 byte cacheline */
>  	NR_BOUNCE,
> -	NR_VMSCAN_WRITE,
> -	NR_VMSCAN_IMMEDIATE,	/* Prioritise for reclaim when writeback ends */
> -	NR_DIRTIED,		/* page dirtyings since bootup */
> -	NR_WRITTEN,		/* page writings since bootup */
>  #if IS_ENABLED(CONFIG_ZSMALLOC)
>  	NR_ZSPAGES,		/* allocated in zsmalloc */
>  #endif
> @@ -162,6 +158,10 @@ enum node_stat_item {
>  	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
>  	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
>  	NR_UNSTABLE_NFS,	/* NFS unstable pages */
> +	NR_VMSCAN_WRITE,
> +	NR_VMSCAN_IMMEDIATE,	/* Prioritise for reclaim when writeback ends */
> +	NR_DIRTIED,		/* page dirtyings since bootup */
> +	NR_WRITTEN,		/* page writings since bootup */
>  	NR_VM_NODE_STAT_ITEMS
>  };
>  
> diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
> index c581d9c04ca5..a1002cfede1d 100644
> --- a/include/trace/events/writeback.h
> +++ b/include/trace/events/writeback.h
> @@ -415,8 +415,8 @@ TRACE_EVENT(global_dirty_state,
>  		__entry->nr_dirty	= global_node_page_state(NR_FILE_DIRTY);
>  		__entry->nr_writeback	= global_node_page_state(NR_WRITEBACK);
>  		__entry->nr_unstable	= global_node_page_state(NR_UNSTABLE_NFS);
> -		__entry->nr_dirtied	= global_page_state(NR_DIRTIED);
> -		__entry->nr_written	= global_page_state(NR_WRITTEN);
> +		__entry->nr_dirtied	= global_node_page_state(NR_DIRTIED);
> +		__entry->nr_written	= global_node_page_state(NR_WRITTEN);
>  		__entry->background_thresh = background_thresh;
>  		__entry->dirty_thresh	= dirty_thresh;
>  		__entry->dirty_limit	= global_wb_domain.dirty_limit;
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index aa9fa1eb8b80..95f38a59f9a0 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -2433,7 +2433,7 @@ void account_page_dirtied(struct page *page, struct address_space *mapping)
>  
>  		mem_cgroup_inc_page_stat(page, MEM_CGROUP_STAT_DIRTY);
>  		__inc_node_page_state(page, NR_FILE_DIRTY);
> -		__inc_zone_page_state(page, NR_DIRTIED);
> +		__inc_node_page_state(page, NR_DIRTIED);
>  		__inc_wb_stat(wb, WB_RECLAIMABLE);
>  		__inc_wb_stat(wb, WB_DIRTIED);
>  		task_io_account_write(PAGE_SIZE);
> @@ -2521,7 +2521,7 @@ void account_page_redirty(struct page *page)
>  
>  		wb = unlocked_inode_to_wb_begin(inode, &locked);
>  		current->nr_dirtied--;
> -		dec_zone_page_state(page, NR_DIRTIED);
> +		dec_node_page_state(page, NR_DIRTIED);
>  		dec_wb_stat(wb, WB_DIRTIED);
>  		unlocked_inode_to_wb_end(inode, locked);
>  	}
> @@ -2750,7 +2750,7 @@ int test_clear_page_writeback(struct page *page)
>  	if (ret) {
>  		mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_WRITEBACK);
>  		dec_node_page_state(page, NR_WRITEBACK);
> -		inc_zone_page_state(page, NR_WRITTEN);
> +		inc_node_page_state(page, NR_WRITTEN);
>  	}
>  	unlock_page_memcg(page);
>  	return ret;
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 4501a9ab1d3f..1f7c1262c0a3 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -612,7 +612,7 @@ static pageout_t pageout(struct page *page, struct address_space *mapping,
>  			ClearPageReclaim(page);
>  		}
>  		trace_mm_vmscan_writepage(page);
> -		inc_zone_page_state(page, NR_VMSCAN_WRITE);
> +		inc_node_page_state(page, NR_VMSCAN_WRITE);
>  		return PAGE_SUCCESS;
>  	}
>  
> @@ -1111,7 +1111,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>  				 * except we already have the page isolated
>  				 * and know it's dirty
>  				 */
> -				inc_zone_page_state(page, NR_VMSCAN_IMMEDIATE);
> +				inc_node_page_state(page, NR_VMSCAN_IMMEDIATE);
>  				SetPageReclaim(page);
>  
>  				goto keep_locked;
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 32c499251174..b58ceb8c872c 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -929,10 +929,6 @@ const char * const vmstat_text[] = {
>  	"nr_page_table_pages",
>  	"nr_kernel_stack",
>  	"nr_bounce",
> -	"nr_vmscan_write",
> -	"nr_vmscan_immediate_reclaim",
> -	"nr_dirtied",
> -	"nr_written",
>  #if IS_ENABLED(CONFIG_ZSMALLOC)
>  	"nr_zspages",
>  #endif
> @@ -967,6 +963,10 @@ const char * const vmstat_text[] = {
>  	"nr_writeback_temp",
>  	"nr_shmem",
>  	"nr_unstable",
> +	"nr_vmscan_write",
> +	"nr_vmscan_immediate_reclaim",
> +	"nr_dirtied",
> +	"nr_written",
>  
>  	/* enum writeback_stat_item counters */
>  	"nr_dirty_threshold",
> -- 
> 2.6.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1429864 — Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node

On Wed, Jun 22, 2016 at 04:40:39PM +0200, Michal Hocko wrote:
> On Tue 21-06-16 15:15:58, Mel Gorman wrote:
> > As reclaim is now node-based, it follows that page write activity
> > due to page reclaim should also be accounted for on the node. For
> > consistency, also account page writes and page dirtying on a per-node
> > basis.
> > 
> > After this patch, there are a few remaining zone counters that may
> > appear strange but are fine. NUMA stats are still per-zone as this is a
> > user-space interface that tools consume. NR_MLOCK, NR_SLAB_*, NR_PAGETABLE,
> > NR_KERNEL_STACK and NR_BOUNCE are all allocations that potentially pin
> > low memory and cannot trivially be reclaimed on demand. This information
> > is still useful for debugging a page allocation failure warning.
> 
> As I've said in other patch. I think we will need to provide
> /proc/nodeinfo to fill the gap.
> 

I added a patch on top that prints the node stats in zoneinfo but only
once for the first populated zone in a node. Doing this or creating a
new file are both potentially surprising but extending zoneinfo means
there is a greater chance that a user will spot the change.

-- 
Mel Gorman
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1429872 — Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node

On 06/23/2016 03:57 PM, Mel Gorman wrote:
> On Wed, Jun 22, 2016 at 04:40:39PM +0200, Michal Hocko wrote:
>> On Tue 21-06-16 15:15:58, Mel Gorman wrote:
>>> As reclaim is now node-based, it follows that page write activity
>>> due to page reclaim should also be accounted for on the node. For
>>> consistency, also account page writes and page dirtying on a per-node
>>> basis.
>>>
>>> After this patch, there are a few remaining zone counters that may
>>> appear strange but are fine. NUMA stats are still per-zone as this is a
>>> user-space interface that tools consume. NR_MLOCK, NR_SLAB_*, NR_PAGETABLE,
>>> NR_KERNEL_STACK and NR_BOUNCE are all allocations that potentially pin
>>> low memory and cannot trivially be reclaimed on demand. This information
>>> is still useful for debugging a page allocation failure warning.
>>
>> As I've said in other patch. I think we will need to provide
>> /proc/nodeinfo to fill the gap.
>>
>
> I added a patch on top that prints the node stats in zoneinfo but only
> once for the first populated zone in a node. Doing this or creating a
> new file are both potentially surprising but extending zoneinfo means
> there is a greater chance that a user will spot the change.

BTW, there should already be /sys/devices/system/node/nodeX/vmstat 
providing the per-node stats, right?

Changing zoneinfo so that some zones have some stats that others don't 
seems to me like it can break some scripts...

[toc] | [prev] | [next] | [standalone]

#1429947 — Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node

On Thu, Jun 23, 2016 at 04:06:09PM +0200, Vlastimil Babka wrote:
> On 06/23/2016 03:57 PM, Mel Gorman wrote:
> >On Wed, Jun 22, 2016 at 04:40:39PM +0200, Michal Hocko wrote:
> >>On Tue 21-06-16 15:15:58, Mel Gorman wrote:
> >>>As reclaim is now node-based, it follows that page write activity
> >>>due to page reclaim should also be accounted for on the node. For
> >>>consistency, also account page writes and page dirtying on a per-node
> >>>basis.
> >>>
> >>>After this patch, there are a few remaining zone counters that may
> >>>appear strange but are fine. NUMA stats are still per-zone as this is a
> >>>user-space interface that tools consume. NR_MLOCK, NR_SLAB_*, NR_PAGETABLE,
> >>>NR_KERNEL_STACK and NR_BOUNCE are all allocations that potentially pin
> >>>low memory and cannot trivially be reclaimed on demand. This information
> >>>is still useful for debugging a page allocation failure warning.
> >>
> >>As I've said in other patch. I think we will need to provide
> >>/proc/nodeinfo to fill the gap.
> >>
> >
> >I added a patch on top that prints the node stats in zoneinfo but only
> >once for the first populated zone in a node. Doing this or creating a
> >new file are both potentially surprising but extending zoneinfo means
> >there is a greater chance that a user will spot the change.
> 
> BTW, there should already be /sys/devices/system/node/nodeX/vmstat providing
> the per-node stats, right?
> 
> Changing zoneinfo so that some zones have some stats that others don't seems
> to me like it can break some scripts...
> 

I suspect a lot of scripts that read zoneinfo just blindly record it.
Similarly, there is no guarantee that a smart script knows to look in
the per-node vmstat files either. This is a question of "wait see what
breaks".

-- 
Mel Gorman
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1427818 — [PATCH 24/27] mm, page_alloc: Remove fair zone allocation policy

The fair zone allocation policy interleaves allocation requests between
zones to avoid an age inversion problem whereby new pages are reclaimed
to balance a zone. Reclaim is now node-based so this should no longer be
an issue and the fair zone allocation policy is not free. This patch
removes it.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/linux/mmzone.h |  5 ----
 mm/internal.h          |  1 -
 mm/page_alloc.c        | 75 +-------------------------------------------------
 mm/vmstat.c            |  4 +--
 4 files changed, 2 insertions(+), 83 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 79fb9f6efc55..2c331f0afd04 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -110,7 +110,6 @@ struct zone_padding {
 enum zone_stat_item {
 	/* First 128 byte cacheline (assuming 64 bit words) */
 	NR_FREE_PAGES,
-	NR_ALLOC_BATCH,
 	NR_ZONE_LRU_BASE, /* Used only for compaction and reclaim retry */
 	NR_ZONE_LRU_ANON = NR_ZONE_LRU_BASE,
 	NR_ZONE_LRU_FILE,
@@ -512,10 +511,6 @@ struct zone {
 	atomic_long_t		vm_stat[NR_VM_ZONE_STAT_ITEMS];
 } ____cacheline_internodealigned_in_smp;
 
-enum zone_flags {
-	ZONE_FAIR_DEPLETED,		/* fair zone policy batch depleted */
-};
-
 enum pgdat_flags {
 	PGDAT_CONGESTED,		/* zone has many dirty pages backed by
 					 * a congested BDI
diff --git a/mm/internal.h b/mm/internal.h
index 4abb2336e127..863b12ca7aa2 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -469,7 +469,6 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,
 #define ALLOC_HIGH		0x20 /* __GFP_HIGH set */
 #define ALLOC_CPUSET		0x40 /* check for correct cpuset */
 #define ALLOC_CMA		0x80 /* allow allocations from CMA areas */
-#define ALLOC_FAIR		0x100 /* fair zone allocation */
 
 enum ttu_flags;
 struct tlbflush_unmap_batch;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 78e5abc41857..1b5080475481 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2592,7 +2592,6 @@ struct page *buffered_rmqueue(struct zone *preferred_zone,
 			else
 				page = list_first_entry(list, struct page, lru);
 
-			__dec_zone_state(zone, NR_ALLOC_BATCH);
 			list_del(&page->lru);
 			pcp->count--;
 
@@ -2618,15 +2617,10 @@ struct page *buffered_rmqueue(struct zone *preferred_zone,
 		spin_unlock(&zone->lock);
 		if (!page)
 			goto failed;
-		__mod_zone_page_state(zone, NR_ALLOC_BATCH, -(1 << order));
 		__mod_zone_freepage_state(zone, -(1 << order),
 					  get_pcppage_migratetype(page));
 	}
 
-	if (atomic_long_read(&zone->vm_stat[NR_ALLOC_BATCH]) <= 0 &&
-	    !test_bit(ZONE_FAIR_DEPLETED, &zone->flags))
-		set_bit(ZONE_FAIR_DEPLETED, &zone->flags);
-
 	__count_zone_vm_events(PGALLOC, zone, 1 << order);
 	zone_statistics(preferred_zone, zone, gfp_flags);
 	local_irq_restore(flags);
@@ -2837,40 +2831,18 @@ bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
 }
 
 #ifdef CONFIG_NUMA
-static bool zone_local(struct zone *local_zone, struct zone *zone)
-{
-	return local_zone->node == zone->node;
-}
-
 static bool zone_allows_reclaim(struct zone *local_zone, struct zone *zone)
 {
 	return node_distance(zone_to_nid(local_zone), zone_to_nid(zone)) <
 				RECLAIM_DISTANCE;
 }
 #else	/* CONFIG_NUMA */
-static bool zone_local(struct zone *local_zone, struct zone *zone)
-{
-	return true;
-}
-
 static bool zone_allows_reclaim(struct zone *local_zone, struct zone *zone)
 {
 	return true;
 }
 #endif	/* CONFIG_NUMA */
 
-static void reset_alloc_batches(struct zone *preferred_zone)
-{
-	struct zone *zone = preferred_zone->zone_pgdat->node_zones;
-
-	do {
-		mod_zone_page_state(zone, NR_ALLOC_BATCH,
-			high_wmark_pages(zone) - low_wmark_pages(zone) -
-			atomic_long_read(&zone->vm_stat[NR_ALLOC_BATCH]));
-		clear_bit(ZONE_FAIR_DEPLETED, &zone->flags);
-	} while (zone++ != preferred_zone);
-}
-
 /*
  * get_page_from_freelist goes through the zonelist trying to allocate
  * a page.
@@ -2881,10 +2853,6 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 {
 	struct zoneref *z = ac->preferred_zoneref;
 	struct zone *zone;
-	bool fair_skipped = false;
-	bool apply_fair = (alloc_flags & ALLOC_FAIR);
-
-zonelist_scan:
 	/*
 	 * Scan zonelist, looking for a zone with enough free.
 	 * See also __cpuset_node_allowed() comment in kernel/cpuset.c.
@@ -2899,23 +2867,6 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 			!__cpuset_zone_allowed(zone, gfp_mask))
 				continue;
 		/*
-		 * Distribute pages in proportion to the individual
-		 * zone size to ensure fair page aging.  The zone a
-		 * page was allocated in should have no effect on the
-		 * time the page has in memory before being reclaimed.
-		 */
-		if (apply_fair) {
-			if (test_bit(ZONE_FAIR_DEPLETED, &zone->flags)) {
-				fair_skipped = true;
-				continue;
-			}
-			if (!zone_local(ac->preferred_zoneref->zone, zone)) {
-				if (fair_skipped)
-					goto reset_fair;
-				apply_fair = false;
-			}
-		}
-		/*
 		 * When allocating a page cache page for writing, we
 		 * want to get it from a node that is within its dirty
 		 * limit, such that no single node holds more than its
@@ -2986,23 +2937,6 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 		}
 	}
 
-	/*
-	 * The first pass makes sure allocations are spread fairly within the
-	 * local node.  However, the local node might have free pages left
-	 * after the fairness batches are exhausted, and remote zones haven't
-	 * even been considered yet.  Try once more without fairness, and
-	 * include remote zones now, before entering the slowpath and waking
-	 * kswapd: prefer spilling to a remote zone over swapping locally.
-	 */
-	if (fair_skipped) {
-reset_fair:
-		apply_fair = false;
-		fair_skipped = false;
-		reset_alloc_batches(ac->preferred_zoneref->zone);
-		z = ac->preferred_zoneref;
-		goto zonelist_scan;
-	}
-
 	return NULL;
 }
 
@@ -3753,7 +3687,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
 {
 	struct page *page;
 	unsigned int cpuset_mems_cookie;
-	unsigned int alloc_flags = ALLOC_WMARK_LOW|ALLOC_FAIR;
+	unsigned int alloc_flags = ALLOC_WMARK_LOW;
 	gfp_t alloc_mask = gfp_mask; /* The gfp_t that was actually used for allocation */
 	struct alloc_context ac = {
 		.high_zoneidx = gfp_zone(gfp_mask),
@@ -5954,9 +5888,6 @@ static void __paginginit free_area_init_core(struct pglist_data *pgdat)
 		zone_seqlock_init(zone);
 		zone_pcp_init(zone);
 
-		/* For bootup, initialized properly in watermark setup */
-		mod_zone_page_state(zone, NR_ALLOC_BATCH, zone->managed_pages);
-
 		if (!size)
 			continue;
 
@@ -6809,10 +6740,6 @@ static void __setup_per_zone_wmarks(void)
 		zone->watermark[WMARK_LOW]  = min_wmark_pages(zone) + tmp;
 		zone->watermark[WMARK_HIGH] = min_wmark_pages(zone) + tmp * 2;
 
-		__mod_zone_page_state(zone, NR_ALLOC_BATCH,
-			high_wmark_pages(zone) - low_wmark_pages(zone) -
-			atomic_long_read(&zone->vm_stat[NR_ALLOC_BATCH]));
-
 		spin_unlock_irqrestore(&zone->lock, flags);
 	}
 
diff --git a/mm/vmstat.c b/mm/vmstat.c
index b58ceb8c872c..ec96d20eeb52 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -920,7 +920,6 @@ int fragmentation_index(struct zone *zone, unsigned int order)
 const char * const vmstat_text[] = {
 	/* enum zone_stat_item countes */
 	"nr_free_pages",
-	"nr_alloc_batch",
 	"nr_zone_anon_lru",
 	"nr_zone_file_lru",
 	"nr_mlock",
@@ -1626,10 +1625,9 @@ int vmstat_refresh(struct ctl_table *table, int write,
 		val = atomic_long_read(&vm_zone_stat[i]);
 		if (val < 0) {
 			switch (i) {
-			case NR_ALLOC_BATCH:
 			case NR_PAGES_SCANNED:
 				/*
-				 * These are often seen to go negative in
+				 * This is often seen to go negative in
 				 * recent kernels, but not to go permanently
 				 * negative.  Whilst it would be nicer not to
 				 * have exceptions, rooting them out would be
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1427819 — [PATCH 18/27] mm: Move most file-based accounting to the node

There are now a number of accounting oddities such as mapped file pages
being accounted for on the node while the total number of file pages are
accounted on the zone. This can be coped with to some extent but it's
confusing so this patch moves the relevant file-based accounted.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 arch/s390/appldata/appldata_mem.c         |  2 +-
 arch/tile/mm/pgtable.c                    |  8 +++---
 drivers/base/node.c                       | 10 ++++----
 drivers/staging/android/lowmemorykiller.c |  4 +--
 fs/fs-writeback.c                         |  4 +--
 fs/fuse/file.c                            |  8 +++---
 fs/nfs/internal.h                         |  2 +-
 fs/nfs/write.c                            |  2 +-
 fs/proc/meminfo.c                         | 10 ++++----
 include/linux/mmzone.h                    | 12 ++++-----
 include/trace/events/writeback.h          |  6 ++---
 mm/filemap.c                              | 10 ++++----
 mm/migrate.c                              | 12 ++++-----
 mm/page-writeback.c                       | 42 +++++++++++++------------------
 mm/page_alloc.c                           | 34 ++++++++++++-------------
 mm/shmem.c                                | 12 ++++-----
 mm/swap_state.c                           |  4 +--
 mm/util.c                                 |  4 +--
 mm/vmscan.c                               | 16 ++++++------
 mm/vmstat.c                               | 12 ++++-----
 20 files changed, 103 insertions(+), 111 deletions(-)

diff --git a/arch/s390/appldata/appldata_mem.c b/arch/s390/appldata/appldata_mem.c
index edcf2a706942..598df5708501 100644
--- a/arch/s390/appldata/appldata_mem.c
+++ b/arch/s390/appldata/appldata_mem.c
@@ -102,7 +102,7 @@ static void appldata_get_mem_data(void *data)
 	mem_data->totalhigh = P2K(val.totalhigh);
 	mem_data->freehigh  = P2K(val.freehigh);
 	mem_data->bufferram = P2K(val.bufferram);
-	mem_data->cached    = P2K(global_page_state(NR_FILE_PAGES)
+	mem_data->cached    = P2K(global_node_page_state(NR_FILE_PAGES)
 				- val.bufferram);
 
 	si_swapinfo(&val);
diff --git a/arch/tile/mm/pgtable.c b/arch/tile/mm/pgtable.c
index c606b0ef2f7e..7cc6ee7f1a58 100644
--- a/arch/tile/mm/pgtable.c
+++ b/arch/tile/mm/pgtable.c
@@ -49,16 +49,16 @@ void show_mem(unsigned int filter)
 		global_node_page_state(NR_ACTIVE_FILE)),
 	       (global_node_page_state(NR_INACTIVE_ANON) +
 		global_node_page_state(NR_INACTIVE_FILE)),
-	       global_page_state(NR_FILE_DIRTY),
-	       global_page_state(NR_WRITEBACK),
-	       global_page_state(NR_UNSTABLE_NFS),
+	       global_node_page_state(NR_FILE_DIRTY),
+	       global_node_page_state(NR_WRITEBACK),
+	       global_node_page_state(NR_UNSTABLE_NFS),
 	       global_page_state(NR_FREE_PAGES),
 	       (global_page_state(NR_SLAB_RECLAIMABLE) +
 		global_page_state(NR_SLAB_UNRECLAIMABLE)),
 	       global_node_page_state(NR_FILE_MAPPED),
 	       global_page_state(NR_PAGETABLE),
 	       global_page_state(NR_BOUNCE),
-	       global_page_state(NR_FILE_PAGES),
+	       global_node_page_state(NR_FILE_PAGES),
 	       get_nr_swap_pages());
 
 	for_each_zone(zone) {
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 897b6bcb36be..ec733919bc6b 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -116,18 +116,18 @@ static ssize_t node_read_meminfo(struct device *dev,
 		       "Node %d AnonHugePages:  %8lu kB\n"
 #endif
 			,
-		       nid, K(sum_zone_node_page_state(nid, NR_FILE_DIRTY)),
-		       nid, K(sum_zone_node_page_state(nid, NR_WRITEBACK)),
-		       nid, K(sum_zone_node_page_state(nid, NR_FILE_PAGES)),
+		       nid, K(node_page_state(pgdat, NR_FILE_DIRTY)),
+		       nid, K(node_page_state(pgdat, NR_WRITEBACK)),
+		       nid, K(node_page_state(pgdat, NR_FILE_PAGES)),
 		       nid, K(node_page_state(pgdat, NR_FILE_MAPPED)),
 		       nid, K(node_page_state(pgdat, NR_ANON_MAPPED)),
 		       nid, K(i.sharedram),
 		       nid, sum_zone_node_page_state(nid, NR_KERNEL_STACK) *
 				THREAD_SIZE / 1024,
 		       nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)),
-		       nid, K(sum_zone_node_page_state(nid, NR_UNSTABLE_NFS)),
+		       nid, K(node_page_state(pgdat, NR_UNSTABLE_NFS)),
 		       nid, K(sum_zone_node_page_state(nid, NR_BOUNCE)),
-		       nid, K(sum_zone_node_page_state(nid, NR_WRITEBACK_TEMP)),
+		       nid, K(node_page_state(pgdat, NR_WRITEBACK_TEMP)),
 		       nid, K(sum_zone_node_page_state(nid, NR_SLAB_RECLAIMABLE) +
 				sum_zone_node_page_state(nid, NR_SLAB_UNRECLAIMABLE)),
 		       nid, K(sum_zone_node_page_state(nid, NR_SLAB_RECLAIMABLE)),
diff --git a/drivers/staging/android/lowmemorykiller.c b/drivers/staging/android/lowmemorykiller.c
index 93dbcc38eb0f..45a1b4ec4ca3 100644
--- a/drivers/staging/android/lowmemorykiller.c
+++ b/drivers/staging/android/lowmemorykiller.c
@@ -91,8 +91,8 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc)
 	short selected_oom_score_adj;
 	int array_size = ARRAY_SIZE(lowmem_adj);
 	int other_free = global_page_state(NR_FREE_PAGES) - totalreserve_pages;
-	int other_file = global_page_state(NR_FILE_PAGES) -
-						global_page_state(NR_SHMEM) -
+	int other_file = global_node_page_state(NR_FILE_PAGES) -
+						global_node_page_state(NR_SHMEM) -
 						total_swapcache_pages();
 
 	if (lowmem_adj_size < array_size)
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 989a2cef6b76..fd68f8efb440 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -1771,8 +1771,8 @@ static struct wb_writeback_work *get_next_work_item(struct bdi_writeback *wb)
  */
 static unsigned long get_nr_dirty_pages(void)
 {
-	return global_page_state(NR_FILE_DIRTY) +
-		global_page_state(NR_UNSTABLE_NFS) +
+	return global_node_page_state(NR_FILE_DIRTY) +
+		global_node_page_state(NR_UNSTABLE_NFS) +
 		get_nr_dirty_inodes();
 }
 
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 9154f8679024..2382f22a2a8b 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1452,7 +1452,7 @@ static void fuse_writepage_finish(struct fuse_conn *fc, struct fuse_req *req)
 	list_del(&req->writepages_entry);
 	for (i = 0; i < req->num_pages; i++) {
 		dec_wb_stat(&bdi->wb, WB_WRITEBACK);
-		dec_zone_page_state(req->pages[i], NR_WRITEBACK_TEMP);
+		dec_node_page_state(req->pages[i], NR_WRITEBACK_TEMP);
 		wb_writeout_inc(&bdi->wb);
 	}
 	wake_up(&fi->page_waitq);
@@ -1642,7 +1642,7 @@ static int fuse_writepage_locked(struct page *page)
 	req->inode = inode;
 
 	inc_wb_stat(&inode_to_bdi(inode)->wb, WB_WRITEBACK);
-	inc_zone_page_state(tmp_page, NR_WRITEBACK_TEMP);
+	inc_node_page_state(tmp_page, NR_WRITEBACK_TEMP);
 
 	spin_lock(&fc->lock);
 	list_add(&req->writepages_entry, &fi->writepages);
@@ -1756,7 +1756,7 @@ static bool fuse_writepage_in_flight(struct fuse_req *new_req,
 		spin_unlock(&fc->lock);
 
 		dec_wb_stat(&bdi->wb, WB_WRITEBACK);
-		dec_zone_page_state(page, NR_WRITEBACK_TEMP);
+		dec_node_page_state(page, NR_WRITEBACK_TEMP);
 		wb_writeout_inc(&bdi->wb);
 		fuse_writepage_free(fc, new_req);
 		fuse_request_free(new_req);
@@ -1855,7 +1855,7 @@ static int fuse_writepages_fill(struct page *page,
 	req->page_descs[req->num_pages].length = PAGE_SIZE;
 
 	inc_wb_stat(&inode_to_bdi(inode)->wb, WB_WRITEBACK);
-	inc_zone_page_state(tmp_page, NR_WRITEBACK_TEMP);
+	inc_node_page_state(tmp_page, NR_WRITEBACK_TEMP);
 
 	err = 0;
 	if (is_writeback && fuse_writepage_in_flight(req, page)) {
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 6b89fdf2c7fa..722731e16648 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -653,7 +653,7 @@ void nfs_mark_page_unstable(struct page *page, struct nfs_commit_info *cinfo)
 	if (!cinfo->dreq) {
 		struct inode *inode = page_file_mapping(page)->host;
 
-		inc_zone_page_state(page, NR_UNSTABLE_NFS);
+		inc_node_page_state(page, NR_UNSTABLE_NFS);
 		inc_wb_stat(&inode_to_bdi(inode)->wb, WB_RECLAIMABLE);
 		__mark_inode_dirty(inode, I_DIRTY_DATASYNC);
 	}
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 3087fb6f1983..4715549be0c3 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -887,7 +887,7 @@ nfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg,
 static void
 nfs_clear_page_commit(struct page *page)
 {
-	dec_zone_page_state(page, NR_UNSTABLE_NFS);
+	dec_node_page_state(page, NR_UNSTABLE_NFS);
 	dec_wb_stat(&inode_to_bdi(page_file_mapping(page)->host)->wb,
 		    WB_RECLAIMABLE);
 }
diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index 076afb43fc56..6cb9ea36d0fc 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -40,7 +40,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 	si_swapinfo(&i);
 	committed = percpu_counter_read_positive(&vm_committed_as);
 
-	cached = global_page_state(NR_FILE_PAGES) -
+	cached = global_node_page_state(NR_FILE_PAGES) -
 			total_swapcache_pages() - i.bufferram;
 	if (cached < 0)
 		cached = 0;
@@ -136,8 +136,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 #endif
 		K(i.totalswap),
 		K(i.freeswap),
-		K(global_page_state(NR_FILE_DIRTY)),
-		K(global_page_state(NR_WRITEBACK)),
+		K(global_node_page_state(NR_FILE_DIRTY)),
+		K(global_node_page_state(NR_WRITEBACK)),
 		K(global_node_page_state(NR_ANON_MAPPED)),
 		K(global_node_page_state(NR_FILE_MAPPED)),
 		K(i.sharedram),
@@ -150,9 +150,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 #ifdef CONFIG_QUICKLIST
 		K(quicklist_total_size()),
 #endif
-		K(global_page_state(NR_UNSTABLE_NFS)),
+		K(global_node_page_state(NR_UNSTABLE_NFS)),
 		K(global_page_state(NR_BOUNCE)),
-		K(global_page_state(NR_WRITEBACK_TEMP)),
+		K(global_node_page_state(NR_WRITEBACK_TEMP)),
 		K(vm_commit_limit()),
 		K(committed),
 		(unsigned long)VMALLOC_TOTAL >> 10,
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 6b1fea6cde9a..9924b46e3a13 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -115,20 +115,14 @@ enum zone_stat_item {
 	NR_ZONE_LRU_ANON = NR_ZONE_LRU_BASE,
 	NR_ZONE_LRU_FILE,
 	NR_MLOCK,		/* mlock()ed pages found and moved off LRU */
-	NR_FILE_PAGES,
-	NR_FILE_DIRTY,
-	NR_WRITEBACK,
 	NR_SLAB_RECLAIMABLE,
 	NR_SLAB_UNRECLAIMABLE,
 	NR_PAGETABLE,		/* used for pagetables */
 	NR_KERNEL_STACK,
 	/* Second 128 byte cacheline */
-	NR_UNSTABLE_NFS,	/* NFS unstable pages */
 	NR_BOUNCE,
 	NR_VMSCAN_WRITE,
 	NR_VMSCAN_IMMEDIATE,	/* Prioritise for reclaim when writeback ends */
-	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
-	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
 	NR_DIRTIED,		/* page dirtyings since bootup */
 	NR_WRITTEN,		/* page writings since bootup */
 #if IS_ENABLED(CONFIG_ZSMALLOC)
@@ -162,6 +156,12 @@ enum node_stat_item {
 	NR_ANON_MAPPED,	/* Mapped anonymous pages */
 	NR_FILE_MAPPED,	/* pagecache pages mapped into pagetables.
 			   only modified from process context */
+	NR_FILE_PAGES,
+	NR_FILE_DIRTY,
+	NR_WRITEBACK,
+	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
+	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
+	NR_UNSTABLE_NFS,	/* NFS unstable pages */
 	NR_VM_NODE_STAT_ITEMS
 };
 
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index 73614ce1d204..c581d9c04ca5 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -412,9 +412,9 @@ TRACE_EVENT(global_dirty_state,
 	),
 
 	TP_fast_assign(
-		__entry->nr_dirty	= global_page_state(NR_FILE_DIRTY);
-		__entry->nr_writeback	= global_page_state(NR_WRITEBACK);
-		__entry->nr_unstable	= global_page_state(NR_UNSTABLE_NFS);
+		__entry->nr_dirty	= global_node_page_state(NR_FILE_DIRTY);
+		__entry->nr_writeback	= global_node_page_state(NR_WRITEBACK);
+		__entry->nr_unstable	= global_node_page_state(NR_UNSTABLE_NFS);
 		__entry->nr_dirtied	= global_page_state(NR_DIRTIED);
 		__entry->nr_written	= global_page_state(NR_WRITTEN);
 		__entry->background_thresh = background_thresh;
diff --git a/mm/filemap.c b/mm/filemap.c
index d50619adfd7f..b99035dd2288 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -209,9 +209,9 @@ void __delete_from_page_cache(struct page *page, void *shadow)
 
 	/* hugetlb pages do not participate in page cache accounting. */
 	if (!PageHuge(page))
-		__dec_zone_page_state(page, NR_FILE_PAGES);
+		__dec_node_page_state(page, NR_FILE_PAGES);
 	if (PageSwapBacked(page))
-		__dec_zone_page_state(page, NR_SHMEM);
+		__dec_node_page_state(page, NR_SHMEM);
 
 	/*
 	 * At this point page must be either written or cleaned by truncate.
@@ -549,9 +549,9 @@ int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask)
 		 * hugetlb pages do not participate in page cache accounting.
 		 */
 		if (!PageHuge(new))
-			__inc_zone_page_state(new, NR_FILE_PAGES);
+			__inc_node_page_state(new, NR_FILE_PAGES);
 		if (PageSwapBacked(new))
-			__inc_zone_page_state(new, NR_SHMEM);
+			__inc_node_page_state(new, NR_SHMEM);
 		spin_unlock_irqrestore(&mapping->tree_lock, flags);
 		mem_cgroup_migrate(old, new);
 		radix_tree_preload_end();
@@ -658,7 +658,7 @@ static int __add_to_page_cache_locked(struct page *page,
 
 	/* hugetlb pages do not participate in page cache accounting. */
 	if (!huge)
-		__inc_zone_page_state(page, NR_FILE_PAGES);
+		__inc_node_page_state(page, NR_FILE_PAGES);
 	spin_unlock_irq(&mapping->tree_lock);
 	if (!huge)
 		mem_cgroup_commit_charge(page, memcg, false, false);
diff --git a/mm/migrate.c b/mm/migrate.c
index 1582c07205c6..d3fe4cfc2808 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -505,15 +505,15 @@ int migrate_page_move_mapping(struct address_space *mapping,
 	 * are mapped to swap space.
 	 */
 	if (newzone != oldzone) {
-		__dec_zone_state(oldzone, NR_FILE_PAGES);
-		__inc_zone_state(newzone, NR_FILE_PAGES);
+		__dec_node_state(oldzone->zone_pgdat, NR_FILE_PAGES);
+		__inc_node_state(newzone->zone_pgdat, NR_FILE_PAGES);
 		if (PageSwapBacked(page) && !PageSwapCache(page)) {
-			__dec_zone_state(oldzone, NR_SHMEM);
-			__inc_zone_state(newzone, NR_SHMEM);
+			__dec_node_state(oldzone->zone_pgdat, NR_SHMEM);
+			__inc_node_state(newzone->zone_pgdat, NR_SHMEM);
 		}
 		if (dirty && mapping_cap_account_dirty(mapping)) {
-			__dec_zone_state(oldzone, NR_FILE_DIRTY);
-			__inc_zone_state(newzone, NR_FILE_DIRTY);
+			__dec_node_state(oldzone->zone_pgdat, NR_FILE_DIRTY);
+			__inc_node_state(newzone->zone_pgdat, NR_FILE_DIRTY);
 		}
 	}
 	local_irq_enable();
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index a2b24d5ea43a..aa9fa1eb8b80 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -471,20 +471,12 @@ static unsigned long node_dirty_limit(struct pglist_data *pgdat)
  */
 bool node_dirty_ok(struct pglist_data *pgdat)
 {
-	int z;
 	unsigned long limit = node_dirty_limit(pgdat);
 	unsigned long nr_pages = 0;
 
-	for (z = 0; z < MAX_NR_ZONES; z++) {
-		struct zone *zone = pgdat->node_zones + z;
-
-		if (!populated_zone(zone))
-			continue;
-
-		nr_pages += zone_page_state(zone, NR_FILE_DIRTY);
-		nr_pages += zone_page_state(zone, NR_UNSTABLE_NFS);
-		nr_pages += zone_page_state(zone, NR_WRITEBACK);
-	}
+	nr_pages += node_page_state(pgdat, NR_FILE_DIRTY);
+	nr_pages += node_page_state(pgdat, NR_UNSTABLE_NFS);
+	nr_pages += node_page_state(pgdat, NR_WRITEBACK);
 
 	return nr_pages <= limit;
 }
@@ -1574,10 +1566,10 @@ static void balance_dirty_pages(struct address_space *mapping,
 		 * written to the server's write cache, but has not yet
 		 * been flushed to permanent storage.
 		 */
-		nr_reclaimable = global_page_state(NR_FILE_DIRTY) +
-					global_page_state(NR_UNSTABLE_NFS);
+		nr_reclaimable = global_node_page_state(NR_FILE_DIRTY) +
+					global_node_page_state(NR_UNSTABLE_NFS);
 		gdtc->avail = global_dirtyable_memory();
-		gdtc->dirty = nr_reclaimable + global_page_state(NR_WRITEBACK);
+		gdtc->dirty = nr_reclaimable + global_node_page_state(NR_WRITEBACK);
 
 		domain_dirty_limits(gdtc);
 
@@ -1914,8 +1906,8 @@ bool wb_over_bg_thresh(struct bdi_writeback *wb)
 	 * as we're trying to decide whether to put more under writeback.
 	 */
 	gdtc->avail = global_dirtyable_memory();
-	gdtc->dirty = global_page_state(NR_FILE_DIRTY) +
-		      global_page_state(NR_UNSTABLE_NFS);
+	gdtc->dirty = global_node_page_state(NR_FILE_DIRTY) +
+		      global_node_page_state(NR_UNSTABLE_NFS);
 	domain_dirty_limits(gdtc);
 
 	if (gdtc->dirty > gdtc->bg_thresh)
@@ -1959,8 +1951,8 @@ void throttle_vm_writeout(gfp_t gfp_mask)
                  */
                 dirty_thresh += dirty_thresh / 10;      /* wheeee... */
 
-                if (global_page_state(NR_UNSTABLE_NFS) +
-			global_page_state(NR_WRITEBACK) <= dirty_thresh)
+                if (global_node_page_state(NR_UNSTABLE_NFS) +
+			global_node_page_state(NR_WRITEBACK) <= dirty_thresh)
                         	break;
                 congestion_wait(BLK_RW_ASYNC, HZ/10);
 
@@ -1988,8 +1980,8 @@ int dirty_writeback_centisecs_handler(struct ctl_table *table, int write,
 void laptop_mode_timer_fn(unsigned long data)
 {
 	struct request_queue *q = (struct request_queue *)data;
-	int nr_pages = global_page_state(NR_FILE_DIRTY) +
-		global_page_state(NR_UNSTABLE_NFS);
+	int nr_pages = global_node_page_state(NR_FILE_DIRTY) +
+		global_node_page_state(NR_UNSTABLE_NFS);
 	struct bdi_writeback *wb;
 
 	/*
@@ -2440,7 +2432,7 @@ void account_page_dirtied(struct page *page, struct address_space *mapping)
 		wb = inode_to_wb(inode);
 
 		mem_cgroup_inc_page_stat(page, MEM_CGROUP_STAT_DIRTY);
-		__inc_zone_page_state(page, NR_FILE_DIRTY);
+		__inc_node_page_state(page, NR_FILE_DIRTY);
 		__inc_zone_page_state(page, NR_DIRTIED);
 		__inc_wb_stat(wb, WB_RECLAIMABLE);
 		__inc_wb_stat(wb, WB_DIRTIED);
@@ -2461,7 +2453,7 @@ void account_page_cleaned(struct page *page, struct address_space *mapping,
 {
 	if (mapping_cap_account_dirty(mapping)) {
 		mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_DIRTY);
-		dec_zone_page_state(page, NR_FILE_DIRTY);
+		dec_node_page_state(page, NR_FILE_DIRTY);
 		dec_wb_stat(wb, WB_RECLAIMABLE);
 		task_io_account_cancelled_write(PAGE_SIZE);
 	}
@@ -2716,7 +2708,7 @@ int clear_page_dirty_for_io(struct page *page)
 		wb = unlocked_inode_to_wb_begin(inode, &locked);
 		if (TestClearPageDirty(page)) {
 			mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_DIRTY);
-			dec_zone_page_state(page, NR_FILE_DIRTY);
+			dec_node_page_state(page, NR_FILE_DIRTY);
 			dec_wb_stat(wb, WB_RECLAIMABLE);
 			ret = 1;
 		}
@@ -2757,7 +2749,7 @@ int test_clear_page_writeback(struct page *page)
 	}
 	if (ret) {
 		mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_WRITEBACK);
-		dec_zone_page_state(page, NR_WRITEBACK);
+		dec_node_page_state(page, NR_WRITEBACK);
 		inc_zone_page_state(page, NR_WRITTEN);
 	}
 	unlock_page_memcg(page);
@@ -2798,7 +2790,7 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
 	}
 	if (!ret) {
 		mem_cgroup_inc_page_stat(page, MEM_CGROUP_STAT_WRITEBACK);
-		inc_zone_page_state(page, NR_WRITEBACK);
+		inc_node_page_state(page, NR_WRITEBACK);
 	}
 	unlock_page_memcg(page);
 	return ret;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index cf3523f399e5..6f5120a282c3 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4180,7 +4180,7 @@ EXPORT_SYMBOL_GPL(si_mem_available);
 void si_meminfo(struct sysinfo *val)
 {
 	val->totalram = totalram_pages;
-	val->sharedram = global_page_state(NR_SHMEM);
+	val->sharedram = global_node_page_state(NR_SHMEM);
 	val->freeram = global_page_state(NR_FREE_PAGES);
 	val->bufferram = nr_blockdev_pages();
 	val->totalhigh = totalhigh_pages;
@@ -4202,7 +4202,7 @@ void si_meminfo_node(struct sysinfo *val, int nid)
 	for (zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++)
 		managed_pages += pgdat->node_zones[zone_type].managed_pages;
 	val->totalram = managed_pages;
-	val->sharedram = sum_zone_node_page_state(nid, NR_SHMEM);
+	val->sharedram = node_page_state(pgdat, NR_SHMEM);
 	val->freeram = sum_zone_node_page_state(nid, NR_FREE_PAGES);
 #ifdef CONFIG_HIGHMEM
 	for (zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++) {
@@ -4309,13 +4309,13 @@ void show_free_areas(unsigned int filter)
 		global_node_page_state(NR_INACTIVE_FILE),
 		global_node_page_state(NR_ISOLATED_FILE),
 		global_node_page_state(NR_UNEVICTABLE),
-		global_page_state(NR_FILE_DIRTY),
-		global_page_state(NR_WRITEBACK),
-		global_page_state(NR_UNSTABLE_NFS),
+		global_node_page_state(NR_FILE_DIRTY),
+		global_node_page_state(NR_WRITEBACK),
+		global_node_page_state(NR_UNSTABLE_NFS),
 		global_page_state(NR_SLAB_RECLAIMABLE),
 		global_page_state(NR_SLAB_UNRECLAIMABLE),
 		global_node_page_state(NR_FILE_MAPPED),
-		global_page_state(NR_SHMEM),
+		global_node_page_state(NR_SHMEM),
 		global_page_state(NR_PAGETABLE),
 		global_page_state(NR_BOUNCE),
 		global_page_state(NR_FREE_PAGES),
@@ -4332,6 +4332,11 @@ void show_free_areas(unsigned int filter)
 			" isolated(anon):%lukB"
 			" isolated(file):%lukB"
 			" mapped:%lukB"
+			" dirty:%lukB"
+			" writeback:%lukB"
+			" shmem:%lukB"
+			" writeback_tmp:%lukB"
+			" unstable:%lukB"
 			" all_unreclaimable? %s"
 			"\n",
 			pgdat->node_id,
@@ -4343,6 +4348,11 @@ void show_free_areas(unsigned int filter)
 			K(node_page_state(pgdat, NR_ISOLATED_ANON)),
 			K(node_page_state(pgdat, NR_ISOLATED_FILE)),
 			K(node_page_state(pgdat, NR_FILE_MAPPED)),
+			K(node_page_state(pgdat, NR_FILE_DIRTY)),
+			K(node_page_state(pgdat, NR_WRITEBACK)),
+			K(node_page_state(pgdat, NR_SHMEM)),
+			K(node_page_state(pgdat, NR_WRITEBACK_TEMP)),
+			K(node_page_state(pgdat, NR_UNSTABLE_NFS)),
 			!pgdat_reclaimable(pgdat) ? "yes" : "no");
 	}
 
@@ -4365,19 +4375,14 @@ void show_free_areas(unsigned int filter)
 			" present:%lukB"
 			" managed:%lukB"
 			" mlocked:%lukB"
-			" dirty:%lukB"
-			" writeback:%lukB"
-			" shmem:%lukB"
 			" slab_reclaimable:%lukB"
 			" slab_unreclaimable:%lukB"
 			" kernel_stack:%lukB"
 			" pagetables:%lukB"
-			" unstable:%lukB"
 			" bounce:%lukB"
 			" free_pcp:%lukB"
 			" local_pcp:%ukB"
 			" free_cma:%lukB"
-			" writeback_tmp:%lukB"
 			" node_pages_scanned:%lu"
 			"\n",
 			zone->name,
@@ -4388,20 +4393,15 @@ void show_free_areas(unsigned int filter)
 			K(zone->present_pages),
 			K(zone->managed_pages),
 			K(zone_page_state(zone, NR_MLOCK)),
-			K(zone_page_state(zone, NR_FILE_DIRTY)),
-			K(zone_page_state(zone, NR_WRITEBACK)),
-			K(zone_page_state(zone, NR_SHMEM)),
 			K(zone_page_state(zone, NR_SLAB_RECLAIMABLE)),
 			K(zone_page_state(zone, NR_SLAB_UNRECLAIMABLE)),
 			zone_page_state(zone, NR_KERNEL_STACK) *
 				THREAD_SIZE / 1024,
 			K(zone_page_state(zone, NR_PAGETABLE)),
-			K(zone_page_state(zone, NR_UNSTABLE_NFS)),
 			K(zone_page_state(zone, NR_BOUNCE)),
 			K(free_pcp),
 			K(this_cpu_read(zone->pageset->pcp.count)),
 			K(zone_page_state(zone, NR_FREE_CMA_PAGES)),
-			K(zone_page_state(zone, NR_WRITEBACK_TEMP)),
 			K(node_page_state(zone->zone_pgdat, NR_PAGES_SCANNED)));
 		printk("lowmem_reserve[]:");
 		for (i = 0; i < MAX_NR_ZONES; i++)
@@ -4444,7 +4444,7 @@ void show_free_areas(unsigned int filter)
 
 	hugetlb_show_meminfo();
 
-	printk("%ld total pagecache pages\n", global_page_state(NR_FILE_PAGES));
+	printk("%ld total pagecache pages\n", global_node_page_state(NR_FILE_PAGES));
 
 	show_swap_cache_info();
 }
diff --git a/mm/shmem.c b/mm/shmem.c
index 24463b67b6ef..f01d291012d5 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -312,8 +312,8 @@ static int shmem_add_to_page_cache(struct page *page,
 								 page);
 	if (!error) {
 		mapping->nrpages++;
-		__inc_zone_page_state(page, NR_FILE_PAGES);
-		__inc_zone_page_state(page, NR_SHMEM);
+		__inc_node_page_state(page, NR_FILE_PAGES);
+		__inc_node_page_state(page, NR_SHMEM);
 		spin_unlock_irq(&mapping->tree_lock);
 	} else {
 		page->mapping = NULL;
@@ -335,8 +335,8 @@ static void shmem_delete_from_page_cache(struct page *page, void *radswap)
 	error = shmem_radix_tree_replace(mapping, page->index, page, radswap);
 	page->mapping = NULL;
 	mapping->nrpages--;
-	__dec_zone_page_state(page, NR_FILE_PAGES);
-	__dec_zone_page_state(page, NR_SHMEM);
+	__dec_node_page_state(page, NR_FILE_PAGES);
+	__dec_node_page_state(page, NR_SHMEM);
 	spin_unlock_irq(&mapping->tree_lock);
 	put_page(page);
 	BUG_ON(error);
@@ -1084,8 +1084,8 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
 	error = shmem_radix_tree_replace(swap_mapping, swap_index, oldpage,
 								   newpage);
 	if (!error) {
-		__inc_zone_page_state(newpage, NR_FILE_PAGES);
-		__dec_zone_page_state(oldpage, NR_FILE_PAGES);
+		__inc_node_page_state(newpage, NR_FILE_PAGES);
+		__dec_node_page_state(oldpage, NR_FILE_PAGES);
 	}
 	spin_unlock_irq(&swap_mapping->tree_lock);
 
diff --git a/mm/swap_state.c b/mm/swap_state.c
index c99463ac02fb..c8310a37be3a 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -95,7 +95,7 @@ int __add_to_swap_cache(struct page *page, swp_entry_t entry)
 					entry.val, page);
 	if (likely(!error)) {
 		address_space->nrpages++;
-		__inc_zone_page_state(page, NR_FILE_PAGES);
+		__inc_node_page_state(page, NR_FILE_PAGES);
 		INC_CACHE_INFO(add_total);
 	}
 	spin_unlock_irq(&address_space->tree_lock);
@@ -147,7 +147,7 @@ void __delete_from_swap_cache(struct page *page)
 	set_page_private(page, 0);
 	ClearPageSwapCache(page);
 	address_space->nrpages--;
-	__dec_zone_page_state(page, NR_FILE_PAGES);
+	__dec_node_page_state(page, NR_FILE_PAGES);
 	INC_CACHE_INFO(del_total);
 }
 
diff --git a/mm/util.c b/mm/util.c
index b756ee36f7f0..21cbc0fc2fd4 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -522,7 +522,7 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 
 	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
 		free = global_page_state(NR_FREE_PAGES);
-		free += global_page_state(NR_FILE_PAGES);
+		free += global_node_page_state(NR_FILE_PAGES);
 
 		/*
 		 * shmem pages shouldn't be counted as free in this
@@ -530,7 +530,7 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 		 * that won't affect the overall amount of available
 		 * memory in the system.
 		 */
-		free -= global_page_state(NR_SHMEM);
+		free -= global_node_page_state(NR_SHMEM);
 
 		free += get_nr_swap_pages();
 
diff --git a/mm/vmscan.c b/mm/vmscan.c
index c6e958079398..4501a9ab1d3f 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3594,11 +3594,11 @@ int sysctl_min_unmapped_ratio = 1;
  */
 int sysctl_min_slab_ratio = 5;
 
-static inline unsigned long zone_unmapped_file_pages(struct zone *zone)
+static inline unsigned long node_unmapped_file_pages(struct pglist_data *pgdat)
 {
-	unsigned long file_mapped = node_page_state(zone->zone_pgdat, NR_FILE_MAPPED);
-	unsigned long file_lru = node_page_state(zone->zone_pgdat, NR_INACTIVE_FILE) +
-		node_page_state(zone->zone_pgdat, NR_ACTIVE_FILE);
+	unsigned long file_mapped = node_page_state(pgdat, NR_FILE_MAPPED);
+	unsigned long file_lru = node_page_state(pgdat, NR_INACTIVE_FILE) +
+		node_page_state(pgdat, NR_ACTIVE_FILE);
 
 	/*
 	 * It's possible for there to be more file mapped pages than
@@ -3617,17 +3617,17 @@ static unsigned long zone_pagecache_reclaimable(struct zone *zone)
 	/*
 	 * If RECLAIM_UNMAP is set, then all file pages are considered
 	 * potentially reclaimable. Otherwise, we have to worry about
-	 * pages like swapcache and zone_unmapped_file_pages() provides
+	 * pages like swapcache and node_unmapped_file_pages() provides
 	 * a better estimate
 	 */
 	if (zone_reclaim_mode & RECLAIM_UNMAP)
-		nr_pagecache_reclaimable = zone_page_state(zone, NR_FILE_PAGES);
+		nr_pagecache_reclaimable = node_page_state(zone->zone_pgdat, NR_FILE_PAGES);
 	else
-		nr_pagecache_reclaimable = zone_unmapped_file_pages(zone);
+		nr_pagecache_reclaimable = node_unmapped_file_pages(zone->zone_pgdat);
 
 	/* If we can't clean pages, remove dirty pages from consideration */
 	if (!(zone_reclaim_mode & RECLAIM_WRITE))
-		delta += zone_page_state(zone, NR_FILE_DIRTY);
+		delta += node_page_state(zone->zone_pgdat, NR_FILE_DIRTY);
 
 	/* Watch for any possible underflows due to delta */
 	if (unlikely(delta > nr_pagecache_reclaimable))
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 12022ed481f0..32c499251174 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -924,19 +924,13 @@ const char * const vmstat_text[] = {
 	"nr_zone_anon_lru",
 	"nr_zone_file_lru",
 	"nr_mlock",
-	"nr_file_pages",
-	"nr_dirty",
-	"nr_writeback",
 	"nr_slab_reclaimable",
 	"nr_slab_unreclaimable",
 	"nr_page_table_pages",
 	"nr_kernel_stack",
-	"nr_unstable",
 	"nr_bounce",
 	"nr_vmscan_write",
 	"nr_vmscan_immediate_reclaim",
-	"nr_writeback_temp",
-	"nr_shmem",
 	"nr_dirtied",
 	"nr_written",
 #if IS_ENABLED(CONFIG_ZSMALLOC)
@@ -967,6 +961,12 @@ const char * const vmstat_text[] = {
 	"workingset_nodereclaim",
 	"nr_anon_pages",
 	"nr_mapped",
+	"nr_file_pages",
+	"nr_dirty",
+	"nr_writeback",
+	"nr_writeback_temp",
+	"nr_shmem",
+	"nr_unstable",
 
 	/* enum writeback_stat_item counters */
 	"nr_dirty_threshold",
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1428900 — Re: [PATCH 18/27] mm: Move most file-based accounting to the node

On Tue 21-06-16 15:15:57, Mel Gorman wrote:
> There are now a number of accounting oddities such as mapped file pages
> being accounted for on the node while the total number of file pages are
> accounted on the zone. This can be coped with to some extent but it's
> confusing so this patch moves the relevant file-based accounted.

Same concern about the zoneinfo as for the other patch, but other than
that no issues spotted.
 
> Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  arch/s390/appldata/appldata_mem.c         |  2 +-
>  arch/tile/mm/pgtable.c                    |  8 +++---
>  drivers/base/node.c                       | 10 ++++----
>  drivers/staging/android/lowmemorykiller.c |  4 +--
>  fs/fs-writeback.c                         |  4 +--
>  fs/fuse/file.c                            |  8 +++---
>  fs/nfs/internal.h                         |  2 +-
>  fs/nfs/write.c                            |  2 +-
>  fs/proc/meminfo.c                         | 10 ++++----
>  include/linux/mmzone.h                    | 12 ++++-----
>  include/trace/events/writeback.h          |  6 ++---
>  mm/filemap.c                              | 10 ++++----
>  mm/migrate.c                              | 12 ++++-----
>  mm/page-writeback.c                       | 42 +++++++++++++------------------
>  mm/page_alloc.c                           | 34 ++++++++++++-------------
>  mm/shmem.c                                | 12 ++++-----
>  mm/swap_state.c                           |  4 +--
>  mm/util.c                                 |  4 +--
>  mm/vmscan.c                               | 16 ++++++------
>  mm/vmstat.c                               | 12 ++++-----
>  20 files changed, 103 insertions(+), 111 deletions(-)
> 
> diff --git a/arch/s390/appldata/appldata_mem.c b/arch/s390/appldata/appldata_mem.c
> index edcf2a706942..598df5708501 100644
> --- a/arch/s390/appldata/appldata_mem.c
> +++ b/arch/s390/appldata/appldata_mem.c
> @@ -102,7 +102,7 @@ static void appldata_get_mem_data(void *data)
>  	mem_data->totalhigh = P2K(val.totalhigh);
>  	mem_data->freehigh  = P2K(val.freehigh);
>  	mem_data->bufferram = P2K(val.bufferram);
> -	mem_data->cached    = P2K(global_page_state(NR_FILE_PAGES)
> +	mem_data->cached    = P2K(global_node_page_state(NR_FILE_PAGES)
>  				- val.bufferram);
>  
>  	si_swapinfo(&val);
> diff --git a/arch/tile/mm/pgtable.c b/arch/tile/mm/pgtable.c
> index c606b0ef2f7e..7cc6ee7f1a58 100644
> --- a/arch/tile/mm/pgtable.c
> +++ b/arch/tile/mm/pgtable.c
> @@ -49,16 +49,16 @@ void show_mem(unsigned int filter)
>  		global_node_page_state(NR_ACTIVE_FILE)),
>  	       (global_node_page_state(NR_INACTIVE_ANON) +
>  		global_node_page_state(NR_INACTIVE_FILE)),
> -	       global_page_state(NR_FILE_DIRTY),
> -	       global_page_state(NR_WRITEBACK),
> -	       global_page_state(NR_UNSTABLE_NFS),
> +	       global_node_page_state(NR_FILE_DIRTY),
> +	       global_node_page_state(NR_WRITEBACK),
> +	       global_node_page_state(NR_UNSTABLE_NFS),
>  	       global_page_state(NR_FREE_PAGES),
>  	       (global_page_state(NR_SLAB_RECLAIMABLE) +
>  		global_page_state(NR_SLAB_UNRECLAIMABLE)),
>  	       global_node_page_state(NR_FILE_MAPPED),
>  	       global_page_state(NR_PAGETABLE),
>  	       global_page_state(NR_BOUNCE),
> -	       global_page_state(NR_FILE_PAGES),
> +	       global_node_page_state(NR_FILE_PAGES),
>  	       get_nr_swap_pages());
>  
>  	for_each_zone(zone) {
> diff --git a/drivers/base/node.c b/drivers/base/node.c
> index 897b6bcb36be..ec733919bc6b 100644
> --- a/drivers/base/node.c
> +++ b/drivers/base/node.c
> @@ -116,18 +116,18 @@ static ssize_t node_read_meminfo(struct device *dev,
>  		       "Node %d AnonHugePages:  %8lu kB\n"
>  #endif
>  			,
> -		       nid, K(sum_zone_node_page_state(nid, NR_FILE_DIRTY)),
> -		       nid, K(sum_zone_node_page_state(nid, NR_WRITEBACK)),
> -		       nid, K(sum_zone_node_page_state(nid, NR_FILE_PAGES)),
> +		       nid, K(node_page_state(pgdat, NR_FILE_DIRTY)),
> +		       nid, K(node_page_state(pgdat, NR_WRITEBACK)),
> +		       nid, K(node_page_state(pgdat, NR_FILE_PAGES)),
>  		       nid, K(node_page_state(pgdat, NR_FILE_MAPPED)),
>  		       nid, K(node_page_state(pgdat, NR_ANON_MAPPED)),
>  		       nid, K(i.sharedram),
>  		       nid, sum_zone_node_page_state(nid, NR_KERNEL_STACK) *
>  				THREAD_SIZE / 1024,
>  		       nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)),
> -		       nid, K(sum_zone_node_page_state(nid, NR_UNSTABLE_NFS)),
> +		       nid, K(node_page_state(pgdat, NR_UNSTABLE_NFS)),
>  		       nid, K(sum_zone_node_page_state(nid, NR_BOUNCE)),
> -		       nid, K(sum_zone_node_page_state(nid, NR_WRITEBACK_TEMP)),
> +		       nid, K(node_page_state(pgdat, NR_WRITEBACK_TEMP)),
>  		       nid, K(sum_zone_node_page_state(nid, NR_SLAB_RECLAIMABLE) +
>  				sum_zone_node_page_state(nid, NR_SLAB_UNRECLAIMABLE)),
>  		       nid, K(sum_zone_node_page_state(nid, NR_SLAB_RECLAIMABLE)),
> diff --git a/drivers/staging/android/lowmemorykiller.c b/drivers/staging/android/lowmemorykiller.c
> index 93dbcc38eb0f..45a1b4ec4ca3 100644
> --- a/drivers/staging/android/lowmemorykiller.c
> +++ b/drivers/staging/android/lowmemorykiller.c
> @@ -91,8 +91,8 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc)
>  	short selected_oom_score_adj;
>  	int array_size = ARRAY_SIZE(lowmem_adj);
>  	int other_free = global_page_state(NR_FREE_PAGES) - totalreserve_pages;
> -	int other_file = global_page_state(NR_FILE_PAGES) -
> -						global_page_state(NR_SHMEM) -
> +	int other_file = global_node_page_state(NR_FILE_PAGES) -
> +						global_node_page_state(NR_SHMEM) -
>  						total_swapcache_pages();
>  
>  	if (lowmem_adj_size < array_size)
> diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
> index 989a2cef6b76..fd68f8efb440 100644
> --- a/fs/fs-writeback.c
> +++ b/fs/fs-writeback.c
> @@ -1771,8 +1771,8 @@ static struct wb_writeback_work *get_next_work_item(struct bdi_writeback *wb)
>   */
>  static unsigned long get_nr_dirty_pages(void)
>  {
> -	return global_page_state(NR_FILE_DIRTY) +
> -		global_page_state(NR_UNSTABLE_NFS) +
> +	return global_node_page_state(NR_FILE_DIRTY) +
> +		global_node_page_state(NR_UNSTABLE_NFS) +
>  		get_nr_dirty_inodes();
>  }
>  
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index 9154f8679024..2382f22a2a8b 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -1452,7 +1452,7 @@ static void fuse_writepage_finish(struct fuse_conn *fc, struct fuse_req *req)
>  	list_del(&req->writepages_entry);
>  	for (i = 0; i < req->num_pages; i++) {
>  		dec_wb_stat(&bdi->wb, WB_WRITEBACK);
> -		dec_zone_page_state(req->pages[i], NR_WRITEBACK_TEMP);
> +		dec_node_page_state(req->pages[i], NR_WRITEBACK_TEMP);
>  		wb_writeout_inc(&bdi->wb);
>  	}
>  	wake_up(&fi->page_waitq);
> @@ -1642,7 +1642,7 @@ static int fuse_writepage_locked(struct page *page)
>  	req->inode = inode;
>  
>  	inc_wb_stat(&inode_to_bdi(inode)->wb, WB_WRITEBACK);
> -	inc_zone_page_state(tmp_page, NR_WRITEBACK_TEMP);
> +	inc_node_page_state(tmp_page, NR_WRITEBACK_TEMP);
>  
>  	spin_lock(&fc->lock);
>  	list_add(&req->writepages_entry, &fi->writepages);
> @@ -1756,7 +1756,7 @@ static bool fuse_writepage_in_flight(struct fuse_req *new_req,
>  		spin_unlock(&fc->lock);
>  
>  		dec_wb_stat(&bdi->wb, WB_WRITEBACK);
> -		dec_zone_page_state(page, NR_WRITEBACK_TEMP);
> +		dec_node_page_state(page, NR_WRITEBACK_TEMP);
>  		wb_writeout_inc(&bdi->wb);
>  		fuse_writepage_free(fc, new_req);
>  		fuse_request_free(new_req);
> @@ -1855,7 +1855,7 @@ static int fuse_writepages_fill(struct page *page,
>  	req->page_descs[req->num_pages].length = PAGE_SIZE;
>  
>  	inc_wb_stat(&inode_to_bdi(inode)->wb, WB_WRITEBACK);
> -	inc_zone_page_state(tmp_page, NR_WRITEBACK_TEMP);
> +	inc_node_page_state(tmp_page, NR_WRITEBACK_TEMP);
>  
>  	err = 0;
>  	if (is_writeback && fuse_writepage_in_flight(req, page)) {
> diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
> index 6b89fdf2c7fa..722731e16648 100644
> --- a/fs/nfs/internal.h
> +++ b/fs/nfs/internal.h
> @@ -653,7 +653,7 @@ void nfs_mark_page_unstable(struct page *page, struct nfs_commit_info *cinfo)
>  	if (!cinfo->dreq) {
>  		struct inode *inode = page_file_mapping(page)->host;
>  
> -		inc_zone_page_state(page, NR_UNSTABLE_NFS);
> +		inc_node_page_state(page, NR_UNSTABLE_NFS);
>  		inc_wb_stat(&inode_to_bdi(inode)->wb, WB_RECLAIMABLE);
>  		__mark_inode_dirty(inode, I_DIRTY_DATASYNC);
>  	}
> diff --git a/fs/nfs/write.c b/fs/nfs/write.c
> index 3087fb6f1983..4715549be0c3 100644
> --- a/fs/nfs/write.c
> +++ b/fs/nfs/write.c
> @@ -887,7 +887,7 @@ nfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg,
>  static void
>  nfs_clear_page_commit(struct page *page)
>  {
> -	dec_zone_page_state(page, NR_UNSTABLE_NFS);
> +	dec_node_page_state(page, NR_UNSTABLE_NFS);
>  	dec_wb_stat(&inode_to_bdi(page_file_mapping(page)->host)->wb,
>  		    WB_RECLAIMABLE);
>  }
> diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
> index 076afb43fc56..6cb9ea36d0fc 100644
> --- a/fs/proc/meminfo.c
> +++ b/fs/proc/meminfo.c
> @@ -40,7 +40,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>  	si_swapinfo(&i);
>  	committed = percpu_counter_read_positive(&vm_committed_as);
>  
> -	cached = global_page_state(NR_FILE_PAGES) -
> +	cached = global_node_page_state(NR_FILE_PAGES) -
>  			total_swapcache_pages() - i.bufferram;
>  	if (cached < 0)
>  		cached = 0;
> @@ -136,8 +136,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>  #endif
>  		K(i.totalswap),
>  		K(i.freeswap),
> -		K(global_page_state(NR_FILE_DIRTY)),
> -		K(global_page_state(NR_WRITEBACK)),
> +		K(global_node_page_state(NR_FILE_DIRTY)),
> +		K(global_node_page_state(NR_WRITEBACK)),
>  		K(global_node_page_state(NR_ANON_MAPPED)),
>  		K(global_node_page_state(NR_FILE_MAPPED)),
>  		K(i.sharedram),
> @@ -150,9 +150,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>  #ifdef CONFIG_QUICKLIST
>  		K(quicklist_total_size()),
>  #endif
> -		K(global_page_state(NR_UNSTABLE_NFS)),
> +		K(global_node_page_state(NR_UNSTABLE_NFS)),
>  		K(global_page_state(NR_BOUNCE)),
> -		K(global_page_state(NR_WRITEBACK_TEMP)),
> +		K(global_node_page_state(NR_WRITEBACK_TEMP)),
>  		K(vm_commit_limit()),
>  		K(committed),
>  		(unsigned long)VMALLOC_TOTAL >> 10,
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 6b1fea6cde9a..9924b46e3a13 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -115,20 +115,14 @@ enum zone_stat_item {
>  	NR_ZONE_LRU_ANON = NR_ZONE_LRU_BASE,
>  	NR_ZONE_LRU_FILE,
>  	NR_MLOCK,		/* mlock()ed pages found and moved off LRU */
> -	NR_FILE_PAGES,
> -	NR_FILE_DIRTY,
> -	NR_WRITEBACK,
>  	NR_SLAB_RECLAIMABLE,
>  	NR_SLAB_UNRECLAIMABLE,
>  	NR_PAGETABLE,		/* used for pagetables */
>  	NR_KERNEL_STACK,
>  	/* Second 128 byte cacheline */
> -	NR_UNSTABLE_NFS,	/* NFS unstable pages */
>  	NR_BOUNCE,
>  	NR_VMSCAN_WRITE,
>  	NR_VMSCAN_IMMEDIATE,	/* Prioritise for reclaim when writeback ends */
> -	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
> -	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
>  	NR_DIRTIED,		/* page dirtyings since bootup */
>  	NR_WRITTEN,		/* page writings since bootup */
>  #if IS_ENABLED(CONFIG_ZSMALLOC)
> @@ -162,6 +156,12 @@ enum node_stat_item {
>  	NR_ANON_MAPPED,	/* Mapped anonymous pages */
>  	NR_FILE_MAPPED,	/* pagecache pages mapped into pagetables.
>  			   only modified from process context */
> +	NR_FILE_PAGES,
> +	NR_FILE_DIRTY,
> +	NR_WRITEBACK,
> +	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
> +	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
> +	NR_UNSTABLE_NFS,	/* NFS unstable pages */
>  	NR_VM_NODE_STAT_ITEMS
>  };
>  
> diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
> index 73614ce1d204..c581d9c04ca5 100644
> --- a/include/trace/events/writeback.h
> +++ b/include/trace/events/writeback.h
> @@ -412,9 +412,9 @@ TRACE_EVENT(global_dirty_state,
>  	),
>  
>  	TP_fast_assign(
> -		__entry->nr_dirty	= global_page_state(NR_FILE_DIRTY);
> -		__entry->nr_writeback	= global_page_state(NR_WRITEBACK);
> -		__entry->nr_unstable	= global_page_state(NR_UNSTABLE_NFS);
> +		__entry->nr_dirty	= global_node_page_state(NR_FILE_DIRTY);
> +		__entry->nr_writeback	= global_node_page_state(NR_WRITEBACK);
> +		__entry->nr_unstable	= global_node_page_state(NR_UNSTABLE_NFS);
>  		__entry->nr_dirtied	= global_page_state(NR_DIRTIED);
>  		__entry->nr_written	= global_page_state(NR_WRITTEN);
>  		__entry->background_thresh = background_thresh;
> diff --git a/mm/filemap.c b/mm/filemap.c
> index d50619adfd7f..b99035dd2288 100644
> --- a/mm/filemap.c
> +++ b/mm/filemap.c
> @@ -209,9 +209,9 @@ void __delete_from_page_cache(struct page *page, void *shadow)
>  
>  	/* hugetlb pages do not participate in page cache accounting. */
>  	if (!PageHuge(page))
> -		__dec_zone_page_state(page, NR_FILE_PAGES);
> +		__dec_node_page_state(page, NR_FILE_PAGES);
>  	if (PageSwapBacked(page))
> -		__dec_zone_page_state(page, NR_SHMEM);
> +		__dec_node_page_state(page, NR_SHMEM);
>  
>  	/*
>  	 * At this point page must be either written or cleaned by truncate.
> @@ -549,9 +549,9 @@ int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask)
>  		 * hugetlb pages do not participate in page cache accounting.
>  		 */
>  		if (!PageHuge(new))
> -			__inc_zone_page_state(new, NR_FILE_PAGES);
> +			__inc_node_page_state(new, NR_FILE_PAGES);
>  		if (PageSwapBacked(new))
> -			__inc_zone_page_state(new, NR_SHMEM);
> +			__inc_node_page_state(new, NR_SHMEM);
>  		spin_unlock_irqrestore(&mapping->tree_lock, flags);
>  		mem_cgroup_migrate(old, new);
>  		radix_tree_preload_end();
> @@ -658,7 +658,7 @@ static int __add_to_page_cache_locked(struct page *page,
>  
>  	/* hugetlb pages do not participate in page cache accounting. */
>  	if (!huge)
> -		__inc_zone_page_state(page, NR_FILE_PAGES);
> +		__inc_node_page_state(page, NR_FILE_PAGES);
>  	spin_unlock_irq(&mapping->tree_lock);
>  	if (!huge)
>  		mem_cgroup_commit_charge(page, memcg, false, false);
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 1582c07205c6..d3fe4cfc2808 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -505,15 +505,15 @@ int migrate_page_move_mapping(struct address_space *mapping,
>  	 * are mapped to swap space.
>  	 */
>  	if (newzone != oldzone) {
> -		__dec_zone_state(oldzone, NR_FILE_PAGES);
> -		__inc_zone_state(newzone, NR_FILE_PAGES);
> +		__dec_node_state(oldzone->zone_pgdat, NR_FILE_PAGES);
> +		__inc_node_state(newzone->zone_pgdat, NR_FILE_PAGES);
>  		if (PageSwapBacked(page) && !PageSwapCache(page)) {
> -			__dec_zone_state(oldzone, NR_SHMEM);
> -			__inc_zone_state(newzone, NR_SHMEM);
> +			__dec_node_state(oldzone->zone_pgdat, NR_SHMEM);
> +			__inc_node_state(newzone->zone_pgdat, NR_SHMEM);
>  		}
>  		if (dirty && mapping_cap_account_dirty(mapping)) {
> -			__dec_zone_state(oldzone, NR_FILE_DIRTY);
> -			__inc_zone_state(newzone, NR_FILE_DIRTY);
> +			__dec_node_state(oldzone->zone_pgdat, NR_FILE_DIRTY);
> +			__inc_node_state(newzone->zone_pgdat, NR_FILE_DIRTY);
>  		}
>  	}
>  	local_irq_enable();
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index a2b24d5ea43a..aa9fa1eb8b80 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -471,20 +471,12 @@ static unsigned long node_dirty_limit(struct pglist_data *pgdat)
>   */
>  bool node_dirty_ok(struct pglist_data *pgdat)
>  {
> -	int z;
>  	unsigned long limit = node_dirty_limit(pgdat);
>  	unsigned long nr_pages = 0;
>  
> -	for (z = 0; z < MAX_NR_ZONES; z++) {
> -		struct zone *zone = pgdat->node_zones + z;
> -
> -		if (!populated_zone(zone))
> -			continue;
> -
> -		nr_pages += zone_page_state(zone, NR_FILE_DIRTY);
> -		nr_pages += zone_page_state(zone, NR_UNSTABLE_NFS);
> -		nr_pages += zone_page_state(zone, NR_WRITEBACK);
> -	}
> +	nr_pages += node_page_state(pgdat, NR_FILE_DIRTY);
> +	nr_pages += node_page_state(pgdat, NR_UNSTABLE_NFS);
> +	nr_pages += node_page_state(pgdat, NR_WRITEBACK);
>  
>  	return nr_pages <= limit;
>  }
> @@ -1574,10 +1566,10 @@ static void balance_dirty_pages(struct address_space *mapping,
>  		 * written to the server's write cache, but has not yet
>  		 * been flushed to permanent storage.
>  		 */
> -		nr_reclaimable = global_page_state(NR_FILE_DIRTY) +
> -					global_page_state(NR_UNSTABLE_NFS);
> +		nr_reclaimable = global_node_page_state(NR_FILE_DIRTY) +
> +					global_node_page_state(NR_UNSTABLE_NFS);
>  		gdtc->avail = global_dirtyable_memory();
> -		gdtc->dirty = nr_reclaimable + global_page_state(NR_WRITEBACK);
> +		gdtc->dirty = nr_reclaimable + global_node_page_state(NR_WRITEBACK);
>  
>  		domain_dirty_limits(gdtc);
>  
> @@ -1914,8 +1906,8 @@ bool wb_over_bg_thresh(struct bdi_writeback *wb)
>  	 * as we're trying to decide whether to put more under writeback.
>  	 */
>  	gdtc->avail = global_dirtyable_memory();
> -	gdtc->dirty = global_page_state(NR_FILE_DIRTY) +
> -		      global_page_state(NR_UNSTABLE_NFS);
> +	gdtc->dirty = global_node_page_state(NR_FILE_DIRTY) +
> +		      global_node_page_state(NR_UNSTABLE_NFS);
>  	domain_dirty_limits(gdtc);
>  
>  	if (gdtc->dirty > gdtc->bg_thresh)
> @@ -1959,8 +1951,8 @@ void throttle_vm_writeout(gfp_t gfp_mask)
>                   */
>                  dirty_thresh += dirty_thresh / 10;      /* wheeee... */
>  
> -                if (global_page_state(NR_UNSTABLE_NFS) +
> -			global_page_state(NR_WRITEBACK) <= dirty_thresh)
> +                if (global_node_page_state(NR_UNSTABLE_NFS) +
> +			global_node_page_state(NR_WRITEBACK) <= dirty_thresh)
>                          	break;
>                  congestion_wait(BLK_RW_ASYNC, HZ/10);
>  
> @@ -1988,8 +1980,8 @@ int dirty_writeback_centisecs_handler(struct ctl_table *table, int write,
>  void laptop_mode_timer_fn(unsigned long data)
>  {
>  	struct request_queue *q = (struct request_queue *)data;
> -	int nr_pages = global_page_state(NR_FILE_DIRTY) +
> -		global_page_state(NR_UNSTABLE_NFS);
> +	int nr_pages = global_node_page_state(NR_FILE_DIRTY) +
> +		global_node_page_state(NR_UNSTABLE_NFS);
>  	struct bdi_writeback *wb;
>  
>  	/*
> @@ -2440,7 +2432,7 @@ void account_page_dirtied(struct page *page, struct address_space *mapping)
>  		wb = inode_to_wb(inode);
>  
>  		mem_cgroup_inc_page_stat(page, MEM_CGROUP_STAT_DIRTY);
> -		__inc_zone_page_state(page, NR_FILE_DIRTY);
> +		__inc_node_page_state(page, NR_FILE_DIRTY);
>  		__inc_zone_page_state(page, NR_DIRTIED);
>  		__inc_wb_stat(wb, WB_RECLAIMABLE);
>  		__inc_wb_stat(wb, WB_DIRTIED);
> @@ -2461,7 +2453,7 @@ void account_page_cleaned(struct page *page, struct address_space *mapping,
>  {
>  	if (mapping_cap_account_dirty(mapping)) {
>  		mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_DIRTY);
> -		dec_zone_page_state(page, NR_FILE_DIRTY);
> +		dec_node_page_state(page, NR_FILE_DIRTY);
>  		dec_wb_stat(wb, WB_RECLAIMABLE);
>  		task_io_account_cancelled_write(PAGE_SIZE);
>  	}
> @@ -2716,7 +2708,7 @@ int clear_page_dirty_for_io(struct page *page)
>  		wb = unlocked_inode_to_wb_begin(inode, &locked);
>  		if (TestClearPageDirty(page)) {
>  			mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_DIRTY);
> -			dec_zone_page_state(page, NR_FILE_DIRTY);
> +			dec_node_page_state(page, NR_FILE_DIRTY);
>  			dec_wb_stat(wb, WB_RECLAIMABLE);
>  			ret = 1;
>  		}
> @@ -2757,7 +2749,7 @@ int test_clear_page_writeback(struct page *page)
>  	}
>  	if (ret) {
>  		mem_cgroup_dec_page_stat(page, MEM_CGROUP_STAT_WRITEBACK);
> -		dec_zone_page_state(page, NR_WRITEBACK);
> +		dec_node_page_state(page, NR_WRITEBACK);
>  		inc_zone_page_state(page, NR_WRITTEN);
>  	}
>  	unlock_page_memcg(page);
> @@ -2798,7 +2790,7 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
>  	}
>  	if (!ret) {
>  		mem_cgroup_inc_page_stat(page, MEM_CGROUP_STAT_WRITEBACK);
> -		inc_zone_page_state(page, NR_WRITEBACK);
> +		inc_node_page_state(page, NR_WRITEBACK);
>  	}
>  	unlock_page_memcg(page);
>  	return ret;
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index cf3523f399e5..6f5120a282c3 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4180,7 +4180,7 @@ EXPORT_SYMBOL_GPL(si_mem_available);
>  void si_meminfo(struct sysinfo *val)
>  {
>  	val->totalram = totalram_pages;
> -	val->sharedram = global_page_state(NR_SHMEM);
> +	val->sharedram = global_node_page_state(NR_SHMEM);
>  	val->freeram = global_page_state(NR_FREE_PAGES);
>  	val->bufferram = nr_blockdev_pages();
>  	val->totalhigh = totalhigh_pages;
> @@ -4202,7 +4202,7 @@ void si_meminfo_node(struct sysinfo *val, int nid)
>  	for (zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++)
>  		managed_pages += pgdat->node_zones[zone_type].managed_pages;
>  	val->totalram = managed_pages;
> -	val->sharedram = sum_zone_node_page_state(nid, NR_SHMEM);
> +	val->sharedram = node_page_state(pgdat, NR_SHMEM);
>  	val->freeram = sum_zone_node_page_state(nid, NR_FREE_PAGES);
>  #ifdef CONFIG_HIGHMEM
>  	for (zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++) {
> @@ -4309,13 +4309,13 @@ void show_free_areas(unsigned int filter)
>  		global_node_page_state(NR_INACTIVE_FILE),
>  		global_node_page_state(NR_ISOLATED_FILE),
>  		global_node_page_state(NR_UNEVICTABLE),
> -		global_page_state(NR_FILE_DIRTY),
> -		global_page_state(NR_WRITEBACK),
> -		global_page_state(NR_UNSTABLE_NFS),
> +		global_node_page_state(NR_FILE_DIRTY),
> +		global_node_page_state(NR_WRITEBACK),
> +		global_node_page_state(NR_UNSTABLE_NFS),
>  		global_page_state(NR_SLAB_RECLAIMABLE),
>  		global_page_state(NR_SLAB_UNRECLAIMABLE),
>  		global_node_page_state(NR_FILE_MAPPED),
> -		global_page_state(NR_SHMEM),
> +		global_node_page_state(NR_SHMEM),
>  		global_page_state(NR_PAGETABLE),
>  		global_page_state(NR_BOUNCE),
>  		global_page_state(NR_FREE_PAGES),
> @@ -4332,6 +4332,11 @@ void show_free_areas(unsigned int filter)
>  			" isolated(anon):%lukB"
>  			" isolated(file):%lukB"
>  			" mapped:%lukB"
> +			" dirty:%lukB"
> +			" writeback:%lukB"
> +			" shmem:%lukB"
> +			" writeback_tmp:%lukB"
> +			" unstable:%lukB"
>  			" all_unreclaimable? %s"
>  			"\n",
>  			pgdat->node_id,
> @@ -4343,6 +4348,11 @@ void show_free_areas(unsigned int filter)
>  			K(node_page_state(pgdat, NR_ISOLATED_ANON)),
>  			K(node_page_state(pgdat, NR_ISOLATED_FILE)),
>  			K(node_page_state(pgdat, NR_FILE_MAPPED)),
> +			K(node_page_state(pgdat, NR_FILE_DIRTY)),
> +			K(node_page_state(pgdat, NR_WRITEBACK)),
> +			K(node_page_state(pgdat, NR_SHMEM)),
> +			K(node_page_state(pgdat, NR_WRITEBACK_TEMP)),
> +			K(node_page_state(pgdat, NR_UNSTABLE_NFS)),
>  			!pgdat_reclaimable(pgdat) ? "yes" : "no");
>  	}
>  
> @@ -4365,19 +4375,14 @@ void show_free_areas(unsigned int filter)
>  			" present:%lukB"
>  			" managed:%lukB"
>  			" mlocked:%lukB"
> -			" dirty:%lukB"
> -			" writeback:%lukB"
> -			" shmem:%lukB"
>  			" slab_reclaimable:%lukB"
>  			" slab_unreclaimable:%lukB"
>  			" kernel_stack:%lukB"
>  			" pagetables:%lukB"
> -			" unstable:%lukB"
>  			" bounce:%lukB"
>  			" free_pcp:%lukB"
>  			" local_pcp:%ukB"
>  			" free_cma:%lukB"
> -			" writeback_tmp:%lukB"
>  			" node_pages_scanned:%lu"
>  			"\n",
>  			zone->name,
> @@ -4388,20 +4393,15 @@ void show_free_areas(unsigned int filter)
>  			K(zone->present_pages),
>  			K(zone->managed_pages),
>  			K(zone_page_state(zone, NR_MLOCK)),
> -			K(zone_page_state(zone, NR_FILE_DIRTY)),
> -			K(zone_page_state(zone, NR_WRITEBACK)),
> -			K(zone_page_state(zone, NR_SHMEM)),
>  			K(zone_page_state(zone, NR_SLAB_RECLAIMABLE)),
>  			K(zone_page_state(zone, NR_SLAB_UNRECLAIMABLE)),
>  			zone_page_state(zone, NR_KERNEL_STACK) *
>  				THREAD_SIZE / 1024,
>  			K(zone_page_state(zone, NR_PAGETABLE)),
> -			K(zone_page_state(zone, NR_UNSTABLE_NFS)),
>  			K(zone_page_state(zone, NR_BOUNCE)),
>  			K(free_pcp),
>  			K(this_cpu_read(zone->pageset->pcp.count)),
>  			K(zone_page_state(zone, NR_FREE_CMA_PAGES)),
> -			K(zone_page_state(zone, NR_WRITEBACK_TEMP)),
>  			K(node_page_state(zone->zone_pgdat, NR_PAGES_SCANNED)));
>  		printk("lowmem_reserve[]:");
>  		for (i = 0; i < MAX_NR_ZONES; i++)
> @@ -4444,7 +4444,7 @@ void show_free_areas(unsigned int filter)
>  
>  	hugetlb_show_meminfo();
>  
> -	printk("%ld total pagecache pages\n", global_page_state(NR_FILE_PAGES));
> +	printk("%ld total pagecache pages\n", global_node_page_state(NR_FILE_PAGES));
>  
>  	show_swap_cache_info();
>  }
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 24463b67b6ef..f01d291012d5 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -312,8 +312,8 @@ static int shmem_add_to_page_cache(struct page *page,
>  								 page);
>  	if (!error) {
>  		mapping->nrpages++;
> -		__inc_zone_page_state(page, NR_FILE_PAGES);
> -		__inc_zone_page_state(page, NR_SHMEM);
> +		__inc_node_page_state(page, NR_FILE_PAGES);
> +		__inc_node_page_state(page, NR_SHMEM);
>  		spin_unlock_irq(&mapping->tree_lock);
>  	} else {
>  		page->mapping = NULL;
> @@ -335,8 +335,8 @@ static void shmem_delete_from_page_cache(struct page *page, void *radswap)
>  	error = shmem_radix_tree_replace(mapping, page->index, page, radswap);
>  	page->mapping = NULL;
>  	mapping->nrpages--;
> -	__dec_zone_page_state(page, NR_FILE_PAGES);
> -	__dec_zone_page_state(page, NR_SHMEM);
> +	__dec_node_page_state(page, NR_FILE_PAGES);
> +	__dec_node_page_state(page, NR_SHMEM);
>  	spin_unlock_irq(&mapping->tree_lock);
>  	put_page(page);
>  	BUG_ON(error);
> @@ -1084,8 +1084,8 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
>  	error = shmem_radix_tree_replace(swap_mapping, swap_index, oldpage,
>  								   newpage);
>  	if (!error) {
> -		__inc_zone_page_state(newpage, NR_FILE_PAGES);
> -		__dec_zone_page_state(oldpage, NR_FILE_PAGES);
> +		__inc_node_page_state(newpage, NR_FILE_PAGES);
> +		__dec_node_page_state(oldpage, NR_FILE_PAGES);
>  	}
>  	spin_unlock_irq(&swap_mapping->tree_lock);
>  
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index c99463ac02fb..c8310a37be3a 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -95,7 +95,7 @@ int __add_to_swap_cache(struct page *page, swp_entry_t entry)
>  					entry.val, page);
>  	if (likely(!error)) {
>  		address_space->nrpages++;
> -		__inc_zone_page_state(page, NR_FILE_PAGES);
> +		__inc_node_page_state(page, NR_FILE_PAGES);
>  		INC_CACHE_INFO(add_total);
>  	}
>  	spin_unlock_irq(&address_space->tree_lock);
> @@ -147,7 +147,7 @@ void __delete_from_swap_cache(struct page *page)
>  	set_page_private(page, 0);
>  	ClearPageSwapCache(page);
>  	address_space->nrpages--;
> -	__dec_zone_page_state(page, NR_FILE_PAGES);
> +	__dec_node_page_state(page, NR_FILE_PAGES);
>  	INC_CACHE_INFO(del_total);
>  }
>  
> diff --git a/mm/util.c b/mm/util.c
> index b756ee36f7f0..21cbc0fc2fd4 100644
> --- a/mm/util.c
> +++ b/mm/util.c
> @@ -522,7 +522,7 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
>  
>  	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
>  		free = global_page_state(NR_FREE_PAGES);
> -		free += global_page_state(NR_FILE_PAGES);
> +		free += global_node_page_state(NR_FILE_PAGES);
>  
>  		/*
>  		 * shmem pages shouldn't be counted as free in this
> @@ -530,7 +530,7 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
>  		 * that won't affect the overall amount of available
>  		 * memory in the system.
>  		 */
> -		free -= global_page_state(NR_SHMEM);
> +		free -= global_node_page_state(NR_SHMEM);
>  
>  		free += get_nr_swap_pages();
>  
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index c6e958079398..4501a9ab1d3f 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -3594,11 +3594,11 @@ int sysctl_min_unmapped_ratio = 1;
>   */
>  int sysctl_min_slab_ratio = 5;
>  
> -static inline unsigned long zone_unmapped_file_pages(struct zone *zone)
> +static inline unsigned long node_unmapped_file_pages(struct pglist_data *pgdat)
>  {
> -	unsigned long file_mapped = node_page_state(zone->zone_pgdat, NR_FILE_MAPPED);
> -	unsigned long file_lru = node_page_state(zone->zone_pgdat, NR_INACTIVE_FILE) +
> -		node_page_state(zone->zone_pgdat, NR_ACTIVE_FILE);
> +	unsigned long file_mapped = node_page_state(pgdat, NR_FILE_MAPPED);
> +	unsigned long file_lru = node_page_state(pgdat, NR_INACTIVE_FILE) +
> +		node_page_state(pgdat, NR_ACTIVE_FILE);
>  
>  	/*
>  	 * It's possible for there to be more file mapped pages than
> @@ -3617,17 +3617,17 @@ static unsigned long zone_pagecache_reclaimable(struct zone *zone)
>  	/*
>  	 * If RECLAIM_UNMAP is set, then all file pages are considered
>  	 * potentially reclaimable. Otherwise, we have to worry about
> -	 * pages like swapcache and zone_unmapped_file_pages() provides
> +	 * pages like swapcache and node_unmapped_file_pages() provides
>  	 * a better estimate
>  	 */
>  	if (zone_reclaim_mode & RECLAIM_UNMAP)
> -		nr_pagecache_reclaimable = zone_page_state(zone, NR_FILE_PAGES);
> +		nr_pagecache_reclaimable = node_page_state(zone->zone_pgdat, NR_FILE_PAGES);
>  	else
> -		nr_pagecache_reclaimable = zone_unmapped_file_pages(zone);
> +		nr_pagecache_reclaimable = node_unmapped_file_pages(zone->zone_pgdat);
>  
>  	/* If we can't clean pages, remove dirty pages from consideration */
>  	if (!(zone_reclaim_mode & RECLAIM_WRITE))
> -		delta += zone_page_state(zone, NR_FILE_DIRTY);
> +		delta += node_page_state(zone->zone_pgdat, NR_FILE_DIRTY);
>  
>  	/* Watch for any possible underflows due to delta */
>  	if (unlikely(delta > nr_pagecache_reclaimable))
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 12022ed481f0..32c499251174 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -924,19 +924,13 @@ const char * const vmstat_text[] = {
>  	"nr_zone_anon_lru",
>  	"nr_zone_file_lru",
>  	"nr_mlock",
> -	"nr_file_pages",
> -	"nr_dirty",
> -	"nr_writeback",
>  	"nr_slab_reclaimable",
>  	"nr_slab_unreclaimable",
>  	"nr_page_table_pages",
>  	"nr_kernel_stack",
> -	"nr_unstable",
>  	"nr_bounce",
>  	"nr_vmscan_write",
>  	"nr_vmscan_immediate_reclaim",
> -	"nr_writeback_temp",
> -	"nr_shmem",
>  	"nr_dirtied",
>  	"nr_written",
>  #if IS_ENABLED(CONFIG_ZSMALLOC)
> @@ -967,6 +961,12 @@ const char * const vmstat_text[] = {
>  	"workingset_nodereclaim",
>  	"nr_anon_pages",
>  	"nr_mapped",
> +	"nr_file_pages",
> +	"nr_dirty",
> +	"nr_writeback",
> +	"nr_writeback_temp",
> +	"nr_shmem",
> +	"nr_unstable",
>  
>  	/* enum writeback_stat_item counters */
>  	"nr_dirty_threshold",
> -- 
> 2.6.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1427825 — [PATCH 23/27] mm, vmscan: Add classzone information to tracepoints

This is convenient when tracking down why the skip count is high because it'll
show what classzone kswapd woke up at and what zones are being isolated.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/trace/events/vmscan.h | 51 ++++++++++++++++++++++++++-----------------
 mm/vmscan.c                   | 14 +++++++-----
 2 files changed, 40 insertions(+), 25 deletions(-)

diff --git a/include/trace/events/vmscan.h b/include/trace/events/vmscan.h
index 897f1aa1ee5f..c88fd0934e7e 100644
--- a/include/trace/events/vmscan.h
+++ b/include/trace/events/vmscan.h
@@ -55,21 +55,23 @@ TRACE_EVENT(mm_vmscan_kswapd_sleep,
 
 TRACE_EVENT(mm_vmscan_kswapd_wake,
 
-	TP_PROTO(int nid, int order),
+	TP_PROTO(int nid, int zid, int order),
 
-	TP_ARGS(nid, order),
+	TP_ARGS(nid, zid, order),
 
 	TP_STRUCT__entry(
 		__field(	int,	nid	)
+		__field(	int,	zid	)
 		__field(	int,	order	)
 	),
 
 	TP_fast_assign(
 		__entry->nid	= nid;
+		__entry->zid    = zid;
 		__entry->order	= order;
 	),
 
-	TP_printk("nid=%d order=%d", __entry->nid, __entry->order)
+	TP_printk("nid=%d zid=%d order=%d", __entry->nid, __entry->zid, __entry->order)
 );
 
 TRACE_EVENT(mm_vmscan_wakeup_kswapd,
@@ -98,47 +100,50 @@ TRACE_EVENT(mm_vmscan_wakeup_kswapd,
 
 DECLARE_EVENT_CLASS(mm_vmscan_direct_reclaim_begin_template,
 
-	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags),
+	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags, int classzone_idx),
 
-	TP_ARGS(order, may_writepage, gfp_flags),
+	TP_ARGS(order, may_writepage, gfp_flags, classzone_idx),
 
 	TP_STRUCT__entry(
 		__field(	int,	order		)
 		__field(	int,	may_writepage	)
 		__field(	gfp_t,	gfp_flags	)
+		__field(	int,	classzone_idx	)
 	),
 
 	TP_fast_assign(
 		__entry->order		= order;
 		__entry->may_writepage	= may_writepage;
 		__entry->gfp_flags	= gfp_flags;
+		__entry->classzone_idx	= classzone_idx;
 	),
 
-	TP_printk("order=%d may_writepage=%d gfp_flags=%s",
+	TP_printk("order=%d may_writepage=%d gfp_flags=%s classzone_idx=%d",
 		__entry->order,
 		__entry->may_writepage,
-		show_gfp_flags(__entry->gfp_flags))
+		show_gfp_flags(__entry->gfp_flags),
+		__entry->classzone_idx)
 );
 
 DEFINE_EVENT(mm_vmscan_direct_reclaim_begin_template, mm_vmscan_direct_reclaim_begin,
 
-	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags),
+	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags, int classzone_idx),
 
-	TP_ARGS(order, may_writepage, gfp_flags)
+	TP_ARGS(order, may_writepage, gfp_flags, classzone_idx)
 );
 
 DEFINE_EVENT(mm_vmscan_direct_reclaim_begin_template, mm_vmscan_memcg_reclaim_begin,
 
-	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags),
+	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags, int classzone_idx),
 
-	TP_ARGS(order, may_writepage, gfp_flags)
+	TP_ARGS(order, may_writepage, gfp_flags, classzone_idx)
 );
 
 DEFINE_EVENT(mm_vmscan_direct_reclaim_begin_template, mm_vmscan_memcg_softlimit_reclaim_begin,
 
-	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags),
+	TP_PROTO(int order, int may_writepage, gfp_t gfp_flags, int classzone_idx),
 
-	TP_ARGS(order, may_writepage, gfp_flags)
+	TP_ARGS(order, may_writepage, gfp_flags, classzone_idx)
 );
 
 DECLARE_EVENT_CLASS(mm_vmscan_direct_reclaim_end_template,
@@ -266,16 +271,18 @@ TRACE_EVENT(mm_shrink_slab_end,
 
 DECLARE_EVENT_CLASS(mm_vmscan_lru_isolate_template,
 
-	TP_PROTO(int order,
+	TP_PROTO(int classzone_idx,
+		int order,
 		unsigned long nr_requested,
 		unsigned long nr_scanned,
 		unsigned long nr_taken,
 		isolate_mode_t isolate_mode,
 		int file),
 
-	TP_ARGS(order, nr_requested, nr_scanned, nr_taken, isolate_mode, file),
+	TP_ARGS(classzone_idx, order, nr_requested, nr_scanned, nr_taken, isolate_mode, file),
 
 	TP_STRUCT__entry(
+		__field(int, classzone_idx)
 		__field(int, order)
 		__field(unsigned long, nr_requested)
 		__field(unsigned long, nr_scanned)
@@ -285,6 +292,7 @@ DECLARE_EVENT_CLASS(mm_vmscan_lru_isolate_template,
 	),
 
 	TP_fast_assign(
+		__entry->classzone_idx = classzone_idx;
 		__entry->order = order;
 		__entry->nr_requested = nr_requested;
 		__entry->nr_scanned = nr_scanned;
@@ -293,8 +301,9 @@ DECLARE_EVENT_CLASS(mm_vmscan_lru_isolate_template,
 		__entry->file = file;
 	),
 
-	TP_printk("isolate_mode=%d order=%d nr_requested=%lu nr_scanned=%lu nr_taken=%lu file=%d",
+	TP_printk("isolate_mode=%d classzone=%d order=%d nr_requested=%lu nr_scanned=%lu nr_taken=%lu file=%d",
 		__entry->isolate_mode,
+		__entry->classzone_idx,
 		__entry->order,
 		__entry->nr_requested,
 		__entry->nr_scanned,
@@ -304,27 +313,29 @@ DECLARE_EVENT_CLASS(mm_vmscan_lru_isolate_template,
 
 DEFINE_EVENT(mm_vmscan_lru_isolate_template, mm_vmscan_lru_isolate,
 
-	TP_PROTO(int order,
+	TP_PROTO(int classzone_idx,
+		int order,
 		unsigned long nr_requested,
 		unsigned long nr_scanned,
 		unsigned long nr_taken,
 		isolate_mode_t isolate_mode,
 		int file),
 
-	TP_ARGS(order, nr_requested, nr_scanned, nr_taken, isolate_mode, file)
+	TP_ARGS(classzone_idx, order, nr_requested, nr_scanned, nr_taken, isolate_mode, file)
 
 );
 
 DEFINE_EVENT(mm_vmscan_lru_isolate_template, mm_vmscan_memcg_isolate,
 
-	TP_PROTO(int order,
+	TP_PROTO(int classzone_idx,
+		int order,
 		unsigned long nr_requested,
 		unsigned long nr_scanned,
 		unsigned long nr_taken,
 		isolate_mode_t isolate_mode,
 		int file),
 
-	TP_ARGS(order, nr_requested, nr_scanned, nr_taken, isolate_mode, file)
+	TP_ARGS(classzone_idx, order, nr_requested, nr_scanned, nr_taken, isolate_mode, file)
 
 );
 
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4fa1fee5e486..16ad7b4be1e9 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1433,7 +1433,7 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
 	if (!list_empty(&pages_skipped))
 		list_splice(&pages_skipped, src);
 	*nr_scanned = scan;
-	trace_mm_vmscan_lru_isolate(sc->order, nr_to_scan, scan,
+	trace_mm_vmscan_lru_isolate(sc->reclaim_idx, sc->order, nr_to_scan, scan,
 				    nr_taken, mode, is_file_lru(lru));
 	for (scan = 0; scan < MAX_NR_ZONES; scan++) {
 		nr_pages = nr_zone_taken[scan];
@@ -2893,7 +2893,8 @@ unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
 
 	trace_mm_vmscan_direct_reclaim_begin(order,
 				sc.may_writepage,
-				gfp_mask);
+				gfp_mask,
+				sc.reclaim_idx);
 
 	nr_reclaimed = do_try_to_free_pages(zonelist, &sc);
 
@@ -2924,7 +2925,8 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg,
 
 	trace_mm_vmscan_memcg_softlimit_reclaim_begin(sc.order,
 						      sc.may_writepage,
-						      sc.gfp_mask);
+						      sc.gfp_mask,
+						      sc.reclaim_idx);
 
 	/*
 	 * NOTE: Although we can get the priority field, using it
@@ -2972,7 +2974,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
 
 	trace_mm_vmscan_memcg_reclaim_begin(0,
 					    sc.may_writepage,
-					    sc.gfp_mask);
+					    sc.gfp_mask,
+					    sc.reclaim_idx);
 
 	nr_reclaimed = do_try_to_free_pages(zonelist, &sc);
 
@@ -3407,7 +3410,8 @@ static int kswapd(void *p)
 		 * but kcompactd is woken to compact for the original
 		 * request (alloc_order).
 		 */
-		trace_mm_vmscan_kswapd_wake(pgdat->node_id, alloc_order);
+		trace_mm_vmscan_kswapd_wake(pgdat->node_id, classzone_idx,
+						alloc_order);
 		reclaim_order = balance_pgdat(pgdat, alloc_order, classzone_idx);
 		if (reclaim_order < alloc_order)
 			goto kswapd_try_sleep;
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1427827 — [PATCH 20/27] mm, vmscan: Update classzone_idx if buffer_heads_over_limit

If buffer heads are over the limit then the direct reclaim gfp_mask
is promoted to __GFP_HIGHMEM so that lowmem is indirectly freed. With
node-based reclaim, it is also required that the classzone_idx be updated
or the pages will be skipped.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 mm/vmscan.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 1f7c1262c0a3..f4c759b3581b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2571,8 +2571,10 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc,
 	 * highmem pages could be pinning lowmem pages storing buffer_heads
 	 */
 	orig_mask = sc->gfp_mask;
-	if (buffer_heads_over_limit)
+	if (buffer_heads_over_limit) {
 		sc->gfp_mask |= __GFP_HIGHMEM;
+		classzone_idx = gfp_zone(sc->gfp_mask);
+	}
 
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 					gfp_zone(sc->gfp_mask), sc->nodemask) {
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1428865 — Re: [PATCH 20/27] mm, vmscan: Update classzone_idx if buffer_heads_over_limit

On Tue 21-06-16 15:15:59, Mel Gorman wrote:
> If buffer heads are over the limit then the direct reclaim gfp_mask
> is promoted to __GFP_HIGHMEM so that lowmem is indirectly freed. With
> node-based reclaim, it is also required that the classzone_idx be updated
> or the pages will be skipped.
> 
> Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>
> ---
>  mm/vmscan.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 1f7c1262c0a3..f4c759b3581b 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -2571,8 +2571,10 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc,
>  	 * highmem pages could be pinning lowmem pages storing buffer_heads
>  	 */
>  	orig_mask = sc->gfp_mask;
> -	if (buffer_heads_over_limit)
> +	if (buffer_heads_over_limit) {
>  		sc->gfp_mask |= __GFP_HIGHMEM;
> +		classzone_idx = gfp_zone(sc->gfp_mask);
> +	}
>  
>  	for_each_zone_zonelist_nodemask(zone, z, zonelist,
>  					gfp_zone(sc->gfp_mask), sc->nodemask) {

The code that follows that however does update classzone_idx
		/*
		 * Note that reclaim_idx does not change as it is the highest
		 * zone reclaimed from which for empty zones is a no-op but
		 * classzone_idx is used by shrink_node to test if the slabs
		 * should be shrunk on a given node.
		 */
		while (!populated_zone(zone->zone_pgdat->node_zones +
							classzone_idx)) {
			classzone_idx--;
		}

so if we start with a zone which doesn't have the highmem zone we will
not see highmems of other zone AFAIU. I guess this would be unlikely
because highmem systems will be UMA in most cases but the code looks a
bit confusing.

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1427828 — [PATCH 27/27] mm: vmstat: Account per-zone stalls and pages skipped during reclaim

The vmstat allocstall was fairly useful in the general sense but
node-based LRUs change that. It's important to know if a stall was for an
address-limited allocation request as this will require skipping pages from
other zones. This patch adds pgstall_* counters to replace allocstall. The
sum of the counters will equal the old allocstall so it can be trivially
recalculated. A high number of address-limited allocation requests may
result in a lot of useless LRU scanning for suitable pages.

As address-limited allocations require pages to be skipped, it's important
to know how much useless LRU scanning took place so this patch adds
pgskip* counters. This yields the following model

1. The number of address-space limited stalls can be accounted for (pgstall)
2. The amount of useless work required to reclaim the data is accounted (pgskip)
3. The total number of scans is available from pgscan_kswapd and pgscan_direct
   so from that the ratio of useful to useless scans can be calculated.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/linux/vm_event_item.h |  4 +++-
 mm/huge_memory.c              | 19 +++++++++++++++----
 mm/vmscan.c                   | 15 +++++++++++++--
 mm/vmstat.c                   |  3 ++-
 4 files changed, 33 insertions(+), 8 deletions(-)

diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index 8dcb5a813163..0a0503da8c3b 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -23,6 +23,8 @@
 
 enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
 		FOR_ALL_ZONES(PGALLOC),
+		FOR_ALL_ZONES(PGSTALL),
+		FOR_ALL_ZONES(PGSCAN_SKIP),
 		PGFREE, PGACTIVATE, PGDEACTIVATE,
 		PGFAULT, PGMAJFAULT,
 		PGLAZYFREED,
@@ -37,7 +39,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
 #endif
 		PGINODESTEAL, SLABS_SCANNED, KSWAPD_INODESTEAL,
 		KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
-		PAGEOUTRUN, ALLOCSTALL, PGROTATED,
+		PAGEOUTRUN, PGROTATED,
 		DROP_PAGECACHE, DROP_SLAB,
 #ifdef CONFIG_NUMA_BALANCING
 		NUMA_PTE_UPDATES,
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d5dd6533de32..e55bc5b6601d 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2452,6 +2452,17 @@ static bool __collapse_huge_page_swapin(struct mm_struct *mm,
 	return true;
 }
 
+static unsigned long sum_alloc_stalls(void)
+{
+	int zid;
+	unsigned long allocstall = 0;
+
+	for (zid = 0; zid < MAX_NR_ZONES - 1; zid++)
+		allocstall += sum_vm_event(PGSTALL_NORMAL - ZONE_NORMAL + zid);
+
+	return allocstall;
+}
+
 static void collapse_huge_page(struct mm_struct *mm,
 				   unsigned long address,
 				   struct page **hpage,
@@ -2488,7 +2499,7 @@ static void collapse_huge_page(struct mm_struct *mm,
 	}
 
 	swap = get_mm_counter(mm, MM_SWAPENTS);
-	curr_allocstall = sum_vm_event(ALLOCSTALL);
+	curr_allocstall = sum_alloc_stalls();;
 	down_read(&mm->mmap_sem);
 	result = hugepage_vma_revalidate(mm, address);
 	if (result) {
@@ -2934,7 +2945,7 @@ static void khugepaged_wait_work(void)
 		if (!scan_sleep_jiffies)
 			return;
 
-		allocstall = sum_vm_event(ALLOCSTALL);
+		allocstall = sum_alloc_stalls();
 		khugepaged_sleep_expire = jiffies + scan_sleep_jiffies;
 		wait_event_freezable_timeout(khugepaged_wait,
 					     khugepaged_should_wakeup(),
@@ -2943,7 +2954,7 @@ static void khugepaged_wait_work(void)
 	}
 
 	if (khugepaged_enabled()) {
-		allocstall = sum_vm_event(ALLOCSTALL);
+		allocstall = sum_alloc_stalls();
 		wait_event_freezable(khugepaged_wait, khugepaged_wait_event());
 	}
 }
@@ -2954,7 +2965,7 @@ static int khugepaged(void *none)
 
 	set_freezable();
 	set_user_nice(current, MAX_NICE);
-	allocstall = sum_vm_event(ALLOCSTALL);
+	allocstall = sum_alloc_stalls();
 
 	while (!kthread_should_stop()) {
 		khugepaged_do_scan();
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 16ad7b4be1e9..66219c391797 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1388,6 +1388,7 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
 	struct list_head *src = &lruvec->lists[lru];
 	unsigned long nr_taken = 0;
 	unsigned long nr_zone_taken[MAX_NR_ZONES] = { 0 };
+	unsigned long nr_skipped[MAX_NR_ZONES] = { 0, };
 	unsigned long scan, nr_pages;
 	LIST_HEAD(pages_skipped);
 
@@ -1402,6 +1403,7 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
 
 		if (page_zonenum(page) > sc->reclaim_idx) {
 			list_move(&page->lru, &pages_skipped);
+			nr_skipped[page_zonenum(page)]++;
 			continue;
 		}
 
@@ -1430,8 +1432,17 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
 	 * scanning would soon rescan the same pages to skip and put the
 	 * system at risk of premature OOM.
 	 */
-	if (!list_empty(&pages_skipped))
+	if (!list_empty(&pages_skipped)) {
+		int zid;
+
 		list_splice(&pages_skipped, src);
+		for (zid = 0; zid < MAX_NR_ZONES; zid++) {
+			if (!nr_skipped[zid])
+				continue;
+
+			__count_zid_vm_events(PGSCAN_SKIP, zid, nr_skipped[zid]);
+		}
+	}
 	*nr_scanned = scan;
 	trace_mm_vmscan_lru_isolate(sc->reclaim_idx, sc->order, nr_to_scan, scan,
 				    nr_taken, mode, is_file_lru(lru));
@@ -2684,7 +2695,7 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 	delayacct_freepages_start();
 
 	if (global_reclaim(sc))
-		count_vm_event(ALLOCSTALL);
+		__count_zid_vm_events(PGSTALL, classzone_idx, 1);
 
 	do {
 		vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup,
diff --git a/mm/vmstat.c b/mm/vmstat.c
index ec96d20eeb52..f5f20db200c6 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -979,6 +979,8 @@ const char * const vmstat_text[] = {
 	"pswpout",
 
 	TEXTS_FOR_ZONES("pgalloc")
+	TEXTS_FOR_ZONES("pgstall")
+	TEXTS_FOR_ZONES("pgskip")
 
 	"pgfree",
 	"pgactivate",
@@ -1004,7 +1006,6 @@ const char * const vmstat_text[] = {
 	"kswapd_low_wmark_hit_quickly",
 	"kswapd_high_wmark_hit_quickly",
 	"pageoutrun",
-	"allocstall",
 
 	"pgrotated",
 
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1427830 — [PATCH 26/27] mm: vmstat: Replace __count_zone_vm_events with a zone id equivalent

This is partially a preparation patch for more vmstat work but it also
has the slight advantage that __count_zid_vm_events is cheaper to
calculate than __count_zone_vm_events().

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/linux/vmstat.h | 5 ++---
 mm/page_alloc.c        | 2 +-
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h
index c31f8dc6121c..1cab6dd300ac 100644
--- a/include/linux/vmstat.h
+++ b/include/linux/vmstat.h
@@ -107,9 +107,8 @@ static inline void vm_events_fold_cpu(int cpu)
 #define count_vm_vmacache_event(x) do {} while (0)
 #endif
 
-#define __count_zone_vm_events(item, zone, delta) \
-		__count_vm_events(item##_NORMAL - ZONE_NORMAL + \
-		zone_idx(zone), delta)
+#define __count_zid_vm_events(item, zid, delta) \
+	__count_vm_events(item##_NORMAL - ZONE_NORMAL + zid, delta)
 
 /*
  * Zone and node-based page accounting with per cpu differentials.
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 6d0a527cff3d..f5b4f5a372fc 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2621,7 +2621,7 @@ struct page *buffered_rmqueue(struct zone *preferred_zone,
 					  get_pcppage_migratetype(page));
 	}
 
-	__count_zone_vm_events(PGALLOC, zone, 1 << order);
+	__count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order);
 	zone_statistics(preferred_zone, zone, gfp_flags);
 	local_irq_restore(flags);
 
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1427831 — [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes

Historically dirty pages were spread among zones but now that LRUs are
per-node it is more appropriate to consider dirty pages in a node.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 include/linux/mmzone.h    | 12 +++---
 include/linux/writeback.h |  2 +-
 mm/page-writeback.c       | 94 ++++++++++++++++++++++++-----------------------
 mm/page_alloc.c           | 26 ++++++-------
 4 files changed, 67 insertions(+), 67 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 60a1becfd256..ca9e05ca851d 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -361,12 +361,6 @@ struct zone {
 	struct pglist_data	*zone_pgdat;
 	struct per_cpu_pageset __percpu *pageset;
 
-	/*
-	 * This is a per-zone reserve of pages that are not available
-	 * to userspace allocations.
-	 */
-	unsigned long		totalreserve_pages;
-
 #ifndef CONFIG_SPARSEMEM
 	/*
 	 * Flags for a pageblock_nr_pages block. See pageblock-flags.h.
@@ -684,6 +678,12 @@ typedef struct pglist_data {
 	/* Number of pages migrated during the rate limiting time interval */
 	unsigned long numabalancing_migrate_nr_pages;
 #endif
+	/*
+	 * This is a per-node reserve of pages that are not available
+	 * to userspace allocations.
+	 */
+	unsigned long		totalreserve_pages;
+
 	/* Write-intensive fields used from the page allocator */
 	ZONE_PADDING(_pad1_)
 	spinlock_t		lru_lock;
diff --git a/include/linux/writeback.h b/include/linux/writeback.h
index d0b5ca5d4e08..44b4422ae57f 100644
--- a/include/linux/writeback.h
+++ b/include/linux/writeback.h
@@ -320,7 +320,7 @@ void laptop_mode_timer_fn(unsigned long data);
 static inline void laptop_sync_completion(void) { }
 #endif
 void throttle_vm_writeout(gfp_t gfp_mask);
-bool zone_dirty_ok(struct zone *zone);
+bool node_dirty_ok(struct pglist_data *pgdat);
 int wb_domain_init(struct wb_domain *dom, gfp_t gfp);
 #ifdef CONFIG_CGROUP_WRITEBACK
 void wb_domain_exit(struct wb_domain *dom);
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index a0aa268b6281..a2b24d5ea43a 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -267,26 +267,35 @@ static void wb_min_max_ratio(struct bdi_writeback *wb,
  */
 
 /**
- * zone_dirtyable_memory - number of dirtyable pages in a zone
- * @zone: the zone
+ * node_dirtyable_memory - number of dirtyable pages in a node
+ * @pgdat: the node
  *
- * Returns the zone's number of pages potentially available for dirty
- * page cache.  This is the base value for the per-zone dirty limits.
+ * Returns the node's number of pages potentially available for dirty
+ * page cache.  This is the base value for the per-node dirty limits.
  */
-static unsigned long zone_dirtyable_memory(struct zone *zone)
+static unsigned long node_dirtyable_memory(struct pglist_data *pgdat)
 {
-	unsigned long nr_pages;
+	unsigned long nr_pages = 0;
+	int z;
+
+	for (z = 0; z < MAX_NR_ZONES; z++) {
+		struct zone *zone = pgdat->node_zones + z;
+
+		if (!populated_zone(zone))
+			continue;
+
+		nr_pages += zone_page_state(zone, NR_FREE_PAGES);
+	}
 
-	nr_pages = zone_page_state(zone, NR_FREE_PAGES);
 	/*
 	 * Pages reserved for the kernel should not be considered
 	 * dirtyable, to prevent a situation where reclaim has to
 	 * clean pages in order to balance the zones.
 	 */
-	nr_pages -= min(nr_pages, zone->totalreserve_pages);
+	nr_pages -= min(nr_pages, pgdat->totalreserve_pages);
 
-	nr_pages += node_page_state(zone->zone_pgdat, NR_INACTIVE_FILE);
-	nr_pages += node_page_state(zone->zone_pgdat, NR_ACTIVE_FILE);
+	nr_pages += node_page_state(pgdat, NR_INACTIVE_FILE);
+	nr_pages += node_page_state(pgdat, NR_ACTIVE_FILE);
 
 	return nr_pages;
 }
@@ -294,29 +303,13 @@ static unsigned long zone_dirtyable_memory(struct zone *zone)
 static unsigned long highmem_dirtyable_memory(unsigned long total)
 {
 #ifdef CONFIG_HIGHMEM
-	int node;
 	unsigned long x = 0;
-	int i;
-
-	for_each_node_state(node, N_HIGH_MEMORY) {
-		for (i = 0; i < MAX_NR_ZONES; i++) {
-			struct zone *z = &NODE_DATA(node)->node_zones[i];
 
-			if (is_highmem(z))
-				x += zone_dirtyable_memory(z);
-		}
-	}
 	/*
-	 * Unreclaimable memory (kernel memory or anonymous memory
-	 * without swap) can bring down the dirtyable pages below
-	 * the zone's dirty balance reserve and the above calculation
-	 * will underflow.  However we still want to add in nodes
-	 * which are below threshold (negative values) to get a more
-	 * accurate calculation but make sure that the total never
-	 * underflows.
+	 * LRU lists are per-node so there is no fast and accurate means of
+	 * calculating dirtyable memory in the highmem zone.
 	 */
-	if ((long)x < 0)
-		x = 0;
+	x = totalhigh_pages;
 
 	/*
 	 * Make sure that the number of highmem pages is never larger
@@ -445,23 +438,23 @@ void global_dirty_limits(unsigned long *pbackground, unsigned long *pdirty)
 }
 
 /**
- * zone_dirty_limit - maximum number of dirty pages allowed in a zone
- * @zone: the zone
+ * node_dirty_limit - maximum number of dirty pages allowed in a node
+ * @pgdat: the node
  *
- * Returns the maximum number of dirty pages allowed in a zone, based
- * on the zone's dirtyable memory.
+ * Returns the maximum number of dirty pages allowed in a node, based
+ * on the node's dirtyable memory.
  */
-static unsigned long zone_dirty_limit(struct zone *zone)
+static unsigned long node_dirty_limit(struct pglist_data *pgdat)
 {
-	unsigned long zone_memory = zone_dirtyable_memory(zone);
+	unsigned long node_memory = node_dirtyable_memory(pgdat);
 	struct task_struct *tsk = current;
 	unsigned long dirty;
 
 	if (vm_dirty_bytes)
 		dirty = DIV_ROUND_UP(vm_dirty_bytes, PAGE_SIZE) *
-			zone_memory / global_dirtyable_memory();
+			node_memory / global_dirtyable_memory();
 	else
-		dirty = vm_dirty_ratio * zone_memory / 100;
+		dirty = vm_dirty_ratio * node_memory / 100;
 
 	if (tsk->flags & PF_LESS_THROTTLE || rt_task(tsk))
 		dirty += dirty / 4;
@@ -470,19 +463,30 @@ static unsigned long zone_dirty_limit(struct zone *zone)
 }
 
 /**
- * zone_dirty_ok - tells whether a zone is within its dirty limits
- * @zone: the zone to check
+ * node_dirty_ok - tells whether a node is within its dirty limits
+ * @pgdat: the node to check
  *
- * Returns %true when the dirty pages in @zone are within the zone's
+ * Returns %true when the dirty pages in @pgdat are within the node's
  * dirty limit, %false if the limit is exceeded.
  */
-bool zone_dirty_ok(struct zone *zone)
+bool node_dirty_ok(struct pglist_data *pgdat)
 {
-	unsigned long limit = zone_dirty_limit(zone);
+	int z;
+	unsigned long limit = node_dirty_limit(pgdat);
+	unsigned long nr_pages = 0;
+
+	for (z = 0; z < MAX_NR_ZONES; z++) {
+		struct zone *zone = pgdat->node_zones + z;
+
+		if (!populated_zone(zone))
+			continue;
+
+		nr_pages += zone_page_state(zone, NR_FILE_DIRTY);
+		nr_pages += zone_page_state(zone, NR_UNSTABLE_NFS);
+		nr_pages += zone_page_state(zone, NR_WRITEBACK);
+	}
 
-	return zone_page_state(zone, NR_FILE_DIRTY) +
-	       zone_page_state(zone, NR_UNSTABLE_NFS) +
-	       zone_page_state(zone, NR_WRITEBACK) <= limit;
+	return nr_pages <= limit;
 }
 
 int dirty_background_ratio_handler(struct ctl_table *table, int write,
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index d62b147fd426..962e98e54fd2 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2917,31 +2917,24 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 		}
 		/*
 		 * When allocating a page cache page for writing, we
-		 * want to get it from a zone that is within its dirty
-		 * limit, such that no single zone holds more than its
+		 * want to get it from a node that is within its dirty
+		 * limit, such that no single node holds more than its
 		 * proportional share of globally allowed dirty pages.
-		 * The dirty limits take into account the zone's
+		 * The dirty limits take into account the node's
 		 * lowmem reserves and high watermark so that kswapd
 		 * should be able to balance it without having to
 		 * write pages from its LRU list.
 		 *
-		 * This may look like it could increase pressure on
-		 * lower zones by failing allocations in higher zones
-		 * before they are full.  But the pages that do spill
-		 * over are limited as the lower zones are protected
-		 * by this very same mechanism.  It should not become
-		 * a practical burden to them.
-		 *
 		 * XXX: For now, allow allocations to potentially
-		 * exceed the per-zone dirty limit in the slowpath
+		 * exceed the per-node dirty limit in the slowpath
 		 * (spread_dirty_pages unset) before going into reclaim,
 		 * which is important when on a NUMA setup the allowed
-		 * zones are together not big enough to reach the
+		 * nodes are together not big enough to reach the
 		 * global limit.  The proper fix for these situations
-		 * will require awareness of zones in the
+		 * will require awareness of nodes in the
 		 * dirty-throttling and the flusher threads.
 		 */
-		if (ac->spread_dirty_pages && !zone_dirty_ok(zone))
+		if (ac->spread_dirty_pages && !node_dirty_ok(zone->zone_pgdat))
 			continue;
 
 		mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
@@ -6692,6 +6685,9 @@ static void calculate_totalreserve_pages(void)
 	enum zone_type i, j;
 
 	for_each_online_pgdat(pgdat) {
+
+		pgdat->totalreserve_pages = 0;
+
 		for (i = 0; i < MAX_NR_ZONES; i++) {
 			struct zone *zone = pgdat->node_zones + i;
 			long max = 0;
@@ -6708,7 +6704,7 @@ static void calculate_totalreserve_pages(void)
 			if (max > zone->managed_pages)
 				max = zone->managed_pages;
 
-			zone->totalreserve_pages = max;
+			pgdat->totalreserve_pages += max;
 
 			reserve_pages += max;
 		}
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1428839 — Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes

On Tue 21-06-16 15:15:54, Mel Gorman wrote:
> Historically dirty pages were spread among zones but now that LRUs are
> per-node it is more appropriate to consider dirty pages in a node.

I think this should deserve a note that a behavior for 32b highmem
systems will change and could lead to early write throttling and
observable stalls as a result because highmem_dirtyable_memory will
always return totalhigh_pages regardless of how much is free resp. on
LRUs so we can overestimate it.

Highmem is usually used for LRU pages but there are other allocations
which can use it (e.g. vmalloc). I understand how this is both an
inherent problem of 32b with a larger high:low ratio and why it is hard
to at least pretend we can cope with it with node based approach but we
should at least document it.

I workaround would be to enable highmem_dirtyable_memory which can lead
to premature OOM killer for some workloads AFAIR.

> Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>

The patch looks ok otherwise to me.
Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  include/linux/mmzone.h    | 12 +++---
>  include/linux/writeback.h |  2 +-
>  mm/page-writeback.c       | 94 ++++++++++++++++++++++++-----------------------
>  mm/page_alloc.c           | 26 ++++++-------
>  4 files changed, 67 insertions(+), 67 deletions(-)
> 
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 60a1becfd256..ca9e05ca851d 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -361,12 +361,6 @@ struct zone {
>  	struct pglist_data	*zone_pgdat;
>  	struct per_cpu_pageset __percpu *pageset;
>  
> -	/*
> -	 * This is a per-zone reserve of pages that are not available
> -	 * to userspace allocations.
> -	 */
> -	unsigned long		totalreserve_pages;
> -
>  #ifndef CONFIG_SPARSEMEM
>  	/*
>  	 * Flags for a pageblock_nr_pages block. See pageblock-flags.h.
> @@ -684,6 +678,12 @@ typedef struct pglist_data {
>  	/* Number of pages migrated during the rate limiting time interval */
>  	unsigned long numabalancing_migrate_nr_pages;
>  #endif
> +	/*
> +	 * This is a per-node reserve of pages that are not available
> +	 * to userspace allocations.
> +	 */
> +	unsigned long		totalreserve_pages;
> +
>  	/* Write-intensive fields used from the page allocator */
>  	ZONE_PADDING(_pad1_)
>  	spinlock_t		lru_lock;
> diff --git a/include/linux/writeback.h b/include/linux/writeback.h
> index d0b5ca5d4e08..44b4422ae57f 100644
> --- a/include/linux/writeback.h
> +++ b/include/linux/writeback.h
> @@ -320,7 +320,7 @@ void laptop_mode_timer_fn(unsigned long data);
>  static inline void laptop_sync_completion(void) { }
>  #endif
>  void throttle_vm_writeout(gfp_t gfp_mask);
> -bool zone_dirty_ok(struct zone *zone);
> +bool node_dirty_ok(struct pglist_data *pgdat);
>  int wb_domain_init(struct wb_domain *dom, gfp_t gfp);
>  #ifdef CONFIG_CGROUP_WRITEBACK
>  void wb_domain_exit(struct wb_domain *dom);
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index a0aa268b6281..a2b24d5ea43a 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -267,26 +267,35 @@ static void wb_min_max_ratio(struct bdi_writeback *wb,
>   */
>  
>  /**
> - * zone_dirtyable_memory - number of dirtyable pages in a zone
> - * @zone: the zone
> + * node_dirtyable_memory - number of dirtyable pages in a node
> + * @pgdat: the node
>   *
> - * Returns the zone's number of pages potentially available for dirty
> - * page cache.  This is the base value for the per-zone dirty limits.
> + * Returns the node's number of pages potentially available for dirty
> + * page cache.  This is the base value for the per-node dirty limits.
>   */
> -static unsigned long zone_dirtyable_memory(struct zone *zone)
> +static unsigned long node_dirtyable_memory(struct pglist_data *pgdat)
>  {
> -	unsigned long nr_pages;
> +	unsigned long nr_pages = 0;
> +	int z;
> +
> +	for (z = 0; z < MAX_NR_ZONES; z++) {
> +		struct zone *zone = pgdat->node_zones + z;
> +
> +		if (!populated_zone(zone))
> +			continue;
> +
> +		nr_pages += zone_page_state(zone, NR_FREE_PAGES);
> +	}
>  
> -	nr_pages = zone_page_state(zone, NR_FREE_PAGES);
>  	/*
>  	 * Pages reserved for the kernel should not be considered
>  	 * dirtyable, to prevent a situation where reclaim has to
>  	 * clean pages in order to balance the zones.
>  	 */
> -	nr_pages -= min(nr_pages, zone->totalreserve_pages);
> +	nr_pages -= min(nr_pages, pgdat->totalreserve_pages);
>  
> -	nr_pages += node_page_state(zone->zone_pgdat, NR_INACTIVE_FILE);
> -	nr_pages += node_page_state(zone->zone_pgdat, NR_ACTIVE_FILE);
> +	nr_pages += node_page_state(pgdat, NR_INACTIVE_FILE);
> +	nr_pages += node_page_state(pgdat, NR_ACTIVE_FILE);
>  
>  	return nr_pages;
>  }
> @@ -294,29 +303,13 @@ static unsigned long zone_dirtyable_memory(struct zone *zone)
>  static unsigned long highmem_dirtyable_memory(unsigned long total)
>  {
>  #ifdef CONFIG_HIGHMEM
> -	int node;
>  	unsigned long x = 0;
> -	int i;
> -
> -	for_each_node_state(node, N_HIGH_MEMORY) {
> -		for (i = 0; i < MAX_NR_ZONES; i++) {
> -			struct zone *z = &NODE_DATA(node)->node_zones[i];
>  
> -			if (is_highmem(z))
> -				x += zone_dirtyable_memory(z);
> -		}
> -	}
>  	/*
> -	 * Unreclaimable memory (kernel memory or anonymous memory
> -	 * without swap) can bring down the dirtyable pages below
> -	 * the zone's dirty balance reserve and the above calculation
> -	 * will underflow.  However we still want to add in nodes
> -	 * which are below threshold (negative values) to get a more
> -	 * accurate calculation but make sure that the total never
> -	 * underflows.
> +	 * LRU lists are per-node so there is no fast and accurate means of
> +	 * calculating dirtyable memory in the highmem zone.
>  	 */
> -	if ((long)x < 0)
> -		x = 0;
> +	x = totalhigh_pages;
>  
>  	/*
>  	 * Make sure that the number of highmem pages is never larger
> @@ -445,23 +438,23 @@ void global_dirty_limits(unsigned long *pbackground, unsigned long *pdirty)
>  }
>  
>  /**
> - * zone_dirty_limit - maximum number of dirty pages allowed in a zone
> - * @zone: the zone
> + * node_dirty_limit - maximum number of dirty pages allowed in a node
> + * @pgdat: the node
>   *
> - * Returns the maximum number of dirty pages allowed in a zone, based
> - * on the zone's dirtyable memory.
> + * Returns the maximum number of dirty pages allowed in a node, based
> + * on the node's dirtyable memory.
>   */
> -static unsigned long zone_dirty_limit(struct zone *zone)
> +static unsigned long node_dirty_limit(struct pglist_data *pgdat)
>  {
> -	unsigned long zone_memory = zone_dirtyable_memory(zone);
> +	unsigned long node_memory = node_dirtyable_memory(pgdat);
>  	struct task_struct *tsk = current;
>  	unsigned long dirty;
>  
>  	if (vm_dirty_bytes)
>  		dirty = DIV_ROUND_UP(vm_dirty_bytes, PAGE_SIZE) *
> -			zone_memory / global_dirtyable_memory();
> +			node_memory / global_dirtyable_memory();
>  	else
> -		dirty = vm_dirty_ratio * zone_memory / 100;
> +		dirty = vm_dirty_ratio * node_memory / 100;
>  
>  	if (tsk->flags & PF_LESS_THROTTLE || rt_task(tsk))
>  		dirty += dirty / 4;
> @@ -470,19 +463,30 @@ static unsigned long zone_dirty_limit(struct zone *zone)
>  }
>  
>  /**
> - * zone_dirty_ok - tells whether a zone is within its dirty limits
> - * @zone: the zone to check
> + * node_dirty_ok - tells whether a node is within its dirty limits
> + * @pgdat: the node to check
>   *
> - * Returns %true when the dirty pages in @zone are within the zone's
> + * Returns %true when the dirty pages in @pgdat are within the node's
>   * dirty limit, %false if the limit is exceeded.
>   */
> -bool zone_dirty_ok(struct zone *zone)
> +bool node_dirty_ok(struct pglist_data *pgdat)
>  {
> -	unsigned long limit = zone_dirty_limit(zone);
> +	int z;
> +	unsigned long limit = node_dirty_limit(pgdat);
> +	unsigned long nr_pages = 0;
> +
> +	for (z = 0; z < MAX_NR_ZONES; z++) {
> +		struct zone *zone = pgdat->node_zones + z;
> +
> +		if (!populated_zone(zone))
> +			continue;
> +
> +		nr_pages += zone_page_state(zone, NR_FILE_DIRTY);
> +		nr_pages += zone_page_state(zone, NR_UNSTABLE_NFS);
> +		nr_pages += zone_page_state(zone, NR_WRITEBACK);
> +	}
>  
> -	return zone_page_state(zone, NR_FILE_DIRTY) +
> -	       zone_page_state(zone, NR_UNSTABLE_NFS) +
> -	       zone_page_state(zone, NR_WRITEBACK) <= limit;
> +	return nr_pages <= limit;
>  }
>  
>  int dirty_background_ratio_handler(struct ctl_table *table, int write,
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index d62b147fd426..962e98e54fd2 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2917,31 +2917,24 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
>  		}
>  		/*
>  		 * When allocating a page cache page for writing, we
> -		 * want to get it from a zone that is within its dirty
> -		 * limit, such that no single zone holds more than its
> +		 * want to get it from a node that is within its dirty
> +		 * limit, such that no single node holds more than its
>  		 * proportional share of globally allowed dirty pages.
> -		 * The dirty limits take into account the zone's
> +		 * The dirty limits take into account the node's
>  		 * lowmem reserves and high watermark so that kswapd
>  		 * should be able to balance it without having to
>  		 * write pages from its LRU list.
>  		 *
> -		 * This may look like it could increase pressure on
> -		 * lower zones by failing allocations in higher zones
> -		 * before they are full.  But the pages that do spill
> -		 * over are limited as the lower zones are protected
> -		 * by this very same mechanism.  It should not become
> -		 * a practical burden to them.
> -		 *
>  		 * XXX: For now, allow allocations to potentially
> -		 * exceed the per-zone dirty limit in the slowpath
> +		 * exceed the per-node dirty limit in the slowpath
>  		 * (spread_dirty_pages unset) before going into reclaim,
>  		 * which is important when on a NUMA setup the allowed
> -		 * zones are together not big enough to reach the
> +		 * nodes are together not big enough to reach the
>  		 * global limit.  The proper fix for these situations
> -		 * will require awareness of zones in the
> +		 * will require awareness of nodes in the
>  		 * dirty-throttling and the flusher threads.
>  		 */
> -		if (ac->spread_dirty_pages && !zone_dirty_ok(zone))
> +		if (ac->spread_dirty_pages && !node_dirty_ok(zone->zone_pgdat))
>  			continue;
>  
>  		mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
> @@ -6692,6 +6685,9 @@ static void calculate_totalreserve_pages(void)
>  	enum zone_type i, j;
>  
>  	for_each_online_pgdat(pgdat) {
> +
> +		pgdat->totalreserve_pages = 0;
> +
>  		for (i = 0; i < MAX_NR_ZONES; i++) {
>  			struct zone *zone = pgdat->node_zones + i;
>  			long max = 0;
> @@ -6708,7 +6704,7 @@ static void calculate_totalreserve_pages(void)
>  			if (max > zone->managed_pages)
>  				max = zone->managed_pages;
>  
> -			zone->totalreserve_pages = max;
> +			pgdat->totalreserve_pages += max;
>  
>  			reserve_pages += max;
>  		}
> -- 
> 2.6.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1428843 — Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes

On Wed 22-06-16 16:15:21, Michal Hocko wrote:
> On Tue 21-06-16 15:15:54, Mel Gorman wrote:
> > Historically dirty pages were spread among zones but now that LRUs are
> > per-node it is more appropriate to consider dirty pages in a node.
> 
> I think this should deserve a note that a behavior for 32b highmem
> systems will change and could lead to early write throttling and
> observable stalls as a result because highmem_dirtyable_memory will
> always return totalhigh_pages regardless of how much is free resp. on
> LRUs so we can overestimate it.
> 
> Highmem is usually used for LRU pages but there are other allocations
> which can use it (e.g. vmalloc). I understand how this is both an
> inherent problem of 32b with a larger high:low ratio and why it is hard
> to at least pretend we can cope with it with node based approach but we
> should at least document it.
> 
> I workaround would be to enable highmem_dirtyable_memory which can lead
> to premature OOM killer for some workloads AFAIR.
[...]
> >  static unsigned long highmem_dirtyable_memory(unsigned long total)
> >  {
> >  #ifdef CONFIG_HIGHMEM
> > -	int node;
> >  	unsigned long x = 0;
> > -	int i;
> > -
> > -	for_each_node_state(node, N_HIGH_MEMORY) {
> > -		for (i = 0; i < MAX_NR_ZONES; i++) {
> > -			struct zone *z = &NODE_DATA(node)->node_zones[i];
> >  
> > -			if (is_highmem(z))
> > -				x += zone_dirtyable_memory(z);
> > -		}
> > -	}

Hmm, I have just noticed that we have NR_ZONE_LRU_ANON resp.
NR_ZONE_LRU_FILE so we can estimate the amount of highmem contribution
to the global counters by the following or similar:

	for_each_node_state(node, N_HIGH_MEMORY) {
		for (i = 0; i < MAX_NR_ZONES; i++) {
			struct zone *z = &NODE_DATA(node)->node_zones[i];

			if (!is_highmem(z))
				continue;

			x += zone_page_state(z, NR_FREE_PAGES) + zone_page_state(z, NR_ZONE_LRU_FILE) - high_wmark_pages(zone);
		}

high wmark reduction would be to emulate the reserve. What do you think?
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1429788 — Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes

On Wed, Jun 22, 2016 at 04:27:57PM +0200, Michal Hocko wrote:
> > which can use it (e.g. vmalloc). I understand how this is both an
> > inherent problem of 32b with a larger high:low ratio and why it is hard
> > to at least pretend we can cope with it with node based approach but we
> > should at least document it.
> > 
> > I workaround would be to enable highmem_dirtyable_memory which can lead
> > to premature OOM killer for some workloads AFAIR.
> [...]
> > >  static unsigned long highmem_dirtyable_memory(unsigned long total)
> > >  {
> > >  #ifdef CONFIG_HIGHMEM
> > > -	int node;
> > >  	unsigned long x = 0;
> > > -	int i;
> > > -
> > > -	for_each_node_state(node, N_HIGH_MEMORY) {
> > > -		for (i = 0; i < MAX_NR_ZONES; i++) {
> > > -			struct zone *z = &NODE_DATA(node)->node_zones[i];
> > >  
> > > -			if (is_highmem(z))
> > > -				x += zone_dirtyable_memory(z);
> > > -		}
> > > -	}
> 
> Hmm, I have just noticed that we have NR_ZONE_LRU_ANON resp.
> NR_ZONE_LRU_FILE so we can estimate the amount of highmem contribution
> to the global counters by the following or similar:
> 
> 	for_each_node_state(node, N_HIGH_MEMORY) {
> 		for (i = 0; i < MAX_NR_ZONES; i++) {
> 			struct zone *z = &NODE_DATA(node)->node_zones[i];
> 
> 			if (!is_highmem(z))
> 				continue;
> 
> 			x += zone_page_state(z, NR_FREE_PAGES) + zone_page_state(z, NR_ZONE_LRU_FILE) - high_wmark_pages(zone);
> 		}
> 
> high wmark reduction would be to emulate the reserve. What do you think?

Agreed with minor modifications. Went with this

        for_each_node_state(node, N_HIGH_MEMORY) {
                for (i = ZONE_NORMAL + 1; i < MAX_NR_ZONES; i++) {
                        struct zone *z;

                        if (!is_highmem_idx(z))
                                continue;

                        z = &NODE_DATA(node)->node_zones[i];
                        x += zone_page_state(z, NR_FREE_PAGES) +
                                zone_page_state(z, NR_ZONE_LRU_FILE) -
                                high_wmark_pages(zone);
                }
        }

-- 
Mel Gorman
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1429807 — Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes

On Thu 23-06-16 13:53:12, Mel Gorman wrote:
> On Wed, Jun 22, 2016 at 04:27:57PM +0200, Michal Hocko wrote:
> > > which can use it (e.g. vmalloc). I understand how this is both an
> > > inherent problem of 32b with a larger high:low ratio and why it is hard
> > > to at least pretend we can cope with it with node based approach but we
> > > should at least document it.
> > > 
> > > I workaround would be to enable highmem_dirtyable_memory which can lead
> > > to premature OOM killer for some workloads AFAIR.
> > [...]
> > > >  static unsigned long highmem_dirtyable_memory(unsigned long total)
> > > >  {
> > > >  #ifdef CONFIG_HIGHMEM
> > > > -	int node;
> > > >  	unsigned long x = 0;
> > > > -	int i;
> > > > -
> > > > -	for_each_node_state(node, N_HIGH_MEMORY) {
> > > > -		for (i = 0; i < MAX_NR_ZONES; i++) {
> > > > -			struct zone *z = &NODE_DATA(node)->node_zones[i];
> > > >  
> > > > -			if (is_highmem(z))
> > > > -				x += zone_dirtyable_memory(z);
> > > > -		}
> > > > -	}
> > 
> > Hmm, I have just noticed that we have NR_ZONE_LRU_ANON resp.
> > NR_ZONE_LRU_FILE so we can estimate the amount of highmem contribution
> > to the global counters by the following or similar:
> > 
> > 	for_each_node_state(node, N_HIGH_MEMORY) {
> > 		for (i = 0; i < MAX_NR_ZONES; i++) {
> > 			struct zone *z = &NODE_DATA(node)->node_zones[i];
> > 
> > 			if (!is_highmem(z))
> > 				continue;
> > 
> > 			x += zone_page_state(z, NR_FREE_PAGES) + zone_page_state(z, NR_ZONE_LRU_FILE) - high_wmark_pages(zone);
> > 		}
> > 
> > high wmark reduction would be to emulate the reserve. What do you think?
> 
> Agreed with minor modifications. Went with this
> 
>         for_each_node_state(node, N_HIGH_MEMORY) {
>                 for (i = ZONE_NORMAL + 1; i < MAX_NR_ZONES; i++) {
>                         struct zone *z;
> 
>                         if (!is_highmem_idx(z))
>                                 continue;
> 
>                         z = &NODE_DATA(node)->node_zones[i];
>                         x += zone_page_state(z, NR_FREE_PAGES) +
>                                 zone_page_state(z, NR_ZONE_LRU_FILE) -
>                                 high_wmark_pages(zone);

I guess you will still need an underflow protection. Because both free +
lru pages might be below high wmark.

			dirtyable += zone_page_state(z, NR_FREE_PAGES) +
					zone_page_state(z, NR_ZONE_LRU_FILE);
			if (dirtyable > high_wmark_pages(zone)
				dirtyable -= high_wmark_pages(zone);

			x += dirtyable;
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1427834 — [PATCH 25/27] mm: page_alloc: Cache the last node whose dirty limit is reached

If a page is about to be dirtied then the page allocator attempts to limit
the total number of dirty pages that exists in any given zone. The call
to node_dirty_ok is expensive so this patch records if the last pgdat
examined hit the dirty limits. In some cases, this reduces the number
of calls to node_dirty_ok().

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 mm/page_alloc.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1b5080475481..6d0a527cff3d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2853,6 +2853,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 {
 	struct zoneref *z = ac->preferred_zoneref;
 	struct zone *zone;
+	struct pglist_data *last_pgdat_dirty_limit = NULL;
+
 	/*
 	 * Scan zonelist, looking for a zone with enough free.
 	 * See also __cpuset_node_allowed() comment in kernel/cpuset.c.
@@ -2885,8 +2887,15 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 		 * will require awareness of nodes in the
 		 * dirty-throttling and the flusher threads.
 		 */
-		if (ac->spread_dirty_pages && !node_dirty_ok(zone->zone_pgdat))
-			continue;
+		if (ac->spread_dirty_pages) {
+			if (last_pgdat_dirty_limit == zone->zone_pgdat)
+				continue;
+
+			if (!node_dirty_ok(zone->zone_pgdat)) {
+				last_pgdat_dirty_limit = zone->zone_pgdat;
+				continue;
+			}
+		}
 
 		mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
 		if (!zone_watermark_fast(zone, order, mark,
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

#1427842 — [PATCH 21/27] mm, vmscan: Only wakeup kswapd once per node for the requested classzone

kswapd is woken when zones are below the low watermark but the wakeup
decision is not taking the classzone into account.  Now that reclaim is
node-based, it is only required to wake kswapd once per node and only if
all zones are unbalanced for the requested classzone.

Note that one node might be checked multiple times if the zonelist is ordered
by node because there is no cheap way of tracking what nodes have already
been visited. For zone-ordering, each node should be checked only once.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
 mm/page_alloc.c |  8 ++++++--
 mm/vmscan.c     | 13 +++++++++++--
 2 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 6f5120a282c3..fc2eaa122770 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3372,10 +3372,14 @@ static void wake_all_kswapds(unsigned int order, const struct alloc_context *ac)
 {
 	struct zoneref *z;
 	struct zone *zone;
+	pg_data_t *last_pgdat = NULL;
 
 	for_each_zone_zonelist_nodemask(zone, z, ac->zonelist,
-						ac->high_zoneidx, ac->nodemask)
-		wakeup_kswapd(zone, order, ac_classzone_idx(ac));
+					ac->high_zoneidx, ac->nodemask) {
+		if (last_pgdat != zone->zone_pgdat)
+			wakeup_kswapd(zone, order, ac_classzone_idx(ac));
+		last_pgdat = zone->zone_pgdat;
+	}
 }
 
 static inline unsigned int
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f4c759b3581b..456f2d209651 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3429,6 +3429,7 @@ static int kswapd(void *p)
 void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx)
 {
 	pg_data_t *pgdat;
+	int z;
 
 	if (!populated_zone(zone))
 		return;
@@ -3442,8 +3443,16 @@ void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx)
 	pgdat->kswapd_order = max(pgdat->kswapd_order, order);
 	if (!waitqueue_active(&pgdat->kswapd_wait))
 		return;
-	if (zone_balanced(zone, order, 0))
-		return;
+
+	/* Only wake kswapd if all zones are unbalanced */
+	for (z = 0; z <= classzone_idx; z++) {
+		zone = pgdat->node_zones + z;
+		if (!populated_zone(zone))
+			continue;
+
+		if (zone_balanced(zone, order, classzone_idx))
+			return;
+	}
 
 	trace_mm_vmscan_wakeup_kswapd(pgdat->node_id, zone_idx(zone), order);
 	wake_up_interruptible(&pgdat->kswapd_wait);
-- 
2.6.4

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.kernel

csiph-web