Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1428843

Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes

Show all headers | View raw

On Wed 22-06-16 16:15:21, Michal Hocko wrote:
> On Tue 21-06-16 15:15:54, Mel Gorman wrote:
> > Historically dirty pages were spread among zones but now that LRUs are
> > per-node it is more appropriate to consider dirty pages in a node.
> 
> I think this should deserve a note that a behavior for 32b highmem
> systems will change and could lead to early write throttling and
> observable stalls as a result because highmem_dirtyable_memory will
> always return totalhigh_pages regardless of how much is free resp. on
> LRUs so we can overestimate it.
> 
> Highmem is usually used for LRU pages but there are other allocations
> which can use it (e.g. vmalloc). I understand how this is both an
> inherent problem of 32b with a larger high:low ratio and why it is hard
> to at least pretend we can cope with it with node based approach but we
> should at least document it.
> 
> I workaround would be to enable highmem_dirtyable_memory which can lead
> to premature OOM killer for some workloads AFAIR.
[...]
> >  static unsigned long highmem_dirtyable_memory(unsigned long total)
> >  {
> >  #ifdef CONFIG_HIGHMEM
> > -	int node;
> >  	unsigned long x = 0;
> > -	int i;
> > -
> > -	for_each_node_state(node, N_HIGH_MEMORY) {
> > -		for (i = 0; i < MAX_NR_ZONES; i++) {
> > -			struct zone *z = &NODE_DATA(node)->node_zones[i];
> >  
> > -			if (is_highmem(z))
> > -				x += zone_dirtyable_memory(z);
> > -		}
> > -	}

Hmm, I have just noticed that we have NR_ZONE_LRU_ANON resp.
NR_ZONE_LRU_FILE so we can estimate the amount of highmem contribution
to the global counters by the following or similar:

	for_each_node_state(node, N_HIGH_MEMORY) {
		for (i = 0; i < MAX_NR_ZONES; i++) {
			struct zone *z = &NODE_DATA(node)->node_zones[i];

			if (!is_highmem(z))
				continue;

			x += zone_page_state(z, NR_FREE_PAGES) + zone_page_state(z, NR_ZONE_LRU_FILE) - high_wmark_pages(zone);
		}

high wmark reduction would be to emulate the reserve. What do you think?
-- 
Michal Hocko
SUSE Labs

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 07/27] mm, vmscan: Remove balance gap Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 09/27] mm, vmscan: By default have direct reclaim only shrink once per node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 06/27] mm, vmscan: Make kswapd reclaim in terms of nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 02/27] mm, vmscan: Move lru_lock to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 10/27] mm, vmscan: Remove duplicate logic clearing node congestion and dirty state Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 11/27] mm: vmscan: Do not reclaim from kswapd if there is any eligible zone Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:20 +0200
  [PATCH 14/27] mm, workingset: Make working set detection node-aware Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
  [PATCH 17/27] mm: Rename NR_ANON_PAGES to NR_ANON_MAPPED Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    Re: [PATCH 17/27] mm: Rename NR_ANON_PAGES to NR_ANON_MAPPED Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:30 +0200
  [PATCH 12/27] mm, vmscan: Make shrink_node decisions more node-centric Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    Re: [PATCH 12/27] mm, vmscan: Make shrink_node decisions more  node-centric Michal Hocko <mhocko@kernel.org> - 2016-06-22 15:30 +0200
    Re: [PATCH 12/27] mm, vmscan: Make shrink_node decisions more  node-centric Vlastimil Babka <vbabka@suse.cz> - 2016-06-22 17:50 +0200
  [PATCH 16/27] mm: Move page mapped accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    Re: [PATCH 16/27] mm: Move page mapped accounting to the node Andrew Morton <akpm@linux-foundation.org> - 2016-06-22 00:50 +0200
      Re: [PATCH 16/27] mm: Move page mapped accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 10:40 +0200
    Re: [PATCH 16/27] mm: Move page mapped accounting to the node Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:40 +0200
  [PATCH 13/27] mm, memcg: Move memcg limit enforcement from zones to nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    Re: [PATCH 13/27] mm, memcg: Move memcg limit enforcement from zones  to nodes Michal Hocko <mhocko@kernel.org> - 2016-06-22 15:20 +0200
  [PATCH 22/27] mm: Convert zone_reclaim to node_reclaim Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
  [PATCH 19/27] mm: Move vmscan writes and file write accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting  to the node Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:50 +0200
      Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting  to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 16:00 +0200
        Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting to  the node Vlastimil Babka <vbabka@suse.cz> - 2016-06-23 16:10 +0200
          Re: [PATCH 19/27] mm: Move vmscan writes and file write accounting  to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 18:10 +0200
  [PATCH 24/27] mm, page_alloc: Remove fair zone allocation policy Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
  [PATCH 18/27] mm: Move most file-based accounting to the node Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:30 +0200
    Re: [PATCH 18/27] mm: Move most file-based accounting to the node Michal Hocko <mhocko@kernel.org> - 2016-06-22 17:30 +0200
  [PATCH 23/27] mm, vmscan: Add classzone information to tracepoints Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
  [PATCH 20/27] mm, vmscan: Update classzone_idx if buffer_heads_over_limit Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
    Re: [PATCH 20/27] mm, vmscan: Update classzone_idx if  buffer_heads_over_limit Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:50 +0200
  [PATCH 27/27] mm: vmstat: Account per-zone stalls and pages skipped during reclaim Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
  [PATCH 26/27] mm: vmstat: Replace __count_zone_vm_events with a zone id equivalent Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
  [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms of nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:40 +0200
    Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:30 +0200
      Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Michal Hocko <mhocko@kernel.org> - 2016-06-22 16:30 +0200
        Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 15:00 +0200
          Re: [PATCH 15/27] mm, page_alloc: Consider dirtyable memory in terms  of nodes Michal Hocko <mhocko@kernel.org> - 2016-06-23 15:20 +0200
  [PATCH 25/27] mm: page_alloc: Cache the last node whose dirty limit is reached Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:50 +0200
  [PATCH 21/27] mm, vmscan: Only wakeup kswapd once per node for the requested classzone Mel Gorman <mgorman@techsingularity.net> - 2016-06-21 16:50 +0200
    Re: [PATCH 21/27] mm, vmscan: Only wakeup kswapd once per node for  the requested classzone Vlastimil Babka <vbabka@suse.cz> - 2016-06-22 18:10 +0200
  Re: [PATCH 03/27] mm, vmscan: Move LRU lists to node Vlastimil Babka <vbabka@suse.cz> - 2016-06-22 15:00 +0200
  Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 12:30 +0200
    Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Michal Hocko <mhocko@kernel.org> - 2016-06-23 13:30 +0200
      Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Mel Gorman <mgorman@techsingularity.net> - 2016-06-23 14:40 +0200
        Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 Michal Hocko <mhocko@kernel.org> - 2016-06-23 14:50 +0200

csiph-web