Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1577587 > unrolled thread

[PATCH 0/3 staging-next] android: Lowmemmorykiller task tree

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree peter enderborg <peter.enderborg@sonymobile.com> - 2017-02-09 14:40 +0100
    [PATCH 1/3 v2 staging-next] android: Collect statistics from  lowmemorykiller peter enderborg <peter.enderborg@sonymobile.com> - 2017-02-09 17:00 +0100
      Re: [PATCH 1/3 v2 staging-next] android: Collect statistics from  lowmemorykiller Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-02-09 21:40 +0100
        Re: [PATCH 1/3 v2 staging-next] android: Collect statistics from  lowmemorykiller peter enderborg <peter.enderborg@sonymobile.com> - 2017-02-10 08:20 +0100
    Re: [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree Michal Hocko <mhocko@kernel.org> - 2017-02-10 11:30 +0100
      Re: [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree peter enderborg <peter.enderborg@sonymobile.com> - 2017-02-13 16:50 +0100

#1577587 — [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree

Lowmemorykiller efficiency problem and a solution.

Lowmemorykiller in android has a severe efficiency problem. The basic
problem is that the registered shrinker gets called very often without
  anything actually happening. This is in some cases not a problem as
it is a simple calculation that returns a value. But when there is
high pressure on memory and we get to start killing processes to free
memory we get some heavy work that does lots of cpu processing and
lock holding for no real benefit. This occurs when we are below the
first threshold level in minfree. We call that waste. To see this
problem we introduce a patch that collects statistics from
lowmemorykiller. We collect the amount of kills, scans, counts and
some other metrics. One of this metrics is called waste. These metrics
are presented in procfs as /proc/lmkstats.

Patchset:
0001-android-Collect-statistics-from-lowmemorykiller.patch
0002-oom-Add-notification-for-oom_score_adj.patch
0003-mm-Remove-RCU-and-tasklocks-from-lmk.patch


Collect-statistics-from-lowmemorykiller.patch
---------------------------------------------
This patch only adds metrics and is there to show
behavour before and after and is a good way to
see that the device is in waste zone.


0002-oom-Add-notification-for-oom_score_adj.patch
------------------------------------------------
This is the prerequisite patch to be able to do
the lowmemorykiller change. It introduces notifiers
for oom_score_adj. It generates notifier events for
process creation and death, and when process values
are changed.  These patches are outside from stageing
drivers and are applied to core functions in e.g. fork.c.

0003-mm-Remove-RCU-and-tasklocks-from-lmk.patch
-----------------------------------------------
This patch is the change of lowmemorykiller. It
builds a tree structure that works as cache for
the task list, but only contains the tasks that
are relevant for the lmk. The key thing here is
that the cache is sorted based on the oom_score_adj
value so the scan and count function can find
the right task with only a tree first operation.
Based on the right task the count can give a
proper reply and give a right estimate of the
amount it will free, and more important when
it is not willing to free anything. This makes
the shrinker not to call the scan function at all,
and when it is called it actually do what it's
supposed to do that is to free up some memory.
I consider this as mm based on the behaviour
changes for the shrinker even if the code is
a driver.

About testing.
Reproduce the problem. For this the first patch is needed and enabeld.
It does not change the lowmemory killer other than it add some metrics.
One counter is called WASTE. This is what this patch-set is about.
In android environment this can be tested directly. On other systems
like fedora a method using the stress package can be used. Apply the
patches. (First with only metrics) then in your

shell: echo 400 > /proc/self/oom_score_adj

Now you have created a shell that has something that can be killed.
In the same shell use stress program. The parameters will be very
dependent on your configuration, but you need to run out of memmory.

Most of the wasted cpu cycles are accounted in kswapd0 task so a compare
of the reduced waste can also be seen in the schedstat for that task.
However activitymanager will get some more work done in kernel space.
Finaly the new version also has the WASTE counter, but this one is
the cost of only a rbtree search.

Cost/Drawback
The impact on the fork call is on a 2ghz arm64 is about 500ns for the
notifier.

[toc] | [next] | [standalone]

#1577725 — [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller

This collects stats for shrinker calls and how much
waste work we do within the lowmemorykiller.

Signed-off-by: Peter Enderborg <peter.enderborg@sonymobile.com>
---
 drivers/staging/android/Kconfig                 | 11 ++++
 drivers/staging/android/Makefile                |  1 +
 drivers/staging/android/lowmemorykiller.c       |  9 ++-
 drivers/staging/android/lowmemorykiller_stats.c | 85 +++++++++++++++++++++++++
 drivers/staging/android/lowmemorykiller_stats.h | 29 +++++++++
 5 files changed, 134 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/android/lowmemorykiller_stats.c
 create mode 100644 drivers/staging/android/lowmemorykiller_stats.h

diff --git a/drivers/staging/android/Kconfig b/drivers/staging/android/Kconfig
index 6c00d6f..96e86c7 100644
--- a/drivers/staging/android/Kconfig
+++ b/drivers/staging/android/Kconfig
@@ -24,6 +24,17 @@ config ANDROID_LOW_MEMORY_KILLER
       scripts (/init.rc), and it defines priority values with minimum free memory size
       for each priority.
 
+config ANDROID_LOW_MEMORY_KILLER_STATS
+    bool "Android Low Memory Killer: collect statistics"
+    depends on ANDROID_LOW_MEMORY_KILLER
+    default n
+    help
+      Create a file in /proc/lmkstats that includes
+      collected statistics about kills, scans and counts
+      and  interaction with the shrinker. Its content
+      will be different depeding on lmk implementation used.
+
+
 source "drivers/staging/android/ion/Kconfig"
 
 endif # if ANDROID
diff --git a/drivers/staging/android/Makefile b/drivers/staging/android/Makefile
index 7ed1be7..d710eb2 100644
--- a/drivers/staging/android/Makefile
+++ b/drivers/staging/android/Makefile
@@ -4,3 +4,4 @@ obj-y                    += ion/
 
 obj-$(CONFIG_ASHMEM)            += ashmem.o
 obj-$(CONFIG_ANDROID_LOW_MEMORY_KILLER)    += lowmemorykiller.o
+obj-$(CONFIG_ANDROID_LOW_MEMORY_KILLER_STATS)    += lowmemorykiller_stats.o
diff --git a/drivers/staging/android/lowmemorykiller.c b/drivers/staging/android/lowmemorykiller.c
index ec3b665..15c1b38 100644
--- a/drivers/staging/android/lowmemorykiller.c
+++ b/drivers/staging/android/lowmemorykiller.c
@@ -42,6 +42,7 @@
 #include <linux/rcupdate.h>
 #include <linux/profile.h>
 #include <linux/notifier.h>
+#include "lowmemorykiller_stats.h"
 
 static u32 lowmem_debug_level = 1;
 static short lowmem_adj[6] = {
@@ -72,6 +73,7 @@ static unsigned long lowmem_deathpending_timeout;
 static unsigned long lowmem_count(struct shrinker *s,
                   struct shrink_control *sc)
 {
+    lmk_inc_stats(LMK_COUNT);
     return global_node_page_state(NR_ACTIVE_ANON) +
         global_node_page_state(NR_ACTIVE_FILE) +
         global_node_page_state(NR_INACTIVE_ANON) +
@@ -95,6 +97,7 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc)
                 global_node_page_state(NR_SHMEM) -
                 total_swapcache_pages();
 
+    lmk_inc_stats(LMK_SCAN);
     if (lowmem_adj_size < array_size)
         array_size = lowmem_adj_size;
     if (lowmem_minfree_size < array_size)
@@ -134,6 +137,7 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc)
         if (task_lmk_waiting(p) &&
             time_before_eq(jiffies, lowmem_deathpending_timeout)) {
             task_unlock(p);
+            lmk_inc_stats(LMK_TIMEOUT);
             rcu_read_unlock();
             return 0;
         }
@@ -179,7 +183,9 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc)
                  other_free * (long)(PAGE_SIZE / 1024));
         lowmem_deathpending_timeout = jiffies + HZ;
         rem += selected_tasksize;
-    }
+        lmk_inc_stats(LMK_KILL);
+    } else
+        lmk_inc_stats(LMK_WASTE);
 
     lowmem_print(4, "lowmem_scan %lu, %x, return %lu\n",
              sc->nr_to_scan, sc->gfp_mask, rem);
@@ -196,6 +202,7 @@ static struct shrinker lowmem_shrinker = {
 static int __init lowmem_init(void)
 {
     register_shrinker(&lowmem_shrinker);
+    init_procfs_lmk();
     return 0;
 }
 device_initcall(lowmem_init);
diff --git a/drivers/staging/android/lowmemorykiller_stats.c b/drivers/staging/android/lowmemorykiller_stats.c
new file mode 100644
index 0000000..673691c
--- /dev/null
+++ b/drivers/staging/android/lowmemorykiller_stats.c
@@ -0,0 +1,85 @@
+/*
+ *  lowmemorykiller_stats
+ *
+ *  Copyright (C) 2017 Sony Mobile Communications Inc.
+ *
+ *  Author: Peter Enderborg <peter.enderborg@sonymobile.com>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2 as
+ *  published by the Free Software Foundation.
+ */
+/* This code is bookkeeping of statistical information
+ * from lowmemorykiller and provide a node in proc "/proc/lmkstats".
+ */
+
+#include <linux/proc_fs.h>
+#include <linux/seq_file.h>
+#include "lowmemorykiller_stats.h"
+
+struct lmk_stats {
+    atomic_long_t scans; /* counter as in shrinker scans */
+    atomic_long_t kills; /* the number of sigkills sent */
+    atomic_long_t waste; /* the numer of extensive calls that did
+                  * not lead to anything
+                  */
+    atomic_long_t timeout; /* counter for shrinker calls that needed
+                * to be cancelled due to pending kills
+                */
+    atomic_long_t count; /* number of shrinker count calls */
+    atomic_long_t unknown; /* internal */
+} st;
+
+void lmk_inc_stats(int key)
+{
+    switch (key) {
+    case LMK_SCAN:
+        atomic_long_inc(&st.scans);
+        break;
+    case LMK_KILL:
+        atomic_long_inc(&st.kills);
+        break;
+    case LMK_WASTE:
+        atomic_long_inc(&st.waste);
+        break;
+    case LMK_TIMEOUT:
+        atomic_long_inc(&st.timeout);
+        break;
+    case LMK_COUNT:
+        atomic_long_inc(&st.count);
+        break;
+    default:
+        atomic_long_inc(&st.unknown);
+        break;
+    }
+}
+
+static int lmk_proc_show(struct seq_file *m, void *v)
+{
+    seq_printf(m, "kill: %ld\n", atomic_long_read(&st.kills));
+    seq_printf(m, "scan: %ld\n", atomic_long_read(&st.scans));
+    seq_printf(m, "waste: %ld\n", atomic_long_read(&st.waste));
+    seq_printf(m, "timeout: %ld\n", atomic_long_read(&st.timeout));
+    seq_printf(m, "count: %ld\n", atomic_long_read(&st.count));
+    seq_printf(m, "unknown: %ld (internal)\n",
+           atomic_long_read(&st.unknown));
+
+    return 0;
+}
+
+static int lmk_proc_open(struct inode *inode, struct file *file)
+{
+    return single_open(file, lmk_proc_show, PDE_DATA(inode));
+}
+
+static const struct file_operations lmk_proc_fops = {
+    .open        = lmk_proc_open,
+    .read        = seq_read,
+    .release    = single_release
+};
+
+int __init init_procfs_lmk(void)
+{
+    proc_create_data(LMK_PROCFS_NAME, 0444, NULL, &lmk_proc_fops, NULL);
+    return 0;
+}
diff --git a/drivers/staging/android/lowmemorykiller_stats.h b/drivers/staging/android/lowmemorykiller_stats.h
new file mode 100644
index 0000000..abeb6924
--- /dev/null
+++ b/drivers/staging/android/lowmemorykiller_stats.h
@@ -0,0 +1,29 @@
+/*
+ *  lowmemorykiller_stats interface
+ *
+ *  Copyright (C) 2017 Sony Mobile Communications Inc.
+ *
+ *  Author: Peter Enderborg <peter.enderborg@sonymobile.com>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2 as
+ *  published by the Free Software Foundation.
+ */
+
+enum  lmk_kill_stats {
+    LMK_SCAN = 1,
+    LMK_KILL = 2,
+    LMK_WASTE = 3,
+    LMK_TIMEOUT = 4,
+    LMK_COUNT = 5
+};
+
+#define LMK_PROCFS_NAME "lmkstats"
+
+#ifdef CONFIG_ANDROID_LOW_MEMORY_KILLER_STATS
+void lmk_inc_stats(int key);
+int __init init_procfs_lmk(void);
+#else
+static inline void lmk_inc_stats(int key) { return; };
+static inline int __init init_procfs_lmk(void) { return 0; };
+#endif
-- 
2.4.2

[toc] | [prev] | [next] | [standalone]

#1577939 — Re: [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller

On Thu, Feb 09, 2017 at 04:42:35PM +0100, peter enderborg wrote:
> This collects stats for shrinker calls and how much
> waste work we do within the lowmemorykiller.
> 
> Signed-off-by: Peter Enderborg <peter.enderborg@sonymobile.com>
> ---
>  drivers/staging/android/Kconfig                 | 11 ++++
>  drivers/staging/android/Makefile                |  1 +
>  drivers/staging/android/lowmemorykiller.c       |  9 ++-
>  drivers/staging/android/lowmemorykiller_stats.c | 85 +++++++++++++++++++++++++
>  drivers/staging/android/lowmemorykiller_stats.h | 29 +++++++++
>  5 files changed, 134 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/staging/android/lowmemorykiller_stats.c
>  create mode 100644 drivers/staging/android/lowmemorykiller_stats.h

What changed from v1?

> @@ -72,6 +73,7 @@ static unsigned long lowmem_deathpending_timeout;
>  static unsigned long lowmem_count(struct shrinker *s,
>                    struct shrink_control *sc)
>  {
> +    lmk_inc_stats(LMK_COUNT);
>      return global_node_page_state(NR_ACTIVE_ANON) +
>          global_node_page_state(NR_ACTIVE_FILE) +
>          global_node_page_state(NR_INACTIVE_ANON) +

Your email client is eating tabs and spitting out spaces, making this
impossible to even consider being merged :(

Please fix your email client, documentation for how to do so is in the
kernel Documentation directory.

thanks,

greg k-h

[toc] | [prev] | [next] | [standalone]

#1578213 — Re: [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller

On 02/09/2017 09:13 PM, Greg Kroah-Hartman wrote:
> On Thu, Feb 09, 2017 at 04:42:35PM +0100, peter enderborg wrote:
>> This collects stats for shrinker calls and how much
>> waste work we do within the lowmemorykiller.
>>
>> Signed-off-by: Peter Enderborg <peter.enderborg@sonymobile.com>
>> ---
>>  drivers/staging/android/Kconfig                 | 11 ++++
>>  drivers/staging/android/Makefile                |  1 +
>>  drivers/staging/android/lowmemorykiller.c       |  9 ++-
>>  drivers/staging/android/lowmemorykiller_stats.c | 85 +++++++++++++++++++++++++
>>  drivers/staging/android/lowmemorykiller_stats.h | 29 +++++++++
>>  5 files changed, 134 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/staging/android/lowmemorykiller_stats.c
>>  create mode 100644 drivers/staging/android/lowmemorykiller_stats.h
> What changed from v1?
Nothing. I thought I found the reason why my tabs are replaced by spaces in transport.

>> @@ -72,6 +73,7 @@ static unsigned long lowmem_deathpending_timeout;
>>  static unsigned long lowmem_count(struct shrinker *s,
>>                    struct shrink_control *sc)
>>  {
>> +    lmk_inc_stats(LMK_COUNT);
>>      return global_node_page_state(NR_ACTIVE_ANON) +
>>          global_node_page_state(NR_ACTIVE_FILE) +
>>          global_node_page_state(NR_INACTIVE_ANON) +
> Your email client is eating tabs and spitting out spaces, making this
> impossible to even consider being merged :(
>
> Please fix your email client, documentation for how to do so is in the
> kernel Documentation directory.
>
> thanks,
>
> greg k-h

[toc] | [prev] | [next] | [standalone]

#1578365

[I have only now see this cover - it answers some of the questions I've
 had to specific patches. It would be really great if you could use git
 send-email to post patch series - it just does the right thing(tm)]

On Thu 09-02-17 14:21:40, peter enderborg wrote:
> Lowmemorykiller efficiency problem and a solution.
> 
> Lowmemorykiller in android has a severe efficiency problem. The basic
> problem is that the registered shrinker gets called very often without
>  anything actually happening.

Which is an inherent problem because lkml doesn't belong to shrinkers
infrastructure.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1579903

On 02/10/2017 11:27 AM, Michal Hocko wrote:
> [I have only now see this cover - it answers some of the questions I've
>  had to specific patches. It would be really great if you could use git
>  send-email to post patch series - it just does the right thing(tm)]
>
> On Thu 09-02-17 14:21:40, peter enderborg wrote:
>> Lowmemorykiller efficiency problem and a solution.
>>
>> Lowmemorykiller in android has a severe efficiency problem. The basic
>> problem is that the registered shrinker gets called very often without
>>  anything actually happening.
> Which is an inherent problem because lkml doesn't belong to shrinkers
> infrastructure.

Not really what this patch address.  I see it as a problem with shrinker
that there no slow-path-free (scan/count) where it should belong.
This patch address a specific problem where lot of cpu are wasted
in low memory conditions.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web