Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1412954 > unrolled thread

[PATCH 06/10] mm, oom: kill all tasks sharing the mm

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 06/10] mm, oom: kill all tasks sharing the mm Michal Hocko <mhocko@kernel.org> - 2016-06-03 11:20 +0200
    Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm David Rientjes <rientjes@google.com> - 2016-06-07 00:30 +0200
      Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm Oleg Nesterov <oleg@redhat.com> - 2016-06-07 02:30 +0200
        Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm Michal Hocko <mhocko@kernel.org> - 2016-06-07 08:40 +0200
        Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm David Rientjes <rientjes@google.com> - 2016-06-08 00:20 +0200
          Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm Michal Hocko <mhocko@kernel.org> - 2016-06-08 08:30 +0200
            Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm David Rientjes <rientjes@google.com> - 2016-06-09 01:00 +0200
              Re: [PATCH 06/10] mm, oom: kill all tasks sharing the mm Michal Hocko <mhocko@kernel.org> - 2016-06-09 08:50 +0200

#1412954 — [PATCH 06/10] mm, oom: kill all tasks sharing the mm

From: Michal Hocko <mhocko@suse.com>

Currently oom_kill_process skips both the oom reaper and SIG_KILL if a
process sharing the same mm is unkillable via OOM_ADJUST_MIN. After "mm,
oom_adj: make sure processes sharing mm have same view of oom_score_adj"
all such processes are sharing the same value so we shouldn't see such a
task at all (oom_badness would rule them out).

We can still encounter oom disabled vforked task which has to be killed
as well if we want to have other tasks sharing the mm reapable
because it can access the memory before doing exec. Killing such a task
should be acceptable because it is highly unlikely it has done anything
useful because it cannot modify any memory before it calls exec. An
alternative would be to keep the task alive and skip the oom reaper and
risk all the weird corner cases where the OOM killer cannot make forward
progress because the oom victim hung somewhere on the way to exit.

There is a potential race where we kill the oom disabled task which is
highly unlikely but possible. It would happen if __set_oom_adj raced
with select_bad_process and then it is OK to consider the old value or
with fork when it should be acceptable as well.
Let's add a little note to the log so that people would tell us that
this really happens in the real life and it matters.

Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/oom_kill.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 2c604a9a8305..22affacaf38b 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -847,8 +847,7 @@ void oom_kill_process(struct oom_control *oc, struct task_struct *p,
 			continue;
 		if (same_thread_group(p, victim))
 			continue;
-		if (unlikely(p->flags & PF_KTHREAD) || is_global_init(p) ||
-		    p->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) {
+		if (unlikely(p->flags & PF_KTHREAD) || is_global_init(p)) {
 			/*
 			 * We cannot use oom_reaper for the mm shared by this
 			 * process because it wouldn't get killed and so the
@@ -857,6 +856,11 @@ void oom_kill_process(struct oom_control *oc, struct task_struct *p,
 			can_oom_reap = false;
 			continue;
 		}
+		if (p->signal->oom_score_adj == OOM_ADJUST_MIN)
+			pr_warn("%s pid=%d shares mm with oom disabled %s pid=%d. Seems like misconfiguration, killing anyway!"
+					" Report at linux-mm@kvack.org\n",
+					victim->comm, task_pid_nr(victim),
+					p->comm, task_pid_nr(p));
 		do_send_sig_info(SIGKILL, SEND_SIG_FORCED, p, true);
 	}
 	rcu_read_unlock();
-- 
2.8.1

[toc] | [next] | [standalone]

#1415521

On Fri, 3 Jun 2016, Michal Hocko wrote:

> From: Michal Hocko <mhocko@suse.com>
> 
> Currently oom_kill_process skips both the oom reaper and SIG_KILL if a
> process sharing the same mm is unkillable via OOM_ADJUST_MIN. After "mm,
> oom_adj: make sure processes sharing mm have same view of oom_score_adj"
> all such processes are sharing the same value so we shouldn't see such a
> task at all (oom_badness would rule them out).
> 
> We can still encounter oom disabled vforked task which has to be killed
> as well if we want to have other tasks sharing the mm reapable
> because it can access the memory before doing exec. Killing such a task
> should be acceptable because it is highly unlikely it has done anything
> useful because it cannot modify any memory before it calls exec. An
> alternative would be to keep the task alive and skip the oom reaper and
> risk all the weird corner cases where the OOM killer cannot make forward
> progress because the oom victim hung somewhere on the way to exit.
> 
> There is a potential race where we kill the oom disabled task which is
> highly unlikely but possible. It would happen if __set_oom_adj raced
> with select_bad_process and then it is OK to consider the old value or
> with fork when it should be acceptable as well.
> Let's add a little note to the log so that people would tell us that
> this really happens in the real life and it matters.
> 

We cannot kill oom disabled processes at all, little race or otherwise.  
We'd rather panic the system than oom kill these processes, and that's the 
semantic that the user is basing their decision on.  We cannot suddenly 
start allowing them to be SIGKILL'd.

[toc] | [prev] | [next] | [standalone]

#1415599

On 06/06, David Rientjes wrote:
>
> > There is a potential race where we kill the oom disabled task which is
> > highly unlikely but possible. It would happen if __set_oom_adj raced
> > with select_bad_process and then it is OK to consider the old value or
> > with fork when it should be acceptable as well.
> > Let's add a little note to the log so that people would tell us that
> > this really happens in the real life and it matters.
> >
>
> We cannot kill oom disabled processes at all, little race or otherwise.

But this change doesn't really make it worse?

Oleg.

[toc] | [prev] | [next] | [standalone]

#1415748

On Tue 07-06-16 01:20:08, Oleg Nesterov wrote:
> On 06/06, David Rientjes wrote:
> >
> > > There is a potential race where we kill the oom disabled task which is
> > > highly unlikely but possible. It would happen if __set_oom_adj raced
> > > with select_bad_process and then it is OK to consider the old value or
> > > with fork when it should be acceptable as well.
> > > Let's add a little note to the log so that people would tell us that
> > > this really happens in the real life and it matters.
> > >
> >
> > We cannot kill oom disabled processes at all, little race or otherwise.
> 
> But this change doesn't really make it worse?

Exactly, the race was always there. We could mitigate it to some degree
by (ab)using oom_lock in __set_oom_adj. But I guess this is just an
overkill.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1416677

On Tue, 7 Jun 2016, Oleg Nesterov wrote:

> On 06/06, David Rientjes wrote:
> >
> > > There is a potential race where we kill the oom disabled task which is
> > > highly unlikely but possible. It would happen if __set_oom_adj raced
> > > with select_bad_process and then it is OK to consider the old value or
> > > with fork when it should be acceptable as well.
> > > Let's add a little note to the log so that people would tell us that
> > > this really happens in the real life and it matters.
> > >
> >
> > We cannot kill oom disabled processes at all, little race or otherwise.
> 
> But this change doesn't really make it worse?
> 

Why is the patch asking users to report oom killing of a process that 
raced with setting /proc/pid/oom_score_adj to OOM_SCORE_ADJ_MIN?  What is 
possibly actionable about it?

[toc] | [prev] | [next] | [standalone]

#1416896

On Tue 07-06-16 15:15:37, David Rientjes wrote:
> On Tue, 7 Jun 2016, Oleg Nesterov wrote:
> 
> > On 06/06, David Rientjes wrote:
> > >
> > > > There is a potential race where we kill the oom disabled task which is
> > > > highly unlikely but possible. It would happen if __set_oom_adj raced
> > > > with select_bad_process and then it is OK to consider the old value or
> > > > with fork when it should be acceptable as well.
> > > > Let's add a little note to the log so that people would tell us that
> > > > this really happens in the real life and it matters.
> > > >
> > >
> > > We cannot kill oom disabled processes at all, little race or otherwise.
> > 
> > But this change doesn't really make it worse?
> > 
> 
> Why is the patch asking users to report oom killing of a process that 
> raced with setting /proc/pid/oom_score_adj to OOM_SCORE_ADJ_MIN?  What is 
> possibly actionable about it?

Well, the primary point is to know whether such races happen in the real
loads and whether they actually matter. If yes we can harden the locking
or come up with a less racy solutions.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1417940

On Wed, 8 Jun 2016, Michal Hocko wrote:

> > Why is the patch asking users to report oom killing of a process that 
> > raced with setting /proc/pid/oom_score_adj to OOM_SCORE_ADJ_MIN?  What is 
> > possibly actionable about it?
> 
> Well, the primary point is to know whether such races happen in the real
> loads and whether they actually matter. If yes we can harden the locking
> or come up with a less racy solutions.

A thread being set to oom disabled while racing with the oom killer 
obviously isn't a concern: it could very well be set to oom disabled after 
the SIGKILL is sent and before the signal is handled, and that's not even 
fixable without unneeded complexity because we don't know the source of 
the SIGKILL.  Please remove the printk entirely.

[toc] | [prev] | [next] | [standalone]

#1418106

On Wed 08-06-16 15:51:20, David Rientjes wrote:
> On Wed, 8 Jun 2016, Michal Hocko wrote:
> 
> > > Why is the patch asking users to report oom killing of a process that 
> > > raced with setting /proc/pid/oom_score_adj to OOM_SCORE_ADJ_MIN?  What is 
> > > possibly actionable about it?
> > 
> > Well, the primary point is to know whether such races happen in the real
> > loads and whether they actually matter. If yes we can harden the locking
> > or come up with a less racy solutions.
> 
> A thread being set to oom disabled while racing with the oom killer 
> obviously isn't a concern: it could very well be set to oom disabled after 
> the SIGKILL is sent and before the signal is handled, and that's not even 
> fixable without unneeded complexity because we don't know the source of 
> the SIGKILL.  Please remove the printk entirely.

OK, if you find it more confusing than useful I will not insist.

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web