Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1440284

Re: bug in memcg oom-killer results in a hung syscall in another process in the same cgroup

Show all headers | View raw

On Sat 09-07-16 16:49:32, Shayan Pooya wrote:
> I came across the following issue in kernel 3.16 (Ubuntu 14.04) which
> was then reproduced in kernels 4.4 LTS:
> After a couple of of memcg oom-kills in a cgroup, a syscall in
> *another* process in the same cgroup hangs indefinitely.
> 
> Reproducing:
> 
> # mkdir -p strace_run
> #  mkdir /sys/fs/cgroup/memory/1
> # echo 1073741824 > /sys/fs/cgroup/memory/1/memory.limit_in_bytes
> # echo 0 > /sys/fs/cgroup/memory/1/memory.swappiness
> # for i in $(seq 1000); do ./call-mem-hog
> /sys/fs/cgroup/memory/1/cgroup.procs & done
> 
> Where call-mem-hog is:
> #!/bin/sh
> set -ex
> echo $$ > $1
> echo "Adding $$ to $1"
> strace -ff -tt ./mem-hog 2> strace_run/$$
> 
> 
> Initially I thought it was a userspace bug in dash as it only happened
> with /bin/sh (which points to dash) and not with bash. I see the
> following hanging processes:
> 
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root     20999  0.0  0.0   4508   100 pts/6    S    16:28   0:00
> /bin/sh ./call-mem-hog /sys/fs/cgroup/memory/1/cgroup.procs
> 
> However, when using strace, I noticed that sometimes there is actually
> a mem-hog process hanging on sbrk syscall (Of course the
> memory.oom_control is 0 and this is not expected).
> Sending an ABRT signal to the waiting strace process then resulted in
> the mem-hog process getting oom-killed by the kernel.

Could you post the stack trace of the hung oom victim? Also could you
post the full kernel log?
-- 
Michal Hocko
SUSE Labs

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

bug in memcg oom-killer results in a hung syscall in another process  in the same cgroup Shayan Pooya <shayan@liveve.org> - 2016-07-10 01:50 +0200
  Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Michal Hocko <mhocko@kernel.org> - 2016-07-11 08:50 +0200
    Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Shayan Pooya <shayan@liveve.org> - 2016-07-11 20:40 +0200
      Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Michal Hocko <mhocko@kernel.org> - 2016-07-12 09:20 +0200
        Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Shayan Pooya <shayan@liveve.org> - 2016-07-12 17:40 +0200
          Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Konstantin Khlebnikov <khlebnikov@yandex-team.ru> - 2016-07-12 18:00 +0200
            Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Oleg Nesterov <oleg@redhat.com> - 2016-07-12 19:00 +0200
            Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Shayan Pooya <shayan@liveve.org> - 2016-07-13 01:10 +0200
              Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Oleg Nesterov <oleg@redhat.com> - 2016-07-14 15:30 +0200
                Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Shayan Pooya <shayan@liveve.org> - 2016-07-14 17:40 +0200
                Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Shayan Pooya <shayan@liveve.org> - 2016-07-15 19:00 +0200
                Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Oleg Nesterov <oleg@redhat.com> - 2016-07-18 16:00 +0200
          Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Michal Hocko <mhocko@kernel.org> - 2016-07-13 10:10 +0200
    Re: bug in memcg oom-killer results in a hung syscall in another  process in the same cgroup Michal Hocko <mhocko@kernel.org> - 2016-07-12 09:20 +0200

csiph-web