Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1543339

crash during oom reaper (was: Re: [PATCH 4/4] [RFC!] mm: 'struct mm_struct' reference counting debugging)

From Michal Hocko <mhocko@kernel.org>
Newsgroups linux.kernel
Subject crash during oom reaper (was: Re: [PATCH 4/4] [RFC!] mm: 'struct mm_struct' reference counting debugging)
Date 2016-12-16 11:20 +0100
Message-ID <sOXMm-2df-27@gated-at.bofh.it> (permalink)
References <sOWwW-19z-41@gated-at.bofh.it> <sOWGD-1t7-59@gated-at.bofh.it> <sOWGD-1t7-57@gated-at.bofh.it> <sOXjk-1Nk-25@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On Fri 16-12-16 10:43:52, Vegard Nossum wrote:
[...]
> I don't think it's a bug in the OOM reaper itself, but either of the
> following two patches will fix the problem (without my understand how or
> why):
> 
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index ec9f11d4f094..37b14b2e2af4 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -485,7 +485,7 @@ static bool __oom_reap_task_mm(struct task_struct *tsk,
> struct mm_struct *mm)
>  	 */
>  	mutex_lock(&oom_lock);
> 
> -	if (!down_read_trylock(&mm->mmap_sem)) {
> +	if (!down_write_trylock(&mm->mmap_sem)) {

__oom_reap_task_mm is basically the same thing as MADV_DONTNEED and that
doesn't require the exlusive mmap_sem. So this looks correct to me.
[...]

> --OR--
> 
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index ec9f11d4f094..559aec0acd21 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -508,6 +508,7 @@ static bool __oom_reap_task_mm(struct task_struct *tsk,
> struct mm_struct *mm)
>  	 */
>  	set_bit(MMF_UNSTABLE, &mm->flags);
> 
> +#if 0
>  	tlb_gather_mmu(&tlb, mm, 0, -1);
>  	for (vma = mm->mmap ; vma; vma = vma->vm_next) {
>  		if (is_vm_hugetlb_page(vma))
> @@ -535,6 +536,7 @@ static bool __oom_reap_task_mm(struct task_struct *tsk,
> struct mm_struct *mm)
>  					 &details);
>  	}
>  	tlb_finish_mmu(&tlb, 0, -1);
> +#endif

same here, nothing different from the madvise... Well, except for the
MMF_UNSTABLE part which will force any page fault on this mm to SEGV.

>  	pr_info("oom_reaper: reaped process %d (%s), now anon-rss:%lukB,
> file-rss:%lukB, shmem-rss:%lukB\n",
>  			task_pid_nr(tsk), tsk->comm,
>  			K(get_mm_counter(mm, MM_ANONPAGES)),
> 
> Maybe it's just the fact that we're not releasing the memory and so some
> other bit of code is not able to make enough progress to trigger the
> bug, although curiously, if I just move the #if 0..#endif inside
> tlb_gather_mmu()..tlb_finish_mmu() itself (so just calling tlb_*()
> without doing the for-loop), it still reproduces the crash.

What is the atual crash?

> Another clue, although it might just be a coincidence, is that it seems
> the VMA/file in question is always a mapping for the exe file itself
> (the reason I think this might be a coincidence is that the exe file
> mapping is the first one and we usually traverse VMAs starting with this
> one, that doesn't mean the other VMAs aren't affected by the same
> problem, just that we never hit them).

You can experiment a bit and exclude PROT_EXEC vmas...
-- 
Michal Hocko
SUSE Labs

Back to linux.kernel | Previous | NextPrevious in thread | Next in thread | Find similar | Unroll thread


Thread

[PATCH 1/4] mm: add new mmgrab() helper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 10:00 +0100
  Re: [PATCH 4/4] [RFC!] mm: 'struct mm_struct' reference counting  debugging Michal Hocko <mhocko@kernel.org> - 2016-12-16 10:10 +0100
    Re: [PATCH 4/4] [RFC!] mm: 'struct mm_struct' reference counting  debugging Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 10:50 +0100
      crash during oom reaper (was: Re: [PATCH 4/4] [RFC!] mm: 'struct  mm_struct' reference counting debugging) Michal Hocko <mhocko@kernel.org> - 2016-12-16 11:20 +0100
        Re: crash during oom reaper (was: Re: [PATCH 4/4] [RFC!] mm: 'struct  mm_struct' reference counting debugging) "Kirill A. Shutemov" <kirill@shutemov.name> - 2016-12-16 11:50 +0100
          Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-16 13:00 +0100
            Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-16 13:20 +0100
            Re: crash during oom reaper "Kirill A. Shutemov" <kirill@shutemov.name> - 2016-12-16 13:50 +0100
              Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-16 14:00 +0100
                Re: crash during oom reaper "Kirill A. Shutemov" <kirill@shutemov.name> - 2016-12-16 14:10 +0100
                Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-16 14:20 +0100
                Re: crash during oom reaper Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-18 14:50 +0100
                Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-18 17:10 +0100
        Re: crash during oom reaper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 14:20 +0100
          Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-16 15:10 +0100
            Re: crash during oom reaper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 15:30 +0100
              Re: crash during oom reaper Michal Hocko <mhocko@kernel.org> - 2016-12-16 15:50 +0100
                Re: crash during oom reaper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 16:00 +0100
          Re: crash during oom reaper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 15:10 +0100
  Re: [PATCH 3/4] mm: use mmget_not_zero() helper Michal Hocko <mhocko@kernel.org> - 2016-12-16 10:30 +0100
  Re: [PATCH 2/4] mm: add new mmget() helper Michal Hocko <mhocko@kernel.org> - 2016-12-16 10:30 +0100
  [PATCH 3/4] mm: use mmget_not_zero() helper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 10:40 +0100
  [PATCH 2/4] mm: add new mmget() helper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 10:50 +0100
  Re: [PATCH 1/4] mm: add new mmgrab() helper Michal Hocko <mhocko@kernel.org> - 2016-12-16 10:50 +0100
  Re: [PATCH 1/4] mm: add new mmgrab() helper Peter Zijlstra <peterz@infradead.org> - 2016-12-16 11:00 +0100
    Re: [PATCH 1/4] mm: add new mmgrab() helper "Kirill A. Shutemov" <kirill@shutemov.name> - 2016-12-16 11:20 +0100
      Re: [PATCH 1/4] mm: add new mmgrab() helper Michal Hocko <mhocko@kernel.org> - 2016-12-16 11:50 +0100
      Re: [PATCH 1/4] mm: add new mmgrab() helper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 12:00 +0100
    Re: [PATCH 1/4] mm: add new mmgrab() helper Vegard Nossum <vegard.nossum@oracle.com> - 2016-12-16 12:20 +0100

csiph-web