Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1427803 > unrolled thread

[PATCH] mm/huge_memory: fix the memory leak due to the race

Started byzhongjiang <zhongjiang@huawei.com>
First post2016-06-21 16:20 +0200
Last post2016-06-22 12:10 +0200
Articles 6 — 4 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH] mm/huge_memory: fix the memory leak due to the race zhongjiang <zhongjiang@huawei.com> - 2016-06-21 16:20 +0200
    Re: [PATCH] mm/huge_memory: fix the memory leak due to the race Michal Hocko <mhocko@kernel.org> - 2016-06-21 17:00 +0200
    Re: [PATCH] mm/huge_memory: fix the memory leak due to the race "Kirill A. Shutemov" <kirill@shutemov.name> - 2016-06-21 17:40 +0200
      Re: [PATCH] mm/huge_memory: fix the memory leak due to the race "Kirill A. Shutemov" <kirill@shutemov.name> - 2016-06-21 18:40 +0200
        Re: [PATCH] mm/huge_memory: fix the memory leak due to the race zhong jiang <zhongjiang@huawei.com> - 2016-06-22 05:10 +0200
        Re: [PATCH] mm/huge_memory: fix the memory leak due to the race zhong jiang <zhongjiang@huawei.com> - 2016-06-22 12:10 +0200

#1427803 — [PATCH] mm/huge_memory: fix the memory leak due to the race

Fromzhongjiang <zhongjiang@huawei.com>
Date2016-06-21 16:20 +0200
Subject[PATCH] mm/huge_memory: fix the memory leak due to the race
Message-ID<rMuH0-6Vn-29@gated-at.bofh.it>
From: zhong jiang <zhongjiang@huawei.com>

with great pressure, I run some test cases. As a result, I found
that the THP is not freed, it is detected by check_mm().

BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512

Consider the following race :

	CPU0                               CPU1
  __handle_mm_fault()
        wp_huge_pmd()
   	    do_huge_pmd_wp_page()
		pmdp_huge_clear_flush_notify()
                (pmd_none = true)
					exit_mmap()
					   unmap_vmas()
					     zap_pmd_range()
						pmd_none_or_trans_huge_or_clear_bad()
						   (result in memory leak)
                set_pmd_at()

because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
and it make the pmd entry to be null. Therefore, The memory leak can occur.

The patch fix the scenario that the pmd entry can lead to be null.

Signed-off-by: zhong jiang <zhongjiang@huawei.com>
---
 mm/huge_memory.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e10a4fe..ef04b94 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1340,11 +1340,11 @@ alloc:
 		pmd_t entry;
 		entry = mk_huge_pmd(new_page, vma->vm_page_prot);
 		entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
-		pmdp_huge_clear_flush_notify(vma, haddr, pmd);
+		pmdp_invalidate(vma, haddr, pmd);	
 		page_add_new_anon_rmap(new_page, vma, haddr, true);
 		mem_cgroup_commit_charge(new_page, memcg, false, true);
 		lru_cache_add_active_or_unevictable(new_page, vma);
-		set_pmd_at(mm, haddr, pmd, entry);
+		pmd_populate(mm, pmd, entry);
 		update_mmu_cache_pmd(vma, address, pmd);
 		if (!page) {
 			add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
-- 
1.8.3.1

[toc] | [next] | [standalone]

#1427848

FromMichal Hocko <mhocko@kernel.org>
Date2016-06-21 17:00 +0200
Message-ID<rMvjH-797-19@gated-at.bofh.it>
In reply to#1427803
[CCing Kirill]

On Tue 21-06-16 22:05:56, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
> 
> with great pressure, I run some test cases. As a result, I found
> that the THP is not freed, it is detected by check_mm().
> 
> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
> 
> Consider the following race :
> 
> 	CPU0                               CPU1
>   __handle_mm_fault()
>         wp_huge_pmd()
>    	    do_huge_pmd_wp_page()
> 		pmdp_huge_clear_flush_notify()
>                 (pmd_none = true)
> 					exit_mmap()
> 					   unmap_vmas()
> 					     zap_pmd_range()
> 						pmd_none_or_trans_huge_or_clear_bad()
> 						   (result in memory leak)
>                 set_pmd_at()
>
> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
> and it make the pmd entry to be null. Therefore, The memory leak can occur.

I do not understand this description. CPU1 is in the exit path with last
mm user gone. So CPU0 is a different process with its own mm. How can
they influence each other. But maybe I am just missing your point.
 
> The patch fix the scenario that the pmd entry can lead to be null.
> 
> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> ---
>  mm/huge_memory.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index e10a4fe..ef04b94 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -1340,11 +1340,11 @@ alloc:
>  		pmd_t entry;
>  		entry = mk_huge_pmd(new_page, vma->vm_page_prot);
>  		entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
> -		pmdp_huge_clear_flush_notify(vma, haddr, pmd);
> +		pmdp_invalidate(vma, haddr, pmd);	
>  		page_add_new_anon_rmap(new_page, vma, haddr, true);
>  		mem_cgroup_commit_charge(new_page, memcg, false, true);
>  		lru_cache_add_active_or_unevictable(new_page, vma);
> -		set_pmd_at(mm, haddr, pmd, entry);
> +		pmd_populate(mm, pmd, entry);
>  		update_mmu_cache_pmd(vma, address, pmd);
>  		if (!page) {
>  			add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
> -- 
> 1.8.3.1

-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1427886

From"Kirill A. Shutemov" <kirill@shutemov.name>
Date2016-06-21 17:40 +0200
Message-ID<rMvWr-7DO-29@gated-at.bofh.it>
In reply to#1427803
On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
> 
> with great pressure, I run some test cases. As a result, I found
> that the THP is not freed, it is detected by check_mm().
> 
> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
> 
> Consider the following race :
> 
> 	CPU0                               CPU1
>   __handle_mm_fault()
>         wp_huge_pmd()
>    	    do_huge_pmd_wp_page()
> 		pmdp_huge_clear_flush_notify()
>                 (pmd_none = true)
> 					exit_mmap()
> 					   unmap_vmas()
> 					     zap_pmd_range()
> 						pmd_none_or_trans_huge_or_clear_bad()
> 						   (result in memory leak)
>                 set_pmd_at()
> 
> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
> and it make the pmd entry to be null. Therefore, The memory leak can occur.
> 
> The patch fix the scenario that the pmd entry can lead to be null.

I don't think the scenario is possible.

exit_mmap() called when all mm users have gone, so no parallel threads
exist.

-- 
 Kirill A. Shutemov

[toc] | [prev] | [next] | [standalone]

#1427942

From"Kirill A. Shutemov" <kirill@shutemov.name>
Date2016-06-21 18:40 +0200
Message-ID<rMwSt-8eX-13@gated-at.bofh.it>
In reply to#1427886
On Tue, Jun 21, 2016 at 11:19:07PM +0800, zhong jiang wrote:
> On 2016/6/21 22:37, Kirill A. Shutemov wrote:
> > On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
> >> From: zhong jiang <zhongjiang@huawei.com>
> >>
> >> with great pressure, I run some test cases. As a result, I found
> >> that the THP is not freed, it is detected by check_mm().
> >>
> >> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
> >>
> >> Consider the following race :
> >>
> >> 	CPU0                               CPU1
> >>   __handle_mm_fault()
> >>         wp_huge_pmd()
> >>    	    do_huge_pmd_wp_page()
> >> 		pmdp_huge_clear_flush_notify()
> >>                 (pmd_none = true)
> >> 					exit_mmap()
> >> 					   unmap_vmas()
> >> 					     zap_pmd_range()
> >> 						pmd_none_or_trans_huge_or_clear_bad()
> >> 						   (result in memory leak)
> >>                 set_pmd_at()
> >>
> >> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
> >> and it make the pmd entry to be null. Therefore, The memory leak can occur.
> >>
> >> The patch fix the scenario that the pmd entry can lead to be null.
> > I don't think the scenario is possible.
> >
> > exit_mmap() called when all mm users have gone, so no parallel threads
> > exist.
> >
>  Forget  this patch.  It 's my fault , it indeed don not exist.
>  But I  hit the following problem.  we can see the memory leak when the process exit.
>  
>  
>  Any suggestion will be apprecaited.

Could you try this:

http://lkml.kernel.org/r/20160621150433.GA7536@node.shutemov.name

-- 
 Kirill A. Shutemov

[toc] | [prev] | [next] | [standalone]

#1428377

Fromzhong jiang <zhongjiang@huawei.com>
Date2016-06-22 05:10 +0200
Message-ID<rMGIa-6js-11@gated-at.bofh.it>
In reply to#1427942
On 2016/6/21 23:29, Kirill A. Shutemov wrote:
> On Tue, Jun 21, 2016 at 11:19:07PM +0800, zhong jiang wrote:
>> On 2016/6/21 22:37, Kirill A. Shutemov wrote:
>>> On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>
>>>> with great pressure, I run some test cases. As a result, I found
>>>> that the THP is not freed, it is detected by check_mm().
>>>>
>>>> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
>>>>
>>>> Consider the following race :
>>>>
>>>> 	CPU0                               CPU1
>>>>   __handle_mm_fault()
>>>>         wp_huge_pmd()
>>>>    	    do_huge_pmd_wp_page()
>>>> 		pmdp_huge_clear_flush_notify()
>>>>                 (pmd_none = true)
>>>> 					exit_mmap()
>>>> 					   unmap_vmas()
>>>> 					     zap_pmd_range()
>>>> 						pmd_none_or_trans_huge_or_clear_bad()
>>>> 						   (result in memory leak)
>>>>                 set_pmd_at()
>>>>
>>>> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
>>>> and it make the pmd entry to be null. Therefore, The memory leak can occur.
>>>>
>>>> The patch fix the scenario that the pmd entry can lead to be null.
>>> I don't think the scenario is possible.
>>>
>>> exit_mmap() called when all mm users have gone, so no parallel threads
>>> exist.
>>>
>>  Forget  this patch.  It 's my fault , it indeed don not exist.
>>  But I  hit the following problem.  we can see the memory leak when the process exit.
>>  
>>  
>>  Any suggestion will be apprecaited.
> Could you try this:
>
> http://lkml.kernel.org/r/20160621150433.GA7536@node.shutemov.name
>
 I fails to open it.  can you  display or add  attachmemts ? :-)  thx

[toc] | [prev] | [next] | [standalone]

#1428643

Fromzhong jiang <zhongjiang@huawei.com>
Date2016-06-22 12:10 +0200
Message-ID<rMNgC-221-27@gated-at.bofh.it>
In reply to#1427942
On 2016/6/21 23:29, Kirill A. Shutemov wrote:
> On Tue, Jun 21, 2016 at 11:19:07PM +0800, zhong jiang wrote:
>> On 2016/6/21 22:37, Kirill A. Shutemov wrote:
>>> On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>
>>>> with great pressure, I run some test cases. As a result, I found
>>>> that the THP is not freed, it is detected by check_mm().
>>>>
>>>> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
>>>>
>>>> Consider the following race :
>>>>
>>>> 	CPU0                               CPU1
>>>>   __handle_mm_fault()
>>>>         wp_huge_pmd()
>>>>    	    do_huge_pmd_wp_page()
>>>> 		pmdp_huge_clear_flush_notify()
>>>>                 (pmd_none = true)
>>>> 					exit_mmap()
>>>> 					   unmap_vmas()
>>>> 					     zap_pmd_range()
>>>> 						pmd_none_or_trans_huge_or_clear_bad()
>>>> 						   (result in memory leak)
>>>>                 set_pmd_at()
>>>>
>>>> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
>>>> and it make the pmd entry to be null. Therefore, The memory leak can occur.
>>>>
>>>> The patch fix the scenario that the pmd entry can lead to be null.
>>> I don't think the scenario is possible.
>>>
>>> exit_mmap() called when all mm users have gone, so no parallel threads
>>> exist.
>>>
>>  Forget  this patch.  It 's my fault , it indeed don not exist.
>>  But I  hit the following problem.  we can see the memory leak when the process exit.
>>  
>>  
>>  Any suggestion will be apprecaited.
> Could you try this:
>
> http://lkml.kernel.org/r/20160621150433.GA7536@node.shutemov.name
 The patch I have seen ,  but I  don not think this patch  can fix so problem . if that race occur,  pmd entry points to
 the huge page will be changed ,  and freeze_page spilt pmd will fail. subsequent vm_bug_on() will fired.

 freeze_page()
     try_to_unmap()
         split_huge_pmd_address() (return fail) result in page_mapcount is not zero
 vm_bug_on()

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web